This document provides definitions and best practices related to the specification of manifest files and similar document formats on the Web.
Some specifications on the Web deal with defining sets of files or resources to be consumed together. A common design pattern is to provide a manifest or configuration file that defines which resources are available and how various resources should be used or to provide various kinds of metadata about a collection of resources. This document provides definitions and best practices related to the specification of manifest files and similar document formats on the Web.
Some specifications on the Web deal with defining sets of files or resources to be consumed together. A common design pattern is to provide a manifest or configuration file that defines which resources are available and how various resources should be used or to provide various kinds of metadata about a collection of resources.
Some examples of specifications like this include Webapp [[APPMANIFEST]], Miniapp [[MINIAPP-MANIFEST]], and ePub [[EPUB-33]].
Manifest file formats usually contain syntactic or user-supplied values that are not localizable. But most manifest file formats also contain at least some localizable content fields. A specification that defines a manifest file format that includes localizable content values needs to provide a means for users to localize the manifest into different languages. This document covers the pros and cons of common designs, along with some best practices related to building and managing manifests.
These (and many other) best practices, along with links to supporting materials, can also be found in the Internationalization Best Practices for Spec Developers [[INTERNATIONAL-SPECS]]. In addition to the best practices found here, additional best practices relating to language metadata on the Web can be found in [[STRING-META]]. Core terminology can also be found in the Internationalization Glossary [[I18N-GLOSSARY]].
There are different ways that manifest files might be used on the Web and these affect the choice of manifest localization strategy.
Packaged Manifest. Some manifests are packaged with other files to form an application or document. Since the manifest is generally consumed as part of the larger set of content and the content is downloaded as a single unit, manifests of this type can use more modular localization strategies, such as using multiple files with names or paths based on the locale.
Shorthand Description Manifest. Some manifests are meant to be used as a shorthand description of an application, document, or other resource. Manifests of this type are often downloaded by the client, sometimes as a precursor to retrieving the actual resource or its component parts. Since retrieving multiple files has latency, size, and storage implications, this type of manifest usually seeks to use a single file or other less-modular approaches.
Another consideration is the language negotiation strategy employed by the specification. If a manifest can be generated on demand (rather than being cached at a well-known URL), then the localization might be applied on a per-request basis. This points more toward a shorthand manifest style. In other cases, where some implementations might be able to negotiate while others cannot, it might be important for the specification to permit different strategies to be used.
Bear in mind that modern operating environments support quite extensive sets of available locales and that application owners often wish to satisfy diverse audiences with a single localized application or document. While examples in specifications necessarily are constrained to perhaps three or four locales, actual applications with 50 or more specific locales are not uncommon.
Manifests need to match a user's language preference to the available localized resources. The user's locale preference can include more than just a primary language subtag. It might include the script (in tags such as zh-Hans, meaning "Chinese as written in the Simplified Han script") or region (in tags such as fr-CA, meaning "French as used in Canada")—or both. Some systems support Unicode Locale Identifiers, an extension to [[BCP47]] that includes tailoring such as calendars, number shaping, collation, and more. For more information, see [[LTLI]].
Matching localized resources to a user's locale preference usually involves "locale fallback", using a mechanism such as [[BCP47]]'s "lookup" matching scheme or [[CLDR]]'s "best match" scheme. These involve comparing the language tags on each resource seeking the longest tag that completely matches the user's preference. There is also, usually, a default locale to use for cases in which none of the localized resources match the user's preference.
Matching of language tags to available resources (such as found in [[[#path-vs-file]]]) or to items in some form of language map (see [[[#language-maps]]]) can require an implementation to examine all of the available values, as the first potential match might not be the best overall match.
For example, consider the following language map:
"localized-resource": {
"en": {"value": "This is English"},
"en-GB": {"value": "This is UK English"},
"en-US": {"value": "This is US English"}
}
If the user's locale preference were en-GB or en-US, the best match is included in the list. Obviously, the entire list must be read to find the best match, since the entry for en could be considered a match for either value.
If the user's locale preference were en-AE (English as used in the United Arab Emirates), the best match in the list is en, although some implementations of best matching might apply out-of-band information to potentially match a longer tag.
If the user's locale preference were fr-FR (French as used in France), none of the values match and some form of defaulting would be needed to select the value to use.
Finally, the user's locale preference could include additional subtags, such as those used to tailor locale settings or such as a language variant. An example of a tailored locale might be en-US-u-hc-h23 (US English but with the "hour cycle" set to 24 hour time). An example of a language variant might be en-GB-oxendict (UK English, but using spelling conventions of the Oxford English Dictionary).
Locale fallback is implemented such that the user's preference is matched against the available list of values, and, if no exact match is found, removing the last subtag and repeating. This is described as "lookup" in the language tag matching part of [[BCP47]] (specifically [[RFC4647]]). Thus, the tag en-GB-oxendict would be compared against the above languge map as follows:
en-GB-oxendict // no match en-GB // matches en-GB en // if en-GB were not present, matches en (default) // no match, use defaults
Natural language values stored in a manifest, like any document format or data structure, require language and direction metadata in order to be used by consumers successfully. For detailed information about the requirements, see [[STRING-META]].
There are four common serializations for string values in a manifest, and these are described in this section. Specifications are intended to use these together to form a complete localization solution.
For non-linguistic fields (that is, strings that contain data that is not human language) language or direction metadata SHOULD NOT be associated with the value. Note that this includes application-internal data values [[INTERNATIONAL-SPECS]].
If a consumer is required to assign a language tag to the data, the value zxx (Non-Linguistic) SHOULD be used. If a consumer is required to assign a base direction to the data, the value auto SHOULD be used.
For localizable text fields that appear in a single language, use a data structure to represent the value. The recommended representation is an object with three fields. The field value contains the actual string. The field lang contains a valid [[BCP47]] language tag. The field dir contains the string's base direction (one of the values ltr, rtl, or auto).
See The Localizable WebIDL Dictionary in [[STRING-META]].
Single-language localizable text fields are at their best if there are only a few natural language strings in a given document. When a document such as a manifest contains many natural language strings, this becomes inefficient. To reduce the complexity of encoding these strings, one workaround is to establish a document-level default for language and base direction. These are separate values, as language does not imply direction. There should still be the ability to override either value on any given string value by using the single-language localizable text field representation found above.
For specifications that can make use of the [[JSON-LD]] @context mechanism, use the @language and @direction fields to supply the document level defaults.
If your specification defines its own document level defaults, provide two optional fields:
language and SHOULD be specified to contain a valid [[BCP47]] language tag. Specifications SHOULD specify that implementations are only require to check if a [[BCP47]] language tag is well-formed.direction and support the values ltr, rtl, or auto.The world is not monolingual. Having documents that contain only a single language would mean providing many iterations of the document, one for each language, in order to localize the manifest. This also might require language negotiation when requesting the manifest.
One way to address this is to allow multilingual values for each localizable text field inside the manifest document.
Language selection is not merely the exact matching of language tag string values to the user's preferred locale. The usual object representation of a localizable text field requires that the object be deserialized in order to discover the language tag associated with the value. This can be inefficient when there are many values in a given manifest file. In these cases, the best practice is to use a language map to organize localizable text values. Such a map exposes the language tag for the purposes of selection, but still uses an object representation on the value side of the map, since both language and direction might need to be overridden for a given string value.
There are several common approaches to creating localized manifests. Each offers certain advantages (and suffers from certain drawbacks):
A unitary manifest approach provides for localization by defining localizable fields using an array of language values.
The advantages of a unitary manifest include:
There are also disadvantages to this approach:
Unitary manifests work best with smaller source files with a limited number of localizable content fields and/or a limited number of languages. Most unitary manifests define some type of “language indexing” scheme (such as found in [[JSON-LD]]) to perform the actual selection.
Locale-specific manifests create a separate file for each supported language or locale. Usually the specification will choose either a path or filename based organizational scheme for allocating and finding the manifest files.
The advantages of locale-specific manifests include:
The disadvantages include:
A common compromise is to have a central manifest file that contains most or all of the non-localizable content, coupled with separate files that contains the localizable content. In some cases, the central manifest file contains defaults for the localizable content. The central manifest might also contain a directory or list of available locales (or the URL of language-specific manifests) so that clients only need to try to retrieve localized files that are actually available.
Some manifests opt to store localized variants of the manifest using modification of the path. Others prefer to modify the file name.
Path-based storage has the advantage that multiple files can be localized together. For example, the localized manifest can include a localized icon, logo, stylesheet, README, and terms-of-service file together with the manifest. New languages can be added by adding a new directory.
Path-based systems have the disadvantage that the filenames are generally “all the same” and the contents of a given file is not obvious from the filename—only by its location in the application hierarchy.
Filename-based storage has the advantage that the contents of the file is obvious from the filename and managing files can sometimes be easier in this regard.
Note that manifest localization does not remove the need for the string content in a manifest to provide language and direction metadata for each separate data value. The manifest file can define a file-wide default for language and direction, but specific files should be able to override either value. See [[STRING-META]] for more information.
Some manifests allow for "sparse population" of values in the localized file. For example, the en-US (English, United States) file might only contain one or two US-specific values, relying on the en (English) file for most of its content. Specifications need to be explicit about whether sparse population is allowed and how it works to avoid misconfiguration errors or missing translations.