LocalizedCDImageTools

Differences between revisions 1 and 29 (spanning 28 versions)
Revision 1 as of 2011-05-18 14:41:19
Size: 4030
Editor: p549B360F
Comment: initial spec
Revision 29 as of 2011-06-28 09:11:22
Size: 13399
Editor: pitti
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from DesktopTeam/Specs/Oneriric/LocalizedCDImageTools
Line 3: Line 4:
 * '''Launchpad Entry''': UbuntuSpec:desktop-o-loco-cds  * '''Launchpad Entry''': UbuntuSpec:desktop-o-cd-localization
Line 10: Line 11:
Tools and a workflow to provide truly localized CD images. This goes beyond merely installing the right translations, we also want to provide localized default bookmarks, input methods, chat protocols, radio stations, or background images. Tools and a workflow to provide truly localized CD images. This goes beyond merely installing the right translations, we also want to localize e. g. default bookmarks, or provide a custom background image.
Line 12: Line 13:
Create a set of tools for providing these as ubuntu-l10n-<language> addon packages (as declarative as possible), and build CD images with this extra customization package. Create a set of tools for providing these as `ubuntu-defaults-`''name'' addon packages (as declarative as possible), and build CD images with this extra customization package. Through this spec, these packages will be referred to as '''defaults packages'''. ''name'' would usually be a language or region name, but it will just as well work for flavour/project names.
Line 24: Line 25:
By providing an official method for localizing CD images we want to make these Loco efforts both more visible and also much easier to implement. By providing an official method for localizing CD images we want to make these Loco efforts both more visible and also much easier to implement. So far we have not made much use of the knowledge that Loco teams have, they are in a much better position than the core developers to know what should be adapted for their region.
Line 30: Line 31:
 * Li downloads the "Ubuntu Chinese Edition" from `cdimage.ubuntu.com` (note that this is only a strawman name, CD mirroring is not part of this spec). gfxboot, ubiquity, and the live session start in Simplified Chinese with the sunpinyin ibus input method ready and active. She opens Firefox and clicks on the "DWNews.com" bookmark in the toolbar (note that this is just an example), a popular Chinese news site. She opens the chat window and is able to add her QQ account.  * Li downloads the "Ubuntu Chinese Edition" from `cdimage.ubuntu.com` (note that this is only a strawman name, CD mirroring is not part of this spec). gfxboot, ubiquity, and the live session start in Simplified Chinese with the sunpinyin ibus input method ready for activating with Ctrl+Space. She opens Firefox and clicks on the "DWNews.com" (note that this is just an example) bookmark in the toolbar, a popular Chinese news site. She opens the chat window and is able to add her QQ account.
Line 32: Line 33:
 * Didier already has an official Ubuntu 11.10 CD. He chooses "French" in Ubiquity,  * Didier already has an official Ubuntu 11.10 CD. He chooses "French" in Ubiquity. Since it is connected to the Internet, Ubiquity installs the French specific localization settings automatically. (''Note: This might become an option in the UI''.)
Line 34: Line 35:
 * Kyle prepares a customized version of Ubuntu for Compu1337, a large OEM. He prepares a `compu1337-default-settings` package as a copy of the `ubuntu-l10n-template` package, sticks in an appropriate `background.jpg`, and adds it to the project seeds. When booting the generated image, the default desktop background image shows a big Compu1337 logo.  * Martin already has a 10.04 LTS installation which he just upgraded to 11.10. In software-center he selects the "Deutsche regionale Anpassungen" package ("German regional adaptions") and ends up with the same localizations than a fresh 11.10 install would have applied.
Line 36: Line 37:
== Assumptions ==  * Kyle prepares a customized version of Ubuntu for Compu1337, a large OEM. He prepares a `compu1337-default-settings` package as a copy of the `ubuntu-defaults-template` package, sticks in an appropriate `background.jpg`, and adds it to the project seeds. When booting the generated image, the default desktop background image shows a big Compu1337 logo.

== Scope ==

This specification only deals with the mechanics of declaring, packaging, and shipping default settings, focussing on localizations, but settings are not limited to that. In particular, the following aspects are not covered:

 * CD mirroring/hosting
 * The actual customizations for any particular language or project.
 * QA for the localized images.

This is also limited to settings which can be customized by just adding an additional defaults package (plus its dependencies). In some cases we might modify Ubuntu packages to look in additional data/override files/directories, but we don't support package removal or code changes with this system. If changes of that scope are desired, they need to happen with custom engineering, just as today.
Line 40: Line 51:
Locale specific browser bookmarks, Unity launchers, radio stations in the music player, pre-configured ibus modules, === Customizable settings ===
Line 42: Line 53:
For the initial version, the following default values will get customizable with defaults packages:

 * Firefox bookmarks
 * Firefox start page
 * Unity launchers
 * Radio stations in Banshee and Rhythmbox
 * Default Firefox search engine
 * Background image
 * Extra packages

=== Workflow ===

 * Install `ubuntu-defaults-builder`.
 * Run `ubuntu-defaults-template fr` to generate a skeleton `ubuntu-defaults-fr` source package. This will have a minimal `debian/` directory, some template files, and a README which points to the documentation.
 * The user adds the data as flat files (e. g. `firefox/bookmarks.txt` or `desktop/background.jpg`), and runs `dch` to add a changelog entry.
 * Running `dpkg-buildpackage` will generate `ubuntu-defaults-fr_`''version''`_all.deb` with the data files shipped in the right locations, and maintainer scripts to set them up.
 * This package can then be uploaded to the Ubuntu archive, or kept in a PPA, as desired.
 * Run CD build script (which will probably be [[https://launchpad.net/live-build|live-build]] in the near future) with the corresponding default locale setting, and extra seeding to include the wanted defaults package.

=== Co-installability ===

Unlike existing metapackages like `xubuntu-desktop`, which are co-installable, the defaults package don't primarily define a set of dependencies, but a set of defaults. By nature it only makes sense to install one at a time, as they are meant to define what an image build result will look like. That is, while you can have an installation with both Ubuntu and Xubuntu packages, you have to decide whether the CD build you are about to do should be an Ubuntu or Xubuntu CD/product.

This will be enforced by

 {{{
  Provides: ubuntu-default-settings
  Conflicts: ubuntu-default-settings
 }}}
Line 45: Line 85:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: In the following sections, '''Source''' is the user-provided file in the defaults package source, while '''Destination''' is where the defaults binary package will ship it.
Line 47: Line 87:
=== UI Changes === The `ubuntu-defaults-builder` package will provide a `dh_ubuntu_defaults` debhelper program which will install all '''Source''' files into their destination location. With this, the logic can be maintained centrally in the `ubuntu-defaults-builder` source, and the defaults packages themselves only require a three-line boilerplate `debian/rules`. `ubuntu-defaults-template` will create an appropriate `debian/control` with the language name in the package description etc. It will also provide a standard `debian/copyright` with no templating, which the user can customize manually if desired.
Line 49: Line 89:
Should cover changes required to the UI, or specific UI that is required to implement this === Firefox bookmarks ===
Line 51: Line 91:
=== Code Changes ===  * '''Source:''' `firefox/bookmarks-toolbar.txt`, `firefox/bookmarks-menu.txt`; simple text files, each line has "URL name"
 * '''Destination:''' Create and ship a [[https://wiki.mozilla.org/Distribution_INI_File|"distribution ini" file]] for firefox with the bookmarks, which will be appended to the default ones which we already have (Firefox and Ubuntu folders), in the toolbar and menu respectively.
Line 53: Line 94:
Code changes should include an overview of what needs to change, and in some cases even the specific details. Caveat: Right now these ini file bookmarks do not support favicons. If this is not sufficient, a fallback approach is to create and ship a `/usr/lib/firefox/omni-l10n.jar` based on the standard `/usr/lib/firefox-4.0.1/omni.jar` and change firefox to prefer using the former if present.
Line 55: Line 96:
=== Migration === `ubuntu-defaults-template` pre-creates this file with just a comment about the format.
Line 57: Line 98:
Not necessary. === Firefox start page ===

 * '''Source:''' `firefox/startpage.txt`; simple text file with just the URL
 * '''Destination:''' Create and ship a [[https://wiki.mozilla.org/Distribution_INI_File|"distribution ini" file]] for firefox with the `browser.startup.homepage` preference set to that value.

`ubuntu-defaults-template` pre-creates this file with just a comment with an example.

=== Default Firefox search engine ===

 * '''Source:''' `firefox/defaultsearch.txt`; simple text file with just the search engine name (e. g. `wikipedia`)
 * '''Destination:''' Create and ship a [[https://wiki.mozilla.org/Distribution_INI_File|"distribution ini" file]] for firefox with the `browser.search.defaultenginename` preference set to that value.

`ubuntu-defaults-template` pre-creates this file with just a comment with some examples (google, wikipedia)

Note that the available search engines live in Firefox itself (it ships translated locale specific search engines), so if one is missing, we'd need to fix it there for the time being.

=== Unity launchers ===

 * '''Source:''' `unity/launchers.txt`; simple text file with one launcher name per line. These correspond to `.desktop` file names.
 * '''Destination:''' Create `/usr/share/glib-2.0/schemas/ubuntu-defaults-NAME.gschema.override` with the current default launcher list of Unity pls `launchers.txt` appended. For unity-2d, ship a gconf schema defaults override for `/desktop/unity-2d/launcher/favorites`.

`ubuntu-defaults-template` pre-creates this file with just a comment about the format and an example.

=== Radio stations in Banshee and Rhythmbox ===

 * '''Source:''' `multimedia/radiostations.txt`; text file with one "Name; Genre; URL" entry per line.
 * '''Destination:''' Divert `rhythmbox-plugin`'s `/usr/lib/rhythmbox/plugins/iradio/iradio-initial.xspf` and replace with above stations for Rhythmbox. Banshee needs a [[https://bugzilla.gnome.org/show_bug.cgi?id=548197|patch to support default radio stations]], with that it also reads XSPF station lists in `/usr/share/banshee/stations/`.

`ubuntu-defaults-template` pre-creates this file with just a comment about the format and an example.

=== Background image ===

 * '''Source:''' `desktop/background.jpg`
 * '''Destination:''' Copy to `/usr/share/backgrounds/`''packagename''`.jpg`. Create `/usr/share/glib-2.0/schemas/ubuntu-defaults-NAME.gschema.override` with setting `org.gnome.desktop.background picture-uri` to that path.

`ubuntu-defaults-template` pre-creates this file as empty. An empty file will be ignored.

=== Extra packages ===

 * '''Source:''' `depends.txt`, `recommends.txt`, one dependency per line
 * '''Destination:''' `debian/control` has `${ubuntu-defaults:Depends}` and `${ubuntu-defaults:Recommends}`. `dh_ubuntu_defaults` replaces these with the standard language/support packs (from `./check-language-support -l `''locale'') plus `depends.txt` appended, and builds `${ubuntu-defaults:Recommends}` from `recommends.txt`.

`ubuntu-defaults-template` pre-creates this file with just a comment about the format and an example.

Requirements for the extra dependencies:

 * They should not conflict to a default installation package, or add applications for a purpose which the default installation already has an application for. For example, don't add thunderbird or pidgin when we already have evolution or empathy.
 * They should preferably be in main. If they are not, consider writing a MIR, or add a justification to a comment in depends.txt or the changelog. Depending on universe packages will be a valid reason to reject an ubuntu-defaults package.

=== Language support ===
 * '''Source:''' `langpacks.txt`, one language code per line; `locale.txt`, one line with the default locale
 * '''Destination:''' Copy to `/usr/share/`''packagename''`/`. The image build script will iterate over `langpacks.txt` and call `check-language-support -l `''language'' to install the required langpacks and support packages, and set the default locale to `locale.txt`.

Defaults packages can't set the default locale, language, and keyboard layout. The installer and casper already assume that they can own these files (`/etc/default/locale` and `/etc/default/keyboard`), and change them according to the user settings. Likewise, for installing them into an existing system we do not want to change the current default, as this would be unexpected.

For these the defaults have to be set in the script which generates the ISOs, i. e. [[https://launchpad.net/live-build|live-build]]. This will set the defaults for gfxboot and casper, which will then write it to the `/etc/default/` files.
Line 61: Line 157:
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage. To be done when implementation is at beta.
Line 63: Line 159:
This need not be added or completed until the specification is nearing beta. We will provide an `ubuntu-defaults-test` package with all possible customizations, and provide documentation what the expected result is. This can then be verified in the live system.
Line 65: Line 161:
== Unresolved issues == == Possible future improvements ==
Line 67: Line 163:
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.  * Translating wubi autorun.inf: Currently outside of what we can do with an extra package.
 * Translating strings in initramfs: Needs shell gettext support in initramfs first, then we can just rebuild the initramfs after installing langpacks.

== References ==

 * ISO unpack/repack script of the French Loco team: http://paste.ubuntu.com/609577/
 * squashfs mangling script of the French Loco team: http://paste.ubuntu.com/609578/
 * Some Lo``Cos use the [[http://uck.sourceforge.net/|Ubuntu Customization Kit]]
  • Launchpad Entry: desktop-o-cd-localization

  • Created: 2011-05-18

  • Contributors: Mark Shuttleworth, Didier Roche, Colin Watson, Kyle Nitzsche

  • Packages affected: ubiquity, cdimage, creation of new packages

Summary

Tools and a workflow to provide truly localized CD images. This goes beyond merely installing the right translations, we also want to localize e. g. default bookmarks, or provide a custom background image.

Create a set of tools for providing these as ubuntu-defaults-name addon packages (as declarative as possible), and build CD images with this extra customization package. Through this spec, these packages will be referred to as defaults packages. name would usually be a language or region name, but it will just as well work for flavour/project names.

With this, concrete localization projects like desktop-o-qin-ubuntu-china will be a lot easier to implement.

Release Note

To be done when implementation is at beta.

Rationale

Our only action after the user chooses his locale is to install the matching language packs and support (dictionaries, etc.), but true localization goes way beyond that: Locos want to customize default bookmarks, locale specific chat protocols, or preconfigured ibus input method modules. Right now Locos both have to figure out how to customize these settings by themselves, which is not easy and also rather intrusive sometimes. Often these changes can't be preserved during upgrades, and there is only little QA and advertising happening on those images.

By providing an official method for localizing CD images we want to make these Loco efforts both more visible and also much easier to implement. So far we have not made much use of the knowledge that Loco teams have, they are in a much better position than the core developers to know what should be adapted for their region.

This also mitigates the eternal CD size resource conflicts on which language packs to ship on the official CD releases.

User stories

  • Li downloads the "Ubuntu Chinese Edition" from cdimage.ubuntu.com (note that this is only a strawman name, CD mirroring is not part of this spec). gfxboot, ubiquity, and the live session start in Simplified Chinese with the sunpinyin ibus input method ready for activating with Ctrl+Space. She opens Firefox and clicks on the "DWNews.com" (note that this is just an example) bookmark in the toolbar, a popular Chinese news site. She opens the chat window and is able to add her QQ account.

  • Didier already has an official Ubuntu 11.10 CD. He chooses "French" in Ubiquity. Since it is connected to the Internet, Ubiquity installs the French specific localization settings automatically. (Note: This might become an option in the UI.)

  • Martin already has a 10.04 LTS installation which he just upgraded to 11.10. In software-center he selects the "Deutsche regionale Anpassungen" package ("German regional adaptions") and ends up with the same localizations than a fresh 11.10 install would have applied.
  • Kyle prepares a customized version of Ubuntu for Compu1337, a large OEM. He prepares a compu1337-default-settings package as a copy of the ubuntu-defaults-template package, sticks in an appropriate background.jpg, and adds it to the project seeds. When booting the generated image, the default desktop background image shows a big Compu1337 logo.

Scope

This specification only deals with the mechanics of declaring, packaging, and shipping default settings, focussing on localizations, but settings are not limited to that. In particular, the following aspects are not covered:

  • CD mirroring/hosting
  • The actual customizations for any particular language or project.
  • QA for the localized images.

This is also limited to settings which can be customized by just adding an additional defaults package (plus its dependencies). In some cases we might modify Ubuntu packages to look in additional data/override files/directories, but we don't support package removal or code changes with this system. If changes of that scope are desired, they need to happen with custom engineering, just as today.

Design

Customizable settings

For the initial version, the following default values will get customizable with defaults packages:

  • Firefox bookmarks
  • Firefox start page
  • Unity launchers
  • Radio stations in Banshee and Rhythmbox
  • Default Firefox search engine
  • Background image
  • Extra packages

Workflow

  • Install ubuntu-defaults-builder.

  • Run ubuntu-defaults-template fr to generate a skeleton ubuntu-defaults-fr source package. This will have a minimal debian/ directory, some template files, and a README which points to the documentation.

  • The user adds the data as flat files (e. g. firefox/bookmarks.txt or desktop/background.jpg), and runs dch to add a changelog entry.

  • Running dpkg-buildpackage will generate ubuntu-defaults-fr_version_all.deb with the data files shipped in the right locations, and maintainer scripts to set them up.

  • This package can then be uploaded to the Ubuntu archive, or kept in a PPA, as desired.
  • Run CD build script (which will probably be live-build in the near future) with the corresponding default locale setting, and extra seeding to include the wanted defaults package.

Co-installability

Unlike existing metapackages like xubuntu-desktop, which are co-installable, the defaults package don't primarily define a set of dependencies, but a set of defaults. By nature it only makes sense to install one at a time, as they are meant to define what an image build result will look like. That is, while you can have an installation with both Ubuntu and Xubuntu packages, you have to decide whether the CD build you are about to do should be an Ubuntu or Xubuntu CD/product.

This will be enforced by

  •   Provides: ubuntu-default-settings
      Conflicts: ubuntu-default-settings

Implementation

In the following sections, Source is the user-provided file in the defaults package source, while Destination is where the defaults binary package will ship it.

The ubuntu-defaults-builder package will provide a dh_ubuntu_defaults debhelper program which will install all Source files into their destination location. With this, the logic can be maintained centrally in the ubuntu-defaults-builder source, and the defaults packages themselves only require a three-line boilerplate debian/rules. ubuntu-defaults-template will create an appropriate debian/control with the language name in the package description etc. It will also provide a standard debian/copyright with no templating, which the user can customize manually if desired.

Firefox bookmarks

  • Source: firefox/bookmarks-toolbar.txt, firefox/bookmarks-menu.txt; simple text files, each line has "URL name"

  • Destination: Create and ship a "distribution ini" file for firefox with the bookmarks, which will be appended to the default ones which we already have (Firefox and Ubuntu folders), in the toolbar and menu respectively.

Caveat: Right now these ini file bookmarks do not support favicons. If this is not sufficient, a fallback approach is to create and ship a /usr/lib/firefox/omni-l10n.jar based on the standard /usr/lib/firefox-4.0.1/omni.jar and change firefox to prefer using the former if present.

ubuntu-defaults-template pre-creates this file with just a comment about the format.

Firefox start page

  • Source: firefox/startpage.txt; simple text file with just the URL

  • Destination: Create and ship a "distribution ini" file for firefox with the browser.startup.homepage preference set to that value.

ubuntu-defaults-template pre-creates this file with just a comment with an example.

Default Firefox search engine

  • Source: firefox/defaultsearch.txt; simple text file with just the search engine name (e. g. wikipedia)

  • Destination: Create and ship a "distribution ini" file for firefox with the browser.search.defaultenginename preference set to that value.

ubuntu-defaults-template pre-creates this file with just a comment with some examples (google, wikipedia)

Note that the available search engines live in Firefox itself (it ships translated locale specific search engines), so if one is missing, we'd need to fix it there for the time being.

Unity launchers

  • Source: unity/launchers.txt; simple text file with one launcher name per line. These correspond to .desktop file names.

  • Destination: Create /usr/share/glib-2.0/schemas/ubuntu-defaults-NAME.gschema.override with the current default launcher list of Unity pls launchers.txt appended. For unity-2d, ship a gconf schema defaults override for /desktop/unity-2d/launcher/favorites.

ubuntu-defaults-template pre-creates this file with just a comment about the format and an example.

Radio stations in Banshee and Rhythmbox

  • Source: multimedia/radiostations.txt; text file with one "Name; Genre; URL" entry per line.

  • Destination: Divert rhythmbox-plugin's /usr/lib/rhythmbox/plugins/iradio/iradio-initial.xspf and replace with above stations for Rhythmbox. Banshee needs a patch to support default radio stations, with that it also reads XSPF station lists in /usr/share/banshee/stations/.

ubuntu-defaults-template pre-creates this file with just a comment about the format and an example.

Background image

  • Source: desktop/background.jpg

  • Destination: Copy to /usr/share/backgrounds/packagename.jpg. Create /usr/share/glib-2.0/schemas/ubuntu-defaults-NAME.gschema.override with setting org.gnome.desktop.background picture-uri to that path.

ubuntu-defaults-template pre-creates this file as empty. An empty file will be ignored.

Extra packages

  • Source: depends.txt, recommends.txt, one dependency per line

  • Destination: debian/control has ${ubuntu-defaults:Depends} and ${ubuntu-defaults:Recommends}. dh_ubuntu_defaults replaces these with the standard language/support packs (from ./check-language-support -l locale) plus depends.txt appended, and builds ${ubuntu-defaults:Recommends} from recommends.txt.

ubuntu-defaults-template pre-creates this file with just a comment about the format and an example.

Requirements for the extra dependencies:

  • They should not conflict to a default installation package, or add applications for a purpose which the default installation already has an application for. For example, don't add thunderbird or pidgin when we already have evolution or empathy.
  • They should preferably be in main. If they are not, consider writing a MIR, or add a justification to a comment in depends.txt or the changelog. Depending on universe packages will be a valid reason to reject an ubuntu-defaults package.

Language support

  • Source: langpacks.txt, one language code per line; locale.txt, one line with the default locale

  • Destination: Copy to /usr/share/packagename/. The image build script will iterate over langpacks.txt and call check-language-support -l language to install the required langpacks and support packages, and set the default locale to locale.txt.

Defaults packages can't set the default locale, language, and keyboard layout. The installer and casper already assume that they can own these files (/etc/default/locale and /etc/default/keyboard), and change them according to the user settings. Likewise, for installing them into an existing system we do not want to change the current default, as this would be unexpected.

For these the defaults have to be set in the script which generates the ISOs, i. e. live-build. This will set the defaults for gfxboot and casper, which will then write it to the /etc/default/ files.

Test/Demo Plan

To be done when implementation is at beta.

We will provide an ubuntu-defaults-test package with all possible customizations, and provide documentation what the expected result is. This can then be verified in the live system.

Possible future improvements

  • Translating wubi autorun.inf: Currently outside of what we can do with an extra package.
  • Translating strings in initramfs: Needs shell gettext support in initramfs first, then we can just rebuild the initramfs after installing langpacks.

References


CategorySpec

DesktopTeam/Specs/Oneiric/LocalizedCDImageTools (last edited 2011-10-29 18:06:45 by CPE-58-168-95-140)