== Dev Week -- Getting Translations Quicker into Launchpad: Upstream Imports Sharing -- David Planella -- Wed, Jul 13th, 2011 == {{{#!irc [16:02] hey everyone! [16:03] welcome all to the 3rd day of Ubuntu Developer Week [16:03] and welcome to this UDW talk about one of the coolest features of Launchpad Translations: upstream imports sharing [16:03] My name is David Planella, and I work on the Community team at Canonical as the Ubuntu Translations Coordinator [16:04] As part of my job, I get the pleasure of working with the awesome Ubuntu translation teams and with the equally awesome Launchpad developers [16:04] I also use Launchpad for translating Ubuntu on a regular basis, as part of my volunteer contribution to the project, [16:04] which is why I'm excited about this feature and I'd like to share it with you guys [16:05] So, without further ado, let's roll! [16:05] oh, btw, I've set aside some time for questions at the end, but feel free to ask anything during the talk. [16:05] just remember to prepend your questions with QUESTION: and ask them on the #ubuntu-classroom-chat channel [16:06] So... [16:06] [16:06] What is message sharing [16:06] ----------------------- [16:06] [16:06] Before we delve into details, let's start with some basics: what's all this about? [16:07] In short, message sharing is a unique feature in Launchpad with which translations for identical messages are essentially linked into one single message [16:07] This means that as a translator, you just need to translate that message once and your translations will appear automatically in all other places where that message appears. [16:08] The only requirements are that those messages are contained within a template with the same name and are part of different series of a distribution or project in Launchpad. [16:08] This may sound a bit abstract, so let's have a look at an example: [16:08] Let's say you're translating Unity in Ubuntu Oneiric: [16:08] https://translations.launchpad.net/ubuntu/oneiric/+source/unity/+pots/unity/ [16:09] (you translate it from that URL in Launchpad) [16:09] And you translate the message "Quit" into your language [16:09] Instantly your translation will propagate to the Unity translation in _Natty_ (and all other releases): [16:10] So it will appear translated here: [16:10] https://translations.launchpad.net/ubuntu/natty/+source/unity/+pots/unity/ [16:10] without you having had to actually translate it there [16:10] So basically Launchpad is doing the work for you! :-) [16:11] It also works when you want to do corrections: [16:12] Let's say you are not happy with the translation of "Quit" in Unity, and you change it in the Natty series in Launchpad [16:12] So you go to: https://translations.launchpad.net/ubuntu/natty/+source/unity/+pots/unity/ [16:12] and change it [16:13] actually, I could point you to the actual message: [16:13] https://translations.launchpad.net/ubuntu/natty/+source/unity/+pots/unity/ca/2/+translate [16:13] anyway, so you fix the "Quit" translation [16:13] perhaps there was a typo, perhaps you were not happy with the current translation, etc. [16:15] That change will also be propagated to all other Ubuntu series, so that you don't have to manually fix each one of them [16:15] Isn't that cool? [16:15] And as you see, it works both ways: from older series to new ones, and viceversa. The order does not really matter at all. [16:15] we've got a question: [16:15] shnatsel|busy asked: If there was a change in terminology between the series, e.g. I translated "torrent" with one word, then the app changed (some strings changed, some strings added) and now I use a different word, is there a way to prevent the new terminology partly leaking to the older series and making a terrible mess there? [16:16] that's a good point [16:16] however, with the current implementation this will not happen [16:16] message sharing works only with identical messages [16:17] this means that if in the first series the original string was "torrent", [16:17] and in the next series the original string was "torrent new", [16:18] there won't be any message sharing at all, avoiding inadvertently translating strings you don't want to be translated automatically [16:19] gpc asked: So, "torrent" and "torrent." are not the same string? [16:20] that's correct, even with such a small change as this, they're not considered the same string [16:20] they need to be identical [16:20] so rather than a fuzzy match, message sharing works only on identical matches [16:21] fuzzy matching would need further development in Launchpad [16:21] If you're interested on how difficult it would be to implement it, or even in implement it yourself, [16:21] I'd recommend asking on the #launchpad channel [16:21] that's where the Launchpad developers hang out [16:22] and they're always happy to help [16:22] Anyway, let'w move on... [16:22] So far we've only talked about distributions, and in particular Ubuntu. But this equally works within series of projects in Launchpad [16:22] But more on that later on... [16:22] You'll find out more about sharing here as well: [16:22] http://blog.launchpad.net/translations/sharing-translations [16:22] http://danilo.segan.org/blog/launchpad/automatic-translations-sharing [16:23] [16:23] Benefits of messsage sharing [16:23] ---------------------------- [16:23] Continuing with the basics: what good is this for? [16:24] In terms of Launchpad itself, it makes it much more effective to store the data: rather than storing many translations for a same message, only one needs to be stored. [16:24] But most importantly: [16:24] For project maintainers: when they upload a template to a new release series and it gets imported, will instantly get all existing translations from previous releases shared with the new series and translators won’t have to re-do their work. They won’t have to worry about uploading correct versions of translated PO files, and can just care about POT files instead. [16:25] For translators: they no longer have to worry about back-porting translation fixes to older releases anymore, and they can simply translate the latest release: translations will automatically propagate to older releases. Also, this works both ways, so if you are translating a current stable release, newer development release will get those updates too! [16:26] Any other questions so far? [16:27] Ok, let's continue then [16:27] [16:27] What's new in message sharing [16:27] ----------------------------- [16:27] Until recently, message sharing had only worked within the limits of a distribution or of a project. [16:28] That is, messages could be shared amongst all the series in a distribution or amongst all series of a project. [16:28] As cool as it already was, that was it: data could not flow across projects and distributions, and each one of these Launchpad entities behaved like isolated islands with regards to message sharing. [16:29] But before going forward, let me recap quickly on some other basic concepts in Launchpad. When we're talking about message sharing, we're interested mostly in two types of Launchpad entities [16:30] * Projects: these are standalone projects whose translations are exposed in Launchpad. If these projects are packaged in a distribution, we often refer to the actual project at a location such as https://launchpad.net/some-project as to the upstream project. An example is the Chromium project at https://translations.launchpad.net/chromium-browser/. Upstream projects can host their translations in Launchpad or externally. In the latter case, translat [16:30] ions can still be imported into Launchpad, but more on that later on [16:31] * Distributions: these are collections of source packages, each one of which is also exposed for translation. The most obvious example is Ubuntu. Here's an example of the Natty series of the Ubuntu distribution: https://translations.launchpad.net/ubuntu/natty [16:32] So the news, and the main subject of this talk, is that from now on translations can be shared, given the right permissions, across projects and distributions. [16:32] Again, let's take an example: [16:32] • The Synaptic project is translatable in Launchpad: https://translations.launchpad.net/synaptic [16:33] • At the same time, the Ubuntu distribution has got a Synaptic package which is translatable in Launchpad: https://translations.staging.launchpad.net/ubuntu/natty/+source/synaptic/ [16:33] • Now, given that the upstream maintainer has enabled sharing and has set the right permissions, now translators can translate Synaptic in Ubuntu and their translations will seamlessly flow into the upstream project! [16:33] • This works again both ways: if one translates Synaptic in the upstream project, translations will appear in the Ubuntu distribution [16:34] So no more backporting translations or exporting them and committing them back and forth. [16:35] ashams asked: So why there is a separate set of Templates for each release of Ubuntu, i.e. (in Natty: https://translations.launchpad.net/ubuntu/natty/+source/unity/+pots/unity/) in Oneric: https://translations.launchpad.net/ubuntu/oneiric/+source/unity/+pots/unity/) [16:36] This is due to the fact that for each series of Ubuntu you not only get new applications with new templates (or some go away), but also you get different messages in the templates [16:36] This is how releases of projects work, it hasn't got anything to do with message sharing [16:37] i.e. we need different sets of templates for each series, otherwise if we had one single set, it will overwrite the old ones on each release [16:38] Anyway, combine the previous example with automatic translation imports and exports, and project maintainers get a fully automated translation workflow, which is really really awesome :-) [16:38] More on automatic imports/exports: [16:38] https://help.launchpad.net/Translations/YourProject/ImportingTranslations [16:38] http://blog.launchpad.net/general/exporting-translations-to-a-bazaar-branch [16:39] So far we've covered projects hosted in Launchpad - what happens with projects hosted externally? [16:39] If translations of a given project are hosted externally, you won't get the benefit of full integration into Launchpad, but you'll still get some important advantages: [16:40] • Translations will need to be imported from a mirror branch into a Launchpad upstream project [16:40] • They will then be regularly imported to Launchpad Translations [16:40] • From there, they will flow quickly, and on a regular basis, into the Ubuntu distribution [16:40] • Up until here, the end result is the same as for upstream projects hosting translations in Launchpad [16:40] • However, translations will only flow in the direction upstream -> Ubuntu, as we don't have a way to automatically send translations to the external project yet [16:41] • The big benefit here is that translations will be imported reliably and quickly on a regular basis [16:42] For an overview on message sharing across projects and distributions, check out this UDS presentation by Henning Eggers, one of the Launchpad developers who implemented this feature, and myself: [16:42] http://ubuntuone.com/p/skw/ [16:43] [16:43] How to enable message sharing [16:43] ----------------------------- [16:43] The cool thing to know is that within a project or a distribution message sharing is already enabled [16:44] There are no steps that the project maintainers need to follow: every new series will automatically share messages with all the others [16:45] However, if you want to share messages between an upstream project and a distribution (e.g. Ubuntu), there are a set of steps that need to be performed first: [16:45] * Enable translations in the upstream project, setting the right permissions and translations group [16:46] * If the project you want to enable sharing for is hosting translations externally, you'll need to request a bzr import, so that translations can get imported from an external branch [16:47] * Finally, you'll need to link the upstream project to the corresponding source package in Launchpad [16:47] Right now just a few projects and source packages are linked this way, but this cycle we're planning a community effort to enable sharing for all supported packages. [16:48] I've prepared a table with an overview of the supported packages here: [16:48] https://wiki.ubuntu.com/Translations/Projects/LaunchpadUpstreamImports [16:48] And will soon announce how the community can help in enabling these for sharing. [16:48] Stay tuned to the Ubuntu translators list for more info: [16:49] https://wiki.ubuntu.com/Translations/Contact/#Mailing_lists [16:49] Ok, I think that's all I had on content, so let's go for questions! [16:49] [16:49] Q & A [16:49] ----- [16:50] yurchor asked: Why does translation sharing behaves strange (diffs are really weird)? Ex.: https://translations.launchpad.net/ubuntu/natty/+source/avogadro/+pots/libavogadro/uk/+translate?show=new_suggestions [16:51] I think that particular case is a project in which there was some data that needed migration (i.e. Ubuntu translations exported and uploaded in the upstream package) and that did not happen. [16:51] I'd suggest pointing this out in the #launchpad channel, where the Launchpad devs can have a look at it in more detail [16:52] yurchor asked: What if the project does not have repository with translations (like Fedora's libvirt, etc. on Transifex, translations are generated at package creation)? What will be imported from upstream? [16:53] I'm not familiar with how translations are stored in Fedora's libvirt. The only layout that's supported for external repositories is PO files stored in an external version control system that can be imported as a bzr branch (e.g. git, mercurial, svn, etc) [16:54] QUESTION: For example, I'm translating a BitTorrent client. I had a translation of the term "torrent" that I used in all strings that contain it. Then a new major release arrived that has some strings added, some strings removed and some strings (like "New torrent") intact. For the new version, I find a better translation of the term "torrent", and change it in all those strings in the newer series. But then the new translation of "New torrent", wit [16:54] h the new term to describe "torrent" will leak to the older series, while some other strings in there still use the old term. How can I prevent it? [16:55] oh, I understand what you mean now [16:56] Unfortunately, there is no way to detect this in Launchpad, as there is no way to link the "New torrent" to the "torrent" messages [16:56] In this particular case, one thing you can do is to export the translation as a PO file, and replace all the "New torrent" translations with the new term [16:57] and then reupload in Launchpad [16:57] One thing I forgot to mention, and it might be useful if you want to keep the old terminology in older series, [16:57] is that you can explicitly choose individual messages to diverge [16:58] so you can call "torrent" as "a" in a series and "b" in another series, bypassing message sharing [16:59] Ok, I think there is not much time for more questions, so we can probably wrap it up here [16:59] If you've got other questions, feel free to ask me on the #ubuntu-translators channel on Freenode [16:59] So thank you everyone for listening and for participating with your questions [17:00] I hope you enjoyed the talk and see you soon! }}}