Contact information

Open Source

  • Please describe any previous Open Source development experience:

I'm maintaining Xiphos and Sword in Debian/Ubuntu. Due to this effort I've also became upstream developer of Xiphos. I did a few fixes in Ubuntu & bzr plugins. I'm currently Ubuntu Contributing Developer see my contributor application. Most (if not all) of my development experience is on launchpad & PTS.

  • Why are you interested in Open Source?

Hmmm... Because I'm a geek? I've started using Open Source apps because they were free, ad/malware free, stable and usually had more features. After I've switched to Ubuntu and started contributing I've learned a lot from Open Source apps.

Availability

  • How long will the project take? When can you begin work?

2.5 - 3 months. Can start part time before June. Full time from June.

  • How much time do you expect to have for this project? (weekly)

~ 40 hours. Hoping to start coding early.

  • Where will you based during the summer?

UK (UTC+1) or Latvia (UTC+2)

Other

  • Do you have any commitments for the summer? (holidays/work/summer courses)

2-3 weeks for exams & dissertation. 1 week in August + a few weekends for linejuding Volleyball (world tour & national league).

I'm hoping to start early to minimise impact. And merge milestones separately.

  • Have you ever participated in a previous GSoC?

No.

  • Have you applied for any other 2010 Summer of Code projects? If yes, which ones?

No. But I might apply for a couple of Ubuntu projects.

  • Why did you apply for the Google Summer of Code ?

I see this as opportunity to experience what is it like to be Open Source full-time developer. In the future I would like my career to involve developing open source software.

  • Why did you choose Ubuntu as a mentoring organisation?

    • I am using ubuntu since 2007. I am ubuntu developer. I want to fix Bug #1. I love ubuntu.
    • I like the friendly developer & user community around ubuntu. During my previous work on ubuntu I've met and worked with awesome and cool developers =) some of which are mentors for GSoC.

    • Every ubuntu developer has cool Brainstorm ideas which would be really amazing to have for ubuntu development and/or ubuntu users. These cute ideas are often put aside to the when-I-have-spare-time box of each developer. With GSoC they can become reality.
  • Why do you want to participate and why should Ubuntu choose you?

I want to make a significant contribution to Ubuntu. GSoC will give me opportunity to do that. Ubuntu should choose me because I am passionate, have sufficient knowledge and ready to learn fast. I want to become Motu/CoreDev in the future.

Integrating Bzr with 3.0 (Quilt) packages

Summary

This spec makes bzr & bzr-bd awesome for managing 1.0 or DpkgSrc3.0 (quilt) packages. Because currently it's not easy to manage packages using bzr with 3.0 (quilt) all patches applied state. And looking at debdiff of the previous revisions it's mostly diff-of-diff in the debian/patches/ (I can't read that ;-))

Assumptions

Spec does not include multiple tarballs format.

Design

Imagine bzr:$distro/$package branch implemented as a pipeline. You would have.

With package tags on packaging pipes. Such that when you switch to that tag, previous pipes are merged (pumped) resulting in your current commit and debian/patches/series is replica of pipeline looking downward to upstream, and patches are as if you did "bzr pump-patches" to make this commit.

Execpt that our lp:$distro/$package branches are actually 3 merged dsc import branches (upstream, debian commits, ubuntu commits) which you can access by using tags or point revisions (e.g. 1.1.1):

My proposal is to locally "reveal" this branch to create a pipeline view of it. To work on it and allow to fold back as uncommitted state.

Implementation

So we should add bzr reveal-patches command which will inteligently create a pipeline view of the packag branch like this:

  1. Shelve current packaging branch somewhere safe similar to ~/.bzr/pipes or if using lightweight checkout we are safe.
  2. Branch off first upstream revision of the to be revealed packaging range
  3. Make it base of the pipeline.
  4. Looking at the shelved packaging branch start importing each patch as a pipe for equivalent debian revision.
  5. Make at debian pipe for "debian/" packaging excluding debian/patches
  6. For each Ubuntu revision on top of debian
    1. where possible modify existing patch pipe
    2. add/remove patch pipes
    3. and add ubuntu packaging changes on separate pipe above debian
  7. Do this for all requested package version range

Each commit should be tagged along the way: Eg. 01_patch-debian-3.2-1 for a patch pipe, debian-3.2-1 on debian branch and ubuntu-3.2-1ubuntu1 for ubuntu branch.

Note this doesn't re-write history because the real packaging branch is shelved safely away and used for reference.

So far so good, so we have our very nice pipeline view of the package. So how would you build a package while on the debian pipe tagged 3.2-1?

To achive this we will use a read/write content filter to show debian/patches directory. The content filter will export each patch-pipe as quilt series into debian/patches. So that you can invoke bzr-bd using current state & generating original tarballs from the shelved packaging branch.

So for example you are MOTU (Master Of The Unseeded) and need to do a complex merge with 10 patches. You would grab lp:ubuntu/$package. Reveal it from the base revision. Import new debian revisions from lp:debian/$package in revealed state. Do a bzr pump resolving conflicts. Do a build, do testing. Fold the revealed state into lp:ubuntu/$package which will become a single commit/tag, mark uploaded and push to launchad for building.

Implementation

Modify bzrtools patch command to use quilt for importing patches.

Provide new command import-quilt in bzr-pipeline plugin, which takes quilt series file as argument and creates one pipe per patch. Pipes should be named after quilt patch file names. Pipe should store quilt header (e.g. DEP-3 headers) in branch config for example.

Provide new command export-quilt in bzr-pipeline plugin, which takes output directory as argument and creates quilt series there. This should be similar to the current pipe-patches command exept that series file must be created and patches should have headers restored from the pipe store.

If there is no header on export-quilt it should analyse commits in each pipe and guess summary, author, and bug urls from commits.

Stage 1

Provide a "hidden" command / hook, which exports pipes and replaces debian/patches/ quilt series and add result to the tree. This "hidden" command will then be added to run as part of bzr-bd pre-build hook.

Stage 2

Add a new read & write content filter for debian/patches. Such that when you switch to debian pipe and all patches magically are refreshed. Removing a patch should remove pipe & vice-versa.

UI Changes

bzr patch --quilt - new option to use quilt logic for importing a single / multiple patches bzr import-quilt series - new command to import quilt series file as patches bzr export-quilt [destdir] - new command to export pipes as quilt series to destdir (default debian/patches) bzr export-quilt-hook - new hidden command / API for consumption by bzr-bd as pre-build hook bzr reveal - expose packaging branch as pipeline bzr hide - go back to packaging branch bzr fold - fold current committed state of top-level pipe as a commit on packaging branch

Code Changes

Not ready yet. Draft....

Migration

Stage 1 will not have reveal/hide/fold. It will be able to import quilt series as pipes & export them. Which will already be improvement and will be usable for packaging.

Stage 2 will add content filter to eliminate exporting patches.

Stage 3 will have reveal/hide/fold for total Ayatana development.

Test/Demo Plan

Nearing completion of each stage, packaging recepies will we written for developers.

Unresolved issues

This also can be implemented using threads instead of pipeline. Which might be easier to do. But the design goal is to not push threaded/pipeline view to launchpad and not to rewrite public branch history. Our packaging branches have enough information already for this.

GSoC Questions

I've been doing a few merges & syncs. MoM helps a lot, but it's not perfect. With this project implemented it would be easier to start working on Ubuntu because the required steps will be shown by example. And it will make Ubuntu development more fun "Spot what went wrong?" type of game =)

It will provide archive wide QA check of potential future package versions allowing to spot problems earlier in the development cycle.


CategorySpec

GSoC/2010/DmitrijsLedkovs/Pipes (last edited 2010-03-30 00:09:35 by dmitrij.ledkov)