Ubuntu Wiki

Dev Week -- Kubuntu Ninja's - Packagers in Unicorn mode -- apachelogger and vorian -- Thu, Jan 22

UTC -4 (EST)

(02:02:37 PM) vorian: OK THEN!
(02:02:38 PM) ***apachelogger hands Keybuk a cup of tea and wonders if he and vorian are now on schedule :P
(02:02:42 PM) vorian: Kubuntu Ninja’s - Packagers in Unicorn mode
(02:03:00 PM) apachelogger: Aloha/Hola/Salut/Ni Hao/Hello/Servus/Konnichiwa/Ahoy... Ladies, Gentlemen, IRC Bots, and Supernatural beings!
(02:03:00 PM) vorian: we are up, i do beleive
(02:03:14 PM) apachelogger: My name is not vorian (aka Steve Stalcup) and I am not going to show you how ninjas update a package.
(02:04:04 PM) apachelogger: Before we start, please ask questions any time. First question was already ask before this session even started :)
(02:04:16 PM) apachelogger: ...was something like: "what are 'Ninjas in Unicorn mode'"
(02:04:25 PM) apachelogger: Ninjas are magic blue headed monkeys with batwings and a horn looking like a gear on their foreheads, who are mostly talking jibberish so that the other Ubuntu developers don't understand them.
(02:04:43 PM) apachelogger: As for the unicorn mode ... I would really like to tell you, but then I would have to kill you.
(02:04:58 PM) apachelogger: Now, let's get started for real. First of all, a bit of History :P
(02:05:14 PM) apachelogger: KDE has a kind of unique way to publish releases. About one week before the actual release, unofficial packages get distributed amongst the super nice distribution packagers, so they can update their packages and do some final testing to ensure everything is in proper shape for release.
(02:05:41 PM) apachelogger: If we stumble upon serious issues, these get directly corrected in the tarballs... so after that week the users get
(02:05:48 PM) apachelogger: a) binary packages right away
(02:05:55 PM) apachelogger: b) tarballs tested by quite an amount of people on different architectures with different software stacks
(02:06:12 PM) apachelogger: All in all stuff to be happy about \o/
(02:06:29 PM) apachelogger: Well, at least as user. This pre-release publishing for packagers only works as long as the tarballs don't get published right away, but only when the embargo ends. That makes the whole process of getting the packages updated a whole lot more difficult because we can't just dump the tarballs somewhere and ask people who have some spare time to take a glimps at them.
(02:07:10 PM) apachelogger: So eventually in earlier days Kubuntu robot Riddell did manual package coordination via IRC queries (I suppose at least), until once he was not available for a release. Kind apachelogger started working on it instead ...and what do you know, I was so annoyed by the work that I summoned a whole team of specialists in KDE packaging just to handle the release packaging.
(02:07:35 PM) apachelogger: Super high quality packaging of course ... what else would that being a specialist be good for ;-)
(02:07:55 PM) apachelogger: Nowadays the "Ninjas" take care of the packaging. Well, actually it's a set of scripts I created for this task, the Ninjas just run them :P As a matter of fact the ultimate target is to streamline the process so much that we can assign the packaging to a minion guinea pig... so we can lie under the sun in miami beach and consume loads of captain morgan with coke.
(02:08:31 PM) apachelogger: Those of you who follow KDE development might now be wondering if I will give you access to the tarballs of upcoming KDE 4.2.0
(02:08:44 PM) apachelogger: ... well, I am not :P
(02:09:11 PM) apachelogger: That being said, unfortunately I can not show you how the update process works using an example, because only getting a certified Ninja build environment probably takes longer than this session ;-) I suppose it is enough if I just tell you. :P
(02:09:31 PM) apachelogger: First of all let me outline the very basics of a Ninja build environment:
(02:09:57 PM) apachelogger: Currently (in Ninjaland nothing lasts longer than 2 months due to constant improvement to the process, in regards to speed as well as quality) it consists of: a PBuilder enhanced with hooks for automatic execution of list-missing, for dropping to a shell if the build fails, to run apt-get update before fetching the packags and (if wanted) a hook for distributed compiling and one to maintain a very simple local package pool.
(02:10:56 PM) apachelogger: In addition to that every Ninja environment is equiped witht the so called batscripts. A whole suite of scripts only created to streamline the process of updating core KDE packages ... and ... they are written in RUBY ... ha! take that you python lovas :P
(02:11:31 PM) apachelogger: Ruby Ruby Ruby ... oh, I am loosing foucs...
(02:11:57 PM) apachelogger: So. How does it work?
(02:12:21 PM) apachelogger: First of all some poor dood, namely me, will run a script to secretly download the secret original source tarballs from KDE's even more secret server and store them in some as secret location, so that the secret Ninjas can obtain them using a secret config for their not so secret batscripts.
(02:12:44 PM) apachelogger: Just for reference... we also have a secret PPA ;-)
(02:13:07 PM) apachelogger: Then, considering a Ninja becomes bored from watching the uTube, they run another script to download the source tarball and merge it with the packaging branch (in Kubuntu we have a lot of our packages ... all of core KDE ... in bazaar branches on Launchpad).
(02:13:36 PM) apachelogger: Once that is done, the real work starts (and immediately ends again ;-).
(02:14:07 PM) apachelogger: The Ninja bumps some version requirements, eventually applies some other random change and runs a script to build the package.
(02:14:34 PM) apachelogger: Using the pbuilder hooks the Ninja will be able to fix a broken build on-the-fly (e.g. to update .install files because some installation path changed etc.).
(02:15:07 PM) apachelogger: The build script will collect loads of information the Ninjas ignores... eh... does quality assurance with.
(02:15:48 PM) apachelogger: Once the package is ready another script takes care of sending all necessary stuff to the release coordinator, who reviews the changes and passes the finalized package to a core-dev for sponsoring.
(02:16:21 PM) apachelogger: For KDE 4.2.0 that coordinator would be vorian.... talking about vorian... didn't you want to talk about something as well? *hint* *hint*
(02:16:44 PM) vorian: yes! indeed
(02:16:50 PM) apachelogger: don't say :P
(02:16:56 PM) vorian: but first, anyone have questions for apachelogger?
(02:17:38 PM) vorian: I'm not apachelogger (aka Harald Sitter), and I am not going to talk about ninja magic.
(02:17:48 PM) vorian: However I am going to talk about:
(02:17:56 PM) vorian:    _  __ __ _  __   __  _
(02:17:57 PM) vorian:   / |/ // // |/ /  / /.' \
(02:17:57 PM) vorian:  / || // // || /n_/ // o /
(02:17:57 PM) vorian: /_/|_//_//_/|_/ \_,'/_n_/
(02:17:57 PM) vorian:   _ __ ___  __    _  _____ ___   ___  /7
(02:17:57 PM) mariuz: how we can package an qt-snapshot
(02:17:59 PM) vorian:  /// // o |/  \ .' \/_  _// _/ ,' _/ //
(02:18:01 PM) vorian: U // _,'/ o |/ o / / / / _/ _\ `.
(02:18:03 PM) mariuz: ?
(02:18:04 PM) vorian: \_,'/_/  /__,'/_n_/ /_/ /___//___,'()
(02:18:20 PM) vorian: mariuz: great question
(02:19:17 PM) vorian: mariuz: our qt pro's are in #kubuntu-devel atm, the process is a bit complex, too complex for this session
(02:19:32 PM) vorian: so afterwards, feel free to join us in #kubuntu-devel
(02:19:36 PM) mariuz: people ask about arora with flash and is possible only with snapshot
(02:19:41 PM) mariuz: ok
(02:19:53 PM) vorian: ok
(02:19:58 PM) vorian: so Ninja updates
(02:20:08 PM) vorian: Seeing as we don't have much more time, i'll try and be as quick as possible.
(02:20:18 PM) vorian: To give a little taste on how we update packages, we will actually do one right now.
(02:20:32 PM) vorian: excited?!
(02:20:38 PM) mariuz: wet
(02:20:46 PM) vorian: ohmy
(02:20:48 PM) vorian: the name is plasmoid-toggle-compositing, which is a nice little plasmoid that turns your desktop effects on and off.
(02:21:02 PM) vorian: so, if everyone would please get the source, by either 'apt-get source plasmoid-toggle-compositing' or 'pull-lp-source plasmoid-toggle-compositing'
(02:21:14 PM) vorian: when you have the source, version 0.2.1, raise your hand o/ (or both!)
(02:21:37 PM) dinxter: me hands up!
(02:21:49 PM) vorian: yay dinxter
(02:22:02 PM) vorian: who else is gonna give it a shot?
(02:22:08 PM) apachelogger: \o/
(02:22:32 PM) vorian: all set?
(02:22:45 PM) vorian: We are lucky that this package has a watch file, so we will use that to get the new upstream release.
(02:22:45 PM) dinxter: as ready as i'll ever be
(02:22:59 PM) vorian: so, if everyone could cd plasmoid-toggle-compositing-0.2.1
(02:23:08 PM) vorian: let me know when you're there
(02:23:18 PM) apachelogger: there
(02:23:22 PM) dinxter: aye
(02:23:33 PM) vorian: and now run this command: uscan --verbose --report
(02:23:42 PM) vorian: What this does is scan the known source repository for this upstream package. If there is a new version, it will download it.
(02:23:49 PM) vorian: Pretty awesome eh?
(02:24:13 PM) vorian: We are going to pimp this watch file a little bit
(02:24:24 PM) vorian: Using your editor of choice, edit the debian/watch file and put 'debian uupdate' at the very end of the second line.
(02:24:36 PM) vorian: it should look something like: http://ivplasma.googlecode.com/files/toggle-compositing-([\d\.]*).tar.gz debian uupdate
(02:24:51 PM) vorian: let me know when you are finished
(02:25:14 PM) dinxter: done
(02:25:26 PM) apachelogger: +1
(02:25:34 PM) vorian: excellent
(02:25:53 PM) vorian: now run 'uscan --verbose'
(02:26:16 PM) vorian: What is happening?
(02:26:42 PM) dinxter: new source directory and diffs and everything!
(02:27:07 PM) vorian: w00t
(02:27:11 PM) vorian: that's magic
(02:27:24 PM) vorian: now, with most of the packages we deal with, it's not quite this simple
(02:27:57 PM) vorian: Do a "cd ../plasmoid-toggle-compositing-0.2.2" to see the new package
(02:28:06 PM) vorian: We are not going to use pbuilder for this excersize, so please make sure you 'sudo apt-get install debhelper cdbs cmake libplasma-dev quilt' for our building.
(02:28:29 PM) vorian: let me know when your done downloading, (theres quite a bit to pull there)
(02:28:43 PM) dinxter: its there
(02:28:58 PM) vorian: fantastic
(02:29:06 PM) vorian: Ok, now 'debuild -us -uc'
(02:29:11 PM) vorian: and tell me what happens
(02:29:35 PM) apachelogger: apachelogger__
(02:29:41 PM) dinxter: 2 out of 2 hunks FAILED -- rejects in file CMakeLists.txt
(02:29:41 PM) apachelogger is now known as apachelogger__
(02:29:47 PM) apachelogger_ is now known as apachelogger
(02:29:56 PM) vorian: boy thats a bummer eh?
(02:30:16 PM) dinxter: i always expect pain :)
(02:30:29 PM) vorian: lucky for us, the new upstream release fixes what this patch is for
(02:30:44 PM) vorian: please remove the patch and patch directory 'rm -rf debian/patches'
(02:30:56 PM) vorian: Make sure to note in the change log (dch -e) that - patch cmake-find-plasma.patch removed competely, resolved upstream - or somethng to that effect.
(02:31:11 PM) vorian: let me know when thou art done
(02:32:24 PM) dinxter: done
(02:33:09 PM) vorian: dinxter: care to paste your changelog?
(02:33:20 PM) vorian: http://paste.ubuntu.com
(02:33:21 PM) dinxter: hold on, debian/patches in 0.2.2?
(02:33:44 PM) vorian: dinxter: yes, remove the entire directory
(02:35:11 PM) dinxter: just double checking http://paste.ubuntu.com/108347/
(02:35:28 PM) vorian: no problems :)
(02:35:33 PM) ***vorian looks
(02:35:46 PM) vorian: FANTASTIC
(02:35:56 PM) vorian: ~order cookies for dinxter
(02:36:09 PM) vorian: ok, now 'debuild -us -uc'
(02:36:21 PM) mariuz: for me it gives an error
(02:36:43 PM) vorian: what error mariuz
(02:37:40 PM) dinxter: build fine here
(02:37:56 PM) vorian: so dinxter, you have a shiny new deb?
(02:37:57 PM) mariuz: undefined reference to `typeinfo for Plasma::Applet' collect2: ld returned 1 exit status
(02:38:41 PM) vorian: did you install all the packages i mentioned?
(02:39:28 PM) vorian: mariuz: we can sort it our after our session
(02:40:18 PM) vorian: if you look in the directory which holds the two versions of this package, you should see a shiny new deb
(02:40:20 PM) mariuz: yes is all installed ,  ok
(02:40:56 PM) dinxter: shiny deb plasmoid-toggle-compositing_0.2.2-0ubuntu1_amd64.deb
(02:41:00 PM) vorian: and that is the down and dirty, first ninja steps, kind of way we update a package
(02:41:15 PM) vorian: You have updated a Package!
(02:41:30 PM) vorian: What questions do you have?
(02:42:23 PM) apachelogger__: Any kind of questions that is :)
(02:42:24 PM) JontheEchidna: vorian: how would we contribute our updated package?
(02:42:31 PM) JontheEchidna: I assume we can't just hand over debs :P
(02:42:40 PM) vorian: JontheEchidna: that is correct
(02:42:59 PM) vorian: once you know your package is buildable
(02:43:20 PM) vorian: you always want to build it in a pbuilder environemnt (or sbuild)
(02:43:39 PM) vorian: and when you are 100% sure your package is ready
(02:44:07 PM) vorian: you can create a debdiff to submit to Launchpad for updating
(02:44:52 PM) apachelogger__: You might also want to poke someone in #kubuntu-devel to speed up the process :)
(02:44:59 PM) vorian: to create a debdiff, you simply use 'debdiff old_package.dsc new_package.dsc > new.debdiff
(02:45:17 PM) vorian: great question JontheEchidna
(02:45:23 PM) vorian: ~order cookies for JontheEchidna
(02:45:23 PM) ***kubotu slides a whole bunch of world's finest cookies down the bar to JontheEchidna.
(02:45:24 PM) JontheEchidna: :)
(02:46:04 PM) vorian: any other questions (of any kind)?
(02:46:40 PM) vorian: We are always looking for folks who want to help out in Kubuntu Land, so if you are interested in helping out and becoming a ninja, please stop by #kubuntu-devel!
(02:47:31 PM) vorian: if there are no more questions then, i'll just end with
(02:47:34 PM) vorian: ______________ ___    _____    _______   ____  __.  _________._._.
(02:47:34 PM) vorian: \__    ___/   |   \  /  _  \   \      \ |    |/ _| /   _____/| | |
(02:47:35 PM) vorian:   |    | /    ~    \/  /_\  \  /   |   \|      <   \_____  \ | | |
(02:47:35 PM) vorian:   |    | \    Y    /    |    \/    |    \    |  \  /        \ \|\|
(02:47:35 PM) vorian:   |____|  \___|_  /\____|__  /\____|__  /____|__ \/_______  / ____
(02:47:37 PM) vorian:                 \/         \/         \/        \/        \/  \/\/
(02:47:52 PM) vorian: hope to see you all in #kubuntu-devel!
(02:47:56 PM) vorian: and happy packaging
(02:48:25 PM) dinxter: cheers vorian
(02:48:32 PM) apachelogger__: now party in #kubuntu-devel I suppose?
(02:48:59 PM) ***directhex starts rearranging podium
(02:49:06 PM) vorian: yup yup
(02:49:27 PM) vorian: you all have 10 minutes before the next glorious presentation:
(02:49:31 PM) vorian: Packaging software for Mono, for great justice: I’m very pleased we’re having Jo Shields and Mirco Bauer here to give a session about Mono packaging. How is it different? Why is it a lot of fun? Where does the team need help? Find out today!
(02:50:03 PM) directhex: or tomorrow if you're in australia. technically.
(02:50:24 PM) vorian: true
(02:50:56 PM) mariuz: ok thanks for an interesting session