2009-04-30-2
Next Session: TBD |
1 [07:00] <@dholbach> GOOD MORNING EVERYBODY!
2 [07:00] <@dholbach> who do we have here for the Packaging Training session? :)
3 [07:01] <rww> o/
4 [07:01] <genii> It's morning?
5 [07:01] <rww> genii: in crazy UTC people time, yeah. 11pm here :(
6 [07:01] <@dholbach> genii: it is over here - 08:01 right now :-)
7 [07:02] <Rail> moin moin
8 [07:02] <@dholbach> so... do we have any questions already before we kick off the session?
9 [07:03] <Crusher> nope
10 [07:03] <@dholbach> alright... :-)
11 [07:03] * jost86 gets Coffee...
12 [07:03] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted is the one page you absolutely need to bookmark :)
13 [07:04] <@dholbach> it contains links to all important documentation
14 [07:04] <@dholbach> like the Packaging Guide, like the Developer Videos, Ubuntu Developer process documentation, simple bugs to start working on etc
15 [07:05] <@dholbach> ok
16 [07:05] <@dholbach> which version of Ubuntu are you all running?
17 [07:05] <Crusher> jaunty
18 [07:05] <djails> Hardy
19 [07:05] <rww> Jaunty :)
20 [07:05] <jost86> jaunty
21 [07:06] <genii> 8.04.2
22 [07:06] <@dholbach> alright :)
23 [07:06] <zaidka> hardy
24 [07:06] <slangasek> breezy
25 [07:06] <zaidka> kidding.. jaunty
26 [07:06] <maxb> jaunty with a karmic install available for dual-booting :-)
27 [07:06] <sebsebseb> upgraded 8.10, to 9.04 alpha6, to beta, to rc. install that has some issues from alpha6, so will eventualy clean install
28 [07:06] <zaidka> ubuntu cambridge
29 [07:06] <@dholbach> if you want to work on Ubuntu, you absolutely need to have the latest development release
30 [07:07] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases explains how to do that in a sane way :)
31 [07:07] <@dholbach> I personally had chroots (explained on there as well) for a long time, but now I almost exclusively use kvm (used virtualbox in the past)
32 [07:08] <nixternal> woowoo
33 [07:08] <@dholbach> you absolutely need to be able to test packages in the ubuntu version you're about to upload them for :-)
34 [07:08] <nixternal> teach me ol' great one!
35 [07:09] <@dholbach> nixternal: come on... you've been around long enough already :)
36 [07:09] * genii feeds nixternal some pie to quieten him down
37 [07:09] <nixternal> mmmmm
38 [07:09] <@dholbach> for now, let's just keep using the version we're all comfortable with, but please make a note to set up a VM at some stage :-)
39 [07:09] <@dholbach> please run:
40 [07:09] <@dholbach> sudo apt-get install ubuntu-dev-tools devscripts build-essential gnupg
41 [07:10] <@dholbach> it will install a bunch of packages for you we're going to need for the session
42 [07:10] <nixternal> isn't gnupg installed by default already?
43 [07:10] <@dholbach> maybe nixternal can exaplain what they're there for
44 [07:10] <@dholbach> nixternal: just making sure :)
45 [07:10] <nixternal> k, confused me for a sec
46 [07:10] <raylu> if we're packaging for the current release, we won't need the latest dev release, right?
47 [07:10] <zaidka> what! none of them is installed by default
48 [07:10] <genii> I have to beg out, I'll be /away but checking the backscroll later
49 [07:10] <@dholbach> raylu: for now we'll just play around with the tools, set things up and so on
50 [07:11] <nixternal> raylu: correct, though you never know when you will want to start developing for the next release
51 [07:11] <@dholbach> raylu: if you want to upload packages for karmic, you need to have them built and tested on karmic :)
52 [07:11] <@dholbach> raylu: I hope that makes sense
53 [07:11] * nixternal notes to fix it in the dev cycle and backport/sru as needed to the current stable release
54 [07:11] * ara remembers that if there is someone that prefer to make questions in Spanish, they can be posted at #ubuntu-classroom-chat-es
55 [07:11] <nixternal> thanks ara!
56 [07:12] <raylu> dholbach: right; just that you said we "absolutely" need the latest dev release
57 [07:12] * nixternal goes back to watching and hushes up
58 [07:12] <@dholbach> raylu: right, "absolutely" for testing :-)
59 [07:12] <@dholbach> and "in some form" :)
60 [07:12] <@dholbach> ok
61 [07:12] <@dholbach> so this is what the packages are for:
62 [07:12] <rww> "Please select the mail server configuration type that best meets your needs." =/
63 [07:13] <@dholbach> rww: you can purge mailx and the mailserver afterwards - sorry
64 [07:13] <@dholbach> or use --no-install-recommends
65 [07:13] <@dholbach> (in the apt-get call)
66 [07:13] <nixternal> rww: just do the top selection, no configuration for now
67 [07:13] <@dholbach> - gnupg (installed by default as nixternal said) can sign and encrypt files (we use it to sign source packages)
68 [07:14] <@dholbach> - build-essential: will install the bare minimum to be able to compile (very simple packages): the compiler, make, etc.
69 [07:15] <@dholbach> - devscripts: a bunch of tools that make Debian packaging much easier (especially repetitive tasks)
70 [07:15] <@dholbach> - ubuntu-dev-tools: much like devscripts, but more closely tied to Ubuntu
71 [07:15] <@dholbach> any questions so far?
72 [07:15] <jost86> nope
73 [07:16] <teknico> I got here late
74 [07:16] <slangasek> using --no-install-recommends (or setting it in /etc/apt/apt.conf.d/) is highly advisable for a development chroot, not just because it saves download/configuration time, but because that's a more accurate model of how the buildds work
75 [07:16] <teknico> are there logs anywhere?
76 [07:16] <@dholbach> teknico: sudo apt-get install --no-install-recommends ubuntu-dev-tools devscripts build-essential gnupg (is where we are right now)
77 [07:17] <@dholbach> plus a bunch of links that will be available on https://wiki.ubuntu.com/Packaging/Training later on
78 [07:17] <teknico> thx
79 [07:17] <@dholbach> ok, please edit ~/.pbuilderrc in your favourite editor and add the following
80 [07:17] <@dholbach> COMPONENTS="main universe multiverse restricted"
81 [07:17] <@dholbach> and then save the file
82 [07:17] <rww> teknico: http://paste.ubuntu.com/161169/
83 [07:18] <@dholbach> we often use a tool called pbuilder which will build source packages in a minimal environment for us (sets up that environment, installs the build-depends packages, builds the package, etc in an automated fashion)
84 [07:19] <@dholbach> for now just run
85 [07:19] <@dholbach> pbuilder-dist create <your Ubuntu version> (hardy, jaunty, karmic, whatever)
86 [07:19] <@dholbach> it will set up the initial pbuilder
87 [07:19] <@dholbach> the nice thing about pbuilder-dist is, that you can set up different build environments for karmic, jaunty, intrepid, hardy, etc etc :)
88 [07:20] <@dholbach> (also Debian versions)
89 [07:20] <@dholbach> so that's pretty neatr
90 [07:20] <@dholbach> neat
91 [07:20] <Crusher> dholbach: if you already have a pbuilder environment setup, will this overwrite it?
92 [07:20] <@dholbach> https://wiki.ubuntu.com/PbuilderHowto has more info about that tool
93 [07:20] <@dholbach> Crusher: did you use pbuilder-dist?
94 [07:20] <rww> dholbach: are "create" and "<your ubuntu version>" switched around? I'm getting an error
95 [07:20] <Crusher> dholbach: no
96 [07:20] <@dholbach> rww: they are :-)
97 [07:20] <@dholbach> pbuilder-dist <your Ubuntu version> create (hardy, jaunty, karmic, whatever)
98 [07:20] <@dholbach> thanks a lot rww
99 [07:21] * dholbach didn't sleep very well last night *yawn*
100 [07:21] <@dholbach> Crusher: in that case it shouldn't, but you can safely skip the step as well
101 [07:21] <@dholbach> ok... while that's running, let's set up a GPG key
102 [07:21] <@dholbach> who of you does not have a GPG key set up yet?
103 [07:22] <jost86> i might have one... but I dont remember it
104 [07:22] <@dholbach> awesome, looks like we can skip a step :)
105 [07:22] <jost86> well go ahead
106 [07:22] <@dholbach> jost86: what does ls ~/.gnupg/pubring.gpg say?
107 [07:22] <@dholbach> do you have that file?
108 [07:23] <jost86> yep, present
109 [07:23] <@dholbach> super
110 [07:23] <@dholbach> basically I would have run you through the steps of:
111 [07:23] <@dholbach> gpg --gen-key
112 [07:24] <@dholbach> gpg --send-keys <key>
113 [07:24] <@dholbach> and putting it up on Launchpad
114 [07:24] <raylu> i'd "gpg -K" to make sure you actually have a key
115 [07:24] <@dholbach> raylu: good one - great
116 [07:24] <jost86> k, i think i'll figure that out :) thx
117 [07:25] <@dholbach> if you didn't put up your GPG key on Launchpad, you can do it here: https://launchpad.net/people/+me/+editpgpkeys
118 [07:26] <@dholbach> in the beginning you won't need the GPG key very often, as you're not directly allowed to upload packages to Ubuntu yet
119 [07:26] <@dholbach> but I'll get back to the "upload packages part" in a bit
120 [07:26] <@dholbach> if you use bash (default shell), please add something like this to your ~/.bashrc
121 [07:26] <@dholbach> export DEBFULLNAME='Daniel Holbach'
122 [07:26] <@dholbach> export DEBEMAIL='daniel.holbach@ubuntu.com'
123 [07:27] <@dholbach> these are variables that are used by the tools in devscripts and it'll save you typing your name over and over again :)
124 [07:27] <@dholbach> (editing changelog entries is one of the very common use-cases)
125 [07:28] <@dholbach> another thing you need to make sure when you're working on Ubuntu packages is that you have source package repositories enabled in apt
126 [07:28] <@dholbach> so you need something like this in your /etc/apt/sources.list
127 [07:28] <@dholbach> deb-src http://archive.ubuntu.com/ubuntu/ jaunty restricted main multiverse universe
128 [07:28] <@dholbach> (of course replace jaunty with karmic or hardy or whatever you're using)
129 [07:29] <@dholbach> you can also use the System -> Administration -> Software Sources tool
130 [07:29] <@dholbach> (or whatever it's called in English, sorry :-))
131 [07:29] <@dholbach> once you've configured that, you can simply run
132 [07:30] <@dholbach> sudo apt-get update
133 [07:30] <@dholbach> apt-get source hello
134 [07:30] <@dholbach> and it will download you the current source code for that package from the archive
135 [07:30] <@dholbach> any questions up until now?
136 [07:31] <@dholbach> so let's run
137 [07:31] <@dholbach> apt-get source hello
138 [07:31] <@dholbach> it's a very very boring package, but we'll download it anyway now
139 [07:31] <@dholbach> apt is going to download the following files:
140 [07:31] <@dholbach> hello_2.2-3.diff.gz
141 [07:32] <@dholbach> hello_2.2-3.dsc
142 [07:32] <@dholbach> hello_2.2.orig.tar.gz
143 [07:32] <@dholbach> (might be other versions on your system)
144 [07:32] <@dholbach> let's go through them one by one
145 [07:33] <@dholbach> the .orig.tar.gz file is the original pristine untouched tarball that was downloaded from the website of the software authors
146 [07:33] <@dholbach> in this case it's from something like ftp.gnu.org
147 [07:33] <@dholbach> it's important that we don't change it (only very rare cases where it's justified), but merely rename it to something like
148 [07:33] <@dholbach> <software project>_<version>.orig.tar.gz
149 [07:34] <@dholbach> the .diff.gz file contains the compressed set of changes we need to apply to make the package build the Ubuntu or Debian way
150 [07:34] <@dholbach> that's "the packaging"
151 [07:35] <@dholbach> basically we add a directory called ./debian/ to the source which further describes the package, the history of the packaging, the copyright and so on
152 [07:36] <@dholbach> when I said "make it build the Ubuntu/Debian way", I meant that we wrap a build process around the existing build system that the upstream package offers
153 [07:36] <@dholbach> in the standard case of a C+autotools project that would be something like ./configure; make; sudo make install
154 [07:36] <@dholbach> in a python+setuptools case that would be python setup.py install (or something like that)
155 [07:37] <@dholbach> the reason for that is: we want just one way to build packages :-)
156 [07:37] <@dholbach> the .dsc file contains just some bits of metadata, like the md5sum of the files, etc.
157 [07:37] <@dholbach> are there any questions up until now?
158 [07:38] <Crusher> dholbach: under what circumstances do we make native packages?
159 [07:38] <@dholbach> Crusher: good question
160 [07:38] <rww> dholbach: I have a directory hello-2.2 with a bunch of files in it too, that you didn't mention...
161 [07:38] <@dholbach> rww: will get to that in a sec
162 [07:38] <@dholbach> it seems that Crusher has read a few chapters of the packaging guide already :-)
163 [07:38] <rww> dholbach: oh, okay. Just checking I didn't do something wrong :)
164 [07:39] <Crusher> ;)
165 [07:39] <@dholbach> rww: no, you're doing great :-)
166 [07:39] <raylu> after you answer that... what if their source relies on a directory called "debian" already?
167 [07:39] <@dholbach> so in the case of a native package, you just have a <project>_<version>.tar.gz and a <project>_<version>.dsc file
168 [07:40] <@dholbach> which means all Packaging changes are in the .tar.gz file too
169 [07:40] <@dholbach> so you can't easily tell apart which change was done by the packager and which came from upstream
170 [07:41] <@dholbach> also it means that for every small packaging change (update the package description for example) you need to upload the whole tarball (might be several MB)
171 [07:42] <@dholbach> in the other case you just upload the .diff.gz and the .dsc (ie: 1.2-1: upload the .tar.gz, 1.2-2: just upload the .diff.gz)
172 [07:42] <@dholbach> ok, so why do we use that option?
173 [07:43] <@dholbach> in some cases we are upstream of the package (and don't do regular .tar.gz releases) and keep everything in svn or bzr or git or whatever
174 [07:43] <@dholbach> in that case, especially if it's a small package, we just dump everything in that .tar.gz file and make it a native package
175 [07:44] <@dholbach> Crusher: did that make sense?
176 [07:44] <Crusher> dholbach: yes
177 [07:44] <@dholbach> raylu: some upstream maintainers decide to ship their own Debian packaging, so there's a debian/ directory already
178 [07:45] <@dholbach> that's one of the rare cases where we repackage the tarball and rip out the existing packaging
179 [07:45] <@dholbach> it makes reading the .diff.gz much easier
180 [07:45] <nixternal> what about asking upstream to release a prestine tarball without a debian directory?
181 [07:45] <@dholbach> nixternal: very good point, it makes a lot of sense to do that
182 [07:45] <@dholbach> but in the meantime that's probably what you do
183 [07:46] <@dholbach> alright... cracking on and getting to rww's question
184 [07:46] <@dholbach> what about the hello-2.2 directory
185 [07:46] <@dholbach> "apt-get source" (actually it's dpkg-source -x) is a clever tool - what it did for us was
186 [07:46] <@dholbach> - extract the tarball
187 [07:47] <@dholbach> - use the .diff.gz and patch the code with our packaging changes
188 [07:47] <@dholbach> (and check the md5sums along the way)
189 [07:47] <@dholbach> so if you check out the hello-2.2 directory, you'll see that there's a debian/ directory already
190 [07:47] <@dholbach> now let's test-build the package
191 [07:47] <@dholbach> please run
192 [07:47] <@dholbach> pbuilder-dist <your release> build hello_*.dsc
193 [07:48] <@dholbach> this might take some time
194 [07:48] <@dholbach> and I hope you all get addicted of watching build messages fly by and get involved with Ubuntu development :-)
195 [07:49] <raylu> if i wanted that, i'd be using gentoo
196 [07:49] <@dholbach> haha :)
197 [07:49] <@dholbach> ~/pbuilder/*_result/ should now contain the resulting .deb file
198 [07:50] <@dholbach> raylu: right but in our case, you do one fix and 10 million other users benefit from it
199 [07:51] <@dholbach> if you check out https://wiki.ubuntu.com/PackagingGuide/Recipes you'll find a bunch of examples of what you can do with the packaging tools and perform simpler tasks like "upgrade package to a new upstream", "make a small change", etc.
200 [07:51] <@dholbach> once you created a small patch, you can use the Sponsorship Process to get it included in Ubuntu
201 [07:51] <@dholbach> https://wiki.ubuntu.com/SponsorshipProcess has all the gory details
202 [07:51] <@dholbach> so what's that for?
203 [07:52] <@dholbach> you're only allowed to upload Ubuntu source packages directly when you're part of the development team
204 [07:52] <@dholbach> until then you need to get your patches reviewed and then uploaded by somebody else
205 [07:53] <@dholbach> so what you do is: attach the patch to a bug report on Launchpad (or link to the source package you uploaded somewhere)
206 [07:53] <imbrandon> its not as scary as it sounds
207 [07:53] <@dholbach> then subscribe the sponsors team
208 [07:53] <@dholbach> and they'll review the patch, give you feedback, etc.
209 [07:53] <@dholbach> then apply the patch to the source package, sign it with their GPG key and upload it for you
210 [07:53] <@dholbach> imbrandon: exactly
211 [07:53] <@dholbach> the sponsors team is a really friendly bunch and you'll learn a lot that way
212 [07:54] <@dholbach> once you've been through that process a couple of times, people will tell you that you should join the team, and then it's time to send in a short "application"
213 [07:54] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopers has more details about that
214 [07:54] <@dholbach> did that scare you? :-)
215 [07:54] <@dholbach> do you have any questions about the process?
216 [07:55] * jost86 dreams of being rich and famous ;)
217 [07:55] <teknico> o/
218 [07:55] <teknico> the pbuilder command built a .deb package
219 [07:55] <@dholbach> teknico: so you're scared? or do you have a question? :)
220 [07:55] <@dholbach> teknico: yep
221 [07:55] <teknico> the dsc file is no more signed, though
222 [07:56] <teknico> how come?
223 [07:56] <rww> dholbach: "patch" in the above means debdiffs we create with https://wiki.ubuntu.com/PackagingGuide/Recipes/Debdiff , right?
224 [07:57] <@dholbach> teknico: I usually don't use the .dsc files from *result/, but I guess it's to make sure you don't upload them accidentally
225 [07:57] <imbrandon> teknico: correct you'll have to source build and resign after pbuilder if testing goes ok
226 [07:57] <@dholbach> teknico: dput will refuse to upload them for you (unless you force it to)
227 [07:57] <imbrandon> rww: correct
228 [07:57] <@dholbach> rww: a general patch should be fine, but a debdiff (with a nice changelog entry) will speed things up
229 [07:58] <@dholbach> that's one of the most important points: always document your work properly
230 [07:58] <@dholbach> next week we're going to have a session about PPAs, so Dustin Kirkland will talk a bit about how to upload packages to Launchpad
231 [07:59] <@dholbach> that should be interesting and tie in quite well with this session
232 [07:59] <@dholbach> any more questions? :)
233 [07:59] <djails> can you point me to docs on how to package shared libs ?
234 [07:59] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted should contain all the links I mentioned earlier
235 [07:59] <@dholbach> djails: hang on
236 [08:00] <@dholbach> http://www.debian.org/doc/debian-policy/ch-sharedlibs.html is the debian policy section for it
237 [08:00] <@dholbach> and we had a session about that already, let me find it
238 [08:00] <djails> thanks
239 [08:00] <@dholbach> https://wiki.ubuntu.com/MOTU/School/LibraryPackaging
240 [08:01] <djails> awesome !
241 [08:01] <@dholbach> it's been a while since that session, but most things should not have changed
242 [08:01] <@dholbach> also I'd recommend checking out a few very simple other library packages
243 [08:01] <@dholbach> libsexy for example
244 [08:01] <@dholbach> any other questions?
245 [08:02] <sbeattie> dholbach: is there anything specific about packaging perl packages?
246 [08:02] <@dholbach> ok cool - feel free to join #ubuntu-motu and ask all questions there - the people in there are really friendly and will help you out for sure
247 [08:03] <@dholbach> sbeattie: I'm sure there is, I just did one in the past and used perlmodule.mk (part of CDBS) and made sure I used ${perl:Depends} somewhere
248 [08:03] <@dholbach> sbeattie: that's all I know
249 [08:04] <sbeattie> dholbach: okay, thanks.
250 [08:05] <@dholbach> ok... thanks a lot everybody - it's been a great session!
251 [08:05] * Rail is going to request sponsorship for some packages :)
252 [08:05] <@dholbach> I had a lot of fun and hope to see you guys around more often from now on!
253 [08:05] <@dholbach> Rail: awesome!
254 [08:05] <djails> dholbach: thanks, i ll try the REVU process for my pkg
255 [08:05] <Rail> dholbach: thank you very much
256 [08:05] <@dholbach> djails: perfect
257 [08:06] <rww> dholbach: Are the various videos on the ubuntudevelopers youtube account still up-to-date (e.g. http://www.youtube.com/watch?v=zKLabbXTqMc ), or have there been any changes in the last year or so to watch out for?
258 [08:06] <@dholbach> have a great day everybody and hope to see you at the next sessions too :)
259 [08:06] <@dholbach> rww: there should be only small changes like change "hardy" to "jaunty" or something
260 [08:06] <@dholbach> rww: in principle it should be all the same still
261 [08:07] <rww> dholbach: excellent, thanks :)
262 [08:07] <@dholbach> rock on everybody! :-)
263 [08:07] <Crusher> thanks for the session dholbach, you have a great day aswell.
264 [08:07] <jost86> thx
265 [08:10] * thekorn hugs dholbach, great session
266 [08:10] <@dholbach> gracias thekorn
267 [08:11] <dme> dholbach: Very useful, thanks.
Packaging/Training/Logs/2009-04-30-2 (last edited 2009-04-30 08:03:20 by i59F71706)