source: trunk/debathena/NOTES @ 24928

Revision 24928, 25.5 KB checked in by geofft, 13 years ago (diff)
NOTES: A few updates, primarily more thorough build-all instructions Also start making some assumptions that this is for Debathena and not for Athena 10.
1This hierarchy contains Debian/Ubuntu-specific materials, also known
2as "Debathena".  The contents are:
4* debathena - Debathena-specific software packages such as PAM and NSS
5  modules.
7* config - Packages for configuring native system software in a manner
8  appropriate for Athena.
10* meta - Packages which contain nothing but dependencies on other
11  packages and serve as an installation convenience.
13* scripts - Build scripts and supporting materials.
15Debian software used by Debathena:
17  * schroot - Used to manage build chroot environments for each
18    Debian/Ubuntu version.  We use the device schroot type with aufs
19    union mounts, which allows rapid construction of ephemeral copies
20    of template "source" chroots, so that every binary package build
21    is done in a clean environment.
23  * debuild - Used to create Debian source packages from package
24    source directories.
26  * sbuild - Used to build binary packages from source packages inside
27    schroot environments.
29  * equivs - Used to create packages which only contain dependency
30    information.  Somewhat of a dirty hack, since it doesn't keep
31    proper changelogs, but it reduces overhead.
33  * CDBS (Common Debian Build System) - Referenced by debian/rules
34    files in packages.  Contains standard build rules to cut down on
35    per-package boilerplate.
37  * reprepro - Used to upload packages into the apt repositories.
39  * approx - Used to create a local cache of Debian packages on the
40    build server.  This cache is referenced by the build chroots for
41    improved performance.
43The remainder of this file documents procedures useful to Debathena
44developers and maintainers.
46Developers: Preferences setup
49You will probably want a $HOME/.devscripts file containing the
52DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn"
54This will save you from having to specify those options every time you
55run debuild.  The options mean:
57  * Look for original source as a tarfile or create one.
58  * Do not sign the source package.
59  * Do not sign the changes file.
60  * Ignore common version control metadata files when creating diffs.
61  * Ignore .svn paths when creating tarballs.
63You will also want a $HOME/.sbuildrc file containing the following:
65  $nolog = 1;
66  $mailto = 'yourusername';
67  $log_dir = '/tmp/sbuild-logs';
68  $maintainer_name = 'Debathena Project <>';
69  $force_orig_source = 1;
70  $sbuild_mode = "user";
71  1;
73You should also set the environment variable DEBATHENA_APT to
76Developers: Preparing a change
79To prepare a change to a regular package (a source tree containing a
80debian/ subdir), make the edits in a checkout and record a changelog
81entry.  You can either edit debian/changelog using emacs changelog
82mode (C-c C-v to add a new version entry, C-c C-a to add a change
83entry, C-c C-f to finalize the entry) or you can run "dadch" from
84the debathena locker / SVN scripts directory.
86When creating a new version entry, bump the upstream version number
87(to 10.0.0 if it was not already that high) if you are changing the
88main package source.  Otherwise, just bump the Debian version
89component (change 0debathena1 to 0debathena2, for instance).
91Developers: Building a package for test purposes on one platform
94After you have prepared a change, you will want to test that it builds
95and perhaps that it works before committing it.  First, if it is an
96Athena source directory using autoconf, run "daconfiscate" to set up
97the autoconf boilerplate which we don't check in.  Second, run
98"daorig" to copy or create an orig tarball in the parent directory if
99necessary.  Third, run "debuild".  The resulting package will be
100placed in the parent directory.
102In order to test if the package works, you can install it with "dpkg
103-i filename.deb".
105Developers: Building a package for test purposes on all platforms
108If the package you are working on interacts with the native OS in ways
109that might vary from platform to platform, you may want to do a test
110build for all platforms.  You will need to do this on or another machine which has been set up with
112build schroots.
114As above, run daconfiscate (if necessary) and then daorig.  Then run
115"debuild -S" to create a source package.  Now cd into the parent
116directory and identify the .dsc file created by debuild -S; it will
117have a name like debathena-just_9.4.0-0debathena2.dsc.  Run "da
118sbuildhack filename.dsc" to perform the package builds.  Each build
119will take place inside an ephemeral chroot based on a snapshot of a
120template for a particular Debian or Ubuntu version.  If a build fails
121and it's not obvious from the build log why, you may need to create
122your own ephemeral chroot session with a command like "schroot -c
123gutsy-amd64-sbuild /bin/sh" and then run debuild from within the
124package sources.
126If the build is successful, it will create a set of packages with
127names like debathena-just_9.4.0-0debathena2~ubuntu6.06_amd64.deb.
129Developers: Building an equivs package
132Most of the packages under debathena/meta are faked up using equivs.
133To build one, just run:
135  equivs-build --full filename.equivs
137These equivs files make reference to ../common, so you must have a
138checkout of debathena/meta/common alongside the particular
139meta-package you are building.
141Developers: The meaning of metapackages
144If you are adding a new package to the repository, you will probably
145at some point want to add it to one of the metapackages so that it
146doesn't have to be installed by hand.  Here are some descriptions
147which may help identify which metapackage is best:
149* locker: Provides access to Athena locker software--AFS and
150  automounter configuration, locker-related utilities, etc.
152* clients: Provides clients (either locally-written, like athinfo and
153  Discuss, or configurations) for Athena services, as well as
154  Athena-specific utility programs like "jot".  Configurations for
155  graphical client software are generally in the workstation package
156  instead, in order to make this package less intrusive.
158* standard: Implies locker and clients.  Also provides Athena shell
159  customizations and dotfiles.
161* login: Implies standard.  Configurations to merge the MIT user
162  namespace into the local machine namespace for the purpose of user
163  lookups and authentication.
165* login-graphical: Implies login.  Configurations for the graphical login
166  system and graphical client software intended to provide a standard
167  X login experience using Athena home directories.
169* extra-software: Implies a set of Debian packages common to cluster
170  machines and typical workstations.  The resulting software set is
171  rather large, and thus may not be desirable to all workstation
172  configurations.  Only stock Debian packages belong in this
173  metapackage; do not add other Debathena packages to it.
175* workstation: Implies login-graphical and extra-software.  The use case
176  is the same as that of the old "private workstations", so it includes
177  more centrally-managed configuration than standard or login does
178  (unattended updates, etc.). New as of May 2009.
180* thirdparty: Software supported by 3partysw that upstream Ubuntu
181  already happens to be keeping up to date, and is therefore easier to
182  install via packages than through lockers. This package is HUGE (1 GB of
183  network traffic, 3 GB installed), and is also not strongly tested on
184  Debian or on older releases of Ubuntu.
186* cluster: Implies workstation and thirdparty. Configures public cluster
187  machines (login-time LVM snapshots, cleanups between logins, public
188  root password, etc.). The snapshot code requires a particular LVM
189  layout that the PXE (netboot) cluster installer sets up.
191* debian-dev: Intended for developers of the system itself; provides a
192  set of Debian packages used by Debathena for development.
194For the most part a package should be listed in the "Depends:" line of
195a metapackage, but in some cases it is appropriate to hedge by using
196"Recommends:", which will cause aptitude to succeed even if the
197package is unavailable.  For example, a package which doesn't exist in
198all Debian/Ubuntu suites or isn't free can be listed under
199"Recommends:" so that our metapackages still work in all environments.
201Developers: Index of cluster packages
204Some private workstation admins may want specific pieces of cluster
205infrastructure.  With Athena 9.4, they would typically install the
206entire Athena software suite and then turn off features they did not
207want via /etc/athena/rc.conf.  That is not generally possible in
208Athena 10; a debathena-cluster machine acts like a cluster machine in
209all respects.  Instead, the machine owner should install
210debathena-workstation and then install specific packages they might be
211interested in.  At this time, candidate packages include:
213  * debathena-dns-config: Installs a caching DNS resolver and
214    configures the machine to use it.
216  * debathena-tmp-cleaner: Cleans files in /tmp and /var/tmp which
217    have not been recently accessed.
219  * debathena-reactivate: Causes each graphical login to be performed
220    in a separate ephemeral snapshot of the chroot.  This package
221    requires the root filesystem to be an LVM logical volume inside a
222    volume group with 21GB of free space for the snapshots; if that is
223    not the case, it will break graphical logins.  Install with care.
225  * debathena-auto-update: Periodically updates the machine's software
226    and reboots (when no one is logged in) if necessary.
228Cluster packages which are generally not of interest to private
229machine owners include:
231  * debathena-cluster-login-config: Configures a variety of system
232    services to implement the cluster login policy (no tty logins, no
233    user switching, screensaver logout button after 20 minutes, etc.).
235  * debathena-clusterinfo: Looks up the machine's cluster information
236    in Hesiod and caches it on local disk.
238  * debathena-larvnet: Reports to the central larvnet server whether
239    anyone has a graphical login to the machine.
241  * debathena-syslog-config: Configures sysklogd so that some syslog
242    messages are forwarded to a central logging host.
244Release engineer: Bootstrapping the project infrastructure
247  1. Create the package repository (detailed instructions on this
248     pending).  Set the DEBATHENA_APT environment variable to point to
249     the package repository.  Put a copy of the debathena "scripts"
250     subdir in your path.
252  2. Create the build area.
254  3. Run 'dareprepro export' to set up the repository.  Build each
255     equivs package under meta/ using 'equivs-build --full *.equivs'
256     and upload each with 'daupload-equivs-proposed *.changes',
257     followed by 'damove -proposed "" PACKAGE' (replacing PACKAGE with
258     the name of each equivs package you are moving).
260  4. Set up the build server.  The basic structure of the apt
261     repository must work for make-chroot to succeed, so this must
262     happen after step 3.
264  5. For each normal Debian package in dependency order, cd into its
265     directory in the build area and run "da sbuildhack *.dsc" and
266     "daupload-release *_source.changes".  If the package contains
267     only an "Architecture: all" binary package, pass the -A option to
268     both commands.
270     The all-packages script can generate an approximation of the
271     package list in dependency order, but it doesn't work right yet,
272     and ideally it would be possible to do several builds in parallel
273     using a Makefile like the one in scripts/build-server/build-all.
274     Improvements to this machinery are pending.
276  6. For each package under third, run "da ./debathenify-PKG source
277     binary upload".  Any created directories under third/openafs/meta
278     should be chmodded 777 to work around a perl/AFS permissions
279     issue with File::Temp; if this is not done, OpenAFS metapackage
280     builds will fail for other users.
282Release engineer: Updating the apt repository
285  1. ssh to the build server as the builder account and change to the
286     canonical build directory.
288  2. Run "gen-packages" to update the package list.  (Or "gen-packages
289     -c" if you know the AFS checkout of the source tree is up to date;
290     it should update every half hour.)
292  3. Run "ood-packages" to produce a list of out-of-date packages.
294  4. For each out of date package, run "dasource PKG".  Then change to
295     the package directory and run "da sbuildhack *.dsc" and
296     "daupload-release *_source.changes".  If the package contains
297     only an "Architecture: all" binary package, pass the -A option to
298     both commands.
300  5. svn update the meta directory.  If there are new subdirectories,
301     chmod them 777 to work around a perl/AFS permissions issue with
302     File::Temp.  For each updated subdir, change to it, run
303     "equivs-build --full *.equivs", and then "daequivsupload
304     FILENAME.changes" on the produced changes file.
306  6. svn update the third directory.  You can let autodebathenify
307     handle the updated scripts, or you can touch
308     ~/autodebathenify.suppress, make sure it's not running, and run
309     "da ./debathenify-PKG source binary upload" in each updated
310     directory.
312Sometimes you may have to mix up the order of the above steps in order
313to handle build dependencies.
315Release engineer: Setting up a build server
318  1. Install either Debian testing (or stable, if it's new enough, but
319     see the note about sbuild below) or the latest Ubuntu release.
320     The build server must be installed with free space in an LVM
321     volume group.  The build chroots consume 4GB each.
323  2. Install debathena-login as per the the instructions in
326  3. apt-key add
327     /afs/
329  4. Install the packages listed in scripts/build-server/packages
330     (using "aptitude install").
332     Note that currently the build system uses the latest version of
333     sbuild, so if you have not installed Debian testing, you'll need
334     to set up apt pinning to grab sbuild from there. See the
335     apt_preferences man page. You'll also need to grab schroot from
336     Debian experimental.
338  5. Install the aufs-modules-2.6-amd64 package.
340  6. Append to /etc/approx/approx.conf the contents of
341      scripts/build-server/approx.conf.tail.
342     Run: /etc/init.d/approx restart
344  7. Apply scripts/build-server/mount-defaults.patch and
345     scripts/build-server/pam-schroot.patch.
347  8. For each supported DIST (see scripts/ run:
349       scripts/build-server/make-chroot DIST i386
350       scripts/build-server/make-chroot DIST amd64
352     Example: scripts/build-server/make-chroot intrepid i386
354  9. Create a local account for builder with:
356       adduser --uid 1047 --disabled-password builder
358     Make the home directory mode 700.  Install a
359     daemon/ keytab in the home directory as
360     "keytab".  Install a copy of the secret repository-signing key
361     ( in the home directory's keyring with
362     something like:
364       kinit builder
365       gpg --export-secret-keys | \
366         ssh -l builder machinename gpg --import
368     Create a file named .sbuildrc in builder's homedir containing:
370       $mailto = undef;
371       $log_dir = '/tmp/sbuild-logs';
372       $maintainer_name = 'Debathena Project <>';
373       $force_orig_source = 1;
374       $sbuild_mode = "user";
375       1;
377     Create a file named .ssh/config in builder's homedir containing:
379       Host
380         User debuildsvn
382     Add builder to the sbuild group in /etc/group.
384     Copy scripts/build-server/autodebathenify to builder's homedir.
385     Create a file named autodebathenify.config in builder's homedir
386     containing:
389       scripts_dir=/mit/debathena/bin
390       build_dir=/mit/debathena/packages/third
391       packages="lprng openafs"
392       export DEBATHENA_APT=/mit/debathena/apt
394     Copy scripts/build-server/autodebathenify.cron to builder's
395     homedir and install it with "crontab autodebathenify.cron".
397     Copy scripts/build-server/autolivebuilder to builder's
398     homedir. Create a file called autolivebuilder.config in builder's
399     homedir containing:
402       release_version='9.04'
403       release='jaunty'
404       arch='i386'
405       mirror=''
406       gpg_opts=("-u" "0D8A9E8F")
407       live_dir=/net/
409     Create a file named .devscripts in builder's homedir containing:
411       DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn"
413     In builder's homedir, append to .bashrc:
415       add debathena
416       export PATH=$PATH:~/bin
417       export DEBATHENA_APT=/mit/debathena/apt
420Release engineer: Removing a build chroot on the build server
423  1. Run VG=/dev/blah scripts/clean-schroots as root to make sure that
424     the build chroot is not mounted, substituting the name of the
425     volume group for blah.
427  2. Edit /etc/schroot/schroot.conf and delete the section
428     corresponding to the chroot.
430  3. Run lvchange -an blah/chrootname
431     substituting the name of the volume group for blah and the chroot
432     name for chroot.  Example: lvchange -an dink/gutsy-i386-sbuild
434  4. Run lvremove blah/chrootname
436Release engineer: Removing a dist from the apt repository
439  1. Inside the apt repository, edit conf/distributions and remove the
440     distribution section.
442  2. Run dareprepro --delete clearvanished
444Release engineer: Setting up a canonical build area
447  1. Create an empty directory and cd into it.  The canonical build
448     area lives in /mit/debathena/packages/.
450  2. Run gen-packages to create the table of normal Debian packages.
452  3. Run dasource to create subdirs and source packages for each
453     normal Debian package.
455  4. Create checkouts of the meta and third directories:
457     svn co svn+ssh://
458     svn co svn+ssh://
459     chmod 777 meta/*
461     A couple of subdirectories of debathena/meta are normal Debian
462     packages, so this will create redundant copies of those.  Ignore
463     them; they won't be used.
465Release engineer: Adding a new suite or rebuilding a suite
468This process is rarely performed and the infrastructure for it is
469imperfect.  Substitute the name of the new suite for $suite in all
470steps below.
472  1. Make sure the apt repository is up to date with respect to the
473     source tree for the existing dists. Check
474 and
475 More importantly, make
476     sure there is nothing in svn that isn't built. (An easy way to do
477     this is running the check-unbuilt-packages script.)
479  2. Add the new dist to scripts/  (It is not
480     necessary to add the new dist to codes at this point, but it must
481     be present in the gettag conditional.)  If you are building an
482     unreleased distribution, add an additional ~0.1 tag; if you're
483     rebilding, bump that tag.
485  3. Create the new distribution in the apt repository's configuration
486     file by editing /mit/debathena/apt/conf/gen-distributions and
487     running it with output to the 'distributions' file in the same
488     directory.  Create the skeleton of the dist by running "dareprepro
489     export". (You don't need to do this if you're rebuilding.)
491  4. Add an appropriate sources.list.d/*/$suite.list entry in
492     scripts/build-server. Commit and update
493     /mit/debathena/bin/build-server.
495  5. On the build server, create a chroot for the new distribution:
497        /mit/debathena/bin/build-server/make-chroot $suite i386
498        /mit/debathena/bin/build-server/make-chroot $suite amd64
500     This may require downloading and installing a
501     more recent version of the debootstrap package from the
502     -backports dist corresponding to the build server's OS. (You
503     presumably don't need to do this if you're rebuilding.)
505  6. Fire up screen.
507  7. mkdir /mit/debathena/machines/awesome-build-server/stamps.$suite.
509  8. cd into a checkout of debathena/scripts/build-server/build-all.
511  9. Edit Makefile (and check in the edit) so that suite is the new
512     distribution and psuite is the previously most recent Debian or
513     Ubuntu distribution.  psuite is used to generate the order of
514     build-dependencies, so choose the closest distribution.  If you are
515     rebuilding, you can set psuite to suite.
517  10. Run "make".
519  11. Run "make -k all".
520      You can watch the builds happen in the other windows of the
521      screen session.  It's possible to do several builds at once with
522      make -j N.
524  12. debathenify packages will fail out; they must be built by hand.
525      When the build fails on one, cd into third/packagename in the
526      build area and run "./debathenify $suite-amd64 -A source binary
527      upload" and "./debathenify $suite-i386 binary upload".  Then
528      touch $DEBATHENA_BUILD_AREA/stamps.$suite/packagename.done" and
529      restart the build.
531      The newly created third/openafs/meta directories should be
532      chmodded 777 to work around a perl/AFS permissions issue.
534  13. Equivs packages (in meta/) will not be built, since they are only
535      built once and uploaded to every release. Upload all equivs
536      packages by running dareprepro include $suite-development on each
537      .changes file.
539Release engineer: Maintaining autodebathenify
542autodebathenify is a cron job which runs on the builder account and
543keeps the OpenAFS modules and modified packages under debathena/third
544up to date when there are upstream changes.  It relies on the
547  * builder@ATHENA.MIT.EDU has access to a local account named
548    debuildsvn on with read access to the athena
549    repository.
551  * builder@ATHENA.MIT.EDU has write access to the canonical build
552    area and apt repository in the dev cell.
554  * The builder account on the build machine has a copy of the signing
555    private key for the apt repository.
557The cron job will silently exit if it detects that it is already
558running.  If it fails for any other reason, it will send mail to and touch a file named autodebathenify.suppress
560which will prevent it from running again until manual intervention.
561It is important to get autodebathenify running smoothly or the apt
562repository will become out of date with respect to third-party
563packages, which in turn will compromise the user experience.
565Even if you don't notice failure mail, it's good to check on the
566status of autodebathenify from time to time.  To do this, get tickets
567as builder and ssh to linux-build-10 as builder.  Run "ls -l logs" to
568see what's been going on.  A logfile with size around 48K indicates
569that autodebathenify ran normally but didn't find any new work to do.
570A logfile with size around 60 means that autodebathenify.suppress
571exists and the cron job has stopped running.  A longer logfile
572indicates that autodebathenify attempted to build and upload a
575Possible failure cases include:
577  * One of the upstream apt repositories timed out during an "apt-get
578    update" of an sbuild chroot.  If this happens, you can just remove
579    the autodebathenify.suppress file to get the cron job to start
580    running again.
582  * autodebathenify tried to build an upstream version which already
583    exists.  If this happens, the build will fail at upload time.  If
584    this happens, there is a bug in the part of the debathenify-*
585    script which checks whether our apt repository already has the
586    expected version of the built package.
588  * Debian and Ubuntu have .orig files with the same name but
589    different contexts for a particular third-party package.  This has
590    only happened once, with bash (which is currently not configured
591    into autodebathenify for that reason).  We do not have a general
592    mechanism for resolving such issues at this time.
594  * The new upstream version of the package is sufficiently different
595    to cause our modifications to fail to apply, and our debathenify-*
596    script needs to be adapted appropriately.
598The cron job is run out of builder's home directory, but its canonical
599source location is in debathena/scripts/build-server in this
600repository, should it need to be modified.
602After several months of operation, LVM snapshots on the build server
603will start to become slower and slower, as /etc/lvm/cache/.cache grows
604in size.  Turning off the cache entirely seems to cause schroot to
605occasionally fail; the current solution is to rm /etc/lvm/cache/.cache
606every so often.
608Release engineer: apt repository HTTP server setup
611The apt repository server ( is hosted on out of the debathena locker. To replicate it on a
613dedicated server, install Apache and AFS (e.g., debathena-standard
614and apache2), and configure the document root to be
615  /mit/debathena/web_scripts
617There's also a crontab in /mit/debathena/cron_scripts that updates
618the package lists on the website. It expects to be able to edit the
619web_scripts directory; on scripts, it does so with daemon.scripts
622Trac ( expects to use
623for its database.
Note: See TracBrowser for help on using the repository browser.