source: trunk/debathena/NOTES @ 23253

Revision 23253, 25.3 KB checked in by rbasch, 16 years ago (diff)
In the NOTES section on adding a new suite, when installing the initial meta package, use reprepro's "--ignore=wrongdistribution" option.
Line 
1This hierarchy contains Debian/Ubuntu-specific materials, also known
2as "Debathena".  The contents are:
3
4* debathena - Debathena-specific software packages such as PAM and NSS
5  modules.
6
7* config - Packages for configuring native system software in a manner
8  appropriate for Athena.
9
10* meta - Packages which contain nothing but dependencies on other
11  packages and serve as an installation convenience.
12
13* scripts - Build scripts and supporting materials.
14
15Debathena is a SIPB project, and its infrastructure and procedures
16will need to be adapted for Athena 10.  For the moment this file will
17document the Debathena procedures as they are, not as they will be.
18The current procedures do not even use this svn repository yet.
19
20Debian software used by Debathena:
21
22  * schroot - Used to manage build chroot environments for each
23    Debian/Ubuntu version.  We use the lvm-snapshot schroot type,
24    which allows rapid construction of ephemeral copies of template
25    "source" chroots, so that every binary package build is done in a
26    clean environment.
27
28  * debuild - Used to create Debian source packages from package
29    source directories.
30
31  * sbuild - Used to build binary packages from source packages inside
32    schroot environments.
33
34  * equivs - Used to create packages which only contain dependency
35    information.  Somewhat of a dirty hack, since it doesn't keep
36    proper changelogs, but it reduces overhead.
37
38  * CDBS (Common Debian Build System) - Referenced by debian/rules
39    files in packages.  Contains standard build rules to cut down on
40    per-package boilerplate.
41
42  * reprepro - Used to upload packages into the apt repositories.
43
44  * approx - Used to create a local cache of Debian packages on the
45    build server.  This cache is referenced by the build chroots for
46    improved performance.
47
48The remainder of this file documents procedures useful to Athena 10
49developers and the release engineer.
50
51Developers: Preferences setup
52-----------------------------
53
54You will probably want a $HOME/.devscripts file containing the
55following:
56
57DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn"
58
59This will save you from having to specify those options every time you
60run debuild.  Athena 10 scripts do not assume the above preferences,
61but the instructions in this file do.  The options mean:
62
63  * Look for original source as a tarfile or create one.
64  * Do not sign the source package.
65  * Do not sign the changes file.
66  * Ignore common version control metadata files when creating diffs.
67  * Ignore .svn paths when creating tarballs.
68
69You will also want a $HOME/.sbuildrc file containing the following:
70
71  $nolog = 1;
72  $mailto = 'yourusername';
73  $log_dir = '/tmp/sbuild-logs';
74  $maintainer_name = 'Debian-Athena Project <debathena@mit.edu>';
75  $force_orig_source = 1;
76  $sbuild_mode = "user";
77  1;
78
79You should also set the environment variable DEBATHENA_APT to
80"/afs/dev.mit.edu/system/athena10/apt".
81
82Developers: Preparing a change
83------------------------------
84
85To prepare a change to a regular package (a source tree containing a
86debian/ subdir), make the edits in a checkout and record a changelog
87entry.  You can either edit debian/changelog using emacs changelog
88mode (C-c C-v to add a new version entry, C-c C-a to add a change
89entry, C-c C-f to finalize the entry) or you can run "dadch".
90
91When creating a new version entry, bump the upstream version number
92(to 10.0.0 if it was not already that high) if you are changing the
93main package source.  Otherwise, just bump the Debian version
94component (change 0debathena1 to 0debathena2, for instance).
95
96Developers: Building a package for test purposes on one platform
97----------------------------------------------------------------
98
99After you have prepared a change, you will want to test that it builds
100and perhaps that it works before committing it.  First, if it is an
101Athena source directory using autoconf, run "daconfiscate" to set up
102the autoconf boilerplate which we don't check in.  Second, run
103"daorig" to copy or create an orig tarball in the parent directory if
104necessary.  Third, run "debuild".  The resulting package will be
105placed in the parent directory.
106
107In order to test if the package works, you can install it with "dpkg
108-i filename.deb".
109
110Developers: Building a package for test purposes on all platforms
111-----------------------------------------------------------------
112
113If the package you are working on interacts with the native OS in ways
114that might vary from platform to platform, you may want to do a test
115build for all platforms.  You will need to do this on
116linux-build-10.mit.edu or another machine which has been set up with
117build schroots.
118
119As above, run daconfiscate (if necessary) and then daorig.  Then run
120"debuild -S" to create a source package.  Now cd into the parent
121directory and identify the .dsc file created by debuild -S; it will
122have a name like debathena-just_9.4.0-0debathena2.dsc.  Run "da
123sbuildhack filename.dsc" to perform the package builds.  Each build
124will take place inside an ephemeral chroot based on a snapshot of a
125template for a particular Debian or Ubuntu version.  If a build fails
126and it's not obvious from the build log why, you may need to create
127your own ephemeral chroot session with a command like "schroot -c
128gutsy-amd64-sbuild /bin/sh" and then run debuild from within the
129package sources.
130
131If the build is successful, it will create a set of packages with
132names like debathena-just_9.4.0-0debathena2~ubuntu6.06_amd64.deb.
133
134Developers: Building an equivs package
135--------------------------------------
136
137Most of the packages under debathena/meta are faked up using equivs.
138To build one, just run:
139
140  equivs-build --full filename.equivs
141
142These equivs files make reference to ../common, so you must have a
143checkout of debathena/meta/common alongside the particular
144meta-package you are building.
145
146Developers: The meaning of metapackages
147---------------------------------------
148
149If you are adding a new package to the repository, you will probably
150at some point want to add it to one of the metapackages so that it
151doesn't have to be installed by hand.  Here are some descriptions
152which may help identify which metapackage is best:
153
154* locker: Provides access to Athena locker software--AFS and
155  automounter configuration, locker-related utilities, etc.
156
157* clients: Provides clients (either locally-written, like athinfo and
158  Discuss, or configurations) for Athena services, as well as
159  Athena-specific utility programs like "jot".  Configurations for
160  graphical client software are generally in the workstation package
161  instead, in order to make this package less intrusive.
162
163* standard: Implies locker and clients.  Also provides Athena shell
164  customizations and dotfiles.
165
166* login: Implies standard.  Configurations to merge the MIT user
167  namespace into the local machine namespace for the purpose of user
168  lookups and authentication.
169
170* workstation: Implies login.  Configurations for the graphical login
171  system and graphical client software intended to provide a standard
172  X login experience using Athena home directories.  Still in
173  development.
174
175* extra-software: Implies a set of Debian packages common to cluster
176  machines and typical workstations.  The resulting software set is
177  rather large, and thus may not be desirable to all workstation
178  configurations.  Only stock Debian packages belong in this
179  metapackage; do not add other Debathena packages to it.
180
181* cluster: Implies workstation and extra-software.  Also contains
182  configurations for self-maintenance of machines (unattended updates,
183  cleanups between logins, etc.).
184
185* debian-dev: Intended for developers of the system itself; provides a
186  set of Debian packages used by Debathena for development.
187
188For the most part a package should be listed in the "Depends:" line of
189a metapackage, but in some cases it is appropriate to hedge by using
190"Recommends:", which will cause aptitude to succeed even if the
191package is unavailable.  For example, a package which doesn't exist in
192all Debian/Ubuntu suites or isn't free can be listed under
193"Recommends:" so that our metapackages still work in all environments.
194
195Developers: Index of cluster packages
196-------------------------------------
197
198Some private workstation admins may want specific pieces of cluster
199infrastructure.  With Athena 9.4, they would typically install the
200entire Athena software suite and then turn off features they did not
201want via /etc/athena/rc.conf.  That is not generally possible in
202Athena 10; a debathena-cluster machine acts like a cluster machine in
203all respects.  Instead, the machine owner should install
204debathena-workstation and then install specific packages they might be
205interested in.  At this time, candidate packages include:
206
207  * debathena-dns-config: Installs a caching DNS resolver and
208    configures the machine to use it.
209
210  * debathena-tmp-cleaner: Cleans files in /tmp and /var/tmp which
211    have not been recently accessed.
212
213  * debathena-reactivate: Causes each graphical login to be performed
214    in a separate ephemeral snapshot of the chroot.  This package
215    requires the root filesystem to be an LVM logical volume inside a
216    volume group with 21GB of free space for the snapshots; if that is
217    not the case, it will break graphical logins.  Install with care.
218
219  * debathena-auto-update: Periodically updates the machine's software
220    and reboots (when no one is logged in) if necessary.
221
222Cluster packages which are generally not of interest to private
223machine owners include:
224
225  * debathena-cluster-login-config: Configures a variety of system
226    services to implement the cluster login policy (no tty logins, no
227    user switching, screensaver logout button after 20 minutes, etc.).
228
229  * debathena-clusterinfo: Looks up the machine's cluster information
230    in Hesiod and caches it on local disk.
231
232  * debathena-larvnet: Reports to the central larvnet server whether
233    anyone has a graphical login to the machine.
234
235  * debathena-syslog-config: Configures sysklogd so that some syslog
236    messages are forwarded to a central logging host.
237
238Release engineer: Bootstrapping the project infrastructure
239----------------------------------------------------------
240
241  1. Create the package repository (detailed instructions on this
242     pending).  Set the DEBATHENA_APT environment variable to point to
243     the package repository.  Put a copy of the debathena "scripts"
244     subdir in your path.
245
246  2. Create the build area.
247
248  3. Build each equivs package under meta/ using "equivs-build --full
249     *.equivs" and upload each with "daequivsupload *.changes".  This
250     has the side-effect of creating the basic structure of the
251     package repository.
252
253  4. Set up the build server.  The basic structure of the apt
254     repository must work for make-chroot to succeed, so this must
255     happen after step 3.
256
257  5. For each normal Debian package in dependency order, cd into its
258     directory in the build area and run "da sbuildhack *.dsc" and
259     "daupload-release *_source.changes".  If the package contains
260     only an "Architecture: all" binary package, pass the -A option to
261     both commands.
262
263     The all-packages script can generate an approximation of the
264     package list in dependency order, but it doesn't work right yet,
265     and ideally it would be possible to do several builds in parallel
266     using a Makefile like the one in scripts/build-server/build-all.
267     Improvements to this machinery are pending.
268
269  6. For each package under third, run "da ./debathenify-PKG source
270     binary upload".  Any created directories under third/openafs/meta
271     should be chmodded 777 to work around a perl/AFS permissions
272     issue with File::Temp; if this is not done, OpenAFS metapackage
273     builds will fail for other users.
274
275Release engineer: Updating the apt repository
276---------------------------------------------
277
278  1. ssh to the build server as the builder account and change to the
279     canonical build directory.
280
281  2. Run "gen-packages" to update the package list.  (Or "gen-packages
282     -c" if you know the AFS checkout of the source tree is up to date;
283     it should update every half hour.)
284
285  3. Run "ood-packages" to produce a list of out-of-date packages.
286
287  4. For each out of date package, run "dasource PKG".  Then change to
288     the package directory and run "da sbuildhack *.dsc" and
289     "daupload-release *_source.changes".  If the package contains
290     only an "Architecture: all" binary package, pass the -A option to
291     both commands.
292
293  5. svn update the meta directory.  If there are new subdirectories,
294     chmod them 777 to work around a perl/AFS permissions issue with
295     File::Temp.  For each updated subdir, change to it, run
296     "equivs-build --full *.equivs", and then "daequivsupload
297     FILENAME.changes" on the produced changes file.
298
299  6. svn update the third directory.  You can let autodebathenify
300     handle the updated scripts, or you can touch
301     ~/autodebathenify.suppress, make sure it's not running, and run
302     "da ./debathenify-PKG source binary upload" in each updated
303     directory.
304
305Sometimes you may have to mix up the order of the above steps in order
306to handle build dependencies.
307
308Release engineer: Setting up a build server
309-------------------------------------------
310
311  1. The build server must be installed with free space in an LVM
312     volume group.  The build chroots consume 2GB each.  There is a
313     known memory corruption issue with LVM snapshots in the kernel
314     used in Ubuntu Gutsy (which is based on 2.6.22), so use a newer
315     kernel such as the one in Ubuntu Hardy (based on 2.6.24) instead.
316
317  2. Install debathena-standard as per the the instructions in
318     http://debathena.mit.edu/install.
319
320  3. apt-key add /afs/dev.mit.edu/system/athena10/apt/athena10-archive.asc
321
322  4. Install the packages listed in scripts/build-server/packages
323     (using "aptitude install").
324
325     Note that currently the build system uses a new feature of sbuild
326     which is not in the package version in the main Ubuntu Hardy
327     release.  The proper package version and its prerequisite schroot
328     packages have been backported locally for Hardy, and should be
329     installed from /afs/dev.mit.edu/project/release/10/build/backports.
330     This will also require installing the devscripts package from the
331     Hardy backports repository.
332
333  5. Install debathena-login, debathena-ssh-server, and
334     debathena-build-depends (using "aptitude install").
335
336     (Depending on how recently debathena-build-depends was rebuilt,
337     additional packages might need to be installed to satisfy the
338     build-depends of newer packages.  This can be taken care of later
339     when an error occurs building a source package.)
340
341  6. Edit /etc/security/access.conf and add a first line:
342     -:ALL EXCEPT root <developer usernames>:ALL
343
344  7. Edit /etc/pam.d/schroot, comment out "@include common-session",
345     and add:
346
347       # Basic pam_unix session module in place of common-session.
348       session required         pam_unix.so
349
350  8. Edit /etc/group and add the developers to the sbuild group.
351
352  9. Create /etc/passwd entries for each developer with "hesinfo
353     username passwd >> /etc/passwd" and then run pwconv.
354
355     (This is not necessary for the login system on the main root
356     environment, but is for the chroot environments.)
357
358  10. Append to /etc/approx/approx.conf the contents of
359       scripts/build-server/approx.conf.tail.
360      Change the last line from http://debathena.mit.edu/apt to
361       file:///afs/dev.mit.edu/system/athena10/apt
362      Add "$interval 0" above the repository lines (only necessary if
363       the version of approx as reported by "dpkg -l approx" is less
364       than 3.0)
365      Run: /etc/init.d/approx restart
366
367  11. Apply scripts/build-server/mount-defaults.patch.
368
369  12. Edit /etc/lvm/lvm.conf and change "archive = 1" to "archive = 0"
370      in order to avoid generating a bazillion backup files under
371      /etc/lvm/archive.
372
373  13. For each supported DIST (see scripts/debian-versions.sh) run:
374
375        VG=/dev/blah scripts/build-server/make-chroot DIST i386
376        VG=/dev/blah scripts/build-server/make-chroot DIST amd64
377
378      substituting the name of the volume group for blah.  Omit the
379      amd64 line if DIST is sarge.
380
381      Example: VG=/dev/dink scripts/build-server/make-chroot gutsy i386
382
383  14. Create a local account for builder with:
384
385        adduser --uid 1047 --disabled-password builder
386
387      Make the home directory mode 700.  Install a
388      daemon/linux-build-10.mit.edu keytab in the home directory as
389      "keytab".  Install a copy of the secret repository-signing key
390      (athena10@mit.edu) in the home directory's keyring with
391      something like:
392
393        kinit builder
394        gpg --export-secret-keys athena10@mit.edu | \
395          ssh -l builder machinename gpg --import
396
397      Create a file named .sbuildrc in builder's homedir containing:
398
399        $nolog = 1;
400        $mailto = 'source-wash@mit.edu';
401        $log_dir = '/tmp/sbuild-logs';
402        $maintainer_name = 'Debian-Athena Project <debathena@mit.edu>';
403        $force_orig_source = 1;
404        $sbuild_mode = "user";
405        1;
406
407      Create a file named .ssh/config in builder's homedir containing:
408
409        Host svn.mit.edu
410          User debuildsvn
411
412      Add builder to the sbuild group in /etc/group.
413
414      Copy scripts/build-server/autodebathenify to builder's homedir.
415      Create a file named autodebathenify.config in builder's homedir
416      containing:
417
418        error_addr=source-wash@mit.edu
419        scripts_dir=/afs/dev.mit.edu/source/src-svn/debathena/scripts
420        build_dir=/afs/dev.mit.edu/project/release/10/build/third
421        packages="cyrus-sasl2-mit evolution-data-server lprng openafs tcsh"
422        export DEBATHENA_APT=/afs/dev.mit.edu/system/athena10/apt
423
424      Copy scripts/build-server/autodebathenify.cron to builder's
425      homedir and install it with "crontab autodebathenify.cron".
426
427      Create a file named .devscripts in builder's homedir containing:
428
429        DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn"
430
431      In builder's homedir, append to .profile:
432
433        PATH=${PATH}:/afs/dev.mit.edu/source/src-svn/debathena/scripts
434        export DEBATHENA_APT=/afs/dev.mit.edu/system/athena10/apt
435
436      and to .bashrc:
437
438        bld=/afs/dev.mit.edu/project/release/10/build
439
440Release engineer: Removing a build chroot on the build server
441-------------------------------------------------------------
442
443  1. Run VG=/dev/blah scripts/clean-schroots as root to make sure that
444     the build chroot is not mounted, substituting the name of the
445     volume group for blah.
446
447  2. Edit /etc/schroot/schroot.conf and delete the section
448     corresponding to the chroot.
449
450  3. Run lvchange -an blah/chrootname
451     substituting the name of the volume group for blah and the chroot
452     name for chroot.  Example: lvchange -an dink/gutsy-i386-sbuild
453
454  4. Run lvremove blah/chrootname
455
456Release engineer: Removing a dist from the apt repository
457---------------------------------------------------------
458
459  1. Inside the apt repository, edit conf/distributions and remove the
460     distribution section.
461
462  2. Run reprepro -Vb $DEBATHENA_APT --delete clearvanished
463
464Release engineer: Setting up a canonical build area
465---------------------------------------------------
466
467  1. Create an empty directory and cd into it.  The canonical build
468     area lives in /afs/dev.mit.edu/project/release/10/build.
469
470  2. Run gen-packages to create the table of normal Debian packages.
471
472  3. Run dasource to create subdirs and source packages for each
473     normal Debian package.
474
475  4. Create checkouts of the meta and third directories:
476
477     svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/meta
478     svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/third
479     chmod 777 meta/*
480
481     A couple of subdirectories of debathena/meta are normal Debian
482     packages, so this will create redundant copies of those.  Ignore
483     them; they won't be used.
484
485Release engineer: Adding a new suite
486------------------------------------
487
488This process is rarely performed and the infrastructure for it is
489imperfect.  Substitute the name of the new suite for "newdist" in all
490steps below.
491
492  1. Make sure the apt repository is up to date with respect to the
493     source tree for the existing dists.
494
495  2. Add the new dist to scripts/debian-versions.sh.  (It is not
496     necessary to add the new dist to codes at this point, but it must
497     be present in the gettag conditional.)
498
499  3. Create the new distribution in the apt repository's configuration
500     file.  Create the skeleton of the dist by installing at least one
501     equivs package from meta/ with "reprepro -Vb $DEBATHENA_APT
502     --ignore=wrongdistribution include newdistname file.changes".
503
504  4. On the build server, create a chroot for the new distribution as
505     documented above.  This may require downloading and installing a
506     more recent version of the debootstrap package from the
507     -backports dist corresponding to the build server's OS.
508
509  5. Set the DEBATHENA_BUILD_AREA environment variable to point to the
510     build area.
511
512  6. Fire up screen.
513
514  7. mkdir $DEBATHENA_BUILD_AREA/stamps.newdist.
515
516  8. cd into a checkout of debathena/scripts/build-server/build-all.
517
518  9. Edit Makefile (and check in the edit) so that suite is the new
519     distribution and psuite is the previously most recent Debian or
520     Ubuntu distribution.
521
522  10. Run "make deps.mk".
523
524  11. Run "make -k all STAMPS=$DEBATHENA_BUILD_AREA/stamps.newdist".
525      You can watch the builds happen in the other windows of the
526      screen session.  It's possible to do several builds at once with
527      make -j N.
528
529  12. debathenify packages will fail out; they must be built by hand.
530      When the build fails on one, cd into third/packagename in the
531      build area and run "./debathenify newdist-amd64 -A source binary
532      upload" and "./debathenify newdist-i386 binary upload".  Then
533      touch $DEBATHENA_BUILD_AREA/stamps.newdist/packagename.done" and
534      restart the build.
535
536      The newly created third/openafs/meta directories should be
537      chmodded 777 to work around a perl/AFS permissions issue.
538
539Release engineer: Maintaining autodebathenify
540---------------------------------------------
541
542autodebathenify is a cron job which runs on the builder account and
543keeps the OpenAFS modules and modified packages under debathena/third
544up to date when there are upstream changes.  It relies on the
545following:
546
547  * builder@ATHENA.MIT.EDU has access to a local account named
548    debuildsvn on svn.mit.edu with read access to the athena
549    repository.
550
551  * builder@ATHENA.MIT.EDU has write access to the canonical build
552    area and apt repository in the dev cell.
553
554  * The builder account on the build machine has a copy of the signing
555    private key for the apt repository.
556
557The cron job will silently exit if it detects that it is already
558running.  If it fails for any other reason, it will send mail to
559source-wash@mit.edu and touch a file named autodebathenify.suppress
560which will prevent it from running again until manual intervention.
561It is important to get autodebathenify running smoothly or the apt
562repository will become out of date with respect to third-party
563packages, which in turn will compromise the user experience.
564
565Even if you don't notice failure mail, it's good to check on the
566status of autodebathenify from time to time.  To do this, get tickets
567as builder and ssh to linux-build-10 as builder.  Run "ls -l logs" to
568see what's been going on.  A logfile with size around 48K indicates
569that autodebathenify ran normally but didn't find any new work to do.
570A logfile with size around 60 means that autodebathenify.suppress
571exists and the cron job has stopped running.  A longer logfile
572indicates that autodebathenify attempted to build and upload a
573package.
574
575Possible failure cases include:
576
577  * One of the upstream apt repositories timed out during an "apt-get
578    update" of an sbuild chroot.  If this happens, you can just remove
579    the autodebathenify.suppress file to get the cron job to start
580    running again.
581
582  * autodebathenify tried to build an upstream version which already
583    exists.  If this happens, the build will fail at upload time.  If
584    this happens, there is a bug in the part of the debathenify-*
585    script which checks whether our apt repository already has the
586    expected version of the built package.
587
588  * Debian and Ubuntu have .orig files with the same name but
589    different contexts for a particular third-party package.  This has
590    only happened once, with bash (which is currently not configured
591    into autodebathenify for that reason).  We do not have a general
592    mechanism for resolving such issues at this time.
593
594  * The new upstream version of the package is sufficiently different
595    to cause our modifications to fail to apply, and our debathenify-*
596    script needs to be adapted appropriately.
597
598The cron job is run out of builder's home directory, but its canonical
599source location is in debathena/scripts/build-server in this
600repository, should it need to be modified.
601
602After several months of operation, LVM snapshots on the build server
603will start to become slower and slower, as /etc/lvm/cache/.cache grows
604in size.  Turning off the cache entirely seems to cause schroot to
605occasionally fail; the current solution is to rm /etc/lvm/cache/.cache
606every so often.
607
608Release engineer: apt repository HTTP server setup
609--------------------------------------------------
610
611The apt repository server (athena10.mit.edu) is an ops virtual image
612with httpd and AFS installed.  At the moment, the only customization
613is two changes to /etc/httpd/conf/httpd.conf:
614
615  * The DocumentRoot is set to "/afs/dev.mit.edu/system/athena10".
616  * The Directory entry for /var/html/www is also changed to
617    "/afs/dev.mit.edu/system/athena10".
Note: See TracBrowser for help on using the repository browser.