This hierarchy contains Debian/Ubuntu-specific materials, also known as "Debathena". The contents are: * debathena - Debathena-specific software packages such as PAM and NSS modules. * config - Packages for configuring native system software in a manner appropriate for Athena. * meta - Packages which contain nothing but dependencies on other packages and serve as an installation convenience. * scripts - Build scripts and supporting materials. Debathena is a SIPB project, and its infrastructure and procedures will need to be adapted for Athena 10. For the moment this file will document the Debathena procedures as they are, not as they will be. The current procedures do not even use this svn repository yet. Debian software used by Debathena: * schroot - Used to manage build chroot environments for each Debian/Ubuntu version. We use the lvm-snapshot schroot type, which allows rapid construction of ephemeral copies of template "source" chroots, so that every binary package build is done in a clean environment. * debuild - Used to create Debian source packages from package source directories. * sbuild - Used to build binary packages from source packages inside schroot environments. * equivs - Used to create packages which only contain dependency information. Somewhat of a dirty hack, since it doesn't keep proper changelogs, but it reduces overhead. * CDBS (Common Debian Build System) - Referenced by debian/rules files in packages. Contains standard build rules to cut down on per-package boilerplate. * reprepro - Used to upload packages into the apt repositories. * approx - Used to create a local cache of Debian packages on the build server. This cache is referenced by the build chroots for improved performance. The remainder of this file documents procedures useful to Athena 10 developers and the release engineer. Developers: Preferences setup ----------------------------- You will probably want a $HOME/.devscripts file containing the following: DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn" This will save you from having to specify those options every time you run debuild. Athena 10 scripts do not assume the above preferences, but the instructions in this file do. The options mean: * Look for original source as a tarfile or create one. * Do not sign the source package. * Do not sign the changes file. * Ignore common version control metadata files when creating diffs. * Ignore .svn paths when creating tarballs. You will also want a $HOME/.sbuildrc file containing the following: $nolog = 1; $mailto = 'yourusername'; $log_dir = '/tmp/sbuild-logs'; $maintainer_name = 'Debathena Project '; $force_orig_source = 1; $sbuild_mode = "user"; 1; You should also set the environment variable DEBATHENA_APT to "/afs/dev.mit.edu/system/athena10/apt". Developers: Preparing a change ------------------------------ To prepare a change to a regular package (a source tree containing a debian/ subdir), make the edits in a checkout and record a changelog entry. You can either edit debian/changelog using emacs changelog mode (C-c C-v to add a new version entry, C-c C-a to add a change entry, C-c C-f to finalize the entry) or you can run "dadch" from the debathena locker / SVN scripts directory. When creating a new version entry, bump the upstream version number (to 10.0.0 if it was not already that high) if you are changing the main package source. Otherwise, just bump the Debian version component (change 0debathena1 to 0debathena2, for instance). Developers: Building a package for test purposes on one platform ---------------------------------------------------------------- After you have prepared a change, you will want to test that it builds and perhaps that it works before committing it. First, if it is an Athena source directory using autoconf, run "daconfiscate" to set up the autoconf boilerplate which we don't check in. Second, run "daorig" to copy or create an orig tarball in the parent directory if necessary. Third, run "debuild". The resulting package will be placed in the parent directory. In order to test if the package works, you can install it with "dpkg -i filename.deb". Developers: Building a package for test purposes on all platforms ----------------------------------------------------------------- If the package you are working on interacts with the native OS in ways that might vary from platform to platform, you may want to do a test build for all platforms. You will need to do this on linux-build-10.mit.edu or another machine which has been set up with build schroots. As above, run daconfiscate (if necessary) and then daorig. Then run "debuild -S" to create a source package. Now cd into the parent directory and identify the .dsc file created by debuild -S; it will have a name like debathena-just_9.4.0-0debathena2.dsc. Run "da sbuildhack filename.dsc" to perform the package builds. Each build will take place inside an ephemeral chroot based on a snapshot of a template for a particular Debian or Ubuntu version. If a build fails and it's not obvious from the build log why, you may need to create your own ephemeral chroot session with a command like "schroot -c gutsy-amd64-sbuild /bin/sh" and then run debuild from within the package sources. If the build is successful, it will create a set of packages with names like debathena-just_9.4.0-0debathena2~ubuntu6.06_amd64.deb. Developers: Building an equivs package -------------------------------------- Most of the packages under debathena/meta are faked up using equivs. To build one, just run: equivs-build --full filename.equivs These equivs files make reference to ../common, so you must have a checkout of debathena/meta/common alongside the particular meta-package you are building. Developers: The meaning of metapackages --------------------------------------- If you are adding a new package to the repository, you will probably at some point want to add it to one of the metapackages so that it doesn't have to be installed by hand. Here are some descriptions which may help identify which metapackage is best: * locker: Provides access to Athena locker software--AFS and automounter configuration, locker-related utilities, etc. * clients: Provides clients (either locally-written, like athinfo and Discuss, or configurations) for Athena services, as well as Athena-specific utility programs like "jot". Configurations for graphical client software are generally in the workstation package instead, in order to make this package less intrusive. * standard: Implies locker and clients. Also provides Athena shell customizations and dotfiles. * login: Implies standard. Configurations to merge the MIT user namespace into the local machine namespace for the purpose of user lookups and authentication. * login-graphical: Implies login. Configurations for the graphical login system and graphical client software intended to provide a standard X login experience using Athena home directories. * extra-software: Implies a set of Debian packages common to cluster machines and typical workstations. The resulting software set is rather large, and thus may not be desirable to all workstation configurations. Only stock Debian packages belong in this metapackage; do not add other Debathena packages to it. * workstation: Implies login-graphical and extra-software. The use case is the same as that of the old "private workstations", so it includes more centrally-managed configuration than standard or login does (unattended updates, etc.). New as of May 2009. * thirdparty: Software supported by 3partysw that upstream Ubuntu already happens to be keeping up to date, and is therefore easier to install via packages than through lockers. This package is HUGE (1 GB of network traffic, 3 GB installed), and is also not strongly tested on Debian or on older releases of Ubuntu. * cluster: Implies workstation and thirdparty. Configures public cluster machines (login-time LVM snapshots, cleanups between logins, public root password, etc.). The snapshot code requires a particular LVM layout that the PXE (netboot) cluster installer sets up. * debian-dev: Intended for developers of the system itself; provides a set of Debian packages used by Debathena for development. For the most part a package should be listed in the "Depends:" line of a metapackage, but in some cases it is appropriate to hedge by using "Recommends:", which will cause aptitude to succeed even if the package is unavailable. For example, a package which doesn't exist in all Debian/Ubuntu suites or isn't free can be listed under "Recommends:" so that our metapackages still work in all environments. Developers: Index of cluster packages ------------------------------------- Some private workstation admins may want specific pieces of cluster infrastructure. With Athena 9.4, they would typically install the entire Athena software suite and then turn off features they did not want via /etc/athena/rc.conf. That is not generally possible in Athena 10; a debathena-cluster machine acts like a cluster machine in all respects. Instead, the machine owner should install debathena-workstation and then install specific packages they might be interested in. At this time, candidate packages include: * debathena-dns-config: Installs a caching DNS resolver and configures the machine to use it. * debathena-tmp-cleaner: Cleans files in /tmp and /var/tmp which have not been recently accessed. * debathena-reactivate: Causes each graphical login to be performed in a separate ephemeral snapshot of the chroot. This package requires the root filesystem to be an LVM logical volume inside a volume group with 21GB of free space for the snapshots; if that is not the case, it will break graphical logins. Install with care. * debathena-auto-update: Periodically updates the machine's software and reboots (when no one is logged in) if necessary. Cluster packages which are generally not of interest to private machine owners include: * debathena-cluster-login-config: Configures a variety of system services to implement the cluster login policy (no tty logins, no user switching, screensaver logout button after 20 minutes, etc.). * debathena-clusterinfo: Looks up the machine's cluster information in Hesiod and caches it on local disk. * debathena-larvnet: Reports to the central larvnet server whether anyone has a graphical login to the machine. * debathena-syslog-config: Configures sysklogd so that some syslog messages are forwarded to a central logging host. Release engineer: Bootstrapping the project infrastructure ---------------------------------------------------------- 1. Create the package repository (detailed instructions on this pending). Set the DEBATHENA_APT environment variable to point to the package repository. Put a copy of the debathena "scripts" subdir in your path. 2. Create the build area. 3. Run 'reprepro -Vb $DEBATHENA_APT export' to set up the repository. Build each equivs package under meta/ using 'equivs-build --full *.equivs' and upload each with 'daupload-equivs-proposed *.changes', followed by 'damove -proposed "" PACKAGE' (replacing PACKAGE with the name of each equivs package you are moving). 4. Set up the build server. The basic structure of the apt repository must work for make-chroot to succeed, so this must happen after step 3. 5. For each normal Debian package in dependency order, cd into its directory in the build area and run "da sbuildhack *.dsc" and "daupload-release *_source.changes". If the package contains only an "Architecture: all" binary package, pass the -A option to both commands. The all-packages script can generate an approximation of the package list in dependency order, but it doesn't work right yet, and ideally it would be possible to do several builds in parallel using a Makefile like the one in scripts/build-server/build-all. Improvements to this machinery are pending. 6. For each package under third, run "da ./debathenify-PKG source binary upload". Any created directories under third/openafs/meta should be chmodded 777 to work around a perl/AFS permissions issue with File::Temp; if this is not done, OpenAFS metapackage builds will fail for other users. Release engineer: Updating the apt repository --------------------------------------------- 1. ssh to the build server as the builder account and change to the canonical build directory. 2. Run "gen-packages" to update the package list. (Or "gen-packages -c" if you know the AFS checkout of the source tree is up to date; it should update every half hour.) 3. Run "ood-packages" to produce a list of out-of-date packages. 4. For each out of date package, run "dasource PKG". Then change to the package directory and run "da sbuildhack *.dsc" and "daupload-release *_source.changes". If the package contains only an "Architecture: all" binary package, pass the -A option to both commands. 5. svn update the meta directory. If there are new subdirectories, chmod them 777 to work around a perl/AFS permissions issue with File::Temp. For each updated subdir, change to it, run "equivs-build --full *.equivs", and then "daequivsupload FILENAME.changes" on the produced changes file. 6. svn update the third directory. You can let autodebathenify handle the updated scripts, or you can touch ~/autodebathenify.suppress, make sure it's not running, and run "da ./debathenify-PKG source binary upload" in each updated directory. Sometimes you may have to mix up the order of the above steps in order to handle build dependencies. Release engineer: Setting up a build server ------------------------------------------- 1. Install either Debian testing (or stable, if it's new enough, but see the note about sbuild below) or the latest Ubuntu release. The build server must be installed with free space in an LVM volume group. The build chroots consume 2GB each. 2. Install debathena-standard as per the the instructions in http://debathena.mit.edu/install. 3. apt-key add /afs/dev.mit.edu/system/athena10/apt/athena10-archive.asc 4. Install the packages listed in scripts/build-server/packages (using "aptitude install"). Note that currently the build system uses the latest version of sbuild, so if you have not installed Debian testing, you'll need to set up apt pinning to grab sbuild from there. See the apt_preferences man page. 5. Install debathena-standard, debathena-ssh-server, and debathena-build-depends (using "aptitude install"). (Depending on how recently debathena-build-depends was rebuilt, additional packages might need to be installed to satisfy the build-depends of newer packages. This can be taken care of later when an error occurs building a source package.) 6. Append to /etc/approx/approx.conf the contents of scripts/build-server/approx.conf.tail. Run: /etc/init.d/approx restart 7. Apply scripts/build-server/mount-defaults.patch. 8. Edit /etc/lvm/lvm.conf and change "archive = 1" to "archive = 0" in order to avoid generating a bazillion backup files under /etc/lvm/archive. 9. For each supported DIST (see scripts/debian-versions.sh) run: VG=/dev/blah scripts/build-server/make-chroot DIST i386 VG=/dev/blah scripts/build-server/make-chroot DIST amd64 substituting the name of the volume group for blah. Example: VG=/dev/dink scripts/build-server/make-chroot intrepid i386 10. Create a local account for builder with: adduser --uid 1047 --disabled-password builder Make the home directory mode 700. Install a daemon/linux-build-10.mit.edu keytab in the home directory as "keytab". Install a copy of the secret repository-signing key (debathena@mit.edu) in the home directory's keyring with something like: kinit builder gpg --export-secret-keys athena10@mit.edu | \ ssh -l builder machinename gpg --import Create a file named .sbuildrc in builder's homedir containing: $nolog = 1; $mailto = 'source-wash@mit.edu'; $log_dir = '/tmp/sbuild-logs'; $maintainer_name = 'Debathena Project '; $force_orig_source = 1; $sbuild_mode = "user"; 1; Create a file named .ssh/config in builder's homedir containing: Host svn.mit.edu User debuildsvn Add builder to the sbuild group in /etc/group. Copy scripts/build-server/autodebathenify to builder's homedir. Create a file named autodebathenify.config in builder's homedir containing: error_addr=source-wash@mit.edu scripts_dir=/afs/dev.mit.edu/source/src-svn/debathena/scripts build_dir=/afs/dev.mit.edu/project/release/10/build/third packages="cyrus-sasl2-mit evolution-data-server lprng openafs tcsh" export DEBATHENA_APT=/afs/dev.mit.edu/system/athena10/apt Copy scripts/build-server/autodebathenify.cron to builder's homedir and install it with "crontab autodebathenify.cron". Create a file named .devscripts in builder's homedir containing: DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn" In builder's homedir, append to .bashrc: add debathena export PATH=$PATH:~/bin export DEBATHENA_APT=/mit/debathena/apt Release engineer: Removing a build chroot on the build server ------------------------------------------------------------- 1. Run VG=/dev/blah scripts/clean-schroots as root to make sure that the build chroot is not mounted, substituting the name of the volume group for blah. 2. Edit /etc/schroot/schroot.conf and delete the section corresponding to the chroot. 3. Run lvchange -an blah/chrootname substituting the name of the volume group for blah and the chroot name for chroot. Example: lvchange -an dink/gutsy-i386-sbuild 4. Run lvremove blah/chrootname Release engineer: Removing a dist from the apt repository --------------------------------------------------------- 1. Inside the apt repository, edit conf/distributions and remove the distribution section. 2. Run reprepro -Vb $DEBATHENA_APT --delete clearvanished Release engineer: Setting up a canonical build area --------------------------------------------------- 1. Create an empty directory and cd into it. The canonical build area lives in /mit/debathena/packages/. 2. Run gen-packages to create the table of normal Debian packages. 3. Run dasource to create subdirs and source packages for each normal Debian package. 4. Create checkouts of the meta and third directories: svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/meta svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/third chmod 777 meta/* A couple of subdirectories of debathena/meta are normal Debian packages, so this will create redundant copies of those. Ignore them; they won't be used. Release engineer: Adding a new suite ------------------------------------ This process is rarely performed and the infrastructure for it is imperfect. Substitute the name of the new suite for "newdist" in all steps below. 1. Make sure the apt repository is up to date with respect to the source tree for the existing dists. 2. Add the new dist to scripts/debian-versions.sh. (It is not necessary to add the new dist to codes at this point, but it must be present in the gettag conditional.) 3. Create the new distribution in the apt repository's configuration file. Create the skeleton of the dist by running "reprepro -Vb $DEBATHENA_APT export". 4. On the build server, create a chroot for the new distribution as documented above. This may require downloading and installing a more recent version of the debootstrap package from the -backports dist corresponding to the build server's OS. 5. Set the DEBATHENA_BUILD_AREA environment variable to point to the build area. 6. Fire up screen. 7. mkdir $DEBATHENA_BUILD_AREA/stamps.newdist. 8. cd into a checkout of debathena/scripts/build-server/build-all. 9. Edit Makefile (and check in the edit) so that suite is the new distribution and psuite is the previously most recent Debian or Ubuntu distribution. 10. Run "make deps.mk". 11. Run "make -k all STAMPS=$DEBATHENA_BUILD_AREA/stamps.newdist". You can watch the builds happen in the other windows of the screen session. It's possible to do several builds at once with make -j N. 12. debathenify packages will fail out; they must be built by hand. When the build fails on one, cd into third/packagename in the build area and run "./debathenify newdist-amd64 -A source binary upload" and "./debathenify newdist-i386 binary upload". Then touch $DEBATHENA_BUILD_AREA/stamps.newdist/packagename.done" and restart the build. The newly created third/openafs/meta directories should be chmodded 777 to work around a perl/AFS permissions issue. Release engineer: Maintaining autodebathenify --------------------------------------------- autodebathenify is a cron job which runs on the builder account and keeps the OpenAFS modules and modified packages under debathena/third up to date when there are upstream changes. It relies on the following: * builder@ATHENA.MIT.EDU has access to a local account named debuildsvn on svn.mit.edu with read access to the athena repository. * builder@ATHENA.MIT.EDU has write access to the canonical build area and apt repository in the dev cell. * The builder account on the build machine has a copy of the signing private key for the apt repository. The cron job will silently exit if it detects that it is already running. If it fails for any other reason, it will send mail to source-wash@mit.edu and touch a file named autodebathenify.suppress which will prevent it from running again until manual intervention. It is important to get autodebathenify running smoothly or the apt repository will become out of date with respect to third-party packages, which in turn will compromise the user experience. Even if you don't notice failure mail, it's good to check on the status of autodebathenify from time to time. To do this, get tickets as builder and ssh to linux-build-10 as builder. Run "ls -l logs" to see what's been going on. A logfile with size around 48K indicates that autodebathenify ran normally but didn't find any new work to do. A logfile with size around 60 means that autodebathenify.suppress exists and the cron job has stopped running. A longer logfile indicates that autodebathenify attempted to build and upload a package. Possible failure cases include: * One of the upstream apt repositories timed out during an "apt-get update" of an sbuild chroot. If this happens, you can just remove the autodebathenify.suppress file to get the cron job to start running again. * autodebathenify tried to build an upstream version which already exists. If this happens, the build will fail at upload time. If this happens, there is a bug in the part of the debathenify-* script which checks whether our apt repository already has the expected version of the built package. * Debian and Ubuntu have .orig files with the same name but different contexts for a particular third-party package. This has only happened once, with bash (which is currently not configured into autodebathenify for that reason). We do not have a general mechanism for resolving such issues at this time. * The new upstream version of the package is sufficiently different to cause our modifications to fail to apply, and our debathenify-* script needs to be adapted appropriately. The cron job is run out of builder's home directory, but its canonical source location is in debathena/scripts/build-server in this repository, should it need to be modified. After several months of operation, LVM snapshots on the build server will start to become slower and slower, as /etc/lvm/cache/.cache grows in size. Turning off the cache entirely seems to cause schroot to occasionally fail; the current solution is to rm /etc/lvm/cache/.cache every so often. Release engineer: apt repository HTTP server setup -------------------------------------------------- The apt repository server (debathena.mit.edu) is hosted on scripts.mit.edu out of the debathena locker. To replicate it on a dedicated server, install Apache and AFS (e.g., debathena-standard and apache2), and configure the document root to be /mit/debathena/web_scripts There's also a crontab in /mit/debathena/cron_scripts that updates the package lists on the website. It expects to be able to edit the web_scripts directory; on scripts, it does so with daemon.scripts permissions. Trac (http://debathena.mit.edu/trac/) expects to use sql.mit.edu for its database.