This page describes the config-package-dev package, a system that allows easy and efficient creation of configuration packages. Configuration packages are packages whose purpose is to configure a Debian system, rather than add new software. config-package-dev consists of a set of modules for the Common Debian Build System. The config-package-dev system was created by Anders Kaseorg and Tim Abbott as part of the MIT Debian-Athena project.
You will need to download and install the config-package-dev package from the link above.
If you’re not familiar with the process of building Debian packages, you may want to look at our abbrieviated packaging tutorial and the Debian references it links to.
We recommend using config-package-dev with sbuild and schroot, or another framework for building Debian packages in clean environments.
Configuration packages have several advantages over other approaches (such as the preseed file approach and Cfengine) for distributing site-wide configuration such as the MIT-specific configurations distributed as part of Debathena.
These properties make configuration packages effective for distributing site configurations at organizations on the scale of a university. They make it possible to configure site defaults so that interacting with any site services "just works", without restricting the ability of individual users at the site to control their machine.
Discussions of configuration packages on the Debian Wiki may also be of interest, as might be the thread on debian-devel discussing the weaknesses of the system and the technical motivation for the system..
A related project relevant for supporting multiple distributions is the Debathena build system which allows a single package-version pair to appear in an apt repository multiple times (we append tags like ~ubuntu7.04 to the version number at build time). It also contains code to automate the process of building packages for many different architectures; however, it is not documented for release. If you would like us to release it, send us an email at debathena@mit.edu.
The config-package-dev system uses dpkg-divert to move aside configuration files. dpkg-divert is a standard Debian feature that configures dpkg, the Debian package manager, to move any file packaged to be installed at a given path to a different location. Using dpkg-divert for configuration files means that when Debian the upstream upgrades the relevant configuration file, they will not override the site changes. It also means that any Debian-wide updates will be preserved if the configuration package is ever uninstalled (but see Debian #476899). Using a symlink here rather than a direct replacement is necessary for technical reasons related to how Debian unpacks files. The diversion system in config-package-dev has been carefully tested to support uninstallation, upgrades, and cleanly recovering from when you hit ^C in the middle of installing something (or installation fails for some other reason).
Our configuration system has three primary components, each in its own makefile. Typically, users will only interact with replace_files.mk (see the examples below), which includes the other two, but because it uses the other two as subroutines we will explain their architecture first.
The first, divert.mk, is a system for using dpkg-divert to move aside configuration files (and other files; it’s quite useful for installing wrapper scripts as well) and replace them with a symlink to a new version shipped by the configuration package.
If file.debathena appears in this list, then on postinst, file will be diverted to file.debathena-orig, and a symlink file -> file.debathena will be installed. After using our system to divert /etc/ldap/ldap.conf, the state of the system will be:
$ ls -l /etc/ldap/ldap.conf* lrwxrwxrwx 1 root root 19 2007-08-13 17:07 /etc/ldap/ldap.conf -> ldap.conf.debathena -rw-r--r-- 1 root root 347 2007-08-15 17:58 /etc/ldap/ldap.conf.debathena -rw-r--r-- 1 root root 333 2007-08-13 17:27 /etc/ldap/ldap.conf.debathena-orig $ /usr/sbin/dpkg-divert --list | grep ldap diversion of /etc/ldap/ldap.conf to /etc/ldap/ldap.conf.debathena-orig by debathena-ldap-config
Note that DEB_DIVERT_EXTENSION can appear anywhere in the filename. Thus, /usr/share/man/man1/manpage.1.gz would be diverted to /usr/share/man/man1/manpage.debathena-orig.1.gz and a symlink created pointing /usr/share/man/man1/manpage.1.gz to /usr/share/man/man1/manpage.debathena.1.gz, so that man manpage.debathena-orig, man manpage.debathena, and man manpage all behave as expected.
The system just described does not work well for replacing files in directories such as /etc/cron.d where all files in the directory are used -- in the case of diverting /etc/cron.daily/lprng, the original cron job would get run and the new cron job would be run twice. You can banish such files into /usr/share/package by adding the file to DEB_REMOVE_FILES_package. For example, in Debian etch, the lprng package contained a buggy cron job. After removing it with DEB_REMOVE_FILES_lprng-no-cron and installing the lprng-no-cron, one has:
$ /usr/sbin/dpkg-divert --list | grep cron diversion of /etc/cron.daily/lprng to /usr/share/lprng-no-cron/etc++cron.daily++lprng by lprng-no-cron
If you want to instead replace the cron job with a different one, you should just install the new cron just using dh_install (or in the case of cron, it's probably better to use dh_installcron by naming your new cron file debian/foo.cron.daily). Do not give the replacement cron job the same name as the original one, instead give it a different name, like /etc/cron.daily/lprng.divert. If you do use the same name, it will not be possible to install your package in the same apt invocation that installs the package containing the original cron job.
The encoding of the filename using + characters shown in the last example is also used to ensure two packages that divert the same file conflict; if both packages divert /etc/cron.daily/lprng, they will both Provides: and Conflicts: configures-etc++cron.daily++lprng. The encoding used in both cases is a bijective map from legal ASCII filenames to package names. In order to enabled this you need to add ${diverted-files} to your Provides and Conflicts lines of your debian/control file; see the examples below for details.
Also note that when using DEB_DIVERT_FILES you will need to create and install the replacement file file.debathena (probably using dh_install) yourself. See the examples package.install files below for how to do this.
The second system, in check-files.mk, is used to get an unmodified version of a configuration file (verified by the md5sums that Debian ships with each package). If the relevant md5sum does not match or does not exist, it causes a build failure. It supports checking the md5sums for both configuration files and non-configuration files. While all packages ship md5sums for configuration files, some packages do not contain md5sums for non-configuration files (around 3% of Debian packages have this property). If used to check the md5sum for such a non-configuration file, check-files.mk prints a warning but does not halt the build. It correctly follows diversions so that it is compatible with divert.mk.
The third system, in transform-files.mk, provides a convenient wrapper to the first two systems, while sacrificing only a small bit of functionality. Given a list of files of the form path/file.debathena, it will obtain an unmodified copy of path/file using check-files.mk, and then transform this unmodified configuration file by piping it through a (user-supplied) script called debian/transform_file.debathena. It will then install the result as path/file.debathena. It will use divert.mk to divert path/file to path/file.debathena-orig on postinst. Setting DEB_CHECK_FILES_SOURCE_path/file.debathena to some other filename will cause check-files.mk to obtain its unmodified copy from that path; see the debathena-nsswitch-config example below to see how this is used.
The config-package-dev system is controlled through the following makefile variables set in the debian/rules file:
Below are a few example packages from the MIT Debathena project using this configuration package system. These examples are modified from the deployed Debathena system only in that the CDBS interfaces have been improved and the variables have been renamed from DEBATHENA_ to DEB_; the actual binary packages produced are the same. Complete Debian packages for these examples are available here.
This example is the debathena-ldap-config package, as shipped on Debathena. The parts of the control and rules files that are related to the configuration packages system (as opposed to CDBS or Debian in general) are highlighted. The build log from running debuild to build this example is here.
debian/control.in, which generates debian/control:
Source: debathena-ldap-config
Section: debathena-config/net
Priority: extra
Maintainer: Debian-Athena Project <debathena@mit.edu>
Build-Depends: @cdbs@, libldap2-dev
Standards-Version: 3.7.2
Package: debathena-ldap-config
Architecture: all
Depends: debathena-ssl-certificates, libsasl2-modules-gssapi-mit | libsasl2-gssapi-mit, libldap2, ${misc:Depends}
Provides: ${diverted-files}
Conflicts: ${diverted-files}
Description: LDAP configuration for Debian-Athena
This package configures your system to use ldap.mit.edu as its
default LDAP server.
debian/rules:
#!/usr/bin/make -f
DEB_AUTO_UPDATE_DEBIAN_CONTROL = 1
DEB_DIVERT_EXTENSION = .debathena
DEB_TRANSFORM_FILES_debathena-ldap-config += \
/etc/ldap/ldap.conf.debathena
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/rules/config-package.mk
debian/transform_ldap.conf.debathena:
#!/usr/bin/perl -0p s/^#BASE.*$/BASE\tdc=mit, dc=edu/m or die; s,^#URI.*$,URI\tldap://ldap.mit.edu\nTLS_CACERT\t/usr/share/debathena-ssl-certificates/mitCA.pem,m or die;
In this example, we want to replace /etc/nsswitch.conf. On newer Debian distributions, /etc/nsswitch.conf may have been modified by the mdns package’s postinst script and thus one will not be able to obtain an unmodified (stock) version of this script to change. On such systems, Debian also ships an unmodified “template” copy of the nsswitch.conf file, in /usr/share/base-files/nsswitch.conf. This option allow us to use the template copy instead of the installed copy of nsswitch.conf for modification using the transform script. The build log from running debuild to build this example on Debian Etch is here.
Some packages do not ship md5sums for non-configuration files (a complete list is available here) and md5sums of non-configuration files in such packages cannot be verified. base-files happens to be one of these packages, so when building debathena-nsswitch-config, you will likely see a warning and the md5sum of /usr/share/base-files/nsswitch.conf will not be checked. This should be safe, since this template file should never be modified from upstream anyway.
debian/control.in:
Source: debathena-nsswitch-config
Section: debathena-config/net
Priority: extra
Maintainer: Debian-Athena Project <debathena@mit.edu>
Build-Depends: @cdbs@
Standards-Version: 3.7.2
Package: debathena-nsswitch-config
Architecture: all
Depends: debathena-hesiod-config, debathena-afuse-automounter, libnss-nonlocal (>= 1.3~), libnss-afspag, ${misc:Depends}
Provides: ${diverted-files}
Conflicts: ${diverted-files}
Description: nsswitch configuration for Debian-Athena
This packages configures nsswitch to get passwd information from
Athena Hesiod, using libnss-nonlocal, which prevents network users
from authenticating to the same uid as a local user (and similarly
for network groups). It also configures libnss-afspag, which gives
names to AFS pag gids.
.
Installing this package results in all Athena account holders having
an account on your system. Be sure to consider the relevant security
consequences of doing so if you install this package.
debian/rules:
#!/usr/bin/make -f
DEB_AUTO_UPDATE_DEBIAN_CONTROL = 1
DEB_DIVERT_EXTENSION=.debathena
DEB_TRANSFORM_FILES_debathena-nsswitch-config += \
/etc/nsswitch.conf.debathena
ifneq ($(wildcard /usr/share/base-files/nsswitch.conf),)
DEB_CHECK_FILES_SOURCE_/etc/nsswitch.conf.debathena = \
/usr/share/base-files/nsswitch.conf
endif
DEB_DIVERT_FILES_debathena-nsswitch-config += \
/usr/sbin/adduser.debathena \
/usr/sbin/groupadd.debathena \
/usr/sbin/addgroup.debathena \
/usr/sbin/useradd.debathena \
/usr/bin/dpkg.debathena \
/usr/sbin/invoke-rc.d.debathena
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/rules/config-package.mk
debian/transform_nsswitch.conf.debathena:
#!/usr/bin/perl -0p s/^(passwd: .*)$/$1 nonlocal\npasswd_nonlocal: hesiod/m or die; s/^(group: .*)$/$1 afspag nonlocal\ngroup_nonlocal: hesiod/m or die;
This AFS configuration example is an instance of a slightly more complex situation. It uses the DEB_DIVERT_FILES system to replace several configuration files that do not vary between Debian releases with a constant file included in the package distribution. This package has a lot of complexity in it that is unrelated to the config-package-dev system intended to carefully check we're using the official Athena CellServDB; you should feel free to ignore it. The build log from running debuild to build this example on Debian Etch is here.
debian/control.in:
Source: debathena-afs-config
Section: debathena-config/net
Priority: extra
Maintainer: Debian-Athena Project <debathena@mit.edu>
Build-Depends: @cdbs@, wget, openafs-client
Standards-Version: 3.7.2
Package: debathena-afs-config
Architecture: all
Depends: openafs-client, debathena-kerberos-config, debathena-machtype, openafs-krb5, ${misc:Depends}
Recommends: debathena-autofs-config
Provides: ${diverted-files}
Conflicts: ${diverted-files}
Description: AFS configuration for Debian-Athena
This packages configures OpenAFS to access the Athena AFS cell.
debian/rules:
#!/usr/bin/make -f
DEB_AUTO_UPDATE_DEBIAN_CONTROL = 1
DEB_DIVERT_EXTENSION=.debathena
DEB_DIVERT_FILES_debathena-afs-config += \
/etc/openafs/afs.conf.client.debathena \
/etc/openafs/CellAlias.debathena \
/etc/openafs/CellServDB.debathena \
/etc/openafs/SuidCells.debathena \
/etc/openafs/ThisCell.debathena \
/etc/openafs/cacheinfo.debathena
DEB_TRANSFORM_FILES_debathena-afs-config += \
/etc/openafs/afs.conf.debathena
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/rules/config-package.mk
FROM_ATHENA = debian/CellAlias.debathena debian/CellServDB.debathena
clean:: $(FROM_ATHENA)
common-build-indep:: $(FROM_ATHENA)
$(patsubst %.debathena,%.athena,$(FROM_ATHENA)) : %.athena: FORCE
wget http://stuff.mit.edu/afs/athena.mit.edu/service/$(patsubst debian/%.athena,%,$@) -O $@.new
diff -q $@ $@.new # Check that they are the same.
rm -f $@.new
$(FROM_ATHENA) : %.debathena: %.athena %.debathena-extra
cat $^ >| $@.tmp
diff -q $@.tmp $@ || mv $@.tmp $@
rm -f $@.tmp
FORCE:
.PRECIOUS: $(patsubst %.debathena,%.athena,$(FROM_ATHENA))
debian/transform_afs.conf.debathena:
#!/usr/bin/perl -0p
if (m/AFS_SYSNAME/) {
s/^.*AFS_SYSNAME=.*$/AFS_SYSNAME="\$(machtype -S) \$(machtype -C | sed "s\/:\/ \/g")"/m or die;
} else {
s/^AFS_POST_INIT=.*$/AFS_POST_INIT="fs sysname \$(machtype -S) \$(machtype -C | sed "s\/:\/ \/g")"/m or die;
}
s/^OPTIONS=(AUTOMATIC|\$MEDIUM)$/OPTIONS="-stat 10000 -daemons 6 -volumes 200"/m or die;
debian/debathena-afs-config.install:
debian/afs.conf.client.debathena etc/openafs debian/CellAlias.debathena etc/openafs debian/CellServDB.debathena etc/openafs debian/SuidCells.debathena etc/openafs debian/ThisCell.debathena etc/openafs debian/cacheinfo.debathena etc/openafs
While most packages only need the DEB_TRANSFORM_FILES primitive, because it is more convenient, it is slightly less expressive than the DEB_DIVERT_FILES and debian_check_files primitives. Below is another packaging of debathena-ldap-config producing precisely the same binary package as the debathena-ldap-config example above. This packaging uses the DEB_DIVERT_FILES and debian_check_files, to show how they work. The build log from running debuild to build this example on Debian Etch is here.
debian/rules:
#!/usr/bin/make -f
DEB_AUTO_UPDATE_DEBIAN_CONTROL = 1
DEB_DIVERT_EXTENSION=.debathena
DEB_DIVERT_FILES_debathena-ldap-config += \
/etc/ldap/ldap.conf.debathena
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/rules/config-package.mk
common-build-indep:: debian/ldap.conf.debathena
debian/ldap.conf.debathena: $(call debian_check_files,/etc/ldap/ldap.conf)
perl -0pe ' \
s/^#BASE.*$$/BASE\tdc=mit, dc=edu/m and \
s,^#URI.*$$,URI\tldap://ldap.mit.edu\nTLS_CACERT\t/usr/share/debathena-ssl-certificates/mitCA.pem,m or \
die' $< > $@
clean::
rm -f debian/ldap.conf.debathena
debian/debathena-ldap-config.install:
debian/ldap.conf.debathena etc/ldap