source: trunk/doc/build-system @ 17299

This document explains how to build packages from the Athena source
tree, and then describes guidelines for designing the build system for
code we maintain in the Athena source tree.  A discussion of building
packages we get from third parties is in the file "third-party" in
this directory.

This file contains the following sections:

        Building packages from the source tree
        Building the tree
        Generic build system guidelines
        Using Autoconf
        Using a straight Makefile

Building packages from the source tree
--------------------------------------

To build a package from the Athena source tree, you should generally
first attach the source tree that you want.  You can either attach a
locker like "source-8.0" or "source-8.1" to get the sources for a
particular release, or you can attach "source" to get the current
development sources.  Releases prior to 8.1 will not build using the
procedure below.  Note that you can only build whole packages, not
arbitrary subdirectories; package names are listed in
packs/build/packages.

After attaching the correct locker, you can build a package by doing:

        set src = /mit/source           # or /mit/source-8.1, etc.
        cp -r /mit/souce/PACKAGE /tmp/PACKAGE
        cd /tmp/PACKAGE
        sh $src/packs/build/do.sh dist
        sh $src/packs/build/do.sh prepare
        sh $src/packs/build/do.sh all

If you are building the current sources (or sources from a release
other than the one running on the Athena machine you're doing the
build from), they might depend on aspects of the Athena release which
aren't true of the machine you're using.  If this is true, then you
may have to construct a build root and build large parts of the
release.  See the next section on "Build the tree" if that is true.

"do" accepts the following options:

        -s SRCDIR       Specifies the source hierarchy, if not default
        -d DESTDIR      Specifies the install hierarchy, if not /.srvd
        -n              Show what build steps would be done, but don't
                        do them
        -p              Don't change the path (used by rhlinux SRPMs)

The default location for SRCDIR is the source locker you attached
(/mit/source, /mit/source-8.1, etc.).  "do" supports the following
build operations:

        dist            Copy in or create any materials necessary to
                        make source tree suitable for distribution
        prepare         Set up the source tree for building
        clean           Clean any files generated by the "all" operation
        all             Create automatically generated files
        check           Perform automated tests, if any
        install         Install in DESTDIR specified by -d (or default)

Building the tree
-----------------

To build the entire tree, you must create a build root area on a
machine to run a chrooted build inside.  This is slow and somewhat
disruptive of the machine (if you set up AFS to be accessible from the
root area), so it helps to have a machine set aside for the purpose.

To create a build root area, run

        sh /mit/source/packs/build/makeroot.sh PATH [RELEASE]

where PATH is the place you want the build root to be in and the
optional RELEASE argument specifies which release you want the build
to be for (if none is given, the script assumes you want to be
building /mit/source).

To make AFS accessible in the build root, you must turn AFSADJUST off
in /etc/athena/rc.conf, modify /usr/vice/etc/cacheinfo to mount AFS in
PATH/afs, reboot, and then remove /afs and make it a symlink to
PATH/afs.

If you do not want to make AFS accessible in the build root, you will
have to make a copy of the source tree in the build root; for Solaris,
you will also have to make a copy of the sunsoft locker.  There may be
other dependencies on AFS as well.

After you have set up a build root, run:

        chroot PATH /bin/sh /mit/source/packs/build/build.sh

By default, the builds will be done in PATH/build and the installs
done into PATH/.srvd.  "build" takes the following options:

        -s SRCDIR       Specifies the source hierarchy, if not default
        -b BUILDDIR     Specifies the build hierarchy, if not /build
        -d DESTDIR      Specifies the install hierarchy, if not /.srvd
        -k              Keep going if a package fails to build
        -n              Only display a list of packages to be built
        -l              Log build output to a file in BUILDDIR/logs

On Linux, we use a different build script which builds RPMs instead of
doing a traditional build.  To use it, run:

        chroot PATH /bin/sh /mit/source/packs/build/rpm/build-all

By default, the builds will be done in /build/BUILD and the resulting
RPMs will wind up in /build/SRPMS and /build/RPMS.  "build-all" takes
the same options as "build" except that there is no -b option; the -d
option specifies the path for both the build and destination area.

Building the whole tree has the following requirements right now:

        * The system passwd file must have "pop" and "discuss" users.
          Athena machines should typically have these lines in them
          already, but you may have to add them on non-Athena systems.

        * You must have write access to the destination hierarchy, and
          be able to chown files and make setuid root files in that
          tree.

        * On Solaris, you need system:authuser access to the Athena
          cell in order to use the compiler from the sunsoft locker.

Generic build system guidelines
-------------------------------

Here are some desirable properties of a build system for programs or
libraries, regardless of how it is implemented:

        * It should use feature tests, not platform tests.

        * Once the appropriate configuration steps (e.g. running
          configure or imake) have been done, "make all" should
          generate all automatically generated files without ever
          modifying any source files.  (For example, "make all" should
          not try to regenerate configure from configure.in, and a
          build system should not wait until "make install" time to
          generate a file which is going to be installed.)

        * "make install" should install all files appropriate for the
          target directories without modifying any files in the build
          tree.  In particular, it should not depend on the "all"
          target.

          "make install" should use the install command to install
          files, not cp or tar or shell redirection.  To install
          multiple files in the same way, use a single install command
          rather than a for loop.

        * "make clean" should clean all files generated by "make all."
          "make distclean" should also delete files generated by the
          configuration steps.

        * "make check" should run self-tests.  It should not assume
          that the program has been installed, but it should assume
          that the program has been built.  It should not modify the
          build tree in any way; in particular, it should not depend
          on the "all" target.  If there are no self-tests, "make
          check" should not generate an error.

        * In a multi-directory source tree, it should be possible to
          change to a subdirectory of the build tree and run any of
          the aforementioned make targets with the same results as
          having descended into that directory from above.  It is okay
          to assume that previous parts of the build tree have been
          built, but it is not okay to rely on make command-line
          parameters passed in from the parent directory.

Using Autoconf
--------------

For packages in the "athena" hierarchy, we use autoconf.  Autoconf is
not perfect, but it's the only reasonable package out there for doing
feature tests as opposed to platform tests.  See the Autoconf info
pages (available in the default emacs info menu) for information about
using Autoconf in general.

When we build the program for the Athena environment, we will use a
config.site file (already written; you don't need to write it or point
to it) which does the following:

        * Sets the shell variable "athena" to "true".

        * Sets the appropriate compiler (using the CC environment
          variable) as well as other compilation tools (yacc, lex) for
          the platform in question.

        * Sets the appropriate X include path and library path for use
          with AC_PATH_X.

        * Sets the appropriate with_* variables for finding Motif, AFS
          libraries, Kerberos 4, Kerberos 5, com_err, and ss.  If your
          package uses any of those libraries, copy the aclocal.m4
          file from packs/build and use the macros contained within.

        * Sets prefix to /usr/athena, datadir to '${prefix}/lib',
          sbindir to '${prefix}/etc', and sysconfdir to '/etc/athena'.
          You do not need to take these into account in configure.in;
          just put in your Makefile.in:

                prefix=@prefix@
                exec_prefix=@exec_prefix@
                datadir=@datadir@
                sbindir=@sbindir@
                sysconfdir=@sysconfdir@

        * Sets shell variables lbindir and lsbindir to /bin/athena and
          /etc/athena respectively.  If your build system needs to
          install files in /bin/athena or /etc/athena when built for
          the Athena environment, use the following in your configure.in:

                test -z "$lbindir" && lbindir='${bindir}'
                test -z "$lsbindir" && lsbindir='${sbindir}'
                AC_SUBST(lbindir)
                AC_SUBST(lsbindir)

          And then in your Makefile.in:

                prefix=@prefix@
                exec_prefix=@exec_prefix@
                bindir=@bindir@
                sbindir=@sbindir@
                lbindir=@lbindir@
                lsbindir=@lsbindir@

The build environment will ensure that the following things are true:

        * The environment variable ATHENA_SYS is set to the current
          platform's desired ATHENA_SYS value (e.g. "sun4m_54").

        * The environment variable HOSTTYPE is set to the current
          platform name (e.g. "sun").

        * The environment variable CONFIG_SITE points to the Athena
          config.site file.

Using a straight Makefile
-------------------------

For packages in the "packs" hierarchy, and in some parts of the
"third" hierarchy where we construct the build system by hand, we use
a straight Makefile, since feature tests are not necessary for the
packs hierarchy.

The build system will ensure that the same environment variables are
set as for using Autoconf.  In addition, the make variable CC will be
set to the appropriate compiler for the current platform when the
"all" target is invoked.  You should be sparing in your use of C
source code in packages using a straight Makefile, but for small,
portable C programs it's okay.

The subdirectories "platform/$HOSTTYPE" or "arch/$ATHENA_SYS" should
be used for platform or system-dependent components of a package using
a straight Makefile.

Start each Makefile with "# $Id: $" and "SHELL=/bin/sh".  The latter
is to work around a bug in the Irix make where it runs commands under
${SHELL} rather than /bin/sh.