source: trunk/debathena/NOTES @ 22788

Revision 22788, 10.9 KB checked in by tabbott, 16 years ago (diff)
debathena/NOTES: Correct typo in schroot PAM session workaround.
RevLine 
[22689]1This hierarchy contains Debian/Ubuntu-specific materials, also known
2as "Debathena".  The contents are:
3
4* debathena - Debathena-specific software packages such as PAM and NSS
5  modules.
6
7* config - Packages for configuring native system software in a manner
8  appropriate for Athena.
9
10* meta - Packages which contain nothing but dependencies on other
11  packages and serve as an installation convenience.
12
13* scripts - Build scripts and supporting materials.
14
15Debathena is a SIPB project, and its infrastructure and procedures
16will need to be adapted for Athena 10.  For the moment this file will
17document the Debathena procedures as they are, not as they will be.
18The current procedures do not even use this svn repository yet.
19
20Debian software used by Debathena:
21
22  * schroot - Used to manage build chroot environments for each
23    Debian/Ubuntu version.  We use the lvm-snapshot schroot type,
24    which allows rapid construction of ephemeral copies of template
25    "source" chroots, so that every binary package build is done in a
26    clean environment.
27
28  * debuild - Used to create Debian source packages from package
29    source directories.
30
31  * sbuild - Used to build binary packages from source packages inside
32    schroot environments.
33
34  * equivs - Used to create packages which only contain dependency
35    information.  Somewhat of a dirty hack, since it doesn't keep
36    proper changelogs, but it reduces overhead.
37
38  * CDBS (Common Debian Build System) - Referenced by debian/rules
39    files in packages.  Contains standard build rules to cut down on
40    per-package boilerplate.
41
42  * reprepro - Used to upload packages into the apt repositories.
43
44  * approx - Used to create a local cache of Debian packages on the
45    build server.  This cache is referenced by the build chroots for
46    improved performance.
47
[22787]48The remainder of this file documents procedures useful to Athena 10
49developers and the release engineer.
[22689]50
[22787]51Developers: Preferences setup
52-----------------------------
[22689]53
[22787]54You will probably want a $HOME/.devscripts file containing the
55following:
[22689]56
[22787]57DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn"
[22689]58
[22787]59This will save you from having to specify those options every time you
60run debuild.  Athena 10 scripts do not assume the above preferences,
61but the instructions in this file do.  The options mean:
[22689]62
[22787]63  * Look for original source as a tarfile or create one.
64  * Do not sign the source package.
65  * Do not sign the changes file.
66  * Ignore common version control metadata files when creating diffs.
67  * Ignore .svn paths when creating tarballs.
[22689]68
[22787]69You should also set the environment variable DEBATHENA_APT to
70"/afs/dev.mit.edu/system/athena10/apt".
[22689]71
[22787]72Developers: Preparing a change
73------------------------------
[22689]74
[22787]75To prepare a change to a regular package (a source tree containing a
76debian/ subdir), make the edits in a checkout and record a changelog
77entry.  You can either edit debian/changelog using emacs changelog
78mode (C-c C-v to add a new version entry, C-c C-a to add a change
79entry, C-c C-f to finalize the entry) or you can run "dadch".
[22689]80
[22787]81When creating a new version entry, bump the upstream version number
82(to 10.0.0 if it was not already that high) if you are changing the
83main package source.  Otherwise, just bump the Debian version
84component (change 0debathena1 to 0debathena2, for instance).
[22689]85
[22787]86Developers: Building a package for test purposes on one platform
87----------------------------------------------------------------
[22689]88
[22787]89After you have prepared a change, you will want to test that it builds
90and perhaps that it works before committing it.  First, if it is an
91Athena source directory using autoconf, run "daconfiscate" to set up
92the autoconf boilerplate which we don't check in.  Second, run
93"daorig" to copy or create an orig tarball in the parent directory if
94necessary.  Third, run "debuild".  The resulting package will be
95placed in the parent directory.
[22689]96
[22787]97In order to test if the package works, you can install it with "dpkg
98-i filename.deb".
[22689]99
[22787]100Developers: Building a package for test purposes on all platforms
101-----------------------------------------------------------------
[22764]102
[22787]103If the package you are working on interacts with the native OS in ways
104that might vary from platform to platform, you may want to do a test
105build for all platforms.  You will need to do this on
106linux-build-10.mit.edu or another machine which has been set up with
107build schroots.
108
109As above, run daconfiscate (if necessary) and then daorig.  Then run
110"debuild -S" to create a source package.  Now cd into the parent
111directory and identify the .dsc file created by debuild -S; it will
112have a name like debathena-just_9.4.0-0debathena2.dsc.  Run "da
113sbuildhack filename.dsc" to perform the package builds.  Each build
114will take place inside an ephemeral chroot based on a snapshot of a
115template for a particular Debian or Ubuntu version.  If a build fails
116and it's not obvious from the build log why, you may need to create
117your own ephemeral chroot session with a command like "schroot -c
118gutsy-amd64-sbuild /bin/sh" and then run debuild from within the
119package sources.
120
121If the build is successful, it will create a set of packages with
122names like debathena-just_9.4.0-0debathena2~ubuntu6.06_amd64.deb.
123
124Developers: Building an equivs package
125--------------------------------------
126
127Most of the packages under debathena/meta are faked up using equivs.
128To build one, just run:
129
130  equivs-build --full filename.equivs
131
132These equivs files make reference to ../common, so you must have a
133checkout of debathena/meta/common alongside the particular
134meta-package you are building.
135
136Release engineer: Bootstrapping the project infrastructure
137----------------------------------------------------------
138
[22764]139  1. Create the package repository (detailed instructions on this
140     pending).  Set the DEBATHENA_APT environment variable to point to
141     the package repository.  Put a copy of the debathena "scripts"
142     subdir in your path.
143
144  2. Create the build area.
145
146  3. Build each equivs package under meta/ using "equivs-build --full
147     *.equivs" and upload each with "daequivsupload *.changes".  This
148     has the side-effect of creating the basic structure of the
149     package repository.
150
151  4. Set up the build server.  The basic structure of the apt
152     repository must work for make-chroot to succeed, so this must
153     happen after step 3.
154
155  5. For each normal Debian package in dependency order, cd into its
156     directory in the build area and run "da sbuildhack *.dsc" and
[22787]157     "daupload-release *_source.changes".  If the package contains
158     only an "Architecture: all" binary package, pass the -A option to
159     both commands.
[22764]160
161     The all-packages script can generate an approximation of the
162     package list in dependency order, but it doesn't work right yet,
163     and ideally it would be possible to do several builds in parallel
164     using a Makefile like the one in scripts/build-server/build-all.
165     Improvements to this machinery are pending.
166
[22787]167  6. For each package under third, run "da ./debathenify-PKG source
168     binary upload".  This infrastructure depends on the chroots being
169     up to date, so run "all-schroots upgrade-schroot" beforehand if
170     substantial time has passed since they were created.
[22764]171
[22787]172Release engineer: Setting up a build server
173-------------------------------------------
[22689]174
[22729]175  1. The build server must be installed with free space in an LVM
[22787]176     volume group.  The build chroots consume 2GB each.  There is a
177     known memory corruption issue with LVM snapshots in the kernel
178     used in Ubuntu Gutsy (which is based on 2.6.22), so use a newer
179     kernel such as the one in Ubuntu Hardy (based on 2.6.24) instead.
[22729]180
[22743]181  2. Install debathena-standard as per the the instructions in
[22729]182     http://debathena.mit.edu/install.
183
[22743]184  3. apt-key add /afs/dev.mit.edu/system/athena10/apt/athena10-archive.asc
185
186  4. Install the packages listed in
[22729]187     scripts/build-server/packages (using "aptitude install")
188
[22743]189  5. Install debathena-login, debathena-ssh-server, and
[22729]190     debathena-build-depends (using "aptitude install").
191
192     (Depending on how recently debathena-build-depends was rebuilt,
193     additional packages might need to be installed to satisfy the
194     build-depends of newer packages.  This can be taken care of later
195     when an error occurs building a source package.)
196
[22743]197  6. Edit /etc/security/access.conf and add a first line:
[22729]198     -:ALL EXCEPT root <developer usernames>:ALL
199
[22788]200  7. Edit /etc/pam.d/schroot, comment out "@include common-session",
[22787]201     and add:
[22729]202
[22787]203       # Basic pam_unix session module in place of common-session.
204       session required         pam_unix.so
205
206  8. Edit /etc/group and add the developers to the sbuild group.
207
208  9. Create /etc/passwd entries for each developer with "hesinfo
[22729]209     username passwd >> /etc/passwd" and then run pwconv.
210
211     (This is not necessary for the login system on the main root
212     environment, but is for the chroot environments.)
213
[22787]214  10. Append to /etc/approx/approx.conf the contents of
215       scripts/build-server/approx.conf.tail.
216      Change the last line from http://debathena.mit.edu/apt to
217       file:///afs/dev.mit.edu/system/athena10/apt
218      Add "$interval 0" above the repository lines (only necessary if
219       the version of approx as reported by "dpkg -l approx" is less
220       than 3.0)
221      Run: /etc/init.d/approx restart
[22729]222
[22787]223  11. Apply scripts/build-server/10mount.patch.
[22729]224
[22787]225  12. For each supported DIST (see scripts/debian-versions.sh) run:
[22744]226
[22743]227        VG=/dev/blah scripts/build-server/make-chroot DIST i386
228        VG=/dev/blah scripts/build-server/make-chroot DIST amd64
[22729]229
[22743]230      substituting the name of the volume group for blah.  Omit the
231      amd64 line if DIST is sarge.
[22729]232
[22743]233      Example: VG=/dev/dink scripts/build-server/make-chroot gutsy i386
[22729]234
[22787]235Release engineer: Removing a build chroot on the build server
236-------------------------------------------------------------
[22729]237
[22787]238  1. Run VG=/dev/blah scripts/clean-schroots as root to make sure that
239     the build chroot is not mounted, substituting the name of the
240     volume group for blah.
[22729]241
242  2. Edit /etc/schroot/schroot.conf and delete the section
243     corresponding to the chroot.
244
245  3. Run lvchange -an blah/chrootname
246     substituting the name of the volume group for blah and the chroot
247     name for chroot.  Example: lvchange -an dink/gutsy-i386-sbuild
248
[22764]249  4. Run lvremove blah/chrootname
250
[22787]251Release engineer: Setting up a canonical build area
252---------------------------------------------------
[22764]253
254  1. Create an empty directory and cd into it.  The canonical build
255     area lives in /afs/dev.mit.edu/project/release/10/build.
256
[22787]257  2. Run gen-packages to create the table of normal Debian packages.
[22764]258
[22787]259  3. Run dasource to create subdirs and source packages for each
260     normal Debian package.
[22764]261
262  4. Create checkouts of the meta and third directories:
263
264     svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/meta
265     svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/third
266
267     (A couple of subdirectories of debathena/meta are normal Debian
268     packages, so this will create redundant copies of those.  Ignore
269     them; they won't be used.)
Note: See TracBrowser for help on using the repository browser.