[22689] | 1 | This hierarchy contains Debian/Ubuntu-specific materials, also known |
---|
| 2 | as "Debathena". The contents are: |
---|
| 3 | |
---|
| 4 | * debathena - Debathena-specific software packages such as PAM and NSS |
---|
| 5 | modules. |
---|
| 6 | |
---|
| 7 | * config - Packages for configuring native system software in a manner |
---|
| 8 | appropriate for Athena. |
---|
| 9 | |
---|
| 10 | * meta - Packages which contain nothing but dependencies on other |
---|
| 11 | packages and serve as an installation convenience. |
---|
| 12 | |
---|
| 13 | * scripts - Build scripts and supporting materials. |
---|
| 14 | |
---|
| 15 | Debathena is a SIPB project, and its infrastructure and procedures |
---|
| 16 | will need to be adapted for Athena 10. For the moment this file will |
---|
| 17 | document the Debathena procedures as they are, not as they will be. |
---|
| 18 | The current procedures do not even use this svn repository yet. |
---|
| 19 | |
---|
| 20 | Debian software used by Debathena: |
---|
| 21 | |
---|
| 22 | * schroot - Used to manage build chroot environments for each |
---|
| 23 | Debian/Ubuntu version. We use the lvm-snapshot schroot type, |
---|
| 24 | which allows rapid construction of ephemeral copies of template |
---|
| 25 | "source" chroots, so that every binary package build is done in a |
---|
| 26 | clean environment. |
---|
| 27 | |
---|
| 28 | * debuild - Used to create Debian source packages from package |
---|
| 29 | source directories. |
---|
| 30 | |
---|
| 31 | * sbuild - Used to build binary packages from source packages inside |
---|
| 32 | schroot environments. |
---|
| 33 | |
---|
| 34 | * equivs - Used to create packages which only contain dependency |
---|
| 35 | information. Somewhat of a dirty hack, since it doesn't keep |
---|
| 36 | proper changelogs, but it reduces overhead. |
---|
| 37 | |
---|
| 38 | * CDBS (Common Debian Build System) - Referenced by debian/rules |
---|
| 39 | files in packages. Contains standard build rules to cut down on |
---|
| 40 | per-package boilerplate. |
---|
| 41 | |
---|
| 42 | * reprepro - Used to upload packages into the apt repositories. |
---|
| 43 | |
---|
| 44 | * approx - Used to create a local cache of Debian packages on the |
---|
| 45 | build server. This cache is referenced by the build chroots for |
---|
| 46 | improved performance. |
---|
| 47 | |
---|
[22787] | 48 | The remainder of this file documents procedures useful to Athena 10 |
---|
| 49 | developers and the release engineer. |
---|
[22689] | 50 | |
---|
[22787] | 51 | Developers: Preferences setup |
---|
| 52 | ----------------------------- |
---|
[22689] | 53 | |
---|
[22787] | 54 | You will probably want a $HOME/.devscripts file containing the |
---|
| 55 | following: |
---|
[22689] | 56 | |
---|
[22787] | 57 | DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn" |
---|
[22689] | 58 | |
---|
[22787] | 59 | This will save you from having to specify those options every time you |
---|
| 60 | run debuild. Athena 10 scripts do not assume the above preferences, |
---|
| 61 | but the instructions in this file do. The options mean: |
---|
[22689] | 62 | |
---|
[22787] | 63 | * Look for original source as a tarfile or create one. |
---|
| 64 | * Do not sign the source package. |
---|
| 65 | * Do not sign the changes file. |
---|
| 66 | * Ignore common version control metadata files when creating diffs. |
---|
| 67 | * Ignore .svn paths when creating tarballs. |
---|
[22689] | 68 | |
---|
[23046] | 69 | You will also want a $HOME/.sbuildrc file containing the following: |
---|
| 70 | |
---|
| 71 | $nolog = 1; |
---|
| 72 | $mailto = 'yourusername'; |
---|
| 73 | $log_dir = '/tmp/sbuild-logs'; |
---|
| 74 | $maintainer_name = 'Debian-Athena Project <debathena@mit.edu>'; |
---|
| 75 | $force_orig_source = 1; |
---|
| 76 | $sbuild_mode = "user"; |
---|
| 77 | 1; |
---|
| 78 | |
---|
[22787] | 79 | You should also set the environment variable DEBATHENA_APT to |
---|
| 80 | "/afs/dev.mit.edu/system/athena10/apt". |
---|
[22689] | 81 | |
---|
[22787] | 82 | Developers: Preparing a change |
---|
| 83 | ------------------------------ |
---|
[22689] | 84 | |
---|
[22787] | 85 | To prepare a change to a regular package (a source tree containing a |
---|
| 86 | debian/ subdir), make the edits in a checkout and record a changelog |
---|
| 87 | entry. You can either edit debian/changelog using emacs changelog |
---|
| 88 | mode (C-c C-v to add a new version entry, C-c C-a to add a change |
---|
| 89 | entry, C-c C-f to finalize the entry) or you can run "dadch". |
---|
[22689] | 90 | |
---|
[22787] | 91 | When creating a new version entry, bump the upstream version number |
---|
| 92 | (to 10.0.0 if it was not already that high) if you are changing the |
---|
| 93 | main package source. Otherwise, just bump the Debian version |
---|
| 94 | component (change 0debathena1 to 0debathena2, for instance). |
---|
[22689] | 95 | |
---|
[22787] | 96 | Developers: Building a package for test purposes on one platform |
---|
| 97 | ---------------------------------------------------------------- |
---|
[22689] | 98 | |
---|
[22787] | 99 | After you have prepared a change, you will want to test that it builds |
---|
| 100 | and perhaps that it works before committing it. First, if it is an |
---|
| 101 | Athena source directory using autoconf, run "daconfiscate" to set up |
---|
| 102 | the autoconf boilerplate which we don't check in. Second, run |
---|
| 103 | "daorig" to copy or create an orig tarball in the parent directory if |
---|
| 104 | necessary. Third, run "debuild". The resulting package will be |
---|
| 105 | placed in the parent directory. |
---|
[22689] | 106 | |
---|
[22787] | 107 | In order to test if the package works, you can install it with "dpkg |
---|
| 108 | -i filename.deb". |
---|
[22689] | 109 | |
---|
[22787] | 110 | Developers: Building a package for test purposes on all platforms |
---|
| 111 | ----------------------------------------------------------------- |
---|
[22764] | 112 | |
---|
[22787] | 113 | If the package you are working on interacts with the native OS in ways |
---|
| 114 | that might vary from platform to platform, you may want to do a test |
---|
| 115 | build for all platforms. You will need to do this on |
---|
| 116 | linux-build-10.mit.edu or another machine which has been set up with |
---|
| 117 | build schroots. |
---|
| 118 | |
---|
| 119 | As above, run daconfiscate (if necessary) and then daorig. Then run |
---|
| 120 | "debuild -S" to create a source package. Now cd into the parent |
---|
| 121 | directory and identify the .dsc file created by debuild -S; it will |
---|
| 122 | have a name like debathena-just_9.4.0-0debathena2.dsc. Run "da |
---|
| 123 | sbuildhack filename.dsc" to perform the package builds. Each build |
---|
| 124 | will take place inside an ephemeral chroot based on a snapshot of a |
---|
| 125 | template for a particular Debian or Ubuntu version. If a build fails |
---|
| 126 | and it's not obvious from the build log why, you may need to create |
---|
| 127 | your own ephemeral chroot session with a command like "schroot -c |
---|
| 128 | gutsy-amd64-sbuild /bin/sh" and then run debuild from within the |
---|
| 129 | package sources. |
---|
| 130 | |
---|
| 131 | If the build is successful, it will create a set of packages with |
---|
| 132 | names like debathena-just_9.4.0-0debathena2~ubuntu6.06_amd64.deb. |
---|
| 133 | |
---|
| 134 | Developers: Building an equivs package |
---|
| 135 | -------------------------------------- |
---|
| 136 | |
---|
| 137 | Most of the packages under debathena/meta are faked up using equivs. |
---|
| 138 | To build one, just run: |
---|
| 139 | |
---|
| 140 | equivs-build --full filename.equivs |
---|
| 141 | |
---|
| 142 | These equivs files make reference to ../common, so you must have a |
---|
| 143 | checkout of debathena/meta/common alongside the particular |
---|
| 144 | meta-package you are building. |
---|
| 145 | |
---|
[23018] | 146 | Developers: The meaning of metapackages |
---|
| 147 | --------------------------------------- |
---|
| 148 | |
---|
| 149 | If you are adding a new package to the repository, you will probably |
---|
| 150 | at some point want to add it to one of the metapackages so that it |
---|
| 151 | doesn't have to be installed by hand. Here are some descriptions |
---|
| 152 | which may help identify which metapackage is best: |
---|
| 153 | |
---|
| 154 | * locker: Provides access to Athena locker software--AFS and |
---|
| 155 | automounter configuration, locker-related utilities, etc. |
---|
| 156 | |
---|
| 157 | * clients: Provides clients (either locally-written, like athinfo and |
---|
| 158 | Discuss, or configurations) for Athena services, as well as |
---|
| 159 | Athena-specific utility programs like "jot". Configurations for |
---|
| 160 | graphical client software are generally in the workstation package |
---|
| 161 | instead, in order to make this package less intrusive. |
---|
| 162 | |
---|
| 163 | * standard: Implies locker and clients. Also provides Athena shell |
---|
| 164 | customizations and dotfiles. |
---|
| 165 | |
---|
| 166 | * login: Implies standard. Configurations to merge the MIT user |
---|
| 167 | namespace into the local machine namespace for the purpose of user |
---|
| 168 | lookups and authentication. |
---|
| 169 | |
---|
| 170 | * workstation: Implies login. Configurations for the graphical login |
---|
| 171 | system and graphical client software intended to provide a standard |
---|
| 172 | X login experience using Athena home directories. Still in |
---|
| 173 | development. |
---|
| 174 | |
---|
| 175 | * cluster-software: Provides a set of Debian packages common to |
---|
| 176 | cluster machines. The resulting software set is rather large, and |
---|
| 177 | thus may not be desirable to all workstation configurations. Only |
---|
| 178 | stock Debian packages belong in this metapackage; do not add |
---|
| 179 | other Debathena packages to it. |
---|
| 180 | |
---|
| 181 | * cluster: Implies workstation and cluster-software. Also contains |
---|
| 182 | configurations for self-maintenance of machines (unattended updates, |
---|
[23102] | 183 | cleanups between logins, etc.). |
---|
[23018] | 184 | |
---|
| 185 | * debian-dev: Intended for developers of the system itself; provides a |
---|
| 186 | set of Debian packages used by Debathena for development. |
---|
| 187 | |
---|
| 188 | For the most part a package should be listed in the "Depends:" line of |
---|
| 189 | a metapackage, but in some cases it is appropriate to hedge by using |
---|
| 190 | "Recommends:", which will cause aptitude to succeed even if the |
---|
| 191 | package is unavailable. For example, a package which doesn't exist in |
---|
| 192 | all Debian/Ubuntu suites or isn't free can be listed under |
---|
| 193 | "Recommends:" so that our metapackages still work in all environments. |
---|
| 194 | |
---|
[23158] | 195 | Developers: Index of cluster packages |
---|
| 196 | ------------------------------------- |
---|
| 197 | |
---|
| 198 | Some private workstation admins may want specific pieces of cluster |
---|
| 199 | infrastructure. With Athena 9.4, they would typically install the |
---|
| 200 | entire Athena software suite and then turn off features they did not |
---|
| 201 | want via /etc/athena/rc.conf. That is not generally possible in |
---|
| 202 | Athena 10; a debathena-cluster machine acts like a cluster machine in |
---|
| 203 | all respects. Instead, the machine owner should install |
---|
| 204 | debathena-workstation and then install specific packages they might be |
---|
| 205 | interested in. At this time, candidate packages include: |
---|
| 206 | |
---|
| 207 | * debathena-dns-config: Installs a caching DNS resolver and |
---|
| 208 | configures the machine to use it. |
---|
| 209 | |
---|
| 210 | * debathena-tmp-cleaner: Cleans files in /tmp and /var/tmp which |
---|
| 211 | have not been recently accessed. |
---|
| 212 | |
---|
| 213 | * debathena-reactivate: Causes each graphical login to be performed |
---|
| 214 | in a separate ephemeral snapshot of the chroot. This package |
---|
| 215 | requires the root filesystem to be an LVM logical volume inside a |
---|
| 216 | volume group with 21GB of free space for the snapshots; if that is |
---|
| 217 | not the case, it will break graphical logins. Install with care. |
---|
| 218 | |
---|
| 219 | * debathena-auto-update: Periodically updates the machine's software |
---|
| 220 | and reboots (when no one is logged in) if necessary. |
---|
| 221 | |
---|
| 222 | Cluster packages which are generally not of interest to private |
---|
| 223 | machine owners include: |
---|
| 224 | |
---|
| 225 | * debathena-clusterinfo: Looks up the machine's cluster information |
---|
| 226 | in Hesiod and caches it on local disk. |
---|
| 227 | |
---|
| 228 | * debathena-larvnet: Reports to the central larvnet server whether |
---|
| 229 | anyone has a graphical login to the machine. |
---|
| 230 | |
---|
[23162] | 231 | * debathena-syslog-config: Configures sysklogd so that some syslog |
---|
| 232 | messages are forwarded to a central logging host. |
---|
| 233 | |
---|
[22787] | 234 | Release engineer: Bootstrapping the project infrastructure |
---|
| 235 | ---------------------------------------------------------- |
---|
| 236 | |
---|
[22764] | 237 | 1. Create the package repository (detailed instructions on this |
---|
| 238 | pending). Set the DEBATHENA_APT environment variable to point to |
---|
| 239 | the package repository. Put a copy of the debathena "scripts" |
---|
| 240 | subdir in your path. |
---|
| 241 | |
---|
| 242 | 2. Create the build area. |
---|
| 243 | |
---|
| 244 | 3. Build each equivs package under meta/ using "equivs-build --full |
---|
| 245 | *.equivs" and upload each with "daequivsupload *.changes". This |
---|
| 246 | has the side-effect of creating the basic structure of the |
---|
| 247 | package repository. |
---|
| 248 | |
---|
| 249 | 4. Set up the build server. The basic structure of the apt |
---|
| 250 | repository must work for make-chroot to succeed, so this must |
---|
| 251 | happen after step 3. |
---|
| 252 | |
---|
| 253 | 5. For each normal Debian package in dependency order, cd into its |
---|
| 254 | directory in the build area and run "da sbuildhack *.dsc" and |
---|
[22787] | 255 | "daupload-release *_source.changes". If the package contains |
---|
| 256 | only an "Architecture: all" binary package, pass the -A option to |
---|
| 257 | both commands. |
---|
[22764] | 258 | |
---|
| 259 | The all-packages script can generate an approximation of the |
---|
| 260 | package list in dependency order, but it doesn't work right yet, |
---|
| 261 | and ideally it would be possible to do several builds in parallel |
---|
| 262 | using a Makefile like the one in scripts/build-server/build-all. |
---|
| 263 | Improvements to this machinery are pending. |
---|
| 264 | |
---|
[22787] | 265 | 6. For each package under third, run "da ./debathenify-PKG source |
---|
[23084] | 266 | binary upload". Any created directories under third/openafs/meta |
---|
| 267 | should be chmodded 777 to work around a perl/AFS permissions |
---|
| 268 | issue with File::Temp; if this is not done, OpenAFS metapackage |
---|
| 269 | builds will fail for other users. |
---|
[22764] | 270 | |
---|
[23090] | 271 | Release engineer: Updating the apt repository |
---|
| 272 | --------------------------------------------- |
---|
| 273 | |
---|
| 274 | 1. ssh to the build server as the builder account and change to the |
---|
| 275 | canonical build directory. |
---|
| 276 | |
---|
| 277 | 2. Run "gen-packages" to update the package list. (Or "gen-packages |
---|
| 278 | -c" if you know the AFS checkout of the source tree is up to date; |
---|
| 279 | it should update every half hour.) |
---|
| 280 | |
---|
| 281 | 3. Run "ood-packages" to produce a list of out-of-date packages. |
---|
| 282 | |
---|
| 283 | 4. For each out of date package, run "dasource PKG". Then change to |
---|
| 284 | the package directory and run "da sbuildhack *.dsc" and |
---|
| 285 | "daupload-release *_source.changes". If the package contains |
---|
| 286 | only an "Architecture: all" binary package, pass the -A option to |
---|
| 287 | both commands. |
---|
| 288 | |
---|
| 289 | 5. svn update the meta directory. If there are new subdirectories, |
---|
| 290 | chmod them 777 to work around a perl/AFS permissions issue with |
---|
| 291 | File::Temp. For each updated subdir, change to it, run |
---|
| 292 | "equivs-build --full *.equivs", and then "daequivsupload |
---|
| 293 | FILENAME.changes" on the produced changes file. |
---|
| 294 | |
---|
| 295 | 6. svn update the third directory. You can let autodebathenify |
---|
| 296 | handle the updated scripts, or you can touch |
---|
| 297 | ~/autodebathenify.suppress, make sure it's not running, and run |
---|
| 298 | "da ./debathenify-PKG source binary upload" in each updated |
---|
| 299 | directory. |
---|
| 300 | |
---|
| 301 | Sometimes you may have to mix up the order of the above steps in order |
---|
| 302 | to handle build dependencies. |
---|
| 303 | |
---|
[22787] | 304 | Release engineer: Setting up a build server |
---|
| 305 | ------------------------------------------- |
---|
[22689] | 306 | |
---|
[22729] | 307 | 1. The build server must be installed with free space in an LVM |
---|
[22787] | 308 | volume group. The build chroots consume 2GB each. There is a |
---|
| 309 | known memory corruption issue with LVM snapshots in the kernel |
---|
| 310 | used in Ubuntu Gutsy (which is based on 2.6.22), so use a newer |
---|
| 311 | kernel such as the one in Ubuntu Hardy (based on 2.6.24) instead. |
---|
[22729] | 312 | |
---|
[22743] | 313 | 2. Install debathena-standard as per the the instructions in |
---|
[22729] | 314 | http://debathena.mit.edu/install. |
---|
| 315 | |
---|
[22743] | 316 | 3. apt-key add /afs/dev.mit.edu/system/athena10/apt/athena10-archive.asc |
---|
| 317 | |
---|
| 318 | 4. Install the packages listed in |
---|
[22729] | 319 | scripts/build-server/packages (using "aptitude install") |
---|
| 320 | |
---|
[22743] | 321 | 5. Install debathena-login, debathena-ssh-server, and |
---|
[22729] | 322 | debathena-build-depends (using "aptitude install"). |
---|
| 323 | |
---|
| 324 | (Depending on how recently debathena-build-depends was rebuilt, |
---|
| 325 | additional packages might need to be installed to satisfy the |
---|
| 326 | build-depends of newer packages. This can be taken care of later |
---|
| 327 | when an error occurs building a source package.) |
---|
| 328 | |
---|
[22743] | 329 | 6. Edit /etc/security/access.conf and add a first line: |
---|
[22729] | 330 | -:ALL EXCEPT root <developer usernames>:ALL |
---|
| 331 | |
---|
[22788] | 332 | 7. Edit /etc/pam.d/schroot, comment out "@include common-session", |
---|
[22787] | 333 | and add: |
---|
[22729] | 334 | |
---|
[22787] | 335 | # Basic pam_unix session module in place of common-session. |
---|
| 336 | session required pam_unix.so |
---|
| 337 | |
---|
| 338 | 8. Edit /etc/group and add the developers to the sbuild group. |
---|
| 339 | |
---|
| 340 | 9. Create /etc/passwd entries for each developer with "hesinfo |
---|
[22729] | 341 | username passwd >> /etc/passwd" and then run pwconv. |
---|
| 342 | |
---|
| 343 | (This is not necessary for the login system on the main root |
---|
| 344 | environment, but is for the chroot environments.) |
---|
| 345 | |
---|
[22787] | 346 | 10. Append to /etc/approx/approx.conf the contents of |
---|
| 347 | scripts/build-server/approx.conf.tail. |
---|
| 348 | Change the last line from http://debathena.mit.edu/apt to |
---|
| 349 | file:///afs/dev.mit.edu/system/athena10/apt |
---|
| 350 | Add "$interval 0" above the repository lines (only necessary if |
---|
| 351 | the version of approx as reported by "dpkg -l approx" is less |
---|
| 352 | than 3.0) |
---|
| 353 | Run: /etc/init.d/approx restart |
---|
[22729] | 354 | |
---|
[22984] | 355 | 11. Apply scripts/build-server/mount-defaults.patch. |
---|
[22729] | 356 | |
---|
[22787] | 357 | 12. For each supported DIST (see scripts/debian-versions.sh) run: |
---|
[22744] | 358 | |
---|
[22743] | 359 | VG=/dev/blah scripts/build-server/make-chroot DIST i386 |
---|
| 360 | VG=/dev/blah scripts/build-server/make-chroot DIST amd64 |
---|
[22729] | 361 | |
---|
[22743] | 362 | substituting the name of the volume group for blah. Omit the |
---|
| 363 | amd64 line if DIST is sarge. |
---|
[22729] | 364 | |
---|
[22743] | 365 | Example: VG=/dev/dink scripts/build-server/make-chroot gutsy i386 |
---|
[22729] | 366 | |
---|
[23046] | 367 | 13. Create a local account for builder with: |
---|
| 368 | |
---|
| 369 | adduser --uid 1047 --disabled-password builder |
---|
| 370 | |
---|
| 371 | Make the home directory mode 700. Install a |
---|
| 372 | daemon/linux-build-10.mit.edu keytab in the home directory as |
---|
| 373 | "keytab". Install a copy of the secret repository-signing key |
---|
| 374 | (athena10@mit.edu) in the home directory's keyring with |
---|
| 375 | something like: |
---|
| 376 | |
---|
| 377 | kinit builder |
---|
| 378 | gpg --export-secret-keys athena10@mit.edu | \ |
---|
| 379 | ssh -l builder machinename gpg --import |
---|
| 380 | |
---|
| 381 | Create a file named .sbuildrc in builder's homedir containing: |
---|
| 382 | |
---|
| 383 | $nolog = 1; |
---|
| 384 | $mailto = 'source-wash@mit.edu'; |
---|
| 385 | $log_dir = '/tmp/sbuild-logs'; |
---|
| 386 | $maintainer_name = 'Debian-Athena Project <debathena@mit.edu>'; |
---|
| 387 | $force_orig_source = 1; |
---|
| 388 | $sbuild_mode = "user"; |
---|
| 389 | 1; |
---|
| 390 | |
---|
[23050] | 391 | Create a file named .ssh/config in builder's homedir containing: |
---|
| 392 | |
---|
| 393 | Host svn.mit.edu |
---|
| 394 | User debuildsvn |
---|
| 395 | |
---|
[23046] | 396 | Add builder to the sbuild group in /etc/group. |
---|
| 397 | |
---|
| 398 | Copy scripts/build-server/autodebathenify to builder's homedir. |
---|
| 399 | Create a file named autodebathenify.config in builder's homedir |
---|
| 400 | containing: |
---|
| 401 | |
---|
| 402 | error_addr=source-wash@mit.edu |
---|
| 403 | scripts_dir=/afs/dev.mit.edu/source/src-svn/debathena/scripts |
---|
| 404 | build_dir=/afs/dev.mit.edu/project/release/10/build/third |
---|
| 405 | packages="cyrus-sasl2-mit evolution-data-server lprng openafs tcsh" |
---|
| 406 | export DEBATHENA_APT=/afs/dev.mit.edu/system/athena10/apt |
---|
| 407 | |
---|
| 408 | Copy scripts/build-server/autodebathenify.cron to builder's |
---|
| 409 | homedir and install it with "crontab autodebathenify.cron". |
---|
| 410 | |
---|
[23085] | 411 | Create a file named .devscripts in builder's homedir containing: |
---|
| 412 | |
---|
| 413 | DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -us -uc -i -I.svn" |
---|
| 414 | |
---|
[23065] | 415 | In builder's homedir, append to .profile: |
---|
| 416 | |
---|
| 417 | PATH=${PATH}:/afs/dev.mit.edu/source/src-svn/debathena/scripts |
---|
| 418 | export DEBATHENA_APT=/afs/dev.mit.edu/system/athena10/apt |
---|
| 419 | |
---|
| 420 | and to .bashrc: |
---|
| 421 | |
---|
| 422 | bld=/afs/dev.mit.edu/project/release/10/build |
---|
| 423 | |
---|
[22787] | 424 | Release engineer: Removing a build chroot on the build server |
---|
| 425 | ------------------------------------------------------------- |
---|
[22729] | 426 | |
---|
[22787] | 427 | 1. Run VG=/dev/blah scripts/clean-schroots as root to make sure that |
---|
| 428 | the build chroot is not mounted, substituting the name of the |
---|
| 429 | volume group for blah. |
---|
[22729] | 430 | |
---|
| 431 | 2. Edit /etc/schroot/schroot.conf and delete the section |
---|
| 432 | corresponding to the chroot. |
---|
| 433 | |
---|
| 434 | 3. Run lvchange -an blah/chrootname |
---|
| 435 | substituting the name of the volume group for blah and the chroot |
---|
| 436 | name for chroot. Example: lvchange -an dink/gutsy-i386-sbuild |
---|
| 437 | |
---|
[22764] | 438 | 4. Run lvremove blah/chrootname |
---|
| 439 | |
---|
[22985] | 440 | Release engineer: Removing a dist from the apt repository |
---|
| 441 | --------------------------------------------------------- |
---|
| 442 | |
---|
| 443 | 1. Inside the apt repository, edit conf/distributions and remove the |
---|
| 444 | distribution section. |
---|
| 445 | |
---|
| 446 | 2. Run reprepro -Vb $DEBATHENA_APT --delete clearvanished |
---|
| 447 | |
---|
[22787] | 448 | Release engineer: Setting up a canonical build area |
---|
| 449 | --------------------------------------------------- |
---|
[22764] | 450 | |
---|
| 451 | 1. Create an empty directory and cd into it. The canonical build |
---|
| 452 | area lives in /afs/dev.mit.edu/project/release/10/build. |
---|
| 453 | |
---|
[22787] | 454 | 2. Run gen-packages to create the table of normal Debian packages. |
---|
[22764] | 455 | |
---|
[22787] | 456 | 3. Run dasource to create subdirs and source packages for each |
---|
| 457 | normal Debian package. |
---|
[22764] | 458 | |
---|
| 459 | 4. Create checkouts of the meta and third directories: |
---|
| 460 | |
---|
| 461 | svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/meta |
---|
| 462 | svn co svn+ssh://svn.mit.edu/athena/trunk/debathena/third |
---|
[23084] | 463 | chmod 777 meta/* |
---|
[22764] | 464 | |
---|
[23084] | 465 | A couple of subdirectories of debathena/meta are normal Debian |
---|
[22764] | 466 | packages, so this will create redundant copies of those. Ignore |
---|
[23084] | 467 | them; they won't be used. |
---|
[22957] | 468 | |
---|
| 469 | Release engineer: Adding a new suite |
---|
| 470 | ------------------------------------ |
---|
| 471 | |
---|
| 472 | This process is rarely performed and the infrastructure for it is |
---|
| 473 | imperfect. Substitute the name of the new suite for "newdist" in all |
---|
| 474 | steps below. |
---|
| 475 | |
---|
[22958] | 476 | 1. Make sure the apt repository is up to date with respect to the |
---|
| 477 | source tree for the existing dists. |
---|
| 478 | |
---|
| 479 | 2. Add the new dist to scripts/debian-versions.sh. (It is not |
---|
| 480 | necessary to add the new dist to codes at this point, but it must |
---|
| 481 | be present in the gettag conditional.) |
---|
| 482 | |
---|
| 483 | 3. Create the new distribution in the apt repository's configuration |
---|
[22957] | 484 | file. Create the skeleton of the dist by installing at least one |
---|
| 485 | equivs package from meta/ with "reprepro -Vb $DEBATHENA_APT |
---|
| 486 | include newdistname file.changes". |
---|
| 487 | |
---|
[22958] | 488 | 4. On the build server, create a chroot for the new distribution as |
---|
[22957] | 489 | documented above. This may require downloading and installing a |
---|
| 490 | more recent version of the debootstrap package from the |
---|
| 491 | -backports dist corresponding to the build server's OS. |
---|
| 492 | |
---|
[22958] | 493 | 5. Set the DEBATHENA_BUILD_AREA environment variable to point to the |
---|
[22957] | 494 | build area. |
---|
| 495 | |
---|
[22958] | 496 | 6. Fire up screen. |
---|
[22957] | 497 | |
---|
[22958] | 498 | 7. mkdir $DEBATHENA_BUILD_AREA/stamps.newdist. |
---|
[22957] | 499 | |
---|
[22958] | 500 | 8. cd into a checkout of debathena/scripts/build-server/build-all. |
---|
[22957] | 501 | |
---|
[22958] | 502 | 9. Edit Makefile (and check in the edit) so that suite is the new |
---|
[22957] | 503 | distribution and psuite is the previously most recent Debian or |
---|
| 504 | Ubuntu distribution. |
---|
| 505 | |
---|
[22958] | 506 | 10. Run "make deps.mk". |
---|
[22957] | 507 | |
---|
[22958] | 508 | 11. Run "make -k all STAMPS=$DEBATHENA_BUILD_AREA/stamps.newdist". |
---|
| 509 | You can watch the builds happen in the other windows of the |
---|
| 510 | screen session. It's possible to do several builds at once with |
---|
| 511 | make -j N. |
---|
[22957] | 512 | |
---|
[22958] | 513 | 12. debathenify packages will fail out; they must be built by hand. |
---|
[22957] | 514 | When the build fails on one, cd into third/packagename in the |
---|
| 515 | build area and run "./debathenify newdist-amd64 -A source binary |
---|
| 516 | upload" and "./debathenify newdist-i386 binary upload". Then |
---|
| 517 | touch $DEBATHENA_BUILD_AREA/stamps.newdist/packagename.done" and |
---|
| 518 | restart the build. |
---|
| 519 | |
---|
[23084] | 520 | The newly created third/openafs/meta directories should be |
---|
| 521 | chmodded 777 to work around a perl/AFS permissions issue. |
---|
[23077] | 522 | |
---|
[23094] | 523 | Release engineer: Maintaining autodebathenify |
---|
| 524 | --------------------------------------------- |
---|
| 525 | |
---|
| 526 | autodebathenify is a cron job which runs hourly on the builder account |
---|
| 527 | and keeps the OpenAFS modules and modified packages under |
---|
| 528 | debathena/third up to date when there are upstream changes. It relies |
---|
| 529 | on the following: |
---|
| 530 | |
---|
| 531 | * builder@ATHENA.MIT.EDU has access to a local account named |
---|
| 532 | debuildsvn on svn.mit.edu with read access to the athena |
---|
| 533 | repository. |
---|
| 534 | |
---|
| 535 | * builder@ATHENA.MIT.EDU has write access to the canonical build |
---|
| 536 | area and apt repository in the dev cell. |
---|
| 537 | |
---|
| 538 | * The builder account on the build machine has a copy of the signing |
---|
| 539 | private key for the apt repository. |
---|
| 540 | |
---|
| 541 | The cron job will silently exit if it detects that it is already |
---|
| 542 | running. If it fails for any other reason, it will send mail to |
---|
| 543 | source-wash@mit.edu and touch a file named autodebathenify.suppress |
---|
| 544 | which will prevent it from running again until manual intervention. |
---|
| 545 | It is important to get autodebathenify running smoothly or the apt |
---|
| 546 | repository will become out of date with respect to third-party |
---|
| 547 | packages, which in turn will compromise the user experience. |
---|
| 548 | |
---|
| 549 | Even if you don't notice failure mail, it's good to check on the |
---|
| 550 | status of autodebathenify from time to time. To do this, get tickets |
---|
| 551 | as builder and ssh to linux-build-10 as builder. Run "ls -l logs" to |
---|
| 552 | see what's been going on. A logfile with size around 48K indicates |
---|
| 553 | that autodebathenify ran normally but didn't find any new work to do. |
---|
| 554 | A logfile with size around 60 means that autodebathenify.suppress |
---|
| 555 | exists and the cron job has stopped running. A longer logfile |
---|
| 556 | indicates that autodebathenify attempted to build and upload a |
---|
| 557 | package. |
---|
| 558 | |
---|
| 559 | Possible failure cases include: |
---|
| 560 | |
---|
| 561 | * One of the upstream apt repositories timed out during an "apt-get |
---|
[23119] | 562 | update" of an sbuild chroot. If this happens, you can just remove |
---|
| 563 | the autodebathenify.suppress file to get the cron job to start |
---|
| 564 | running again. |
---|
[23094] | 565 | |
---|
| 566 | * autodebathenify tried to build an upstream version which already |
---|
| 567 | exists. If this happens, the build will fail at upload time. If |
---|
| 568 | this happens, there is a bug in the part of the debathenify-* |
---|
| 569 | script which checks whether our apt repository already has the |
---|
| 570 | expected version of the built package. |
---|
| 571 | |
---|
| 572 | * Debian and Ubuntu have .orig files with the same name but |
---|
| 573 | different contexts for a particular third-party package. This has |
---|
| 574 | only happened once, with bash (which is currently not configured |
---|
| 575 | into autodebathenify for that reason). We do not have a general |
---|
| 576 | mechanism for resolving such issues at this time. |
---|
| 577 | |
---|
| 578 | * The new upstream version of the package is sufficiently different |
---|
| 579 | to cause our modifications to fail to apply, and our debathenify-* |
---|
| 580 | script needs to be adapted appropriately. |
---|
| 581 | |
---|
| 582 | The cron job is run out of builder's home directory, but its canonical |
---|
| 583 | source location is in debathena/scripts/build-server in this |
---|
| 584 | repository, should it need to be modified. |
---|
| 585 | |
---|
[23077] | 586 | Release engineer: apt repository HTTP server setup |
---|
| 587 | -------------------------------------------------- |
---|
| 588 | |
---|
| 589 | The apt repository server (athena10.mit.edu) is an ops virtual image |
---|
| 590 | with httpd and AFS installed. At the moment, the only customization |
---|
| 591 | is two changes to /etc/httpd/conf/httpd.conf: |
---|
| 592 | |
---|
| 593 | * The DocumentRoot is set to "/afs/dev.mit.edu/system/athena10". |
---|
| 594 | * The Directory entry for /var/html/www is also changed to |
---|
| 595 | "/afs/dev.mit.edu/system/athena10". |
---|