Changes between Version 3 and Version 4 of AddingOrRebuildingASuite


Ignore:
Timestamp:
05/08/12 14:47:23 (12 years ago)
Author:
jdreed
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • AddingOrRebuildingASuite

    v3 v4  
    1 This process is rarely performed and the infrastructure for it is 
    2 imperfect.  Substitute the name of the new suite for $suite in all 
    3 steps below. 
     1This covers the process of building for a new suite (version, e.g. "precise").  This process is typically begun 1-2 months before the suite's release date. 
    42 
    5   1. Make sure the apt repository is up to date with respect to the 
    6      source tree for the existing dists. Check 
    7      debathena.mit.edu/package-list/development and 
    8      debathena.mit.edu/package-list/proposed. More importantly, make 
    9      sure there is nothing in svn that isn't built. (An easy way to do 
    10      this is running the check-unbuilt-packages script.) 
     3Substitute the codename of the suite for `$suite` in the directions below where applicable. 
    114 
    12      '''NOTE:''' This is very important, because build-all uses the source files that are already there.  So, let's say a build fails, and you commit a fix.  It's still going to use the old .dsc file, until you build that package normally. 
     5== Bootstrapping a new suite == 
    136 
    14   2. Add the new dist to scripts/debian-versions.sh (and svn up into 
    15      the locker).  (It is not necessary to add the new dist to codes 
    16      at this point, but it must be present in the gettag conditional.) 
    17      If you are building an unreleased distribution, add an additional 
    18      ~0.1 tag; if you're rebilding, bump that tag. 
     71. Add the new distribution to `scripts/debian-versions.sh`.  This is done by adding a new case in the `gettag` function.  Assuming the suite has not yet been released, the value should have an additional `~0.1` tag appended to it.  (The new suite should not be added to the `DEBIAN_CODES` variable until it has been released, as the value of that line determines whether to include the new suite when doing a "normal" build of a single package.) 
    198 
    20   3. Create the new distribution in the apt repository's configuration 
    21      file by editing /mit/debathena/apt/conf/gen-distributions and 
    22      running it with output to the 'distributions' file in the same 
    23      directory.  Create the skeleton of the dist by running 
    24      "dareprepro export" on the build server. (You don't need to do 
    25      this if you're rebuilding.) 
     92. Add an appropriate entry in `scripts/build-server/sources.list.d/{debian|ubuntu}/$suite.list`. 
    2610 
    27   4. Add an appropriate sources.list.d/*/$suite.list entry in 
    28      scripts/build-server. Commit and update 
    29      /mit/debathena/bin/build-server. 
     113. Commit your changes from steps 1 and 2 above. 
    3012 
    31   5. On the build server (as root), create a chroot for the new distribution: 
     134. Update (svn up) /mit/debathena/bin/build-server with the changes you just committed. 
    3214 
    33      First, check if /usr/share/debootstrap/scripts/$suite exists. 
    34      If not, it may require downloading and installing a 
    35      more recent version of the debootstrap package from the 
    36      -backports dist corresponding to the build server's OS. (You 
    37      presumably don't need to do this if you're rebuilding.) 
     155. Create the new distribution in the apt repository's configuration file by editing `/mit/debathena/apt/conf/gen-distributions` (not under version control) and running it with output to the `distributions` file in the same directory (you may wish to first run gen-distributions and sanity-check the output on stdout).  Create the skeleton of the dist by running "dareprepro export" on the build server.  
    3816 
    39      You could also just go into that directory and do 
    40      "ln -s gutsy $suite" since it apparently hasn't changed in years. 
    41  
     176. Log in to the build server as root.  Check if `/usr/share/debootstrap/scripts/$suite exists`.  This may require a more recent version of the debootstrap package, or you can just `ln -s gutsy $suite` since apparently it hasn't changed since Gutsy.  Then create the chroots: 
    4218{{{ 
    4319        /mit/debathena/bin/build-server/make-chroot $suite i386 
    … …  
    4521}}} 
    4622 
    47   6. Fire up screen, ideally making use of the "-S" option to identify 
    48      your screen session.  e.g. "screen -S joeuser.oneiric_build". 
     237. Stop.  Go make sure the apt repository is up to date with respect to the source tree for the existing dists. Check http://debathena.mit.edu/package-list/development and http://debathena.mit.edu/package-list/proposed. More importantly, make sure there is nothing in svn that isn't built. (An easy way to do this is running the check-unbuilt-packages script.) 
    4924 
    50   7. mkdir /mit/debathena/machines/awesome-build-server/stamps.$suite. 
     25     '''NOTE:''' This is very important, because build-all uses the source packages that are already there.  So, let's say a build fails, and you commit a fix.  It's still going to use the old .dsc file, until you build that package normally. 
    5126 
    52   8. cd into a checkout (probably on your local machine) of 
    53      debathena/scripts/build-server/build-all. 
     278. Log out of the build server, and log back in as `builder`. 
    5428 
    55   9. Edit Makefile (and check in the edit) so that suite is the new 
    56      distribution and psuite is the previously most recent Debian or 
    57      Ubuntu distribution.  psuite is used to generate the order of 
    58      build-dependencies, so choose the closest distribution.  If you are 
    59      rebuilding, you can set psuite to suite. 
     299. Start `screen`, ideally making use of the "-S" option to identify your screen session.  e.g. `screen -S joeuser.oneiric_build`. 
    6030 
    61   10. cd to /mit/debathena/bin/build-server/build-all, which is a 
    62       totally logical place to run the build from.  Run "make 
    63       deps.mk". 
     3110. mkdir /mit/debathena/machines/awesome-build-server/stamps.$suite 
    6432 
    65   11. Run "make -k all". 
    66       You can watch the builds happen in the other windows of the 
    67       screen session.  It's possible to do several builds at once with 
    68       make -j N (except as of 7/2011 this is a bad idea) 
     3311. On your local machine, cd into a checkout of `debathena/scripts/build-server/build-all`.  Edit the `Makefile` so that the variable `suite` is the codename of the new suite, and `psuite` is the codename of the most recent Debian or Ubuntu distribution.   
     34 
     3512. Check in your changes from the previous step, and update (svn up) /mit/debathena/bin/build-server/build-all 
     36 
     3713. Back in the screen session on the build server, cd to `/mit/debathena/bin/build-server/build-all` and run `make deps.mk` 
     38 
     3914. Run `make -k all`.  This begins the build.  The actual builds happen in window 1 of screen.  It may or may not be possible or a good idea to run several builds at once with `make -j <N> all`. 
     40 
     41Now, the build will almost certainly fail at some point, because of upstream changes, or incompatibilities, or whatever.  When that happens, you will be prompted (in window 1) to acknowledge the error by pressing <Enter>.  Make a note of the package that failed, and the failure.  Let the build continue as much as it can.   
     42 
     43When you go back to fix changes, you need to not only check the fixes in to svn, but you also need to build that package normally and upload it (ideally to -development), because: 
     44a) you want to avoid version skew between suites 
     45b) the build process uses the latest source package in the repo. 
     46 
     47Once you have fixed, built, and uploaded the new package, you can restart the build with `make -k all`.  Lather, rinse, repeat until everything is built. 
     48 
     49== Re-building a suite == 
     50 
     51This procedure is for a full rebuild of a suite, not picking up where a failed build left off. 
     52 
     531. Go-back and re-read the part about the apt repository being up to date in step 7 above. 
     54 
     552. Edit `scripts/debian-versions.sh` and increment the tag for the suite by 0.1.  If you are doing the final build for a released suite, remove the tag entirely. 
     56 
     573. Edit  `debathena/scripts/build-server/build-all/Makefile` and set `psuite` to be the same as `suite`, since you're re-building. 
     58 
     594. Check in the changes from above and svn up in the locker. 
     60 
     615. Either delete `/mit/debathena/machines/awesome-build-server/stamps.$suite` (from the last build) or move it out of the way. 
     62 
     636. Log in to the build server, start up screen, using the "-S" option to identify your screen session with your username. 
     64 
     657. cd /mit/debathena/bin/build-server/build-all and run `make deps.mk`. 
     66 
     678. Run `make -k all`.  See the notes above about what you need to do when fixing a package.