source: trunk/doc/procedures @ 18525

This file contains a description of the process developers should go
through to get changes into the source tree.  Although it discusses
the use of CVS, it is not a CVS tutorial; read the CVS info pages
(available in M-x info in emacs on Athena) for a general introduction
to CVS.  Areas covered in this file are:

  Checking out a working directory
  Preparing changes for review
  Reviewing changes
  Early checkins
  Third-party sources

You should use cvs from the gnu locker with the source repository.
People without write access to the repository can use "cvs -u" (a
local modification to CVS) to access the repository without making
read locks.  If you do not have write access to the repository and you
want to submit a change, follow the guidelines below up and including
sending mail to source-reviewers, and note in your mail that your
reviewer should check in the change because you cannot do so.

Checking out a working directory
--------------------------------

Set CVSROOT to "/afs/dev.mit.edu/source/repository" before trying to
check out a working directory.

The entire source tree is very large.  You can check it out with "cvs
co -P all", but in almost all cases this would be a big waste of space.
Simply check out a subdirectory of the source tree with a command like
"cvs co -P athena/bin/olc".

CVS knows nothing about AFS permissions, so all directories created
will have the same permissions as their parent.  It is generally
safest to do your checkouts in a private area of the filesystem.

You should use the -P option for checkout because the source tree
contains some historical directories (now empty) which will conflict
with builds.

Preparing changes for review
----------------------------

Changes to the doc hierarchy do not typically need to be reviewed;
notification is typically good enough, since no software will break as
a result of changes to the source tree documentation.

For changes to other parts of the tree, you should perform the
following steps while preparing your changes for review:

  1. Do a "cvs update" in your working directory to merge in changes
     other people may have made.  (You can do "cvs -n update" if you
     want to see what needs to be merged in without actually doing the
     merge.)

  2. Be sure to test your changes.

  3. Make sure your changes are made in reviewable chunks to the
     greatest extent possible.  If you have many changes to make of
     several different types, prepare one patch for each type of
     change; in particular, if you have some cosmetic changes to make
     and some functional fixes to make, submit them as two different
     patches if they add up to a significant number of changes.  This
     requirement creates more work for the submitter, but it greatly
     increases the effectiveness of the review process.

  4. Use "cvs diff -u -N" piped to a file to prepare your changes.  (Do
     not cut and paste diffs from an xterm; your tabs will be
     converted to spaces.)  If your change involves reindentation of
     code, you may want to also use the "-w" flag to diff.  If you
     find that your change is clearer when presented as a context diff
     ("-c" instead of "-u"), feel free to submit it that way.

  5. Look over your diffs.  Make sure you haven't been sloppy about
     spacing, punctuation, and naming, and that you have tried to
     conform to the guidelines in the file "standards" in this
     directory

  6. Send your diffs, along with a clear description of the change you
     are making, to source-reviewers@mit.edu.  If the diffs are very
     large (more than 50K), put the changes somewhere world-readable
     (unless the source code in question is restricted) and mail a
     pointer.

  7. If you do not have write access to the source tree and submitted
     your diff using the -w flag, submit it again without the -w flag
     so that the full patch can be checked in by someone with write
     access.

Ideally, at least one person will respond to your mail within a day or
two, either expressing concerns or signing onto your change.  You
should wait at least one day for people to voice their objections.  If
you receive objections or requests for further information from staff
members, you must either satisfy those concerns or resolve the issue
with the release team before committing your change.  If after one
day, you have received no objections and someone has signed onto your
change, you may commit your change.  You may also commit your change
if no one objects within five days, even if no one has signed onto it.

When you check in your change, be sure to include a clear log message.
Explain why you are making the change you are making if it's not
obvious.

Reviewing changes
-----------------

Sometimes you can review a change by looking at the patch.  Other
times you will want to check out a tree and apply the patch, with
"patch -E -p < message-file" if you have the mail message in a file,
or "dsgrep -p -t trn-number source-reviewers | patch -E -p" if what
you have is a transaction number in the source-reviewers discuss
meeting.

When reviewing a change, be sure to make your position on the change
clear.  Say "I object to this change" if you are not merely voicing a
concern, or "I would like these questions answered before this change
is committed" if you have asked questions and are not merely curious.
When your objections are responded to, you should in turn respond in a
timely fashion saying whether your objections have been satisfied or
not.  If the dispute appears intractable, say so, so that the issue
may be brought up before the release team.

If you have reviewed a change carefully and have found nothing wrong
with it, and no one else has responded to the change, you should sign
onto the change rather than remaining silent.  You are encouraged to
try out changes before signing onto them, but in some cases the
inconvenience outweights the benefit of this consideration.

Early checkins
--------------

In some cases it may be appropriate to check in a change in advance of
the normal review period.  The following should be true of those
cases:

  1. The change is obvious and noncontroversial, such as a fix for a
     syntax error.

  2. The problem being fixed is causing an immediate difficulty,
     usually "I'm doing a build of /mit/source and it blows out at
     this point."

The change should still be sent to source-reviewers with a note about
the early checkin.  If the immediate difficulty is "the wash is broken
and I want the next wash to work," then it is good to get a positive
review of the change before checking it in.  Close to a release cycle,
though, that can be ignored.

Third-party sources
-------------------

For modules in the third hierarchy, we generally use the "cvs import"
command to track development from outside.  (To find out if this
applies to a given module, to a "cvs log" of a file in the tree; if
you see a revision 1.1.1.1, then we're using cvs import.)  Do not
check in a new outside version of a third-party package onto the
mainline if it was originally imported with cvs import; it's very
difficult to recover from that particular mistake.  Do, however, check
local changes you made yourself onto the mainline.  Always refer to
doc/third-party before doing an import to see if there are any special
notes on the module you are importing.

Generally, you should only import "clean" third-party source trees
with no modifications.  If you absolutely need to make changes to the
source tree before importing it (check with a release engineer before
deciding that you have to), make a note in the doc/third-party file so
that people doing future imports will know about it.

Before doing an import, run timestamps.pl from the repository CVSROOT
directory.  This script helps work around the problem that CVS does
not version metadata such as timestamps, by crunching all of the
timestamps up near the current time while preserving their relative
order.  Use the "-d" flag to cvs import to use the resulting
timestamps as the time of import.

Also before doing an import, remove any .cvsignore files which might
be present in the tree.  We are builders, not developers, of
third-party software, and want to track the entire distribution
including any auto-generated files.

It is not uncommon for packages to include files and directories which
match the CVS default ignores list (directories named "core",
filenames ending in ".old", etc.).  Use the "-I \!" option to cvs
import to avoid losing files.

Here is an example of how one might import a hypothetical new version
of emacs (assuming the CVSROOT environment variable is set to point at
the Athena repository):

  gtar xzf emacs-22.3.tar.gz
  cd emacs-22.3
  find . -name .cvsignore -print | xargs rm
  perl $CVSROOT/CVSROOT/timestamps.pl
  cvs import -I \! -d -m "Import emacs 22.3." third/emacs vendor emacs-22_3

Make sure to use the literal word "vendor" for the vendor branch name.

After importing, you should take care of files which have been removed
from one version to the next, since CVS doesn't do a good job.  Here
is an example:

  cd /tmp
  cvs co -r vendor third/emacs
  cd third/emacs
  cvs update -j emacs-22_2 -j emacs-22_3
  cvs ci -m "Not present in emacs 22.3."

It is important to remove such files on the vendor branch, so that
diffs against the vendor branch will work properly.

Now you can do the merge.  Here is an example of how you might do one:

  cd /tmp
  rm -rf third/emacs
  cvs co -kk third/emacs
  cd third/emacs
  cvs update -kk -j emacs-22_2 -j emacs-22_3
  # Resolve conflicts.
  cvs ci -m "Merge with emacs 22.3."

Using a "-kk" working directory and merge will prevent merge conflicts
based on RCS IDs; it is unnecessary for a package which does not use
RCS keyword strings.  If the merge results in any files being deleted,
check those files (with "cvs log") to make sure they were in fact
modified on the mainline.  If they weren't, then some CVS oddity is
involved.  Try using "cvs admin -bvendor filename" and then "cvs
update filename" to make the file go away more naturally.

If you add a new piece of third-party software or import a new
version, you should look over doc/third-party and see if any notes
should be added or modified.  This file is instrumental in locating
new versions of software.

Here are some things to pay attention to when adding or updating a
piece of third-party software:

  * If the package's build system does not use autoconf, you will
    probably need to write a Makefile.athena file telling the build
    system how to build it.

  * If the package's build system does use autoconf, you may need to
    write a configure.athena giving special options to pass to the
    configure script.

  * Most packages will need to be taught how to use DESTDIR.  Make
    sure that DESTDIR references don't make it into the installed
    program.

  * If the package installs a file setuid, it needs to specify the
    owner (probably "-o root" if it didn't specify one before).
    Likewise, a setgid program needs a specified group owner, although
    this is usually done already.  Other than that, our fix_owners
    program will coerce unspecified owners and groups to 0.

  * The package should create directories before installing files in
    them.

  * Test your package's build and install.  Preferrably, use the "do"
    command, something like:

      do prepare
      do clean
      do
      do check
      do -d /var/tmp/inst install

    (Replace "do" with "sh /mit/source/packs/build/do.sh" or use a
    shell alias.)  If the package relies on libraries, you can use "-c
    -d /afs/dev.mit.edu/system/<sysname>/srvd-current" in the
    non-install steps to point at them.