wiki:WorkingWithTheRepository

Working With The Repository

Background

The "central" Athena repository is located on git.mit.edu (aka drugstore), accessible via git+ssh. You must be on the athena-commiters [yes, the typo is deliberate] list in order to read or write to any of the Athena repositories on this server.

Our git repository is automatically mirrored to the mit-athena organization on Github, and anyone without drugstore access should clone from there.

NOTE:

  • This document will use the canonical paths. In general, you can replace git+ssh://git.mit.edu/git/athena with https://github.com/mit-athena in a repository URL.
  • This document assumes that the remote named origin is the repository on git.mit.edu. This is the default behavior if you follow the instructions here.

Submodules

Because of the vast number of repositories, we use submodules. The debathena repository is the parent repository, and its .gitmodules file contains all the submodule information. If you're not familiar with submodules, please check out  https://git-scm.com/book/en/v2/Git-Tools-Submodules

Submodule configuration is stored in the .gitmodules file, which is itself under version control. Submodules know both their path, and the last SHA1 they were updated with. In theory, we would update the central repository every time we make a change to a submodule, so it always points to the submodule's HEAD, or at least a reasonable tag. However, we don't. It is therefore vital that you ensure your submodule is up to date and that you're on master before working on it, lest you find yourself attempt to commit to a detached HEAD.

Repository Layout

At the top level of the repository are 4 sub-directories:

  • athena - This typically contains software we write. These are "non-native" packages (i.e. packages that consist of more than just a debian/ directory and some configuration files). Basically, if something is of value on distributions other than Ubuntu and Debian, it should go in here.
  • debathena - This typically contains "native" packages (i.e. packages which are "native" to Ubuntu and Debian, and which would not make sense on other distributions)
  • misc - Things needed as part of the Debathena build infrastructure.
    • build-system - The "new" build system components
    • debathena-common - Along with manual-config (below), used for manual-config packages
    • installer - The installer, both the PXE versions and the script
    • manual-config - The "manual-config" packages. These have the same names as debathena-config packages, but only provide dependencies, and are otherwise empty. Used by a few users so that, for example, they can install something that depends on debathena-printing-config without actually configuring printing.
    • scripts - Various scripts, and the "old" build system components
  • third - A few packages that we build, but for which we are not the upstream source. These count as "non-native" packages.

Initial Repository Cloning

If you're only working on a single project, you can of course clone that project as you would any other Git repository. However, core developers should clone all the submodules at once for simplicity. This is about 150MB or so.

$ git clone git+ssh://git.mit.edu/git/athena/debathena athena-source-tree
$ cd athena-source-tree
$ git submodule init
$ git submodule update

(You can of course replace athena-source-tree with whatever you'd like your local clone to be called.)

This will checkout specific SHA1s for each submodule (specifically, whatever the SHA1 was when it was first added as a submodule), but *will not switch to the master branch*.

Working on a Package

In the example below, we will prepare to work on the getcluster package:

jdreed@infinite-loop:~/src/athena-source-tree$ cd athena/getcluster
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git status
HEAD detached at 3f5c288
nothing to commit, working directory clean

As you can see, this is a deatched HEAD, because the getcluster submodule was at 3f5c288 when initially added to the parent repository. So to work on it, we should ensure we're up to date:

jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git checkout master
Previous HEAD position was 3f5c288... Revert unintended changelog changes from r25887
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git pull
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From git+ssh://git.mit.edu/git/athena/getcluster
   db5520e..772d62f  pristine-tar -> origin/pristine-tar
 * [new tag]         10.2.2-0debathena2 -> 10.2.2-0debathena2
Already up-to-date.

We checked out the master branch, and then did a git pull. There were no changes to fetch, but there was a new branch (pristine-tar) and a new tag.

Branches

Native packages have a single master branch.

Non-native packages have two branches: debian and master. The master branch contains the software itself, but no debian directory. The debian branch contains the software plus a debian/ directory.

There is also a pristine-tar branch, where "pristine" upstream tarballs are stored as part of the build process. You can check it out remotely, but you basically never want to be working on it. Continuing from the getcluster example above:

jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ ls
getcluster  getcluster.1  setup.py  tests.sh
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git branch
  debian
* master
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git checkout origin/pristine-tar 
Note: checking out 'origin/pristine-tar'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 772d62f... pristine-tar data for getcluster-10.2.2.tar.gz
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ ls
getcluster-10.2.1.tar.gz.delta  getcluster-10.2.1.tar.gz.id  getcluster-10.2.2.tar.gz.delta  getcluster-10.2.2.tar.gz.id
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ git checkout master
Previous HEAD position was 772d62f... pristine-tar data for getcluster-10.2.2.tar.gz
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
jdreed@infinite-loop:~/src/athena-source-tree/athena/getcluster$ ls
getcluster  getcluster.1  setup.py  tests.sh