Ticket #97 (closed enhancement: fixed)

Opened 13 years ago

Last modified 12 years ago

Use unionfs instead of LVM snapshots

Reported by: ghudson Owned by: geofft
Priority: normal Milestone: Karmic Deploy (Canceled)
Component: login chroot Keywords:
Cc: Fixed in version:
Upstream bug:

Description

Right now we use LVM snapshots for copy-on-write functionality in two places: on the build server (linux-build-10) and for login chroots. This works pretty well, but it works at the storage layer, which is a somewhat heavyweight approach.

It is also possible to do this at the filesystem layer using unionfs. I do not know a lot about unionfs at this time, but tabbott reports that this approach performs better for large numbers of snapshots.

Unfortunately, schroot does not have support for unionfs at this time. There is a patch at:

 http://lists.alioth.debian.org/pipermail/buildd-tools-devel/2008-January/001964.html

tabbott has a port of it to schroot 1.2.1 if we want to pursue this.

Change History

comment:1 Changed 12 years ago by jdreed

  • Component set to login chroot
  • Milestone set to The Distant Future

comment:2 Changed 12 years ago by jdreed

Is unionfs support better in Jaunty and/or Karmic? If so, we should start looking at this again. Between this and #321, it would be awesome if cluster logins were faster.

comment:3 Changed 12 years ago by broder

I can't keep track of all this filesystem stuff. Isn't aufs the new hotness, though?

comment:4 follow-up: ↓ 7 Changed 12 years ago by tabbott

aufs seems to be the preferred choice. When I last looked at it, unionfs-fuse didn't work (e.g. you could not delete a directory).

Also, Unionfs support was merged into schroot master recently, and will go into a 1.3.0 release in the near future (they did an -rc1 a couple days ago).

comment:5 Changed 12 years ago by geofft

  • Priority changed from minor to major
  • Milestone changed from The Distant Future to IAP 2010

schroot aufs support appears to be mostly stable; we should hack up reactivate to use schroot 1.3.0rc1 (or the latest version in git) and aufs, and make sure it works reaonably so we're ready when 1.3.0 is released.

Since LVM's slowness is probably Debathena's most annoying bug, especially on quickstations, I'm increasing the priority.

Incidentally, I have a feature-ish request into schroot that affects aufs use with multiple users:
 http://lists.alioth.debian.org/pipermail/buildd-tools-devel/2009-August/003622.html
Fixing this would let us do something crazy like a dialup with aufs chroots, but we don't need to care about multiple simultaneous users for clusters.

comment:6 Changed 12 years ago by broder

If someone works on this, they need to keep in mind that (a)ufs can not be used as a simple drop-in replacement for the LVM snapshots. In particular, the auto-update mechanism only works currently because of the assumption that we can modify the master filesystem without it affecting the snapshot.

comment:7 in reply to: ↑ 4 Changed 12 years ago by geofft

  • Status changed from new to accepted
  • Owner set to geofft

I've started working on this in r24125; I've tested this on lola-granola and it works spiffily. It does require a couple of changes to the schroot package -- two to the build system, one to schroot itself to permit union mounts of / -- that I plan on sending upstream. There's also a fair amount of cleanup possible in that code before we send it to production.

comment:8 Changed 12 years ago by geofft

  • Status changed from accepted to development

I've finished up most of what I want this code to do, and uploaded reactivate 2 and schroot 1.3 to -development (and the auto-update that it depends on to -proposed). I've also upgraded next5 and next6.

The one thing we should consider doing at some point before pushing this to production is making use of the remaining disk space since we're no longer using it for LVM snapshots. One option very much in the spirit of the original partitioning is to make it swap space, and make the tmpfs overlay as big as that partition, so it can swap as necessary.

comment:9 Changed 12 years ago by broder

  • Status changed from development to proposed

I move this to proposed today.

comment:10 Changed 12 years ago by broder

  • Status changed from proposed to development

Just kidding.

I moved this back to development, because schroot was prompting about the schroot.conf changes on opus.

I think what happened was that I uploaded schroot to proposed before uploading reactivate, so a handful of workstations grabbed the schroot update (along with its prompt) before getting the reactivate update that un-frobbed schroot.conf.

I'm not sure if uploading the packages in the reverse order would have the desired effect, but even if it does, that's a bit fragile.

Maybe we want to upload a new version of reactivate 1 that moves the athena chroot into /etc/schroot/chroot.d instead of /etc/schroot/schroot.conf, and give that a few days to propogate.

comment:11 Changed 12 years ago by broder

  • Status changed from development to proposed

Oh right - I moved this back into proposed on Sunday night. Geoffrey assured me that the transition works correctly so long as reactivate 2 is uploaded to the apt repo before schroot

comment:12 Changed 12 years ago by broder

  • Status changed from proposed to closed
  • Resolution set to fixed

Right! I can close tickets.

We moved this to production at 9:59 PM tonight.

Note: See TracTickets for help on using tickets.