Summary

The specification describes an enhancement to keep automated versioned history of files under /etc.

Rationale

Over time, configuration changes occur and it is easy to forget why changes were made, or what was in a config file prior to the change. It becomes even more difficult when there are multiple administrators working on a machine over a period of time.

Use cases

  1. Andrew is a systems administrator and his mail server has stopped working. Richard, another administrator, made some changes a week ago but has now gone on holiday to his residence in France. Andrew is unsure what changes Richard made to the mail configuration, but he can use the automated version history to look at the changes made before Richard went away.

  2. Karl has a box which was working and set up the way he likes. After a failed dist-upgrade his system is left in a half configured state. He would find it benefical to be able to roll back to just before the upgrade.

  3. Igor has a few servers upon which he has made identical bespoke modifications to /etc. He would like to be able to store these modifications centrally and then distribute them to his servers when he changes them.

Scope

Design

Implementation

So far I have tried out the following;

in /etc/; 

bzr init
bzr add *
bzr commit -m "Initial Import"

then I added the following to roots crontab; 

0 * * * * cd /etc; bzr commit -v -m "Automated update at `date`" >& /dev/null

Code

Some proof of concept code is avaialable via bzr from http://www.cs1ajb.staff.shef.ac.uk/bzr/auto-bzr/. You can download it using bzr using 

bzr branch http://www.cs1ajb.staff.shef.ac.uk/bzr/auto-bzr/

Data preservation and migration

Outstanding issues

BoF agenda and discussion

Comments

AchimBohnet:

AndrewBeresford:

PaulSchulz:

Eg. In /etc/, using /var/git/etc.git to hold the repository. 

mkdir /var/git
GIT_DIR=/var/git/etc.git git init-db
GIT_DIR=/var/git/etc.git git add .
GIT_DIR=/var/git/etc.git git commit -m "Initial Import"

Huerlisi (Simon Huerlimann):

StefanMichalowski:

TobiasHunger: Are you aware of etckeeper? That is a set of scripts that does exactly what you are describing. The debs are severly outdated in ubuntu thought.

What does etckeeper do? It integrates into apts dpkg pre and post hooks. That is fine for me as hardly ever use dpkg manually anyway. It does version file permissions, users and groups, etc. by using metastore (a tool which exports the additional data into a file which is then put under version control). Currently it does support mercurial and git for its backend, but it should be trivial to add your favourite VCS as a backend.

I've just hacked up a bzr backend for etckeeper - see http://gitweb.samba.org/?p=jelmer/etckeeper.git;a=summary -- JelmerVernooij

etckeeper, with bzr support, is available in Ubuntu Hardy -- DanielHahler

Comments moved from ServerCandy specification

Merge updated package configuration

Does this advance also include other improvements over Debian's conffile handling? Right now, it only guarantees two versions (the old installed version and and the new package version) are available to examine, and leaves many openings for screw-ups in configuration. A better way would seem to be a three-way merge from the packaged version that the running version in /etc descended from. Basically, I see some nice ideas about keeping better track of /etc, that a lot of sysadmins do on their own, but nothing about integration with the package management system, which is where the real opportunity for advance lies

Decision to choose bzr

What was the reason to choose bzr above other source control systems?

Was the main reason that bzr has the nice feature that there's only a single .bzr in the top level directory?

I suggest that the Svk project be considered, as it has the same feature. However, svk sits above Subversion, which has proved itself as a reliable source control system and many server admins are familair with it.

It seems a little odd to trust a relatively brand new source control system on an OS that's going to be supported for 5 years. I would rather see something that people are familiar with, even if it doesn't have all of bzr's sexy features.

I see several reasons to choose bzr over svk:

bzr requirements

(We should probably create a separate spec for the bzr implications)

I'd like to clarify a bit more just what the use cases for this are. Is it just to keep a record of previous configurations and to allow admins to manually diff and roll back?

I think the bzr side of this is practical for dapper -- indeed I might start tracking my /etc in bzr just to see how it goes.

bzr 0.6.2 handles versioning of files, directories and symlinks. On my breezy machine there are no other file types in /etc; if we wanted to handle sockets or similar things in there that would need to be fixed. (But it is rather unclear to me how we could usefully version them; presumably the program that owns them would recreate them and anyhow they should probably be in /var or /dev.)

The main thing that would need to be fixed is to track permissions and ownership. This should not be too hard to do but will require some new code.

At the moment bzr assumes that working copy files are writable - if you try to update a file that's readonly you will get an error. This is probably a feature in typical use but one *might* want to overwrite it for versioning of /etc. This is probably OK if the admin is going to make any reversions manually.

Hardlinked working files would probably also cause confusion but I don't see any of them in /etc either.

Also see http://joey.kitenet.net/code/etckeeper/ - a collection of tools to let /etc be stored in a git, mercurial, darcs, or bzr repository. It hooks into apt (and other package managers including yum and pacman-g2) to automatically commit changes made to /etc during package upgrades. It tracks file metadata that revison control systems do not normally support, but that is important for /etc, such as the permissions of /etc/shadow. It's quite modular and configurable, while also being simple to use if you understand the basics of working with revision control. etckeeper is available in git at git://git.kitenet.net/etckeeper, or in gitweb. It's packaged in Debian, Unbuntu, Fedora, etc.

CategorySpec

VersionControlledEtc (last edited 2010-09-23 04:17:25 by dhcp198-158)