Summary

Keeping /etc under a revision control system allows to maintain traceability over changes made to configuration files. This should be made easy enough and non-intrusive enough so as to make a good candidate for integration in default Server installs, as part of our "Best Practices integration" theme for Ubuntu Server.

Release Note

Ubuntu Server ships by default with a new traceability feature, built on etckeeper and bzr, integrated together using a specific bzr plugin. This transparently allows to keep an history of configuration changes made to your system, whether those changes are made by package updates or manually.

Rationale

Applying a revision control paradigm to configuration files has been a known best practice for Linux sysadmins for a long time, however everyone uses different mechanisms to achieve that result. It is often seen as an intrusive and cumbersome process that might not be worth the hassle of setting it up and sticking to it.

One of the themes we are pushing in Ubuntu Server is integration of industry best practices : make powerful concepts and tools easy enough to use so that we can include them by default and make them part of what "Ubuntu Server" is.

We can solve the perceived drawbacks by integrating a comprehensive revision control system that hooks into the system to provide a comprehensive view on the package installed and the configuration changes applied during the life of the server. It should maintain, even if the sysadmin doesn't commit his configuration changes, a usable history. If successful, it can be made a default feature of Ubuntu Server, which you could even ignore until you need it.

Use Cases

Design

Goals

Proposal

etckeeper, written by Joey Hess and already in universe, provides most of the needed features. It is written in shell script and is easily extensible. It features:

Unfortunately, at this point etckeeper doesn't integrate the ownership/permissions change in the output and result of the VCS tool: differences in ownership/permissions are shown as a diff in the .etckeeper file, and revert operations must be followed by the appropriate manual ownership/permissions restoration.

It can therefore be a little tricky to use for less experienced users, which are the target of this blueprint. To solve this issue, we can leverage bzr extensibility to provide an improved bzr "etckeeper" plugin that will:

We also need to ensure install-time initialization of etckeeper: running "apt-get install etckeeper" should give us a working system (rather than requiring post configuration actions).

Implementation

etckeeper core improvements

The following new features and bugfixes should be added to etckeeper:

Other interesting new features could be:

bzr etckeeper plugin improvements

The plugin should support the following new features:

bzr improvements

Hooks

Small changes to bzr core could help a lot in implementing this. In particular, bzr is currently missing the right plugin hooks to make the bzr-etckeeper plugin painless.

Example: add a hook for new files created during a tree transform so that we can plug the right ownership/permissions.

Dependencies

bzr currently pulls (through "recommends") things that we probably don't want on a server: bzr recommends bzrtools, which recommends graphviz, which depends on a number of X libraries.

To make this a suitable server target, we might need to clean up some of these dependencies.

Was filed as LP:#321852

Targets

Jaunty

Target for the Jaunty cycle would be to provide this feature as an option that can be enabled, simply by running a single install command.

Jaunty+1

For the next development cycle, enabling this feature by default for all server installs would be considered. This leaves enough time to iron out the remaining issues before the next LTS.

Test/Demo Plan

(This need not be added or completed until the specification is nearing beta.)

Unresolved issues

(This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.)

BoF agenda and discussion

A parallel blueprint was written in the past: https://wiki.ubuntu.com/VersionControlledEtc -- ThierryCarrez

An alternative to implementing permissions/ownership into bzr is to implement a framework for storing generic metadata into bzr repository, with hooks to set/get the metadata. After discussion with the bzr team, the advantages over the separate repository + wrapper approach (option 2) are not obvious. By allowing direct bzr access to metadata you run the risk of unsyncing the files and their metadata. -- ThierryCarrez

Storing pristine and local states on DVCS branches

A crucial feature for a DVCS-based /etc is to store local changes in a separate branch from the packaged pristine state. This lets the admin do a VCS diff operation to figure out how their current state differs from the default. Furthermore, when a package upgrade changes the default config, the local changes can be merged in using a VCS merge operation.

This would be a huge improvement over the current situation where dpkg basically forces you to choose between the new pristine config or the old locally-modified config, and merge any differences manually with no information about which differences are which (a 2-way merge instead of a 3-way merge). -- AndersKaseorg

Interesting feature, which IIUC depends on dpkg to be able to commit original configuration files to a "pristine" branch. It is on the etckeeper roadmap. -- ThierryCarrez

Other design options

Option 1: Using etckeeper

etckeeper provides the following commands (with bzr):

It would need to be improved/completed in the following areas:

Implementation would provide a metapackage that depends on etckeeper and provides the following extra features:

Drawbacks:

Option 2: A bzr wrapper ("Captain's Log")

We might prefer to provide a simpler tool, less likely to break your system, and that doesn't require prior VCS knowledge. This new Python tool (which I call "Captain's log") wraps around bzrlib and tightly integrates an ownership/permissions database into the output. By exposing a limited subset of VCS functionality, it makes sure you don't enter disruptive VCS commands. It would provide the following features:

The wrapper approach allows more advanced features like:

Future extensions could go beyond the "easy traceability" goal to reach more branching/sharing features: this would be done by proxying more bzr commands.

Drawbacks:

Option 3: Use an enhanced bzr natively

To avoid the risky direct bzr commands on the repository, the bzr wrappers in Option 2 (and 2.1) prevent direct usage of bzr over the repository, but that restricts what you can do with the tool. So another option is to implement permissions/ownership directly into bzr, to make sure all options support it in the same way they already support the "x" bit. This feature would make bzr the clear VCS choice for putting anything other than sources under revision control.

Implementation would then just provide:

Drawbacks:

See bug 67589 for an existing bzr feature request about this.

Option 4: A wrapper (or enhanced etckeeper) around an enhanced bzr

This option takes the best of options 2 [.1] and 3. It provides a specific interface for the less-experienced user while still allowing direct usage of a permissions-aware bzr.


CategorySpec

EtcUnderRevisionControlSpec (last edited 2009-01-28 15:16:56 by ttx)