systemd - An alternative boot manager

systemd is a system and session manager for Linux, compatible with SysV and LSB init scripts. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux cgroups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic.

See the systemd home page for further information.

Warning! Experimental code

systemd is under active development. Until now (1/2012) it has been shipped in Fedora 15+16, OpenSUSE 12.1, Mageia 2 (Alpha).

Current versions of systemd require that you customize your kernel as well as running a non-standard init process. These are pretty intrusive changes.

Installing systemd in Ubuntu may limit the amount of help and support available to you. If you have a commercial support agreement then installing systemd would almost certainly invalidate it. Even if you rely on forums etc, you will probably have to reproduce problems on a standard Ubuntu build before anyone can help you much.

If you want to quickly try out systemd, it may be a good idea to create a "sandpit" system for the purpose (e.g. a virtual machine that you can easily re-install or delete afterwards). If you are installing systemd on a system containing data that you care about, please take a full backup first, and make a plan for restoring from backup in the event that the system ends up unbootable.

Status as at December 2010

systemd on Ubuntu can boot the system to the desktop with networking configured and a reasonable number of services running. Some services are not started. This is because they have been converted to "native" upstart jobs in Ubuntu. systemd parses traditional sysvinit boot scripts (under /etc/init.d) but does not parse native upstart jobs (under /etc/init).

Package systemd-extra-units ships additional units (configuration files) that replace the native upstart jobs. Additional services could be started by adding further units to this package.

Ubuntu 12.04 running systemd (termcast)

Personal Package Archive location

systemd and related packages are available on this PPA To use the PPA, first add it to your software sources list as follows.

add-apt-repository ppa:pitti/systemd
apt-get update

Kernel requirements

systemd requires the directory /sys/fs/cgroup as a mountpoint. It doesn't exist in the current Ubuntu kernel (2.6.35). It can't be created with mkdir either because sysfs doesn't allow that. To create the directory this patch from the 2.6.36 kernel must be backported.

The following kernel config options must be selected. The standard Ubuntu kernel meets these config requirements.

General setup  --->
     [*] Control Group support
Device Drivers --->
     Generic Driver Options  --->
          [*] Maintain a devtmpfs filesystem to mount at /dev
          [*]   Automount devtmpfs at /dev, after the kernel mounted the rootfs (NEW)
File systems --->
     < > Kernel automounter support
     <*> Kernel automounter version 4 support (also supports v3)

A suitably patched kernel is available on the PPA. You can install it as follows.

apt-get install linux-image-2.6.35-23-generic=2.6.35-23.41ppa1 linux-headers-2.6.35-23-generic=2.6.35-23.41ppa1 linux-headers-2.6.35-23=2.6.35-23.41ppa1

If you prefer to build your own kernel you will need to apply the patch mentioned above and select the appropriate config options.

Installing systemd

systemd can be installed from the PPA as follows.

apt-get install systemd libpam-systemd systemd-gui systemd-extra-units

This results in systemd being installed alongside upstart. If you were going to try replacing upstart altogether then you would need the additional package systemd-sysv, which provides replacements for commands such as reboot.

Boot loader configuration

After installation, the machine will still boot under upstart by default. To boot under systemd, the following argument must be specified on the kernel command line:

init=/bin/systemd

Note that systemd resides in /bin rather than /sbin.

To make a boot menu entry that boots with systemd, first find the menuentry stanza in /boot/grub/grub.cfg for your patched kernel. Copy it to /etc/grub.d/40_custom and then:

• modify the menu entry name

• modify the kernel command line to include init=/bin/systemd

Here is an example /etc/grub.d/40_custom file. Note that you cannot just copy this example literally because the UUID will not be correct for your system.

exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Ubuntu, 2.6.35-23-generic with systemd' --class ubuntu --class gnu-linux --class gnu --class os {
        recordfail
        insmod part_msdos
        insmod ext2
        set root='(hd0,msdos1)'
        search --no-floppy --fs-uuid --set 552e0817-f1b2-4c99-b2b0-4e1ac4fd9a01
        linux   /boot/vmlinuz-2.6.35-23-generic root=UUID=552e0817-f1b2-4c99-b2b0-4e1ac4fd9a01 ro   crashkernel=384M-2G:64M,2G-:128M init=/bin/systemd
        initrd  /boot/initrd.img-2.6.35-23-generic
}

To boot under systemd by default, edit /etc/default/grub and specify the custom menu entry for systemd as the default one. Continuing with the example above, the relevant line would be changed to read as follows.

GRUB_DEFAULT="Ubuntu, 2.6.35-23-generic with systemd"

After modifying /etc/default/grub or any files in /etc/grub.d the following command is needed to bring the changes into effect.

update-grub

/etc/mtab

systemd prints the following warning on boot:

/etc/mtab is not a symlink or not pointing to /proc/self/mounts. This is not supported anymore. Please make sure to replace this file by a symlink to avoid incorrect or misleading mount(8) output.

It is advisable to do as suggested and replace /etc/mtab. It is not only mount that will behave incorrectly otherwise, but also df and probably most other commands that look at the list of mounted filesystems. This change can be made as follows.

ln -fs /proc/self/mounts /etc/mtab

Using systemd

Booting

To boot under systemd, select the boot menu entry that you created for the purpose. If you didn't bother to create one, just select the entry for your patched kernel, edit the kernel command line directly in grub and add init=/bin/systemd.

If a normal boot under systemd is not successful then it is worth trying with the following parameters:

init=/bin/systemd systemd.unit=emergency.service

• systemd.unit= specifies the target state that the system should boot to (similar to specifying a run level under sysvinit).

• emergency.service launches an emergency bash shell on the console without attempting to start any other services.

Controlling systemd once booted

The main command used to control systemd is systemctl. Some of its subcommands are as follows.

• systemctl list-units - List all units (where unit is the term for a job/service)

• systemctl start [NAME...] - Start (activate) one or more units
• systemctl stop [NAME...] - Stop (deactivate) one or more units
• systemctl enable [NAME...] - Enable one or more unit files
• systemctl disable [NAME...] - Disable one or more unit files
• systemctl reboot - Shut down and reboot the system

For the complete list, see systemctl(1).

systemadm is the GUI equivalent to systemctl, if you like that sort of thing.

Remote filesystem mounts

If you have NFS mounts listed in /etc/fstab then systemd will attempt to mount them but will typically do so too early, before networking has been configured. To get the timing correct we need to tell systemd explicitly that the mount depends on networking and on rpc.statd. To do this, create a file under /lib/systemd/system named <mount-unit-name>.mount with contents as follows.

[Unit]
Description=<mountpoint>
Wants=network.target statd.service
After=network.target statd.service

[Mount]
What=<server>:<share>
Where=<mountpoint>
Type=nfs
StandardOutput=syslog
StandardError=syslog

In the above

• mount-unit-name is the full path to the mountpoint in an escaped format. For example, a mount unit for /usr/local must be named usr-local.mount.

• mountpoint is the local mountpoint

• server:share specify the remote filesystem in the same manner as for /etc/fstab

See systemd.unit(5) and systemd.mount(5) for further details.

A similar approach will probably be required for other remote filesystem types such as nfs4 and cifs

Known issues

This kernel panic is seen sometimes when running with systemd. It occurs most commonly on shutdown.

[   79.173073] BUG: unable to handle kernel NULL pointer dereference at 0000001c
[   79.175071] IP: [<c01443b4>] set_next_entity+0x14/0xe0
[   79.176046] *pde = 2d918067 *pte = 00000000
[   79.176046] Oops: 0000 [#1] SMP
[   79.176046] last sysfs file: /sys/module/pcie_aspm/parameters/policy
[   79.176046] Modules linked in: snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm binfmt_misc snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore snd_page_alloc ppdev parport_pc i2c_piix4 parport psmouse serio_raw autofs4 floppy pcnet32 mii [last unloaded: vboxguest]
[   79.176046]
[   79.176046] Pid: 1138, comm: systemctl Not tainted 2.6.35-22-generic #35ppa1 /VirtualBox
[   79.176046] EIP: 0060:[<c01443b4>] EFLAGS: 00010082 CPU: 0
[   79.176046] EIP is at set_next_entity+0x14/0xe0
[   79.176046] EAX: ef1ea720 EBX: ef1ea720 ECX: edbed19c EDX: 00000000
[   79.176046] ESI: 00000000 EDI: 00000000 EBP: e3927ee4 ESP: e3927ed0
[   79.176046]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[   79.176046] Process systemctl (pid: 1138, ti=e3926000 task=e396f230 task.ti=e3926000)
[   79.176046] Stack:
[   79.176046]  133e53b7 00000000 ef1ea720 00000000 00000000 e3927efc c014457d c5808700
[   79.176046] <0> 00000000 c5808700 00000000 e3927f54 c05c6bb1 c08c3700 c08c3700 e396f230
[   79.176046] <0> c05d8aa0 c08c3700 c08c3700 00000001 e3927f28 c08c3700 c08c3700 c016abff
[   79.176046] Call Trace:
[   79.176046]  [<c014457d>] ? pick_next_task_fair+0xbd/0x100
[   79.176046]  [<c05c6bb1>] ? schedule+0x4f1/0x7a0
[   79.176046]  [<c016abff>] ? switch_task_namespaces+0x1f/0x60
[   79.176046]  [<c014f079>] ? do_exit+0x1b9/0x340
[   79.176046]  [<c014f23e>] ? do_group_exit+0x3e/0xa0
[   79.176046]  [<c014f2b8>] ? sys_exit_group+0x18/0x20
[   79.176046]  [<c05c9114>] ? syscall_call+0x7/0xb
[   79.176046] Code: e4 83 45 ec 08 8b 45 ec 8b 00 85 c0 89 45 e8 75 d8 e9 77 ff ff ff 90 55 89 e5 83 ec 14 89 5d f4 89 75 f8 89 7d fc 0f 1f 44 00 00 <8b> 4a 1c 89 c6 89 d3 85 c9 0f 85 95 00 00 00 8b 46 40 8b 90 4c
[   79.176046] EIP: [<c01443b4>] set_next_entity+0x14/0xe0 SS:ESP 0068:e3927ed0
[   79.176046] CR2: 000000000000001c
[   79.176046] ---[ end trace 19d7c4bdf6f52705 ]---
[   79.176046] Fixing recursive fault but reboot is needed!

This is in the process scheduler code and seems to be related somehow to destroying a cpu-cgroup. Miklos Vajna reproduced a very similar looking panic in a recent kernel.org kernel and raised it on LKML here.

Implementation issues

Packaging of units

A unit properly belongs in the package of the daemon that it starts. systemd units are already included in some upstream packages and some Debian Experimental packages. They will probably appear in Ubuntu packages in due course, unless Ubuntu maintainers deliberately remove them.

The systemd-extra-units package is intended only to make systemd usable in the short term by shipping some important units that don't yet exist in other Ubuntu packages. It should be scaled back as and when units start to appear in their proper places, and should eventually be dropped.

Dependencies on things not yet available in Ubuntu

Upstream systemd depends on recent upstream changes to several other packages that are not yet available in Ubuntu. These have been worked around by reverting the relevant changes in systemd or disabling the relevant feature. In brief these are:

• systemd 15 wants libnotify >= 0.7 and vala 0.11

  • Relevant changes have been reverted in order to build with libnotify 0.5.0 and vala 0.9.

• systemd wants /sbin/agetty but Debian/Ubuntu renames it to /sbin/getty

  • /sbin/agetty will be added as a link in Debian wheezy (Debian bug 603786).

• systemd invokes agetty -s where the option -s was added by recent upstream commits in util-linux.

  • Units getty@.service and serial-getty@.service have been patched to remove this option.

• systemd-fsck invokes fsck -l where the option -l was added by recent upstream commits in util-linux

  • systemd-fsck has been patched to remove this option.

Networking

NetworkManager.service just starts NetworkManager and does not wait for it to bring up a network interface. This seems appropriate, but network.target should not become active until a network connection is up. An extra unit, wait-for-network.service, has been created to delay network.target.

The appropriate way to wait for a network connection with NetworkManager appears to be nm-online, but this binary is not shipped in the Debian or Ubuntu packages for some reason. For systemd-extra-units 0.2 a simple shell loop has been used. This is effective, but hardly in the spirit of systemd.

TODO: Ask whether nm-online could be shipped in the network-manager package, or re-implement something similar if not.

NOTE: The network-manager package from Debian experimental does ship nm-online and will be uploaded to wheezy as soon as squeeze is released.

Remote filesystem mounts

systemd reads /etc/fstab and automatically creates a mount unit for each entry. The automatically generated dependencies for these units are insufficient in the case of remote filesystems. The correct dependencies can be specified explicitly in a mount unit configuration file (see above) but ideally this should be handled automatically.

Proposal: When parsing an /etc/fstab entry, systemd should look for a unit template named after the filesystem type (e.g. nfs@.mount for an NFS mount). If a template is found it should be used, with appropriate substitutions. Otherwise systemd should fall back to creating the mount unit with default settings as per the current behaviour.

TODO: Write a patch for this and propose it upstream.

portmap

The upstart job for portmap saves state using pmap_dump and restores it using pmap_set. The sysvinit script in Debian does the same thing. These actions appear redundant because portmap itself saves state to /var/run/portmap_mapping after each change. My guess is that this is all based on instructions in the upstream source README, which may be out of date or just misleading.

TODO: Confirm whether pmap_dump/pmap_set really serve some purpose and include them in the systemd unit if so.

anacron

/etc/cron.d/anacron invokes anacron via the upstart job, which won't work while booted under systemd. A change to the anacron package would be needed to address this.

/etc/default

Existing sysvinit style scripts read configuration in the form of variable assignments from a file under /etc/default. A policy decision is needed on whether systemd units will do the same.

Advantages: Separates configuration from code, simplifies package upgrade (i.e. same reasoning that applies to the existing sysvinit scripts).

Disadvantages: Extra complexity. Some may argue that systemd units are simple enough to be treated entirely as configuration files.

/etc/default files can be read using the EnvironmentFile= parameter in a systemd service unit. However, this is not fully compatible with the shell. In particular the quoting rules differ. It is not possible to write an environment variable assignment containing whitespace or literal quotes in a way that both a legacy init script and systemd will understand.

TODO: Decide whether systemd will use /etc/default files. Needs discussion with Debian maintainers. There would be little benefit in Ubuntu going it's own way on this.

TODO: Finish off patch for shell compatible quoting support and submit upstream.

Other TODO Items

• Units for NFS v4 support:
  • gssd
  • idmapd
  • rpc_pipefs
• Unit for sshd

• Unit(s) for static network configuration without NetworkManager.

• Units for remaining native Upstart jobs in a default Ubuntu install that don't currently have systemd equivalents:
  • apport
  • cups
  • log dmesg after boot
  • failsafe X session
  • irqbalance
  • bring up virtual network devices
• Desktop integration for systemadm
• Plymouth integration