systemd - An alternative boot manager

systemd is a system and session manager for Linux, compatible with SysV and LSB init scripts. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux cgroups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic.

See the systemd home page for further information.

Warning! Experimental code

systemd is under active development in Ubuntu although the rough plan would be to default to systemd during development of 15.04. If you want to help it's best to be running 15.04. (14.10 might be doable as well..)

Installing systemd in Ubuntu may limit the amount of help and support available to you. If you have a commercial support agreement then installing systemd would almost certainly invalidate it. Even if you rely on forums etc, you will probably have to reproduce problems on a standard Ubuntu build before anyone can help you much.

If you want to quickly try out systemd, it may be a good idea to create a "sandpit" system for the purpose (e.g. a virtual machine that you can easily re-install or delete afterwards). If you are installing systemd on a system containing data that you care about, please take a full backup first, and make a plan for restoring from backup in the event that the system ends up unbootable.

Personal Package Archive location

systemd and related packages are available on this PPA To use the PPA, first add it to your software sources list as follows.

add-apt-repository ppa:pitti/systemd
apt-get update

Installing systemd

systemd can be installed from the PPA as follows.

apt-get install systemd libpam-systemd systemd-ui

This results in systemd being installed alongside upstart.

Boot loader configuration

After installation, the machine will still boot under upstart by default. To boot under systemd, the following argument must be specified on the kernel command line:

init=/lib/systemd/systemd

Note that the systemd binary resides now in /lib/systemd/ and /bin/systemd is just a symlink to it.

To boot under systemd by default, edit /etc/default/grub and change the following line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash init=/lib/systemd/systemd"

After modifying any grub related configuration files like /etc/default/grub the following command is needed to bring the changes into effect.

update-grub

/etc/mtab

systemd prints the following warning on boot:

/etc/mtab is not a symlink or not pointing to /proc/self/mounts. This is not supported anymore. Please make sure to replace this file by a symlink to avoid incorrect or misleading mount(8) output.

It is advisable to do as suggested and replace /etc/mtab. It is not only mount that will behave incorrectly otherwise, but also df and probably most other commands that look at the list of mounted filesystems. This change can be made as follows.

ln -fs /proc/self/mounts /etc/mtab

Using systemd

Booting

To boot under systemd, select the boot menu entry that you created for the purpose. If you didn't bother to create one, just select the entry for your patched kernel, edit the kernel command line directly in grub and add init=/lib/systemd/systemd.

If a normal boot under systemd is not successful then it is worth trying with the following parameters:

init=/lib/systemd/systemd systemd.unit=emergency.service

• systemd.unit= specifies the target state that the system should boot to (similar to specifying a run level under sysvinit).

• emergency.service launches an emergency bash shell on the console without attempting to start any other services.

Controlling systemd once booted

The main command used to control systemd is systemctl. Some of its subcommands are as follows.

• systemctl list-units - List all units (where unit is the term for a job/service)

• systemctl start [NAME...] - Start (activate) one or more units
• systemctl stop [NAME...] - Stop (deactivate) one or more units
• systemctl enable [NAME...] - Enable one or more unit files
• systemctl disable [NAME...] - Disable one or more unit files
• systemctl reboot - Shut down and reboot the system

For the complete list, see systemctl(1).

systemadm is the GUI equivalent to systemctl, if you like that sort of thing.

Remote filesystem mounts

If you have NFS mounts listed in /etc/fstab then systemd will attempt to mount them but will typically do so too early, before networking has been configured. To get the timing correct we need to tell systemd explicitly that the mount depends on networking and on rpc.statd. To do this, create a file under /lib/systemd/system named <mount-unit-name>.mount with contents as follows.

[Unit]
Description=<mountpoint>
Wants=network.target statd.service
After=network.target statd.service

[Mount]
What=<server>:<share>
Where=<mountpoint>
Type=nfs
StandardOutput=syslog
StandardError=syslog

In the above

• mount-unit-name is the full path to the mountpoint in an escaped format. For example, a mount unit for /usr/local must be named usr-local.mount.

• mountpoint is the local mountpoint

• server:share specify the remote filesystem in the same manner as for /etc/fstab

See systemd.unit(5) and systemd.mount(5) for further details.

A similar approach will probably be required for other remote filesystem types such as nfs4 and cifs

Useful snippets

• To see which other units a service depends on:

  •    $ systemctl list-dependencies $service

systemd quickstart for upstart users

• See SystemdForUpstartUsers

Implementation issues

Packaging of units

A unit properly belongs in the package of the daemon that it starts. systemd units are already included in some upstream packages and some Debian Experimental packages. They will probably appear in Ubuntu packages in due course, unless Ubuntu maintainers deliberately remove them.

The systemd-extra-units package is intended only to make systemd usable in the short term by shipping some important units that don't yet exist in other Ubuntu packages. It should be scaled back as and when units start to appear in their proper places, and should eventually be dropped.

Dependencies on things not yet available in Ubuntu

Upstream systemd depends on recent upstream changes to several other packages that are not yet available in Ubuntu. These have been worked around by reverting the relevant changes in systemd or disabling the relevant feature. In brief these are:

• systemd 15 wants libnotify >= 0.7 and vala 0.11

  • Relevant changes have been reverted in order to build with libnotify 0.5.0 and vala 0.9.

• systemd wants /sbin/agetty but Debian/Ubuntu renames it to /sbin/getty

  • /sbin/agetty will be added as a link in Debian wheezy (Debian bug 603786).

• systemd invokes agetty -s where the option -s was added by recent upstream commits in util-linux.

  • Units getty@.service and serial-getty@.service have been patched to remove this option.

• systemd-fsck invokes fsck -l where the option -l was added by recent upstream commits in util-linux

  • systemd-fsck has been patched to remove this option.

Networking

NetworkManager.service just starts NetworkManager and does not wait for it to bring up a network interface. This seems appropriate, but network.target should not become active until a network connection is up. An extra unit, wait-for-network.service, has been created to delay network.target.

The appropriate way to wait for a network connection with NetworkManager appears to be nm-online, but this binary is not shipped in the Debian or Ubuntu packages for some reason. For systemd-extra-units 0.2 a simple shell loop has been used. This is effective, but hardly in the spirit of systemd.

TODO: Ask whether nm-online could be shipped in the network-manager package, or re-implement something similar if not.

NOTE: The network-manager package from Debian experimental does ship nm-online and will be uploaded to wheezy as soon as squeeze is released.

Remote filesystem mounts

systemd reads /etc/fstab and automatically creates a mount unit for each entry. The automatically generated dependencies for these units are insufficient in the case of remote filesystems. The correct dependencies can be specified explicitly in a mount unit configuration file (see above) but ideally this should be handled automatically.

Proposal: When parsing an /etc/fstab entry, systemd should look for a unit template named after the filesystem type (e.g. nfs@.mount for an NFS mount). If a template is found it should be used, with appropriate substitutions. Otherwise systemd should fall back to creating the mount unit with default settings as per the current behaviour.

TODO: Write a patch for this and propose it upstream.

portmap

The upstart job for portmap saves state using pmap_dump and restores it using pmap_set. The sysvinit script in Debian does the same thing. These actions appear redundant because portmap itself saves state to /var/run/portmap_mapping after each change. My guess is that this is all based on instructions in the upstream source README, which may be out of date or just misleading.

TODO: Confirm whether pmap_dump/pmap_set really serve some purpose and include them in the systemd unit if so.

anacron

/etc/cron.d/anacron invokes anacron via the upstart job, which won't work while booted under systemd. A change to the anacron package would be needed to address this.

/etc/default

Existing sysvinit style scripts read configuration in the form of variable assignments from a file under /etc/default. A policy decision is needed on whether systemd units will do the same.

Advantages: Separates configuration from code, simplifies package upgrade (i.e. same reasoning that applies to the existing sysvinit scripts).

Disadvantages: Extra complexity. Some may argue that systemd units are simple enough to be treated entirely as configuration files.

/etc/default files can be read using the EnvironmentFile= parameter in a systemd service unit. However, this is not fully compatible with the shell. In particular the quoting rules differ. It is not possible to write an environment variable assignment containing whitespace or literal quotes in a way that both a legacy init script and systemd will understand.

TODO: Decide whether systemd will use /etc/default files. Needs discussion with Debian maintainers. There would be little benefit in Ubuntu going it's own way on this.

TODO: Finish off patch for shell compatible quoting support and submit upstream.

Other TODO Items

• Units for NFS v4 support:
  • gssd
  • idmapd
  • rpc_pipefs
• Unit for sshd

• Unit(s) for static network configuration without NetworkManager.

• Units for remaining native Upstart jobs in a default Ubuntu install that don't currently have systemd equivalents:
  • apport
  • cups
  • log dmesg after boot
  • failsafe X session
  • irqbalance
  • bring up virtual network devices
• Desktop integration for systemadm
• Plymouth integration

Workarounds

Workaround #2: Failures on software-upgrades

The post-installation script of packages uses /sbin/start and /sbin/stop to start and stop (running) services in an upstart environment. Both binaries are symlinks to /sbin/initctl shipped with upstart package.

Unfortunately, software-upgrades break when switching to systemd as an alternative init-system on Ubuntu systems, more precisely the software-management tool dpkg breaks.

Example

Bug #1008837 "cups fails to install/upgrade with systemd"

A little demo

# mv /sbin/start /sbin/start.orig

# apt-get install --reinstall openssh-server
[OUTPUT]
/var/lib/dpkg/info/openssh-server.postinst: 429: /var/lib/dpkg/info/openssh-server.postinst: start: not found

Workaround: Using an initctl replacement

On IRC I discussed with Michael Biebl and others on the topic and Michael had concerns using /bin/true as a symlink to /sbin/start and /sbin/stop. In the end, the idea of an initctl replacement was born.

• Rename the original /sbin/initctl binary:

# mv /sbin/initctl /sbin/initctl.upstart

• Edit a new /sbin/initctl replacement file (Thanks mbiebl and grawity):

#!/bin/sh

service=$1
action=${0##*/}

if [ -e /sys/fs/cgroup/systemd ] &&
   [ -e /lib/systemd/system/${service}.service ] ||
   [ -e /etc/systemd/system/${service}.service ] ; then

        if [ -n "$DPKG_MAINTSCRIPT_PACKAGE" ]; then
                # If we are called by a maintainer script, chances are
                # good that a new or updated service was installed.
                # Reload daemon to pick up any changes.
                systemctl daemon-reload
        fi

        systemctl $action ${service}.service
else
        if [ "$action" = initctl ] ; then
                initctl.upstart $@
        else
                initctl.upstart $action $service
        fi
fi

• Make the new script executable:

# chmod +x /sbin/initctl

• Create and list a divert for the original initctl (this prevents e.g. its removal on upgrades):

# dpkg-divert --divert /sbin/initctl.upstart /sbin/initctl

# dpkg-divert --truename /sbin/initctl
/sbin/initctl.upstart

Software-upgrades touching start/stop of daemons should now work fine within an upstart or systemd environment.

CategoryBoot