JauntyKernelTeamProcess

Summary

Review of the current kernel team processes for all stages of the release cycle.

Release Note

There are no end user impacts in this process definition work. The intent of this proposal is to improve internal processes.

Rationale

The Intrepid development cycle exposed a number of areas where the kernel team could improve the process. For example, regressions were introduced because patches or drivers were not carried foreward from the previous release. Processes, particularly decision processes, must be documented and published during the cycle. The kernel team and the work expected of it continues to expand and the current process is not scaling well.

Assumptions

The intent is to make no assumptions about "How Things are/were Done". Many new members of the team have little experience of the Debian/Ubuntu history.

Development Cycle

The development cycle starts with the opening of a new source tree and ends when support for the release expires.

Initial Planning

The initial planning has two goals. First, it defines and documents exactly what the kernel for this development cycle includes and, more importantly, what it drops. Each change will include a rationale for the change. Second, the result is published to all interested parties.

A key part of planning is to report the status of this work to the UDS where it will be reviewed.

The result of this work is not cast in stone. It will change as development proceeds but any changes in direction will also be published. The final version, including the rationale for each change will be published in the release documentation.

Kernel Re-base

The initial part of this work is more or less mechanical once the baseline kernel is chosen from the upstream. The final step is to isolate the differences between the current release tree and the mainline tree used for the re-base. These differences are Ubuntu specific changes that must be triaged.

SAUCE Patch Triage

Patches isolated during the re-base fit into the following categories:

  • Upstream features, such as the RT patches, may not in the mainline yet. These patches will either have to be re-based to current version as well or forward ported to the new version.
  • Some patches are backports of upstream fixes. Where possible, they will be replaced by their upstream equivalent.
  • Other patches are Ubuntu specific bug fixes. These patches will have to be forward ported to the new version.
  • There are Ubuntu specific configurations and code that are necessary for the distribution but will not meet the upstream requirements for mainline inclusion. These may or may not be forward ported.
  • Some upstream features/options will be disabled in Ubuntu kernels. Disabled features are not supported.

This is a notification point that follows the process defined below. The documentation sent out in the notification shall document each change from the "vanilla" kernel, including a rationale for the change. Note that each change does not mean every changeset. The granularity of this is at the level of user visible subsystem or feature.

  • Upstream versions of patches are preferred if their functional behavior is identical to the Ubuntu patch they replace. The update of the old patch to a mainline version is recorded.
  • Any patch that is dropped without an upstream replacement is documented with both a rationale and an indication of what other parts of the distribution would be affected, for example, dropping support for a device or system.
  • Additions that come from the new upstream that may impact other packages in the distribution.

Firmware Load Triage

Firmware comes from an outside source, most often the hardware vendor. It has its own release and retirement schedule.

  • New firmware or updated firmware must be identified and included in the build
  • Deprecated firmware must be identified for deletion. The notification should identify possible uses.
  • The license must be reviewed for legal issues that would conflict with Ubuntu policy.
  • The reviews, particularly of legal issues, follow a TBD process involving the TBD stakeholders.

This is a notification point that follows the process defined below.

Configuration (re)Definition

The kernel configuration file .config controls what is included in the kernel. Since the kernel is what this file defines it to be, its contents must be controlled during the life of the release.

  • The set of enabled modules must match the list of Ubuntu supported drivers and subsystems.
  • When enabling a feature, any firmware, user space programs, and licensing issues must be identified.
  • When disabling a feature, all dependencies to that feature must be identified and where they exist, an alternative feature/method/workaround must be identified.
  • Enabling or disabling a feature requires prior public review.
  • The set of supported configuration files may be re-factored into common and flavor specific portions to inprove its management.

This is a notification point that follows the process defined below.

Development

Detail TBD

Freeze Milestones

There are alpha and beta releases. The process is TBD.

A new release will most likely cause a change to the ABI. Since this type of change affects 3rd party vendors, ABI changes in any of these releases will trigger a notification to the affected stakeholders. The notification will include the specifics of the change.

Update Process

Changes to final releases are restricted to bugfixes and security patches.

SRU Release Points

  • Security patches are released in a timely manner subject to CVE procedures.
  • Non-CVE bugfixes are aggregated into timely update releases on a TBD schedule.
  • Any update that changes the ABI triggers a notification event to the affected stakeholders.

Retirement

Source tree and Launchpad bugs process TBD.

Review and Notification

At various stages of the development cycle, formal, documented reviews of certain items such as patches or firmware take place. The results of these reviews and the decisions made must then be communicated to the appropriate people and groups.

Review Steps

  • Identify components
  • Decision criteria for inclusion/removal, i.e. ubuntu patch superceded by upstream
  • Licensing. Defined review process including legal.
  • Stakeholders. Who is interested? Who will be inconvenienced/incensed?

Notification Process

  • Who gets notified?
  • Communications path: email, release notes, wiki page
  • When? How often?

Kernel Specific Issues

Kernel Version Choice

The release cycle is long enough that it is possible that at least one, if not two kernel releases from upstream could occur within the development cycle. This brings up the following issues:

  • Early in the cycle is better than later. What are the bounds?
  • A decision to re-base depends on the value of new upstream work.
  • The impact on the rest of the distribution must be known. Possible regressions in user-land packages should be identified and documented.
  • At what point do we refuse to take a new kernel version? i.e. Alpha 3 milestone?

Feature Configuration Management

Kernel configuration for the various flavors of kernel are somewhat ad hoc and could use some structure.

Some embedded distributions have broken the kernel configuration file into generic, board/arch specific options, and "feature" options. They also have tools to merge and "layer" these configuration fragments into a final configuration for the build. This may be a useful model for re-factoring server/generic/desktop/notebook etc. configurations into something manageable.

Kernel Subsystem and User Land Dependencies

In the past, kernel features were added/enabled but there was no tracking of the user land part of the feature. The user land development fell behind, making the kernel component unusable. There should be a process in place to track the kernel and user land tasks. One option is to create open general release bugs in Launchpad with tasks to track both the kernel and user land work.

ABI Management

ABI changes can cause breakage in 3rd party packages. A process for identifying ABI changes and notifying impacted 3rd parties is defined.

Vanilla Kernel Packaging

The often requested Vanilla Kernel is an Ubuntu packaging of what is the current Linux release. This is not a supported kernel but it is useful for the isolation of reported bugs and the testing of upstream patches and features. Given the volatile nature of the Linux 2.6 HEAD, the packaging has to be controlled.

  • The packaging process should be as automatic as possible from the git pull to the push to the PPA archive.

  • There must be a process in place to decide when to generate a package. For example, daily packaging of potentially unstable or unbuildable kernel trees is wasteful and packaging very late in the release cycle limits testing availability.
  • The package list and kernel configuration variations may not be as extensive as for the supported kernels.

Unresolved issues

BoF agenda and discussion

References

CategorySpec

KernelTeam/Specs/JauntyKernelTeamProcess (last edited 2008-12-01 22:56:40 by dsl081-061-011)