Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
If you are a native english speaker, please correct my pidgin english.

• Launchpad Entry: build-process-guidelines

• Packages affected: many

Proposal for inclusion in a common guideline for the software build process in Ubuntu and derivatives

Summary

Get rid of many hidden bugs. Enhance software maintainability. Let's rewrite all makefiles of the whole distro!
Summary of Recursive Make Considered Harmful:

• The author argues to switch from a recursive build process to a plain master makefile that includes sub-projects makefiles, rather than to call 'make' recursive in sub-directories. He clearly proves that

• many bugs are caused or hidden by the common (wrong) practice of using recursive makefiles.

• software module dependencies are commonly tweaked, artificially left out, or falsely introduced, to fix what essentially comes from using 'make' recursively. The result is wrong software.

• Using flat makefiles enhances maintainability, because dependencies will be correct.

Release Note

• Many formerly hidden bugs found during the software build process.
• Eliminated bugs introduced by the common (wrong) practice to use "recursive make".
• Corrected many dependencies, enhanced maintainability of source code.

Rationale

Right now (I'm using Ubuntu release "Gutsy"), I suspect many packages are build in a "recursive make"-style. This includes some very essential packages, e.g. APT, DPKG, and probably many others.
Once you know "recursive make" is the root of many bugs, you start wondering why your system doesn't crash twice as much.

Use Cases

FIXME

Assumptions

• "Recursive make"-style is common practice in software that make up Ubuntu and derivatives (I assume).
• This assumption has to be approved by s/o who is more envolved.
• As the paper mentioned above is only 10 years young, and usually things need about 20 years from academia to production, I assume this holds true for Ubuntu distros.
• The paper was updated lately, about 10 years after it's first release. Why? Because "recursive make" is still common practice (this I know for sure).
• Many of the software in the distros is much older than the paper.
• Genuine software authors will switch to flat makefiles once they realize their software has less bugs on distros that do builds the right way. Usually they are happy about qualified bug reports (maintainers find bugs during build, and they are more qualified than average).

Examples

The first two lines of the makefile of APT - this tool is essential for the consistency of the whole system - let me conclude APT is build in a "recursive make"-style:

# This is the top level make file for APT, it recurses to each lower
# level make file and runs it with the proper target

At the time of this writing, I'm having trouble with broken package dependencies. APT is great if its database is clean, but once things go wrong you're off. Noone wants to remove hundreds of essential packages without the proposal of installing an upgraded version of these packages. Something is fundamentaly broken, not necessarily in APT itself, but in the whole distro. After reading the abovementioned paper and catch a glimpse of the makefiles of APT and DPKG, I suspect (among other reasons) one root of evil is the use of "recursive make". I.e. not only APT's input, the package DB, might be wrong because maintainers occasionally slip false package dependencies in it. I suspect APT/DPKG is wrong in that they are build the wrong way, and therefore can not function correct (i.e. identify errornous input as such).

Design

You can have subsections that better describe specific parts of the issue. FIXME

Research for automation

Search for a tool that automaticaly rewrites makefiles.

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: FIXME

Automation

* If available, try to use automation tools that rewite makefiles. Probably such a tool fails, but hopefully can give hints in such a case. * Use software dependency tools to identify differences between the real ones and those expressed in the sources.

Code rewiew

Identify packages that use "recursive make". Follow the ordering of package importance given by the distro (essential/basic ones first, etc.)

UI Changes

No changes to the UI are required, except occasionally in developer build automation tools.

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

• Makefiles
• Interface defining files (for C/C++: header files)
• Interface references in source files
• Code changes resulting from bugs found by changing the build process
• Code changes resulting from corrected dependencies

Migration

Include:

• data migration, if any
• redirects from old URLs to new ones, if any
• how users will be pointed to the new way of doing things, if necessary.

FIXME

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during CD testing, and to show off after release.

This need not be added or completed until the specification is nearing beta.

* Change a package sources build style and see if it reveals a bug. FIME EVEN MORE

Outstanding Issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. FIXME, TOO

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

CategorySpec