Summary

In this blueprint we propose extending python-distutils-extra by a "do what I mean" mode where it automatically chooses the right destination path and action for various known file types such as Python modules, exectuable scripts, icons, desktop files, etc. The goal is that the typical setup.py only has to contain meta information like description, version, and author.

We also discuss Debian packaging generation.

Release Note

Ubuntu 9.10 now provides a fully automatic Python distutils based build system for Python applications in the python-distutils-extra package. Following a "convention over configuration" approach, this automatically determines most build and install actions for gettext, GtkBuilder, Qt UI, Python packages/modules, etc. For simple cases, setup.py only needs to contain package metadata, and POTFILES.in, setup.cfg, and MANIFEST.in are not required at all.

Rationale

Python's native and very popular build systems is "distutils" which comes shipped with Python itself. It is fairly limited, though, and requires the user to learn a lot about build systems. python-distutils-extra adds some functionality for i18n, but there are still some standard use cases missing.

With our goal of supporting the opportunistic programmer and our pre-selection of technologies, a lot of the build system design can be made implicit ("convention over configuration"), which will greatly reduce the amount of learning necessary for installing a project and building a package.

User stories

Jockey

The current Jockey project needs a very large and redundant setup.py, setup.cfg, and POTFILES.in, and MANIFEST.in.

With the improved distutils-extras, setup.cfg, POTFILES.in, and MANIFEST.in can be dropped completely, and setup.py can be dropped to the metadata:

from DistUtilsExtra.auto import setup

setup(
    name='jockey',
    version='0.5.1',
    description='UI for managing third-party and non-free drivers',
    url='https://launchpad.net/jockey',
    license='GPL v2 or later',
    author='Martin Pitt',
    author_email='martin.pitt@ubuntu.com',
)

Assumptions

Scope

This project is by and large an upstream build system extension. It does not deal in any way with distribution, code hosting, revision control, Launchpad, etc. These need to be handled in a higher level in the stack.

Design

Project structure

setup.py

For a standard project which uses well-known file types and a standard directory layout, setup.py should only contain metadata about the project which cannot be inferred from the source code.

Required fields:

Optional fields:

sdist

DistUtilsExtra.auto already knows which files in the source tree are "source" and which ones merely build products/noise. This can be turned into a reasonable implicit default MANIFEST.in. An existing file supersedes the automatic implicit one.

Debian packaging

Implementation

distutils-extra extensions

Current python-distutils-extra needs to learn about compiling PyKDE .ui files to .py files with pykdeuic4.

The existing functionality in distutils should be proposed upstream as a PEP and ideally merged into standard distutils gradually.

distutils-extra automatic setup.py

DistUtilsExtra.auto will support the following file types and automatically determine the default values in setup.py:

DistUtilsExtra.auto's goals of "convention over configuration" conflicts with the upstream paradigm of "explicit is better than implicit", and thus will most likely not be considered upstream. We will maintain this project in Debian/Ubuntu for now.

Debian packaging

Based on the information in setup.py and the source code, we can also generate a working Debian packaging.

The van.pydeb project (packages, svn) [already provides a some mechanics, so this should be used by distutils-extra if appropriate.

python-distutils-extra will provide a new command python-mkdebian which generates a debian/ directory with the necessary files from the information contained in the Python .egg-info:

changelog

compat

rules

control

Test/Demo Plan

python-distutils-extra contains an automatic test suite which covers all functionality of the DistUtilsExtra.auto automatic build system. It is run on package build.

For a manual test, check out lp:jockey (upstream trunk), and confirm that it does not have any of setup.cfg, MANIFEST.in, POTFILES.in, and only a very minimal setup.py. Test the following:


CategorySpec

DesktopTeam/Specs/Karmic/AutomagicPythonBuildSystem (last edited 2009-07-29 08:00:04 by pD9EB3788)