Launchpad Entry: server-maverick-uds-web20-workloads
Created: 2010-05-24
Contributors: ClintByrum
Packages affected:
Summary
This specification outlines improvements to Maverick's support of common scalable web application (aka "Web 2.0") stacks such as memcache and gearman.
Release Note
Better support for memcached is now included. The following packages are promoted to main: Memcache service (memcached), language bindings for C (libmemcached2), PHP (php5-memcache, php5-memcached). Additionally there are new language bindings with libmemcached2 in universe for python, perl, and ruby.
gearman bindings for python, php, and java have been added to universe.
Rationale
In order to assist users deploying scalable web applications on top of Ubuntu Server we should offer strong support for common backend server technologies.
User stories
As a WebOps developer I want to deploy memcache servers and memcache enabled applications quickly with a minimum of configuration and packaging effort, knowing that they will be supported through the life cycle of the product. I identify a need for caching, install on my development environment, and then roll out to production using identical packages.
As a WebOps developer I want to write my applications to use gearman to distribute load to backend servers and complete unrelated tasks without blocking. I use apt to install the appropriate language binding and gearman-job-server.
Assumptions
n/a
Design
memcached
memcached
The memcached server is extremely stable and in widespread use. It should be promoted to main.
language bindings
- libmemcached2 - low level C library, considered the best client implementation
- php5-memcached - promote to main, considered higher quality than php5-memcache
- python-libmemcached - needs packaging for universe in maverick, main in later releases
- libmemcached-ruby - needs packaging for universe in maverick, main in later releases
- libmemcached-libmemcached-perl - Memcached::libmemcached CPAN module, needs packaging, universe now, main later
- libcache-memcached-libmemcached-perl - Cache::Memcached::libmemcached in CPAN, High level abstraction CPAN module, needs packaging, universe now, main later
- non libmemcached based language bindings
- php5-memcache - PECL module for memcache access (does not conflict with php5-memcached), popularity would suggest promoting to main even though php5-memcached is considered "better".
- python-memcache - pure python memcache library - leave in universe
- libcache-memcached-perl - pure python memcache library - leave in universe
- libmemcache-client-ruby - pure ruby memcache library - leave in universe
gearman
gearman-job-server is already in universe. Perl bindings are also already in universe.
python-libgearman
pypi package. Package and add to universe.
php5-gearman
PECL module, add to universe.
gearman-java
Java client libraries for gearman, add to universe.
varnish
This will remain in universe. Latest version needs to be synced to deal with ESI bugs in 2.1.0 which was shipped with Lucid.
Implementation
Please see: server-maverick-uds-web20-workloads whiteboard for Work Items
Test/Demo Plan
"This need not be added or completed until the specification is nearing beta."
Unresolved issues
rabbitmq-stomp
The rabbitmq-stomp adapter is still considered experimental and is only available as a checkout from a mercurial server. We should produce packages when an official (even if just beta/alpha) release is made from the project.
UDS Session agenda and discussion
Discussion plan:
- Web accelerators:
- - varnish: All good!
- Key/value store in RAM:
- memcached (in universe dapper+):
- promote to main: yes
- Client libraries to promote to main: php5-memcache, python-memcache, libmemcached-tools, libcache-memcached-perl, libmemcache-client-ruby, libmemcached2.
- only if based on libmemcached2, demote others
libmemcache (without the D) is stale & old - should be deleted
- libmemcachedN is the right thing to package
- libmemcached4 is coming
- if we update, we need to rebuild
- Java client library to package in universe:
- spymemcached
- Package spymemcached in universe ?
- not in debian either
- compares how to Ehcache?
- back Ehcache
- Package spymemcached in universe ?
- memcached (in universe dapper+):
- web frameworks and their stacks (deployment tools):
- ruby: RoR
- TBD with lucas
- python:
- - django
- already in main
- following debian, uptodate
- uptodate, following debian
- - django
- perl: catalyst: uptodate, follow debian
- java
- refer to seperate java session
- php
- - symfony (1.0 in universe, deprecated upstream, clint already updating to 1.4) - zend framework (in universe, up to date) (we do have two versions: libzend-framework-php and zendframework) - cakephp (in universe, up to date)
- ruby: RoR
- Batch systems:
- gearman (universe):
- lucid 0.10, debian 0.12, latest 0.13
- gearman-job-server is the current supported upstream
- promotions/packaging only if they depend on libgearman TBD: - python client library - php client library - java client library - perl client library
- gearman (universe):
- Messaging:
- AMQP: rabbitmq (main)
- do we package the StOMP adapter? If not, do so
- perl, ruby, python client libraries available
- ZeroMQ (0MQ)
- In Debian since somewhat recently
- Ensure we're following debian
- AMQP: rabbitmq (main)
- HA Proxy/Nginx/Lighttpd
- Loadbalancing / director specialty
- Security concerns?
- Lighttpd
- spare time project, community driven
- ubuntu uptodate
- HA proxy
- no url rewrites currently
- outperforms apache's mod_proxy
- 1.3.22, maverick is up-to-date
- Nginx:
- async, very performant
- closed development - no public source control repository, no security backports/test suite available
- currently in universe at 0.7.65 (latest upstream)
- Search/indexing:
- Solr/Lucene (discussed in java stacks) - up-to-date in universe
- sphinx popular in MySQL world
packaged as sphinxsearch in Universe 0.9.8 (0.9.9 current & in maverick)
- Xapian (loosing market share).
- Hadoop (discussed in separate session)