Summary

Currently setting up spam and anti-virus filtering takes a lot of editing of configuration files and is not easy for a new Ubuntu Server users to get set up and working. In this spec we will create a new package that will provide an out of the box working setup that provides a safe, functional mail filtering system. The design principle is that the default configuration will be conservative and prefer incomplete filtering over false positives.

This spec has several pieces:

Release Note

Easy Mail Filtering: A new feature in Ubuntu Server Edition 10.10 (Maverick Meercat) is easy setup of spam and anti-virus filtering with Postfix, Amavisd-new, Spamassassin, Clamav, and Postgrey. By installing a single package (maybe also at install time) you can have an integrated working setup that is safe to use (by design the false-positive risk is low, but non-zero).

Rationale

Currently Ubuntu Server offers a good capability for mail filtering, but it requires editing multiple configuration files. This is documented, but error prone and intimidating for new server admins. We want to make this easy and reliable in line with Ubuntu 's core values.

User stories

Bob has an Exchange server that is creaking under the load of running spam filtering and anti-virus checks in addition to the load generated just by Exchange. He's heard a lot of Linux and wants to see if it can help. He considers putting a Linux border MTA to handle spam and anti-virus checks. He installs Ubuntu Server and selects the mail filter subtask in the installer and then (after considering slitting his wrists while configuring LDAP to integrate with the user list in Exchange - Note: consider a future spec for this in the future) he is amazed how everything is integrated and working with spam and virus checks.

Scott often hangs out in #ubuntu-server and answers questions about how to do mail server related tasks. He's pleased to find that instead of pointing people at documentation and answering questions that are almost all repetition from previous questions he's answered, now he can just tell them to install amavisd-postfix and both he and the user are happy.

Elana has been tasked with establishing the mail infrastructure for a small business. They've previously used their Internet Service Provider's (ISP) mail services, but are tired of too much spam coming in and their mail getting tossed into recipient spam folders due to the poor reputation. The business already uses Ubuntu on the desktops and so Elana chooses Ubuntu Server to minimize support complexity. She reviews the Ubuntu Server Guide and discovers she can have a fully functional single tier mail stack by installing both dovecot-postfix and amavis-postfix on a single server.

Assumptions

None of note.

Design

This is intended to provide a gateway MTA in a two tier mail architecture or in combination with dovecot-postfix provide a complete one tier mail system.

Implementation

amavis-postfix will be a metapackage with configuration of amavisd-new and postfix in it's postinst.

postfix-rbl-add will be another script similar to postfix-filter/policy-add. There is a separate effort to develop a more general solution to postfix master.cf configuration. If this matures in the Lucid cycle in time, it will be used instead, but this effort can be completed without it.

Debconf will be used to gather additional information to get to a fully functional system.

Migration

None, for new installs only

Documentation

Update Ubuntu Server Guide to describe the new simplified process, but don't remove the details on how to set up manually (this should just work, but also shouldn't be magical). Need to review existing Postfix setup docs in the community section of help.ubuntu.com as many are out of date.

Test/Demo Plan

tbd.

Unresolved issues

There were quite a number of good suggestions for this spec. Not all of them can be implemented for Maverick (a best estimate of what can be done is described). Left over ideas will be moved to a new spec for Maverick +1. Tasksel integration is desirable, but unlikely to be reachable in the Maverick cycle.

BoF agenda and discussion

UDS discussion notes

Lucid:

== Upstream ==
Postfix recommends amavisd-new as a filtering option and then stack other tools with it.

== Goals ==
Provide API to /etc/postfix/master.cf
Amavisd-new already covered
Separate mail stacks for use in different environments (someone wants mail filtering, someone doesn't, same goes for SASL, etc...)

== Stack ==
* MTA
* Antispam
  * RBL
  * RulesDuJour
  * Bayesian
  * Greylisting (postgrey in Main)
  * SPF/DKIM
  * Other possible rules:
    * reject_invalid_hostname 
    * reject_non_fqdn_hostname
    * reject_non_fqdn_sender
    * reject_unknown_sender_domain
    * etc
* AntiVirus
    * ClamAV
* Spam/Ham training by the user
* What do to with:
  * Spam mails
  * Virus mails
   * Special case for .exe files (very annoying for users)
* Mail delivery
  * Sieve is a system for server-side, user-defined mail filtering on header or content. It is defined by RFC 5228, and there is an implementation of the protocol in Dovecot. Sieve is thus available in the current Ubuntu mail stack, but needs to be configured by the mail user.
  * ManageSieve is a server that allows certain mail clients to create Sieve scripts on the server. It is not currently configured, but there are several implementations in Ubuntu's archive.
== Decision tree ==
Decision for mail task

 Single server?
  Y: -> everything is installed
  N: Filter or MTA ?
    Filter: Postfix + Amavisd + ...
    MTA: Postfix + Dovecot + ...
  
== Discoverabilty ==
 --> talk about it in the server-tips session and ask Ivanka :D 
 --> add url for additional info in each debconf question
 --> Tasksel option(s)
 
== Documentation ==
 * Update Server Guide with new additions, but leave old instructions.
  * Advertise the Server Guide in the conf files
  * Explain the general policy: why it's this way by default, and what steps to do to improve filtering
 * Update help.u.c mail sections for current postfix, dovecot, etc.

Maverick

= Maverick Mail Integration =

Principle:
 By default, keep all the emails once accepted by the site.

== Use cases ==

Front-end system: spam detection
Backend system: Delivery agent
All-in-one: FE+BE.

MX forwarder only.

== Done in Lucid ==

- dovecot-postfix package
- amavisd-postfix (bug 524341)


== Ideas for Maverick ==

Include amavisd-postfix package into Maverick (Front-end).
Write postfix-rbl-add (Front-end).

Postgrey MIR (Front-end)
Postgrey configuration changes in amavisd-postfix:
 - edit postfix main.cf
Integrate postgrey into amavisd-postfix
Whitelisting option:
 - use postgrey maitnainer scripts to setup postfix to use postgrey - dependency issue?
 - amavisd-postix recommends postgrey.

Evaluate SPF policy support (Front-end):
 - some MIR are required - 2-3 packages
 - add SPF policy to SMTP policy daemon in amavisd-new-postfix package.
 - ship an alternate configuration file.
DKIM in amavisd-new (Front-end):
 - whitelisting from DKIM signature.

Debconf integration for additional questions to be asked during install:
 - some people don't want postgrey
 - add debconf question to amavisd-postfix postinst.
 
Provide -ldap versions that integrate with non-existing Ubuntu DIT

Invesigate renaming dovecot-postfix and amavsid-postfix to more meaninful names:
 - dovecot-postifx (backend): deliver, mail-stack, authentication
 - amavis-postfix (front-end): reception, mail-stack, mail-stack-filtering, border
 - send rfc to the ubuntu-server mailing list.

Get postfix SRU exception.

Sync up with Debian dovecot maintainer to update configuration files to use conf.d/.
Document how the pieces fit together.  (Part of the Server Guide) - sommer


CategorySpec

MaverickServerMoreMailIntegration (last edited 2010-05-25 01:47:35 by kitterman)