Archiveopteryx: The IMAP Server You Always Wanted

Archiveopteryx (aox) is a highly scalable PostgreSQL-backed IMAP/POP server. As described on its website:

Archiveopteryx is an Internet mail server, optimised to support long-term archival storage. It seeks to make it practical not only to manage large archives, but to use the information therein on a daily basis instead of relegating it to offline storage.

and

Archiveopteryx is designed to impose no limits on the size or usage of the archive to the extent of the server hardware's capabilities.

With aox it's possible to have millions of mail in an IMAP folder without experiencing performance issues. My own Postgres configuration is moderately tuned and performs spectacularly. The fact that the mail is in a database means large operations such as marking many thousands of mail as read or moving to different IMAP folders is a very fast and inexpensive operation.

Notable features:

• Automatic deduplication
• Retention: Specify which messages must be deleted/retained, including by search.
• Undelete: Search for accidentally deleted messages and recover them.
• Export: Search for messages and export them.
• Easy backup: just a SQL dump!
• LDAP authentication
• Bleeding edge RFC suppport -- many new IMAP features land here first!

For those curious how this compares to DBMail read here.

One thing to note about aox is the goal of "long-term archival storage". Aox intends for you to be able to read your email with a mail client written 20 years from now. Your mail will be safely stored and RFC compliant. Mail clients of the future should not have to implement thousands of quirks and workarounds to deal with malformed messages from poorly written mail clients of old. This means that your mail in some circumstances may be modified to be standards compliant: headers changed slightly, encoding fixed, etc. This will not be noticable when read unless it happens on a PGP-Signed mail in which case it could break the signature. I occasionally see this happen, but it's not been a major concern of mine. If this doesn't worry you, forge ahead!

Installation & Setup

On FreeBSD:

# pkg install archiveopteryx

We will now initiate the installer. If the Postgres server is local, the use of the pgsql user will initiate the aox database and accounts. If the database is not local I will leave it as an excercise to the reader to permit remote database access by the installer to create the required accounts and database.

Make sure the citext extension is available.

On FreeBSD this is in the postgresql93-contrib package for Postgres 9.3 servers.

# /usr/local/libexec/aox/installer
Connecting to Postgres server /tmp/.s.PGSQL.5432 as Unix user pgsql.
Creating the 'aox' PostgreSQL user.
Creating the 'aoxsuper' PostgreSQL user.
Creating the 'archiveopteryx' database.
Adding citext to the 'archiveopteryx' database.
Loading the database schema.
SET
SET
CREATE TABLE
INSERT 0 1
CREATE EXTENSION
CREATE TABLE
CREATE INDEX
CREATE FUNCTION
(yadda yadda this goes on for a bit)
...
Granting database privileges.
Generating default /usr/local/etc/archiveopteryx/archiveopteryx.conf
Generating default /usr/local/etc/archiveopteryx/aoxsuper.conf
Setting ownership and permissions on
/usr/local/etc/archiveopteryx/archiveopteryx.conf
Done.

Add to /etc/rc.conf:

archiveopteryx_enable="YES"

Let's start it up and create a user. This will add a user and prompt for the password. The username can be anything you want, or the email address if that's more convenient.

# service archiveopteryx start
# aox add user -p <username> <email-address>

You may want to use your own SSL/TLS key. Concat the certificate, key, and chain into a single file and add an entry to /usr/local/etc/archiveopteryx/archiveopteryx.conf:

tls-certificate = /usr/local/etc/archiveopteryx/yourcert.pem

Restart the service and you're ready to connect your mail client, webmail, etc to the server and start using it!

Again, the other intracacies of email hosting are left as an exercise to the user: LMTP delivery from your favorite MTA, spam filtering, valid matching A and PTR records, SPF, DKIM, etc.

I guarantee this setup is easier than competing IMAP servers with significantly less confusing knobs to turn.

Other tips

Import mail from mbox or maildir with:

# aox import

Importing lots of mail? Turning off the SQL index might help speed things up:

# aox tune database mostly-writing

Turn it back on when finished (be patient!):

# aox tune database mostly-reading

A nightly cron will clean up the database by removing emails that have expired beyond the configured undelete-time:

0 0 * * *  root  /usr/local/bin/aox vacuum

Don't care about enforcing IMAP Quotas? Turn them off in archiveopteryx.conf; they're expensive:

use-imap-quota = off

The official aox releases are rare. The authors are do a full audit of the codebase each release which takes significant time. Following git is encouraged if you need a certain bugfix or feature. The following IMAP features missed the 3.2.0 release because their reliability couldn't be vetted in time:

THREAD=ORDEREDSUBJECT
THREAD=REFS
THREAD=REFERENCES

Their absence means you won't see mail threads in webmail clients like Roundcube but it's easy to patch them back in.