Email: Going from your localmail box to a distributed mailbox

Email: Going from your localmail box to a distributed mailbox

Maybe after reading the title you're thinking that I'm going to talk about IMAP, well yes I will, but not only.

The motivation of the topic in this post is simple, I wanted a way to read my local email even when not at home. When all this saga started my local configuration was simple, as my MUA I used sylpheed (in my opinion on of the best email clients you can have) with a localmail using the MH format, then I would fetch my email from 5 different pop3 accounts and read them locally.

Since I've been always a command line friend, my first option (and for long time was the only one) using mutt to read my email. Now you probably know mutt, it's a nice email client, if you don't have to open attachments in non-text format. When I had to do that, I would need to save and then use sftp to get it. Not to mention that sometimes I would have charset issues, and well...everyone made a strange face when seeing me read email in the terminal.
If you have no idea on how mutt looks, well here is a screenshot I took:

Anyway I have to face it I'm starting to be more friendly with good looking GUIs, maybe it's OS X fault or my project manager that is always hitting my head when the user interface isn't good looking, or even both. In the end I got myself thinking last sunday, why not install a webmail system?

So I started thinking on what I would need:

• Webmail uses IMAP servers, so I would need to setup an IMAP server
• To sync my MUA with the webmail, sylpheed would start using the IMAP account also
• Port my MH mailbox to Maildir (the common format among IMAP servers)
• Select a nice webmail application
• Configure https so we can login in the webmail in a secure way
• Finding a way of downloading the email from pop accounts and storing it in the IMAP account

Plenty of things, isn't it? Well yes, but everything gets easily done! If anyone wants to see the final result, you can check my new vhost.

So as IMAP server I selected Courier because I'm already had to work with it, it's a piece of software pretty quick to install and configure. A really cool think I like on Courier is that you can select various ways of identifying users, from local passwd accounts to LDAP directories. It's all up to you.

You'll need Courrier and it's authentication library, both can be found at courier's download page. You can almost run it out of the box, with some minor configuration such as generating certificates so you can use IMAP over SSL and telling what's the users mailbox dir. If you don't know how to generate certificates, you might want to check out this document.
Make sure you're authentication courier service is also running otherwise you won't be able to authenticate.

After getting courier running I had to port my MH mailbox, after a small search on google I found a script called mh2maildir which does exactly that converts mh mailboxes to maildir mailboxes preserving your MH mailbox (although I suggest you to backup). So mainly, what you want, is to run the following command:

So by now you have your MH replicated on the Maildir folder, so you start courier imap server, along with it's authentication service and you configure on your MUA a new IMAP Account. You should now be able to see your Mailbox over the IMAP connection.

Now it comes the part where we select a webmail application I knew two:

• Horde IMP
• SquirrelMail

My preference was for IMP because as a user I enjoyed more the experience with IMP, although after installing Horde framework and seeing the big pain in the ass that it was to configure I start thinking in alternatives.
After browsing a bit in fresheat I found an interesting project called RoundCube.

It seemed fast to install and configure and the look reminded me OS X, so I gave it a try. It took me around 5 minutes to configure and get it running, it's not as solid as IMP nor as all the features but the fundamental one are present and it has style… For me, right now, that's enough!

RoundCube is based in php and it can use has backend one of the following databases: mySQL, posgreSQL or sqlite.

So then I got https support on apache, had to recompile it enabling SSL, generate the certificate and finally create a virtual host for the service. It's pretty easy to install a virtual host with SSL after you have modssl compiled, you just have do to the following:

• Go to httpd.conf and uncomment the line referring to Including the apache ssl configuration, I'm talking about something like: Include conf/extra/httpd-ssl.conf. Now this file has all the information you need to have configured, you just need to check a few variables which are:
• Servername
• SSLCertificateFile
• SSLCertificateKeyFile
• DocumentRoot
• ErrorLog (recommended)
• TransferLog (recommended)

And in your vhost also had a location configuration saying that SSL is required for everything inside DocumentRoot, like for example:

<Location "DOCUMENT_ROOT_HERE">
SSLRequired %{SSL_CIPHER_USEKEYSIZE} >= 128
</Location>

Now a problem I had in this was, that I wasn't defining the port 443 in the virtualhost declaration resulting in a strange browser error "server terminated connection" without leaving any messages in the logs. So if you are having some kind of this errors, check if the port 443 is defined!
Now when you'll restart your apache webserver the key's certificate password will be asked and then https will also start.

Finally there's one thing missing, finding a way of getting the mails from pop accounts to the imap account, well that's not that hard, if you ever used mutt, pine or any other email client that didn't actually had pop3 support enabled you're familiar with fetchmail and procmail.

Fetchmail is the application that accesses the pop accounts (or any other type of email accounts) and downloads the mails. First step is to configure your .fetchmailrc file that will be put in your home directory. The syntax is pretty easy, imagine that you have two pop3 accounts one at mail.server.net and another one at mail.server2.net that uses SSL, your remote user on both is pabrantes with password, password and local user is username, here's the example:

Now has you can see we are passing the mails to procmail so we have to define what procmail will do with them. The procmail definitions are stored in another file called .procmailrc and it uses regular expressions to know what to do. Here's a small example:

Now the 1st part defines some variables, definitley not difficult and not much to say. Then we have a rule that looks for Subjects that contain the word Fenix, and stores them in the .Fenix folder as a new message.

You can then add more rules below and it will run one by one, and if none match then store the email at DEFAULT location.

Finally you have to get fetchmail running also as a daemon so from time to time it downloads your email and you can access via your MUA on the imap account or via webmail. To do that, you can simply type on the shell

This will make fetchmail check your pop accounts each 10 minutes (600 seconds).

The final result was this:

More comfortable no? Many of the exterior logins with my user will now disappear from the logs since I won't be logging to read my email!

So you want to try to do it yourself? Here's all the direct links for the software I used:

• Apache WebServer
• Courrier Authentication lib
• Courrier-Imap
• Fetchmail
• PHP
• Procmail
• Roundcube

You might also find interesting the following references:

• Creating a self-signed certificate
• Generating an SSL Certificate with Apache + mod_ssl
• Procmail basics

Hope you found this lecture interesting.