Cover V11, I07

Article
Figure 1
Figure 2
Figure 3

jul2002.tar

Building a Web Mail Server with SquirrelMail

Brent Bice

About a year ago, I was told that the IPSec-compliant VPN, the SSH server, and a Portmaster 3 with a digital toll-free service were all entirely unacceptable for some of our users to use to check their email. When I asked what would be acceptable, I was told that the users should be able to check their email from anywhere with any Internet connection (no matter what bizarre, proprietary protocols were used) using anyone's Web browser.

Naturally, the first thing I did was to surf to http://www.freshmeat.net/ to see whether the open source community had a Web-to-email gateway. Okay, I admit it -- I even looked at several commercial packages. Being the good, paranoid admin, I added a few additional requirements of my own. The system had to work with Apache (so it would be reliable) and mod_ssl (to help prevent users from sharing their passwords with the entire Internet). It also could not require mailboxes to be stored locally, which meant that the server could not be required to run on the same server as the POP3/IMAP server. This way, even if the Web mail server was compromised, there would be no password or shadow files or mailboxes for a cracker to download, and there could be a firewall between the Web mail server and the rest of my servers.

There were a few packages that fit this bill, but once I found SquirrelMail and started using it, it was no contest. SquirrelMail (http://www.squirrelmail.org) is an open source Web mail package written in PHP4. SquirrelMail provides a uniform, full-featured interface for users to send, check, and compose email through an ordinary Web browser. All SquirrelMail pages appear as pure HTML 4.0 (with no extraneous Javascript to garble the reception), so SquirrelMail is compatible with a wide range of browsers. The SquirrelMail Inbox and Folder list are shown in Figure 1.

Because SquirrelMail is an IMAP email client, users can have many email folders. Users can also set up filters to automatically separate messages into different folders (a great feature for those of us who read mailing lists). There are anti-SPAM filters, a spell checker, and even a plugin for translating messages into foreign languages. If you have an LDAP or "Directory Services" server, SquirrelMail can use it to look up email addresses, or even provide a corporate roster.

It even has support for displaying messages sorted by thread as well as sorted by date, as long as your IMAP server suports it. Because SquirrelMail is an IMAP client, most of the processing and handling of the email messages takes place on the IMAP server, so it's still very fast, even if you're accessing it from overseas or across a 56k dialup. Moving a message with a 15-meg PowerPoint attachment from your inbox to some other folder is just as fast across a 56k dialup as across a T1.

Installation of SquirrelMail itself is extremely easy. But since I usually use Solaris or Solaris x86, I find that configuring, building, and installing the Web server takes more effort than installing SquirrelMail, so I'll start with setting up the Web server. If you're comfortable using rpm files or already have a PHP- and mod_ssl-enabled Web server, just skip to the section on installing and configuring SquirrelMail. If you'd rather build your own, read on. I'll include a list of URLs where you can find the bits and pieces necessary in the resources section at the end of the article.

Building the Web Server

PHP will need to be able to make IMAP and LDAP function calls, so to begin, we'll need those libraries compiled and installed somewhere before we build PHP. If you're going to have hundreds or thousands of users, you should consider also having MySQL installed. If you already have all these things installed, skip forward to Install Apache.

tar xvfz imap-2001.FINAL.tar.Z
cd imap2001-RELEASE-CANDIDATE.1
make slx   (or make <system_type> -- read Makefile to find your type)
    slx is glibc-linux
    gso is GCC Solaris

Next, build and install OpenLDAP
tar xvfz openldap-stable-20010926.tgz
cd openldap-2.0.15
./configure --prefix=/usr/local/openldap2.0.15
make depend
make
If all you want openldap for is the library needed for PHP, then you can just build it without threads and without a fancy, fast back-end like BerkeleyDB. In that case, instead of the above config line, you can use one like:

./configure --prefix=/usr/local/openldap2.0.15 --without-threads \
            --enable-ldbm --with-ldbm_api=ndbm
Install Apache
Before we build PHP, it's a good idea to start with a simple Apache install, test it, add PHP, test it again, then add mod_ssl.

tar xvfz apache_1.3.20.tar.gz
cd apache_1.3.20
./configure --prefix=/opt/apache --enable-module=most
make
su
make install
Edit /opt/apache/conf/httpd.conf; check settings such as Port, ServerName, DocumentRoot, etc. Then start it with the /opt/apache/bin/apachectl start command, point your browser at the new server, and see whether you get the default Apache welcome Web page. You can stop the server with the /opt/apache/bin/apachectl stop command.

Adding the PHP Module

Once you have a working Apache server, you can add the PHP module. I compiled and installed PHP as a static module. (See the README files in the PHP distribution for help making it a dynamically loadable. You'll need to build Apache slightly differently, so check those READMEs, too.) Also, since you never know what optional PHP features a plugin might require, you might as well enable any PHP features you can (assuming you already have the required libraries installed on your system). The IMAP and LDAP features, however, are mandatory for SquirrelMail. And, if you'll have hundreds or thousands of users, you'll probably want MySQL to store user preferences and address books in.

tar xvfz php-4.2.1.tar.gz
cd php-4.2.1
./configure  --with-mysql --with-apache=../apache_1.3.20 --enable-track-vars \
             --with-imap=../imap-2001.RELEASE-CANDIDATE.1 \
             --with-config-file-path=/opt/apache \
             --with-ldap=/usr/local --with-gd --with-jpeg-dir=/usr/local \
             --with-jpeg --with-xpm-dir=/usr/local --with-xpm \
             --with-ttf=/usr/local --with-snmp=/opt/ucd-snmp
make
su
make install
make su-install
Now rebuild Apache with PHP enabled:

cd ../apache_1.3.20
./configure --prefix=/opt/apache --enable-module=most \
            --activate-module=src/modules/php4/libphp4.a
make
su
make install
cp ../php-4.2.1/php.ini-dist /opt/apache/php.ini (or wherever you have
Apache installed).
Edit /opt/apache/conf/httpd.conf, search for "And for PHP 4.x, use:" and uncomment the AddType lines below it. Restart the Apache server and test it to see whether PHP is working:

/opt/apache/bin/apachectl start
Create a PHP file (/opt/apache/htdocs/bobo.php in my case) with one line:

<? phpinfo(); ?>
Point your browser at that file -- http://localhost/bobo.php in my case. If you see the PHP Info page, you're ready to move on to mod_ssl.

Install openssl if you don't already have it.

tar xvfz openssl-0.9.6b.tar.gz
cd openssl-0.9.6b
./config --openssldir=/opt/ssl
make
su
make install
Now, install mod_ssl.

tar xvfz mod_ssl-2.8.4-1.3.20.tar.gz
cd mod_ssl-2.8.4-1.3.20
Set up your environment to use your openssl installation. Mine is installed in /opt/ssl for this example.

setenv SSL_BASE /opt/ssl
setenv PATH /opt/ssl/bin:$PATH
rehash
./configure --with-apache=../apache_1.3.20
Now, go back into Apache and reconfigure and recompile it with the mod_ssl module enabled.

cd ../apache_1.3.20
./configure --prefix=/opt/apache --enable-module=most \
            --activate-module=src/modules/php4/libphp4.a \
            --enable-module=ssl
make
make TYPE=test certificate
      Answer the questions.  Here's what I did for my test server:
      Signature Algorithm - RSA
      Country Name - US
      State - California
      Locality - San Mateo
      Organization Name - Persistence Software Inc.
      Organizational Unit Name - Sysadmins
      Common Name (ie. www.domain.com) - localhost.localdomain
      Email address - sysadmins@persistence.com
      Certif. Validity - 365 days
      Certificate Version - 3
      Encrypt the private key now? - N
Note that because the server key files are NOT encrypted, you must ensure that they are NEVER compromised (obtained in any fashion by someone else). If you do encrypt them, you'll be prompted for a password every time you restart the Web server (including when you reboot the system and your auto-start scripts try to start your snazzy Web server):

su
setenv PATH /opt/ssl/bin:/usr/local/bin:$PATH":/usr/ccs/bin"
setenv SSL_BASE /opt/ssl
rehash
Move the existing httpd.conf file aside so make install creates a new one with the SSL features in it.

mv /opt/apache/conf/httpd.conf /opt/apache/conf/httpd.conf.premodssl
make install
Edit /opt/apache/conf/httpd.conf. I changed these settings:

ServerAdmin sysadmins@persistence.com
Port 80
<IfDefine SSL>
Listen 80
Listen 443
</IfDefine>

<VirtualHost _default_:443>    (used to be 8443)

DirectoryIndex index.html index.php
Remember to uncomment the PHP4 AddType lines again. Note that the above config lines tell the server to accept both SSL and non-SSL connections. You may not want to do this if you want your users always to use SSL to protect their username/password pairs. Alternatively, you can leave it supporting both, provided you have a firewall between the Web server and the outside world to permit only SSL connections to the Web server from the outside.

Now, kill the running non-SSL server and start up the SSL server:

/opt/apache/bin/apachectl stop
/opt/apache/bin/apachectl startssl
Now try accessing bobo.php again, this time using a URL such as "https://localhost.localdomain/bobo.php". If you get the PHP info page, then congratulations on setting up your PHP and mod_ssl Web server! At some point, you may want to request a signed certificate from one of the many certificate authorities and tweak your httpd.conf file to use it. In the Apache conf/ssl.csr directory, you'll find a certificate signing request file for your server and a README.csr file explaining how to use it to request a signed certificate.

At this point, you could install SquirrelMail. But there's one last tweak we should make to your PHP Web server. The Zend PHP Optimizer makes SquirrelMail (and other PHP scripts) much faster. It's worth the few moments it takes to install the Optimizer. Download the correct Zend binary for your version of PHP and your OS, then extract and install it. At the time of writing, this was as simple as extracting a gzipped tar file, running an install shell script, and telling it where your Apache server is installed and the location of your DocumentRoot.

Next, restart Apache, and try looking at bobo.php again. It should work and also show you the version of the ZendOptimizer you're now running.

Installing SquirrelMail

Now that you have an amazingly fast SSL- and PHP-aware Web server, let's install SquirrelMail. Go to your DocumentRoot and extract SquirrelMail. At the time of this writing, SquirrelMail 1.2.6 is the latest "stable" version, but don't be shy about trying newer versions. There are many improvements, and some of the plug-ins only work with the latest version. Although it isn't considered production code, I have been running various non-production versions of SquirrelMail for a long time now and have so far only encountered the occasional cosmetic problem (with fixes usually available before I find the problems). Of course, you should test the newer versions before turning your users loose on them.

I usually extract the latest CVS version and rename the directory to "squirrel.new" and extract the latest production (or the latest CVS version I've tested and trust) and rename it to just "squirrel" so that I have both versions available for use on the server. After extracting SquirrelMail, change the owner of the squirrel tree to be the same as the User option in your httpd.conf file.

For instance, on my example server I did:

cd /opt/apache/htdocs
tar xvfz ~bbice/squirrelmail-1.2.5.tar.gz
mv squirrelmail-1.2.5 squirrel.new
chown -R nobody squirrel.new
chgrp -R nobody squirrel.new
Next, move the data directory to somewhere outside the DocumentRoot. It will contain attachments that your users are sending while they're still formulating the message, their address books, and their user preferences. So you want to be sure that data won't be served up to anyone by your Web server. As always, double-check the INSTALL file to see if there are any additional recommendations. On my example server I did:

mv squirrel.new/data /opt/apache
mkdir /opt/apache/data/attachments
chmod 730 /opt/apache/data/attachments
chgrp apache /opt/apache/data/attachments
Next, cd to squirrel/config and run the conf.pl Perl script. You'll see a menu of configuration options (see Figure 2). Press "D" to set the mail server options to a set of defaults tailored for your mail server. For instance, because I use an IMAP server based on the UW IMAP code, I select "D", then enter "uw". Back at the main menu, go through all the config options and double-check them. If you have trouble seeing the configured values, enter "C" to turn off the color option.

Since you moved the data directory, you MUST change the "Data Directory" option in the General Options menu. I prefer to set the Attachment Directory to a separate directory from the data directory so that I can periodically look for attachment files that never got sent and purge them (e.g., /opt/apache/data/ and /opt/apache/data/attachments/). The attachments directory should not be owned by the user the Web server runs as, but should be mode 730 and owned by the group the Web server runs as.

If you install multiple versions of SquirrelMail, you must do these steps for each installation. It is possible, most of the time, for different versions of SquirrelMail to share the same data directory, but it's not recommended. So far, I've had no trouble with multiple versions sharing one data directory.

Once you have verified all the options (especially the IMAP-, SMTP-, and LDAP-related ones), save your changes and exit conf.pl. Try pointing your Web browser at the index.php file in the squirrel distribution. In my case:

https://localhost.localdomain/squirrel.new/index.php or
https://localhost.localdomain/squirrel/index.php
You should see a SquirrelMail login prompt (Figure 3). If you can't log in, double-check your IMAP server settings. You should be able to log in and should be presented with a list of messages (assuming there are some in your mailbox and your IMAP server is reachable and functioning properly). The user interface is pretty self-evident, so I'll skip it and move on to plug-ins.

Plug-ins

Take a look at http://www.squirrelmail.org/plugins.php for a list of plug-ins available for SquirrelMail. There are more plug-ins than you can shake a stick at. If there's something you don't like about SquirrelMail's interface or there's some feature you wish it had, chances are it's already been implemented with a plug-in. Some of my favorites are the Filters plug-in (automatically separate your inbox into separate folders and optionally filter out SPAM), the LDAPquery plug-in (an interface to the LDAP servers allowing you to search and find more data than just user names and email addresses), Squirrelspell (a spell checker), Attachment Handlers, Printer Friendly, DeleteMoveNext, and SPAMcop.

Some of these plug-ins became so popular that they have been added to the core of the latest version of SquirrelMail. If you see a plug-in listed on the SquirrelMail plug-ins page that says "Added to core", do not download and install -- it should be built in. For instance, the Filters plug-in is still listed on the page in case people running old versions of SquirrelMail want to get a newer version of this plug-in, but it's part of the SquirrelMail core for the latest versions of SquirrelMail and installed by default. You simply need to enable it, not install it.

To install a plug-in, simply cd to the "squirrel/plugins" folder and extract all the plug-ins you'd like to install. In each plug-in folder you should look for README or INSTALL files. Usually they will just give you tips on what settings (if any) you either must or can tweak in a setup file (usually named config.php or setup.php). Then you run conf.pl again, go to the "Plugins" menu, and enable the plug-in. That's all there is to it.

The Filters plug-in is a little more complicated. After reading the README file, you'll see that there is one variable that you should change for best performance on SPAM filtering. This variable tells the plug-in how to identify the one line in your email headers that contains the first hop into your network. The plug-in checks the IP the message came from (found on this line in the message header) to see whether it's in any of the SPAM databases the user wants checked, like RSS, DUL, or RBL. There are options you can play with that change how long DNS queries should be cached, and even an optional multi-threaded bulk DNS query program you can compile and install for extra-fast SPAM checking.

The LDAPquery plug-in also needs a little configuring. You must tell the plug-in about the attributes your LDAP server has, what the user should be allowed to search by, etc. There are some examples of these settings in the config.php file in the "ldapquery" directory, so it's not too painful.

Some of these plug-ins also have options that can be set by each user. You can see the settings by logging in to SquirrelMail and clicking on the Options link (and possibly another link on the Options page for an entire page of settings for that plug-in).

If your mailboxes are very large like mine, or if you apply a lot of filters using the Filter plug-in, you may find that the PHP scripts take longer than a minute to run. PHP, by default, won't allow a script to run more than 60 seconds. If you encounter this problem, you can edit the php.ini file and edit the "max_execution_time" variable to increase the maximum time a script is permitted to run. The anti-SPAM filters can also be optimized by compiling and installing the "bulkquery" program that is included with the Filters plug-in. In my case, this change made the SPAM filters run anywhere from 2 to 10 times faster depending on the latency of my Internet connection (high-latency connections got the best benefit).

Database Backend

You may also want to enable the MySQL backend for user preferences and address books. This was put in primarily for the use of servers handling very large numbers of users. Rather than having a file for every user in the data directory (which gets cumbersome with thousands of users), you can store this data in a MySQL database. This also allows you to set up a cluster of SquirrelMail servers that all use the same MySQL database to save and load user preferences and address books!

To do this, first create a database to contain the user preferences and address books, and create a table for each. Then grant permission to access these tables to the SquirrelMail servers. These tables could be in separate databases, and even on separate machines, but for simplicity I'll put them in a single database for my example. (Be sure to read the doc/db-backend.txt file in case it's been updated since this writing.) See the MySQL Web site for instructions on downloading and installing MySQL. Once it's running, as the MySQL user, run:

mysql -p                (supply a password if needed)
CREATE DATABASE sm;
USE sm;
CREATE TABLE userprefs (user CHAR(128) NOT NULL DEFAULT '',
                        prefkey CHAR(64) NOT NULL DEFAULT '',
                        prefval BLOB NOT NULL DEFAULT '',
                        primary key (user,prefkey));
CREATE TABLE address (
   owner varchar(128) DEFAULT '' NOT NULL,
   nickname varchar(16) DEFAULT '' NOT NULL,
   firstname varchar(128) DEFAULT '' NOT NULL,
   lastname varchar(128) DEFAULT '' NOT NULL,
   email varchar(128) DEFAULT '' NOT NULL,
   label varchar(255),
   PRIMARY KEY (owner,nickname),
   KEY firstname (firstname,lastname)
);
GRANT ALL ON sm.* TO sm@localhost IDENTIFIED BY "mypassword";
Next, cd to the functions directory and do:

cp prefs.php prefs.php.orig
cp db_prefs.php prefs.php
Lastly, cd to the config directory and run "perl conf.pl" again, select "Database", then "DSN for Address Book". Enter in a DSN (or Data Source Name) for your database, such as:

mysql://sm:mypassword@mysqlhostname/sm
Do the same thing for "DSN for Preferences". The general form for these DSNs is:

dbtype://user:password@hostname/dbname
Next, try to log in to SquirrelMail again, and you should be using the MySQL database to load and save user preferences and address books. Congratulations! You have a Web mail server capable of serving ridiculous numbers of users.

Although I first chose SquirrelMail as simple way to solve a problem for my users, it's becoming my favorite email client. With the addition of anti-SPAM filters in the Filters plug-in (where I can get far more aggressive with filtering than I can do corporate-wide on our firewall), the spamcop plug-in for reporting SPAM, the LDAPquery plug-in for looking up roster info, and now a calendar plug-in, SquirrelMail has become very full-featured.

Resources

http://www.squirrelmail.org/
http://www.squirrelmail.org/plugins.php
http://www.washington.edu/imap/
http://www.openldap.org/
http://www.openssl.org/
http://www.mysql.org/
http://httpd.apache.org/
http://www.php.net/
http://www.modssl.org/
http://www.zend.com/
Brent Bice has worked as a programmer, network admin, or systems admin on a variety of UNIX-based systems since 1989. He is currently employed at Persistence Software Inc as Senior System/Network Admin and can be reached at: bbice@persistence.com.