 Mail
Routing Using LDAP
Enrique Flores R.
In an effort to mitigate the spam and virus problems in our user
community, the IT group at Cypress Semiconductor Corporation has
implemented Brightmail Solution Suite, an anti-spam/anti-virus mailwall
solution. Brightmail's product refers to spam and infected
email as "graymail". For each user, the mailwall software
creates individual graymail mailboxes where illegitimate email is
sidelined. Through the use of a Web interface, users are given the
option to delete or retrieve their graymail messages. Retrieved
graymail, or email not filtered by the mailwall rule sets, is forwarded
to our site mail hub, where it is either spooled for local users
or relayed to the appropriate remote site. Figure 1 details the
specifics of our mail flow. Figure 2 is a flowchart depicting the
sidelining process.
In our environment, we store graymail in an NFS-mounted qtree
on one of our Network Appliance filers. Because of the NFS component
of our mailwall configuration, we chose maildrop as the Sendmail
local mailer for graymail delivery. Maildir was the chosen format
for the graymail mailboxes. Graymail is first sidelined in /usr/local/mailwall/gmspool.
Next, Sendmail queues it in /var/spool/graymqueue before maildrop
delivers it to the mailboxes on the NFS mount located at /var/graymail.
Every ten minutes, a "harvester" program collects graymail,
connects to Sendmail, and sends the messages to the NFS-mounted
graymail mailboxes.
We began to experience serious performance problems with our mailwall
systems soon after introducing this service. Further investigation
showed that the maildrop mailer was re-queueing graymail addressed
to unknown users. The logs recorded the following error:
Deferred: local mailer (/usr/local/bin/maildrop) exited with EX_TEMPFAIL
After a short time, the graymqueue directory would grow extremely
large with graymail that was failing to de-queue. The re-queueing
taking place was rapidly depleting system resources and halting service.
The only way to temporarily recover was to create a new graymqueue
directory and restart Sendmail. This procedure had to be repeated
each time the system's backlog reached a dangerous level.
Using LDAP
To alleviate this condition, we chose to address the root of the
problem -- processing mail to unknown users. We configured our
mail systems to handle only mail directed to valid users or aliases.
As a result, maildrop would only process graymail addressed to real
users. To do this, we chose LDAP to handle the validation and routing
of email. In our corporation, all our users have entries (accounts)
in our LDAP systems. By referencing our LDAP systems, only mail
verified by LDAP would be accepted.
LDAP
Namespace
We recently completed a redesign of our Directory Information
Tree (DIT). A DIT is a namespace that provides the means by which
directory data is named and referenced. As part of our redesign,
we created several branch points (organizational units) and categorized
data into functional units. Figure 3 is a diagram of our current
directory layout.
The root of our tree is o=cypress.com. From that point
down, Our DIT is divided into five units:
- ou=People, o=cypress.com -- Where corporate user
data is stored. For this Sendmail configuration, each user entry
was modified to include mail routing information.
- ou=Groups, o=cypress.com -- Holds group data.
- ou=Applications, o=cypress.com -- Where we store
application-specific data. This is where we store configuration
information for our LDAP-enabled applications.
- ou=Resources, o=cypress.com -- Stores resource information
such as printers, conference rooms, phone bridge codes, etc.
- ou=Services, o=cypress.com -- The branch that contains
data for our network services. NIS information and Sendmail configuration
data would be stored here.
We have future plans to LDAP-enable our NIS domain (mis.cypress.com).
For this reason, we created an organizational unit (ou) named ou=mis.cypress.com
under the ou=Services branch. We also created branch points
for our current NIS services. In the near future, our NIS maps will
be stored under their respective ou.
With Sendmail's extensive LDAP support (as of v8.12), we
also plan to store Sendmail configuration data, such as aliases,
maps, and classes in LDAP. With this in mind, an ou=Sendmail
branch under ou=Services was created, as well as ou's
for aliases, maps, and classes. For our configuration, we only implemented
the storage of aliases in LDAP. We have left maps and classes for
a future project.
Aliases
For this Sendmail/LDAP project, I converted our external aliases
file to LDIF (LDAP Data Interchange Format) format. See Listing
1 for a list of objectclasses and attributes used for our aliases.
These are all standard objectclasses and attributes present in our
directory server software; no schema modifications were necessary.
We plan to store internal (NIS) aliases under ou=aliases,
ou=mis.cypress.com, ou=Services, o=cypress.com.
External aliases are stored under ou=aliases, ou=Sendmail,
ou=Services, o=cypress.com.
In preparation for storing NIS maps in LDAP, I designed the format
for our NIS aliases map at the same time as the creation of the
format for our external aliases. The LDIF formats for our internal
and external aliases differ slightly. See Listing 2 for LDIF examples
of internal and external mail aliases.
The distinguishing elements between internal and external LDIF
entries are the objectclasses. Internal entries include the nisMap,
nisObject, and mailGroup objectclasses. External entries only include
the mailGroup objectclass. In our example, this is the characteristic
that separates internal and external aliases in LDAP.
Because our Sendmail configuration is only concerned with incoming/external
mail, I configured Sendmail to deliver exclusively to aliases with
the mailGroup objectclass. When Sendmail is searching the directory
for aliases, it will ignore any entries possessing the "nis"
objectclasses, despite the fact that these entries also contain
the mailGroup objectclass. A detailed explanation of how this is
done will follow later in this article.
Routing
The second piece of our configuration deals with mail routing.
All user entries under the ou=People, o=cypress.com branch
were modified to include mail routing information. Refer to Listing
1 for a list of objectclasses and attributes that have been added.
As an example of a user entry with mail routing attributes, I have
included a portion of my LDAP entry in Listing 2. Later in this
article is an explanation of how the routing decisions are made.
There is no distinction made between internal and external users.
All our users in LDAP possess an @cypress.com mail address
and accept mail from external sources.
Software
LDAP and Sendmail
The fundamental software used for our Sendmail/LDAP configuration
is as follows:
- Solaris 8
- Netscape Directory Server 4.15
- Sendmail 8.12.1
Solaris 8 was used because of its native support for LDAP. The
LDAP libraries are supplied by the SUNWlldap package. These libraries
are referenced during the compilation of Sendmail.
For the LDAP server, we used our existing directory server software,
Netscape Directory Server (v4.15). I made no changes to the directory
server's schema. I took advantage of the software's standard
objectclasses and attributes relevant to email. Please note that
the configuration information detailed here can also be applied
to open source directory software (e.g., OpenLDAP). We chose not
to use open source LDAP software because of current dependencies
and standardization on Netscape Directory Server.
The LDAP features present in Sendmail 8.12.1 made this version
of Sendmail appropriate for our needs. In version 8.12.1, LDAP can
be used for aliases, maps, and classes by either specifying your
own LDAP map specification or using the built-in default LDAP map
specification. LDAP can also be used for mail routing purposes.
This feature enables LDAP-based rerouting of a particular address
to either a different host or a different address. As mentioned
previously, we implemented LDAP for aliases and routing.
In this article, I am only detailing the installation and configuration
of Sendmail. For information regarding the installation of Netscape
Directory Server, visit: http://www.iplanet.com.
Supporting Software
In addition to a directory server and Sendmail, the following
software plays a supporting role in the Sendmail configuration:
- libgcc 3.0.3 libraries
- Berkeley DB 3.2.9
I downloaded Solaris packages for this software at:
ftp://ftp.sunfreeware.com/pub/freeware/sparc/8/libgcc-3.0.3-sol8-sparc-local.gz
ftp://ftp.sunfreeware.com/pub/freeware/sparc/8/db-3.2.9-sol8-sparc-local.gz
Both of these packages were uncompressed and then installed with the
following command (installs in /usr/local):
# pkgadd -d libgcc-3.0.3-sol8-sparc-local
# pkgadd -d db-3.2.9-sol8-sparc-local
The libgcc libraries are needed because of the library requirements
found in Berkeley DB. For this reason, the libgcc package was installed
first. Berkeley DB provides the database formats that can be used
for Sendmail alias files and general maps (aliases, virtusertable,
etc.).
NEWDB is the format used in our configuration. NEWDB is included
automatically in the Sendmail build if the Build script can find
a library named libdb.a or libdb.so. In order for the Build script
to be cognizant of NEWDB library and header files needed for compilation,
the following links were created:
# ln -s /usr/local/BerkeleyDB.3.2/lib/libdb.a /usr/lib/libdb.a
# ln -s /usr/local/BerkeleyDB.3.2/include/db.h /usr/include/db.h
# ln -s /usr/local/BerkeleyDB.3.2/include/db_cxx.h /usr/include/db_cxx.h
Once the links are made, the next step is to build Sendmail.
Compiling and Installing Sendmail 8.12.1
The current release of Sendmail can be found at http://www.sendmail.org.
I downloaded the gzipped source at:
ftp://ftp.sendmail.org/pub/sendmail/sendmail.8.12.1.tar.gz.
Uncompress and untar the source with the following command:
# gunzip -c sendmail.8.12.1.tar.gz | tar xvf -
A directory named "sendmail-8.12.1" will be created. Before
beginning the compiling process, it is important to read through the
various README files. There are new features in this version of Sendmail,
and the documentation should be studied before moving forward.
Security
Starting with version 8.12, Sendmail is more security conscious.
By default, the software will install without set-user-ID root.
To do this, the software must run as a set-group-ID program. Additionally,
there are two Sendmail processes that must be started. One is a
daemon process, and the other a submission program. There are also
two cf files that are used -- sendmail.cf for the daemon and
submit.cf for the submission program.
As part of the security enhancements to the software, a user account
and a group must be created specifically for the Sendmail processes.
It is recommended that the user and group both be named "smmsp"
with a uid and gid of 25. You must ensure that the security configuration
meets the needs and policies of your own organization. I suggest
the following permissions for the files and spool areas used by
Sendmail:
-r-xr-sr-x root smmsp /usr/lib/sendmail
drwxrwx--- smmsp smmsp /var/spool/clientmqueue
drwx------ root root /var/spool/mqueue
-r--r--r-- root root /etc/mail/sendmail.cf
-r--r--r-- root root /etc/mail/submit.cf
For more information regarding the security features of version 8.12,
refer to the "SECURITY" document in the source distribution.
Compiling
The instructions and syntax detailed here are for Solaris 8 only.
Refer to the site configuration document in devtools/Site/README
for additional information.
To build Sendmail with LDAPMAP (map used for LDAP features), the
following two entries must be made in the site.config.m4 file:
APPENDDEF('confMAPDEF', '-DLDAPMAP')
APPENDDEF('confLIBS', '-lldap')
The next step is to change to the sendmail/ directory. In this directory,
compile and install Sendmail by running:
# ./Build
# ./Build install
Sendmail will install in /usr/lib. The next step is to create a sendmail.cf
file (the submit.cf file is created and placed in /etc/mail during
the compile). The Sendmail download comes with generic cf files that
can be used. One of these can be used to test the startup of Sendmail.
Once the sendmail.cf file is created, start the Sendmail processes
by entering:
# /usr/lib/sendmail -L sm-mta -bd -q15m
# /usr/lib/sendmail -L sm-msp-queue -Ac -q15m
To verify the version of Sendmail and the features it was compiled
with, run the following:
# /usr/lib/sendmail -d0.1 -bt
Version 8.12.1
Compiled with: DNSMAP LDAPMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8
MIME8TO7 NAMED_BIND NDBM NETINET NETINET6 NETUNIX NEWDB NIS
NISPLUS PIPELINING SCANF USERDB XDEBUG
============ SYSTEM IDENTITY (after readcf) ============
(short domain name) $w = qbert
(canonical domain name) $j = qbert.cypress.com
(subdomain name) $m = cypress.com
(node name) $k = qbert
========================================================
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
>
For each of the associated Sendmail utilities (makemap, mailstats,
etc.), read the README file in the utility's directory. Before
installing a utility, be sure to make a backup copy of the utility
being replaced. To install, run ./Build followed by ./Build
install.
Sendmail Configuration
Now that the steps to building Sendmail have been listed, I can
expand on our configuration details. There are two mc files that
make up our mailwall system. One configuration is for our Sendmail
process that accepts and filters incoming mail. The second configuration
is for the process that moves graymail to the graymail mailboxes.
Configuration Details
The first mc file is configured to accept incoming mail on the
standard SMTP port, port 25. This configuration interfaces with
our mailwall filter through Sendmail's MILTER API. The mailwall
filter program is named "bmifilter" and it has been configured
to operate on port 5513 (defined by INPUT_MAIL_FILTER). Listing
3 contains the mc file specifics for our port 25 Sendmail instance.
Before the LDAP pieces were added to our system, mail not caught
by the mailwall filter was relayed to our site mail hub. With LDAP-based
routing configured, mail is relayed to the user's specific
"mailhost" value, as indicated in their LDAP entry. In
the mc file, LDAP routing is defined by the FEATURE() command
in the following manner:
FEATURE('ldap_routing', <mailHost>, <mailRoutingAddress>, <bounce>, <detail>)
- <mailHost> is a map definition describing how
to look up an alternative mail host for a particular address.
- <mailRoutingAddress> is a map definition describing
how to look up an alternative address for a particular address.
- <bounce> argument, if present and not the word
"passthru", dictates that mail should be bounced if neither a
mailHost nor mailRoutingAddress is found.
- <detail> indicates which actions to take if the
address contains +detail information. strip tries the lookup
with the +detail and if no matches are found, strips the +detail
and tries the lookup again. preserve does the same as "strip",
but if a mailRoutingAddress match is found, the +detail information
is copied to the new address.
In our mc file, the <bounce> and <detail>
parameters are left blank. For the <mailHost> and <mailRoutingAddress>
attributes, the LDAP search will look through LDAP entries containing
the "mailRecipient" objectclass. At our site, the mailhost
and mailroutingaddress values are set. Therefore, mail is relayed
to the listed mailhost using the mailroutingaddress. A user's
mailrouting address is his uid plus his mailhost (e.g., uid@mailhost.mis.cypress.com).
This information was gathered from our existing aliases file and
imported into all LDAP user entries.
Sendmail makes routing decisions based on the following matrix:
mailHost is mailRoutingAddress is Results in
----------- --------------------- ----------
set to a "local" host set mail delivered to
mailRoutingAddress
set to a "local" host not set delivered to
original address
set to a remote host set mailRoutingAddress
relayed to mailHost
set to a remote host not set original address
relayed to mailHost
not set set mail delivered to
mailRoutingAddress
not set not set delivered to
original address *OR*
bounced as unknown user
ALIAS_FILE
Mail not routed by the ldap_routing FEATURE is handed to the local
mailer. Mail that gets this far is mail addressed to an LDAP entry
that does not contain the mailRecipient objectclass (i.e., a mail
alias). At this point, the ALIAS_FILE definition is consulted to
determine where to deliver the message. An LDAP search query is
the first parameter in the alias definition. The query reads:
ldap:-k (&(objectClass=mailGroup)(!(objectclass=nisMap))(mail=%0)) -v mgrpRFC822MailMember
%0 is the address that has been passed to the local mailer.
The search filter is prepended with a &, which equates
to the Boolean operator AND. The search filter is translated
as follows:
Find an entry that contains:
- A mail attribute set to the address in question (mail=%0),
and
- An objectclass of mailGroup (objectclass=mailGroup), and
- Does NOT contain an objectclass of nisMap (!(objectclass=nisMap))
- Once such an entry is found, deliver the message to the address(es)
set in the mgrpRFC822MailMember attribute.
This search filter will ensure that only mail for external aliases
is accepted. Recall that both internal and external aliases contain
the mailGroup objectclass, but external aliases do not contain the
nisMap objectclass. Therefore, this filter will only find external
mail aliases. Anything not found in LDAP will be searched for in
/etc/mail/aliases (the second parameter in ALIAS_FILE).
Other LDAP Particulars
confLDAP_DEFAULT_SPEC defines the default LDAP settings.
This value contains LDAP-specific settings such as -h host
and -p port. Our example is set to use the standard LDAP
port (port 389) and to search starting from the root of the DIT
(-b o=cypress.com). Redundancy has also been designed. By setting
the host portion to -h ldap1 ldap2 ldap3, three directory
servers will be searched in sequence. If ldap1 is not found, then
ldap2 is contacted. Finally, queries will be sent to ldap3 if ldap2
is not available.
The domain routed via LDAP is defined by LDAPROUTE_DOMAIN().
Alternatively, LDAPROUTE_DOMAIN_FILE() can be used to list
a file that will hold the domain information. In Listing 3, I have
included a commented-out example of the LDAPROUTE_DOMAIN_FILE
syntax.
Graymail Sendmail
The second configuration runs on port 5512 and is responsible
for processing graymail. This configuration is very similar to the
first, with the exception of ldap_routing. In this instance,
LDAP is strictly used as an ALIAS_FILE reference. Since graymail
is being delivered to both aliases (mailGroup objectclass) and regular
users (mailRecipient objectclass), both types of recipients must
be included in the ALIAS_FILE search query. Listing 3 also lists
the mc file for our port 5512 Sendmail instance.
According to our configuration, if LDAP does not return a result,
then the next source looked at is /etc/mail/aliases. If nothing
is found in the local aliases file, then the local user relay (LUSER_RELAY)
handles the message. LUSER_RELAY is set to local:nobody.
This setting results in graymail being sent to "nobody"
if the recipient is not found in LDAP or in the local aliases. Since
"nobody" is set to /dev/null in the aliases file, mail
not validated locally or by LDAP is deleted.
Also note that an SMTP MAILER is not included in this mc file.
The only MAILER configured is "local". We only want a
local mailer because the job of this process is to deliver to graymail
mailboxes on a "local" NFS mount (/var/graymail). When
the cf file is built, by default mail.local is set as the local
mailer (Mlocal). The cf file must be edited manually in order to
replace mail.local with maildrop. Listing 3 contains the local mailer
definition (path, flags, etc.) from our cf file.
Testing LDAP Interaction
Once the cf file is built, Sendmail's interaction with LDAP
can be tested in the following manner:
# /usr/lib/sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> /map ldapmh ehf@cypress.com
map_lookup: ldapmh (ehf@cypress.com) returns mailhost.mis.cypress.com (0)
> /map ldapmra ehf@cypress.com
map_lookup: ldapmra (ehf@cypress.com) returns ehf@mailhost.mis.cypress.com (0)
In this example, I am looking up my (ehf) mailhost and mailroutingaddress
attributes through the ldapmh and ldapmra maps. These
maps were made available by the LDAPMAP definition added to the site.config.m4
file during the build procedure. A successful match will return the
respective value provided by LDAP. An unsuccessful match will return
the statement "no match".
Conclusion
The positive aspect of this configuration is the ability to centrally
manage mail routing and alias information along with the rest of
our user profiles. LDAP provides an easy and convenient way to administer
user information. Our heavy reliance on LDAP is also a negative
point. In our situation, if LDAP is not available, mail cannot be
routed and aliases cannot be expanded. Mail from the outside world
stops completely. Our significant dependency on directory data demands
that we provide a highly available LDAP service. To prevent any
mail service outage, the failover configuration implemented by confLDAP_DEFAULT_SPEC()
will ensure a very low probability that Sendmail will be unable
to contact a directory server. Figures 4 and 5 diagram the final
architecture for mail flow and graymail sidelining.
Incorporating LDAP into our Sendmail build solved our problem
with re-queuing graymail. By deleting unsolicited mail with LUSER_RELAY(),
we were able to greatly diminish the amount of processed graymail.
Most importantly, by only routing mail addressed to users and aliases
found in LDAP, the objective of validating and processing legitimate
mail was met.
References
1. T. Bialaski, M. Haines, Solaris and LDAP Naming Services:
Deploying LDAP in the Enterprise, Sun Microsystems Press, 2001.
2. Sendmail 8.12.1 documentation, "Sendmail Configuration
Files".
3. http://www.sendmail.org
4. http://www.brightmail.com
Enrique Flores is a Sr. UNIX Systems Administrator at Cypress
Semiconductor Corporation in San Jose, California. He has worked
with LDAP since 1998, and began working with Sendmail in 2000. Enrique
received his undergraduate degree from UCLA in 1997 and was enrolled
in graduate studies at SUNY, Albany until the end of 1998. He can
be contacted at: ehf@cypress.com.
| 
