Mail and Sendmail Administration: Part 1
Bruce H. Hunter Mail has two sides -- the obvious front ends like the mail commands and the mysterious mechanisms that get mail delivered. Much of what goes on in UNIX is a mystery to the users, but in the case of mail, and particularly sendmail, the mystery may even extend to the systems staff. Let's see if we can shed a little light on what makes mail work. From the user perspective, mail starts with the mail front-end commands, which are programs in their own right. The first, and most familiar, of the mail commands is /usr/ucb/mail (most users just use mail, but in fact they are using the Berkeley version). The AT&T version, /bin/mail, is still extant, but because it lacks some of the features we have all come to love in the Berkeley versions, relatively few people use it. Both of these programs accept mail and (seem to) deliver mail, though actual delivery agent(s) are separate, I will explain later. Mail at its simplest is intended for mail collection and delivery on a local system. The need for a more sophisticated delivery agent arose when UUCP appeared. With UUCP the issue of address resolution had to be taken care of. There were now addresses like
% mail jaeger!bhunter
With the form system!user, the "!" or bang character separated the system name from the user name (in that order). But UUCP is a switched network, which means that it uses the telephone lines. Very soon users were using mail hops, sending mail by way of other systems to keep down the cost of transmission. One system would send to another and that one to another until the message would either be delivered or time out and get erased. Now the form could be:
apex!acme!stalker!jaeger!bhunter
Berkeley added new dimensions and problems to mail. Berkeley programs are readily customizable, both for the user and at system level. The customizing is usually done with special files, and mail is no exception. ~/.mailrc A user's mail command gets customized in the users $HOME/.mailrc. rc stands for run command, and mailrc is the run command that tells mail how to behave. It is here that aliases are set up and features are defined. Options that can be specified include the use of vi as the editor within mail, mail notification (biff), and a dozen or so others like cc and subject lines. Here is a very simple .mailrc file.
17% cat .mailrc set append dot autoprint set folder=/usr/system/bhunter/mcdmail alias george gsmith@flsm1 alias dave dhill@flsm1 alias chad chad@linus.mitre.ORG alias thegroup chad,dave,george
The set commands set such attributes as append, which appends new mail to end of the user's mail file; dot, which indicates that a period on a line (by itself) is the EOF; and autoprint, which prints each file to the screen as fast as the last is erased. ~/.forward Another innovative Berkeley file, the ~/.forward file, forwards user mail. The file only need contain the next name-address pair in standard UNIX uname@host.org_name.org_ext format. The Constituents of a Mail Message Mail comes and goes in two parts, the message and the envelope, just like surface mail. The message is what the user types in, the envelope is what we see as header information, plus a little more. If you are fast enough, you can occasionally catch an undelivered piece of mail in /var/spool/mqueue. In the mqueue directory are two files for a given message: in the df* file is the message itself (data), while in the qf* file is the queue information or envelope. The following is a sample of the contents of an mqueue directory:
mfg# file dfAA07738 qfAA07738 dfAA07738: English text qfAA07738: ascii text
Figure 1 shows a sample qf file. Notice the prefixes to the header lines:
D the data file name M message (printed by mailq command) S sender address H header definition R recipient(s) E error address
And here's the sample:
mfg# cat dfAA07738 WARNING WARNING WARNING AUSSIE File Server svr05 down 5-6PM Monday downtime 60 minutes to add DISK SPACE
And surprise of surprises there is no magic -- just the message. The two make up the mail message with some or all of the header information prepending the message when it winds up in [var | usr]/spool/mail/user_name. Mail Agents The mail agent has two functions: it collects mail and makes the final delivery. Some mail agents in current use include /usr/ucb/mail (the most familiar, BSD mail); /bin/mail (AT&T mail, the original); mailx; mailtool; elm; and mh. If a mail agent can deliver mail locally, it will, but today's networked computing paradigm almost always involves the Internet and/or UUCP. To get mail the through the network requires a mail routing agent such as sendmail, mmdf, or smail. The most well-known, of course, is sendmail, which originated at UCB and was written by Eric Allman. Once past the mail agent, the message still needs media and protocols in order to be delivered. Mail delivery agents at this level include Ethernet-TCP/IP (SMTP); UUCP (circuit switched lines or hardwire); and /bin/mail (for local delivery). Basic Mail Files Mail, like most facilities in UNIX, has several files and directories. The repository for delivered mail is /var/spool/mail/*. This directory includes a file for each user (by the user's name) that contains the header information followed by the message. Each new message is appended to the file, and each starts with the string "From." As noted earlier, a queuing place for mail is /var/spool/mqueue. The error message file is usually here as well. The Basic sendmail System All systems but one in a mail "domain" have a very simple sendmail (configuration) file. Each will try, without overexerting itself, to deliver the mail to a known host. If sendmail can clearly see that it must use the network for delivery or if it can't figure out how to make the delivery, it sends the mail on to the one different system, the mail master. The mail master has a very complex sendmail.cf file capable of performing the address translation required for getting mail to a relay host if need be. Relay Hosts Relay hosts have direct connections to the Internet. Messages needing the Internet for deliverey are routed to the relay host, and the relay host sends it on its way. The relay host works in the other direction, also, receiving and forwarding incoming mail from the Internet or UUNET. Mail Working with DNS Maintaining organization-wide host tables became a nightmare a long time ago, so most sites have begun to use DNS. With DNS each local domain takes care of its own host files and the DNS mechanism effectively merges all the local host files to maintain a virtual single host table that is accurate and up-to-date. sendmail has kept up with the times and can take advantage of DNS to effect timely mail delivery. An Introduction to sendmail.cf The key to sendmail is the configuration file(s) sendmail.cf, which routes mail by analyzing mail addresses and rewriting them for final delivery. Given the universe of possible addressing conventions used now and in the past, this is no small task. Most of us have gotten used to seeing the format
user@host.domain.domain_type
as in
schultz@peanuts.determined.com
but how about those UUCP addresses like
determined!peanuts!schultz
The ordering is different and so are the delimiters. Mail Delimiters It's the mail delimiters like "@" and "!" and "." and "::" that make it difficult to go from one addressing scheme to another. sendmail manages this through the definitions section typically found at the beginning of sendmail.cf files. The delimiters definition is one of the first and serves to show how definitions work. The single-character mnemonic operator D is a define. What is defined is called a macro and the macro for delimiters is o. Therefore in sendmail.cf the definition of the o delimiter macro is:
Do.:%@!^/[]
Note the lack of white space -- there is nothing to make this easy to read; parsing is done by having only one character for each mnemonic. Addressing Translations To reiterate, mail is sent in two parts, the message and the envelope. The header information must deal with addressing schemes and sendmail must do the address transformations. Table 1 shows a few of the many possible addressing schemes as well as the format into which they must be transformed in order to work on UNIX and with sendmail and other mail and routing agents. Performing these address transformations is not a simple task. The process involves dozens of macros and definitions, several rule sets governing how the transformations take place, and a score or so of mnemonics and variables, as well as a few constructs like conditionals. One observant system programmer noted to me that sendmail.cf's notations, rules, and rule sets are not unlike a make file. Internal Names (Canonical Form) and "Focus" Mail addresses must have a consistent form while internal to sendmail. In addition, a "focus" -- represented by the <> operators -- is added to underscore, in a virtual sense, part of the address. The following are internal names with focus applied to the domain part of the name, leaving the user and host names and the token @ unfocused.
sam@<apex.uucp> sam@<apex.dnet> sam@<apex.ibm>
The form represented here is called canonical form. This form is required by one of the sendmail rulesets, ruleset 0, for message delivery. Mail Routing Made Sane Before plunging any deeper into sendmail.cf, I want to look at the mail agents again and discuss some of their functions. Users do not always know the domain name of the person to whom they want to send mail -- and even if they do, they are not likely to know the hostname of the recipient's machine. If there are 10,000 nodes in an organization, how is one to learn the full user@host.domain name? Part of the problem can be resolved by the company's using a domain name service and well known aliases. If I am in an organization and know the name or acronym of a department, why can't I use it to get mail to a user in that organization? If the design tool group is known as CAD, for example, I should be able to use
jsmith@cad
to get to John Smith in that group. This is done, at least in part, with aliasing. The mail master, a computer used for the final gathering and delivery of mail, would have an alias of cad. In the host's files it would look like
123.21.60.122 svr08 cad
Now all systems can find cad even though no machine by that name really exists. Should the mail master move from svr08, only the alias would have to be changed (on all systems) to make it work. Mail Aliasing and Mail Groups Mail aliasing also works at the user's level in the mail setup files like ~/.mailrc. These aliases are used to set up mailing lists as well.
alias group,jsmith@ws05,anguyen@ws48,swesson@ws357
Mail forwarding is done by way of the users .forward file and through aliases. The ~/.forward file simply contains the user_name@host.domain where the mail is to be forwarded. Aliases may also be used for alternate names (hunter, bruce, bhunter). A domain-wide list of user aliases can be maintained and distributed via NIS with a single central aliases file that is used by mail. Traditionally, this file is /etc/aliases, but if you replace that file with a YP map, mail delivery gets much simpler and more effective. For example, if I have a machine called jaeger, few people will know that name and fewer still will know the name of the mail master, svr04. I can use aliases to tie in the two:
bhunter:bhunter@svr04.
To simplify even further, don't have a /var/spool/mail directory on every machine. Instead, set up one huge mail directory on one server and NFS-mount it to all the domain's systems. With all of this aliased to the domain's organizational name or TLA (three-letter acronym), all of the following will work:
bhunter bhunter@jaeger bhunter@svr4 bhunter@XYZ
Mail Setup for Server and Workstations Mail was originally intended to exist on one system which would service all users. Almost all mail was for local delivery, and little, if any, came in from the outside. Today's paradigm is tens to hundreds of workstations using several servers, all to form a single domain. Larger organizations will require several domains. Within a domain, mail could exist with each system having its own mail spool area (like usr/spool/mail or /var/spool/mail), but this would create many problems, among them delivering mail to users whose system names are unknown or who move from workstation to workstation; mailing to the domain without knowing the system name; keeping the /var or /usr areas from filling up with unread mail; and being unable to spool error messages in a single place where they can be acted upon. It is easier in the long run to have a single mail server -- a system that not only is the mail master but also has all the mail on a single disk partition, disk pack, or concatenated virtual disk. With that system's name aliased to the domain or organization name, addressing mail from the outside is painless. That one system also has access to the YP aliases map and therefore can find any user. All that remains is to mount the master's mail directory on each workstation. Keeping the directory names simple helps make the mount simple also
mount mlsvr:/var/spool/mail /var/spool/mail
Mail Debugging with mqueue A large central mail server makes tracking down delivery problems easier because it puts all the error messages in one place in the mqueue directory. If the master has a problem delivering a message, it will register its complaint in the logfile in mqueue (see Figure 2). If a user has problems getting or sending mail and you know there's no problem with his/her sendmail configuration file, check /var/spool/mqueue for error messages. Look at the times required to complete the deliveries as an indicator of potential problems. Generally speaking, mail delivery is stopped or hampered when the recipient system is not up or not on the network; addressing is erroneous; the wrong user name is used (a solid argument for first initial-last name user naming conventions); or the user-name/address/aliases combination is not correct in the mail aliases file. Coming Soon In the next issue (March/April 1993), I'll deal in more detail with sendmail administration and related issues.
About the Author
Bruce H. Hunter is the co-author, with Karen Hunter, of UNIX Systems Advanced Administration and Management Handbook (Macmillan: 1991).
|