Managing sendmail in a Mixed UUCP/PPP Environment
Arthur Donkers Doing business over the Internet is common practice today, and electronic mail is the most common communication method. It is thus very important that email arrive at the destination quickly and reliably, by whatever means possible. This article describes a setup for an electronic mail system that automatically chooses between SMTP or UUCP delivery, depending on which is available at the time of sending. Why Mix UUCP and Dialup PPP? Why bother setting up sendmail for a mixed UUCP and dialup PPP environment? Configuring sendmail is difficult as it is, even with a fixed delivery policy. There are a few reasons for this self-inflicted torture. My organization, Le Reseau, operates a dialup PPP connection to the Internet. This connection is used for downloading software via ftp and accessing the booming number of Web services. Apart from this, users may also send mail directly to the addressee by using SMTP. SMTP reduces the number of intermediate hops a message must go through before reaching its destination; in fact, most of the time the message goes directly to the destination via SMTP. Direct delivery is important to message privacy. We do a lot of business via the Internet, so a lot of confidential information is exchanged through email, such as credit card numbers. A direct, interactive connection to the destination meansthat this confidential information can be encrypted. There is no store-and-forward of the data or the crypting keys, so there is less chance of losing your confidential information. Because our dialup PPP connection is established by hand, we have to have some sort of fallback delivery for normal mail when our Internet connection is down. We chose UUCP for this because it batches the mail and sends it out at regular intervals. Combining these techniques lets us send mail through SMTP when the connection is up, and through UUCP when the connection is down. This is all done transparently to the users. Another reason for this combined setup is the emerging use of portable computers. By equipping a portable computer with some version of UNIX, we enable people to travel and carry their work with them. Travelers are not connected to any network, but with UUCP, they may still send electronic mail. UUCP batches and stores their mail until they can dial out and send the batch. Once back in the office, travelers can connect the portable to the network and send all their mail automatically using SMTP. The Current Setup The situation in the Netherlands regarding service providers is comparable to that in the United States. A number of providers handle the routing of TCP/IP packets and store-and-forward mail and news for you. Most of them offer some sort of dialup SLIP or PPP, but some also offer a UUCP service. At our end, we established a mail/news gateway using software available on the net. We implemented our gateway using Linux, sendmail, Taylor UUCP, and other publicly available packages. This setup's advantage is source code availability. Whenever you need to adapt something, you can do it yourself. Bug fixes are also available quickly through the vast Internet community. Our experiences with this kind of software are very positive. However the techniques described in this article are not limited to net-available, source code software. Anyone running UNIX with some sort of UUCP and dialup Internet connection can apply these techniques. The key factor in this setup is sendmail 8.6.9, which can be built for many UNIX platforms. By applying the different changes for sendmail, you can also use this setup on SunOS, System V, and other UNIX flavors. How Does sendmail Work? As explained in the sendmail sidebar, sendmail does not do any delivering. That task is delegated to one of the available delivery agents. sendmail determines what agent to select and calls it with the appropriate interface. It does this both for local delivery of incoming mail, and for remote delivery of outgoing mail. sendmail's input derives from a user entering mail on the local system or from the outside world. A user runs a special mail program, such as ELM, to enter the mail message. The mail program supplies the necessary message headers without any intervention from the user other than entering the message body and the recipient's name or address. After completing the message, the mailer offers it to sendmail for further processing. Remote mail is handled very much the same way. For instance, if a message is offered through SMTP, it is sent to the sendmail daemon running on the system. Through a special dialog the message is offered to the daemon, which then processes it. With UUCP there is no dialog, but the complete message is offered to sendmail through an intermediate mailer. This transforms the message into a digestible form. Delivery Agents: SMTP vs UUCP The bottom part of sendmail is the set of available delivery agents. sendmail itself contains a delivery agent for sending mail through SMTP. This delivery agent sends mail across TCP/IP networks. This agent is also used to receive mail when sendmail runs as a daemon. When running as a daemon, sendmail listens on TCP port 25 for incoming mail. If a foreign host wants to send mail, it connects to our machine's port 25. As soon as a connection is established, a dialog starts to exchange mail. sendmail uses this dialog to send outgoing mail, so sendmail must contain this delivery agent. UUCP's delivery agent, uux, is not built-in: UUCP must call it with the appropriate parameters. uux invokes the rmail program, which communicates UUCP and sendmail. SMTP's delivery agents deliver mail directly to the destination or to the destination's mail handling host. UUCP's delivery agent uses a store-and-forward mechanism to send the mail. First the mail is batched on the local host. At regular intervals, determined by the system administrator on the local host, the mail is sent to an upstream host, which might use SMTP or UUCP to forward the message. This method repeats until the message reaches its destination. Specifying a delivery agent can be done from the master configuration file, which is the input for the M4 macro processor. Several mailers have been predefined and can be selected with the MAILER() function. You can also define your own delivery agent. You must then define the different fields of the mailer specification yourself. The example in Figure 1 uses the predefined SMTP and UUCP mailers. For local delivery the procmail system is used; this allows automatic mail processing depending on the contents of the message. After processing the master configuration file, M4 generates the sendmail.cf file. sendmail uses sendmail.cf to specify the configuration. In Figure 2, the selected local delivery agent is procmail. The program delivery agent (Mprog) is a bonus generated by the MAILER(local) function. sendmail uses this mailer when it sends mail to a program. A well-known example would be delivering mail to a pipe specified in the .forward file in the user's $HOME directory. Each mailer specification contains a number of fields. The M field specifies the name of the delivery agent. Each configuration should at least contain a local and a prog delivery agent. The P field specifies the path to the agent's executable. [IPC] refers to the built-in SMTP mailer. The F field specifies the capabilities of the agent, and sendmail uses this field to determine what the agent can do. The S= field defines the rule set used to rewrite the sender's address so the agent can handle it. The R= field defines the rule set used to rewrite the recipient's address. These delivery agents are generic and should not be changed. The decision on how to deliver a message happens during another step of the delivery process. Rule Sets, or, Which Way to Go? A sendmail configuration file consists of many rule sets and subsets, but an intimate knowledge of every rule set is not necessary. The most important ones are rule sets 0, 1, 2, 3, and 4. These sets are interrelated, as shown in Figure 3. Rule set 0 selects a delivery agent based on the recipient's address. This rule set produces three pieces of information:
The delivery agent's name must be one of the agents specified in the Magent rules (M field). The receiving host's name does not necessarily have to match the name of the host to which the message is sent. In UUCP, for instance, the receiving host's name is not the same as the name of the UUCP hub which will store-and-forward the message. This is the rule set I adapt to suit my site's needs. Rule sets 1, 2, and 4 rewrite the sender's and recipient's address so it can be handled by sendmail and the delivery agent. All addresses are pre-cooked by rule set 3. Rule 3 splits addresses into digestible chunks, strips all comments from the address specification, and puts the different parts in the proper place for processing by rules 1 and 2. Customizing Rule 0 Figure 4 contains the customized settings for rule set 0. This customization is divided into two parts. The first one expands the local part of the rule set. This part is typically used for anything related to local delivery. The first three rules come directly from the sendmail configuration. They primarily detect special addressing types related to the local domain. I added a new fourth rule because, apart from our local domain, reseau.nl, we are part of another domain, xs4all.nl. xs4all.nl is the domainname of one of our providers, and at the same time it is an alias of its dial-in host. We have therefore added the name of the xs4all.nl host to our /etc/host table. That way we can always access it by this name, despite the state of our nameserver. The complication this presents is that sendmail is always able to resolve the name xs4all and will decide to contact this host using SMTP. This is often not the case, so mail to this host could get stuck in the mail queue for a long time. This fourth rule forces all mail going to the xs4all domain to instead go through our UUCP channel, thus guaranteeing its timely delivery. Using this method might not be ideal, but at least it always delivers the mail with a maximum delay of one hour, our UUCP polling interval. The second part of the customization applies to remote network delivery. It rewrites remote addresses and decides how to contact the remote host. All hosts on the local network are handled by the first two rules. If the address contains our local domain name, mail is delivered through SMTP to the host on the network. The same goes for mail addressed to the domain itself, which is sent to our mail gateway, also through SMTP. When mail is addressed to a nonlocal host, the last four rules activate. The decision whether we can contact this remote host or not is taken by the name resolver. Whenever we can resolve the MX (Mail eXchange) name for the destination, we decide it is also possible to contact the host through SMTP. The destination's MX name might be the destination itself, or the address of a host that handles all SMTP traffic for the destination. This setup has its implications for our nameserver and dialup connection. Exactly how do we use the nameserver for this? The trick lies in the third rule:
R$* << @ $* $~P >> $* $: $1 << @ $[ $2 $3 $: $2 $3.NOTFOUND $] >> $4
We try to make the destination host canonical through the special symbol $~P. This queries the nameserver. If the nameserver can resolve the hostname, $~P will be replaced by the $2 $3 expression before the $: within the [] expression on the right-hand side of the rule. If the nameserver cannot resolve the name, $~P will be replaced by the expression after the $: within the [] expression, which appends the nonexisting domain NOTFOUND to the address. To prevent this rule from looping, the $: that begins the right-hand side evaluates the rule only once. The next rule in this part is used as a safety net. It handles all mail addressed to unknown hosts within the local domain. This mail is rerouted to the local delivery agent, preventing any improperly addressed mail from getting lost. The last two rules select the appropriate delivery agent, based on the presence of the NOTFOUND pseudodomain. If NOTFOUND is present, the UUCP delivery agent is chosen and the mail is forwarded to the upstream UUCP relay. If the pseudodomain is not present, mail gets directly sent to the destination's MX address. Relationship between BIND and sendmail Using the nameserver to decide whether we can contact the destination directly or not has some implications for the nameserver's setup. A number of problems can occur and, according to Murphy, definitely will occur. The nameserver caches any information it has retrieved. This means that whenever mail is sent to an address still in the cache, sendmail would decide that it should use the SMTP delivery whether or not our Internet link is up. The nameserver supplies the address from the cache despite the state of the Internet link. If the link is down, sendmail times out on the delivery, and the mail sticks in the mail queue until the queue is rerun when the Internet link is up. A second problem relates to the nameserver query itself. If the link is down and the destination address is not present in the nameserver cache, a query for this address times out after about 30 seconds. While querying the nameserver, sendmail would not be able do anything else. This causes problems when we want to serve our mailing list. The mail gateway also serves our Dutch Linux mailing list, a list of about 100 addresses. If each addressee on the mailing list takes 30 seconds to process, processing the complete list takes almost one hour. To handle this, it would be necessary to reconfigure the nameserver so that it quickly times out when the Internet link is down and takes its normal time to resolve whenever the link is up. The first problem, starting and stopping the nameserver whenever the link goes up or down, is the more difficult. The second problem can be solved by using two different nameserver configurations: one for an established Internet link, and one for a standalone situation. This technique is described in the sidebar on BIND. But how can we start and stop the nameserver? This is where PPP comes to the rescue. Establishing and Tearing Down the Internet Link Our Internet link is based on PPP. We use an implementation of PPP that lets us execute a special script whenever the link is established. When we tear down the link, we execute another script. We use these two special scripts to reconfigure the nameserver. The first script, pppconnect (Listing 1), uses the pppchat program to talk to the modem. This pppchat program is very similar to the expect-send strings used to configure UUCP. A pppchat script for communication with the modem appears in Figure 6. The IP addresses of the local and remote hosts are dynamic and determined while establishing the connection. The script sets the default route to the remote host so all Internet traffic is sent to this host, then stores a few more options for the PPP link in the default options file (Figure 5). These options tell PPP that the modem should be locked to prevent UUCP from dialing out, and to use the hardware handshake signals. To establish the connection, I call pppconnect manually. pppconnect puts itself in the background and calls pppchat. Once the connection is established, the PPP daemon calls the ip-up script (Listing 2). This script is executed whenever the IP part of the PPP link is started. The lines related to mirroring are of no importance to this article, but the line
/etc/ns start net
is. This line restarts the nameserver in Internet mode (see the sidebar on BIND). To tear down the Internet link, the script sends a HUP signal to the PPP daemon. The daemon then executes the ip-down script (Listing 3) and hangs up the modem. The line
/etc/ns start minimal
restarts the nameserver in local (minimal) mode. Any query to the nameserver for a remote host quickly times out without delaying mailing list processing. Further Improvements Of course, there is room for further improvement. The question is whether a dial-on-demand PPP would solve the problem. Currently there is an experimental implementation of a dial-on-demand daemon for Linux. This daemon can be configured to dial the provider whenever a connection to the Internet is desired. The problem with this is that any nameserver query will then result in a call to the provider. This can mean that the connection is alive during the greater part of the day, resulting in a huge phone bill. Another improvement might be the requeueing of mail. If many messages are stored in the UUCP mail queue, it might be best to send these when the Internet connection is established. A last improvement might be another configuration of the nameserver. Restarting the nameserver each time a connection is established or torn down is not a very elegant solution. Conclusion This setup works! For more than half a year, all mail has been sent either by SMTP or UUCP. By tweaking the sendmail configuration file, you can change the behavior of this program to suit your needs. Good documentation is essential in this process.
About the Author
Arthur Donkers graduated from the Delft Universiy of Technology with a degree in Electrical Engineering, with a major in Computer Architecture. He has since worked for a number of major software houses in the Netherlands, specializing in projects relating to data communications, especially the integration of multi-vendor network systems. The last four years he worked as an independent consultant for his own company, Le Reseau (The Network). He sees electronic communication as a means of connecting people quickly and reliably across hardware and software boundaries and, most important of all, across social boundaries.
|