Putting cron to Work for You

R. King Ables

The cron program can be used for many tasks other than the automatic work performed by the system -- such as initiating UUCP connections, running calendar, rotating the log files, and running accounting -- and specified in the system crontab file supplied by the vendor. In fact, you can use cron to automate any non-interactive function that is to be performed on a regular basis (such as hourly, daily, weekly, or monthly) simple by writing shell scripts for these commands and adding or deleting the functions as necessary. This capability allows you to be proactive in managing your systems (find the problems before your users do). You can set up a crontab file to run shell scripts that monitor various aspects of your system or your network and notify people when necessary. Using NFS or rdist (or even rcp), you can keep one master copy of each script that is accessible by all your hosts, so that when you add checks, all the hosts pick up the information.

This article examines a few examples of scripts to monitor specific things and talks about other functions you might want to monitor. I leave it as an exercise for the reader to use the examples given to write new scripts to monitor any of the other functions that you might find useful in your local environment.

Checking Disk Space

Every machine on your network probably has some amount of local disk space. Regardless of whether it is user disk space or system disk space, you want to make sure you're using your space efficiently and that there is enough.

You may want to find any very old or very large files or core files (an image of a running program which is left when some catastrophic error occurs) on your system. Core files tend to be very large and, as a rule, useful only for debugging a program.

The find command will produce a list of files which fit a set of criteria specified on the command line (for detailed information, see the man page in section 1 for find). To find any files larger than one megabyte or older than 30 days, you could use the following command:

find / -type f -size +2048 -o -mtime +30 \
-print

The -type argument is used to specify the type of the file (in this case, a plain text file). The -size argument is used to specify the size of file we wish to find. In this case, +2048 means any file larger than 2048 512-byte blocks, one megabyte (-2048 would mean any file smaller than one megabyte, and 2048 would mean any file exactly one megabyte). The -mtime argument is used to specify the time since the file was last modified. -mtime +30 means find files that were modified more than 30 days ago (-30 would mean find files that were modified less than 30 days ago). The -o argument is used to indicate that either test on each side of the -o argument can be true for the file to meet the criteria (in this case we are saying give us the filename if its size is greater than one megabyte or if it was modified more than 30 days ago).

Note that this will list all files that fit the search criteria whether they are on the local machine or remotely mounted (for example, via NFS). Generally, you do not want to be told about files that reside on another machine (the search that you run on that machine will find them). The find command has an argument, -xdev, which restricts its search to those directory trees which are contained on the same device as the target file or files named on the command line. For example:

find / -xdev -print

will only list files contained in your root partition. If your /usr partition is a separately mounted filesystem, it will not be traversed as it is not on the same device as the root filesystem. In order to also search your /usr partition, you must list it explicitly in the command. Other filesystems, both local and remote, that are mounted at mountpoints within a filesystem to be searched will not be searched when using the -sdev argument since it restricts the search to the physical device.

In order to find files larger than a megabyte, files older than 30 days, and all core files on the local system, one might formulate the command this way:

find / /usr /home -xdev -type f -size +2048 -o -mtime +30 -o -name core -print

Sometimes, however, you want to know more about the disk space usage on your system than just on the level of individual files. You might like to see which filesystems are close to full or which users are using the most space. The dfck script (meaning df check, using the df command) in Listing 1 can be used to check each filesystem on a machine and print a message for any that are full over a specified threshold. The script runs the Berkeley df command to make a list of local filesystems and their percentages of capacity used and pulls out the ones that are above the limit. Running this script each day on each host will give you some warning when a filesystem is going to be running out of space.

If you don't have a quota mechanism on your network (which allows you to limit the disk space available to individual users), then you may want to find out which users are occupying more than a certain amount of disk space. The duck script (meaning du check, using the du command) in Listing 2 prints a block count for each user directory found on the system which contains more than a customizable limit. This script can be run on all file servers and the output combined for an overall view of the biggest users of disk space.

A common problem with user disk space occurs when users keep old mail in their mailbox file in a system mailbox area. You can spot abusers of this with the mboxck script (which works much like the duck script does) shown in Listing 3.

Checking Queues for Problems

When jobs hang up in mail queues and print queues, it's often the case that until someone doesn't get output or doesn't get a reply from a mail message, no one is aware that anything is wrong.

It is often useful to check these queues on a periodic basis, perhaps several times a day (this would be a good candidate for work to add to a cron.hourly script) to make sure work is flowing smoothly. The scripts shown in Listings 1 and 2 can be altered to issue the commands used to print out the mail queue or printer queue (or any other queues on your system) and print out the result if the queues are not empty or have more than a certain number of entries.

Synchronizing the Time and Date of Hosts

With the advent of NFS and other remote filesystems, it is even more important for each host on your network to agree on the current time. If your workstation's clock is slow compared to its fileserver's clock (so that the write date of files, as seen from a workstation, could appear to be in the future!), anything that depends on write dates can get very confused. Problems with the make command are especially hard to figure out in this case.

You might want to synchronize the time of your hosts on a periodic basis, at least once per day. The Berkeley rdate program (included in SVR4) can be used to reset the time and date of a machine to be the same as some other host whose time is known to be correct. Note that it is more important that all your machines agree on the time than that they be right!

A simple crontab entry to synchronize the time and date of a machine at 6:15 A.M. each morning would look like:

15 6 * * * /usr/ucb/rdate server >/dev/null
\ 2>&1

where server is the server with the "correct" time. Note that in SunOS, rdate is in /usr/ucb, while in SVR4 it is in /usr/bin. Alternatively, you could add the rdate command to your cron.daily script along with all the other checks you wish to perform on a daily basis.

Updating or Backing Up System Files

If you are not running NIS, then you must be either maintaining several system files separately or using rdist to distribute across your network (see "rdist to the Rescue" in this issue). The sendmail alias file and host table file are good candidates for this. You might distribute the passwd file as well, although distributing files containing information that users are able to change can be problematic (they can only make the change on the "master" host that contains the version that gets distributed, otherwise their changes get overwritten the next time the master file is distributed).

Even if you do run NIS, there are other files that might be easier to manage if you distributed a single master copy to the other hosts on a regular basis. The printcap file (which describes local printers) and even the crontab file itself are two such files. Other local files, such as a company on-line phone list, can also be distributed regularly.

Some sites even use rdist to make "hot" backup copies of their NIS map files that they could use if the machine containing the master copies of the files crashes or suffers some other catastrophic failure.

Security Checks

Running periodic routine security checks is another very useful function that cron allows you to automate. There are many potential security holes in any UNIX system (several were discussed in "Beware the Trojan Horse," Sys Admin, vol. 1, no. 1).

One easy -- and important -- thing to check is that all accounts have a password. On a system that does not use the /etc/shadow file (which is used to protect passwords), any user can look at the password field. While they cannot obtain the password, they can find out if a password is set for an account. Your system should never have any account without a password, and most current password changing programs don't allow it, but on some systems it is possible to set the password to be null. To print out account names with null passwords, use:

awk -F: '{if ($2 == "")\ print $1}' </etc/passwd

This command could be built into a script which checked other attributes of a system as well. Alternatively, the cops package, which contains many useful tools for checking security, provides commands that cron could run automatically to monitor the security of your system. cops is available via anonymous ftp from cert.sei.cmu.edu.

Putting It All Together

So now you have a handful of shell scripts that check different things on your system. In general, the scripts generate output only when there is something wrong that someone should know about.

When cron runs programs, any output generated is sent, via mail, to the user for whom cron is running the program (in the case of the system crontab file, all output is mailed to root). In most cases this will get the notification to the right person or persons.

In a large installation, however, different people will have responsibilities for different aspects of management, so that notice of a security problem might go to a different person than notice that a filesystem is full or a print queue is hung. As it happens, not only can you break up the cron tasks by hourly, daily, weekly, etc., you can also break them up by topic: security management, disk space management, queue management, etc. cron gives you the freedom to set things up the way that best suits you.

You may want to construct a cron.hourly script which simply runs other scripts:

#!/bin/sh
#
# cron.hourly
#
/usr/local/cron/security.hourly
/usr/local/cron/queues.hourly
/usr/local/cron/filesystem.hourly

with the specific checks for each topic in these scripts. Note that the choice of /usr/local/cron as a place to keep cron-related scripts and programs is arbitrary. Only the crontab itself has a specific location, all other files can be managed to your own preference.

The three scripts presented so far -- duck, dfck, and mboxck -- check for full filesystem problems. All three report on system attributes that should probably be checked on a daily basis (you might wish to run dfck even more often if you are really tight on disk space). To accomplish this, I've constructed a filesystem.daily script (see Listing 4) that will run each of these scripts and allow you to mail it to any user or alias or multiple users you choose rather than simply to the root user.

The filesystem.daily script shows how freely you can customize the automatic activities that cron will perform for you. When you find yourself performing some new task on a regular basis, you can write a script for it, then add the script to the list of programs executed by the cron script that is the most appropriate for the particular type of operation.

For More Information

I have only hit the high points on the uses of cron. The sixth chapter of the O'Reilly system administration book (Frisch, leen: Essential System Administration, Sebastopol, CA: O'Reilly & Associates, 1991. ISBN: 0-937175-74-9) takes a much more detailed look at other useful aspects of the cron daemon and the crontab file. the book not only covers cron very well, but many other topics in system administration as well.

About the Author

R. King Ables has been a UNIX user since 1980 and has been managing systems or developing system management and networking tools since 1983. He is currently doing system and network management development for HaL Computer Systems in Austin, TX.