The Filesystem Activity Monitor

William Genosa

Monitoring the filesystem sizes of multiple machines can be a tedious job for system administrators. Database systems grow gradually as records are added and often cause filesystems to decrease in space. In contrast, development systems shared by programmers can use up disk space rapidly. The Filesystem Activity Monitor (Listing 1) allows system administrators to project when a filesystem will run out of space, alerts the system administrator to changes, and in some cases removes files to create space.

I wrote the program on a 3B2 running AT&T System V Ver 3.2.2. The 3B2 is one of six computers internetworked with ethernet and running TCP/IP. One of the 3B2 computers is designated as a print server, and all of the printers are connected to the print server. The print server's /usr/spool/lp directory is remotely mounted on the other five computers using Remote Filesystem Sharing (RFS). RFS allows computers to share filesystems and is similar to Sun Microsystems NFS in that respect.

The Program Structure

The Filesystem Activity Monitor is built around the df utility. The command df -t displays two lines of information for each filesystem. The first line of output displays the name of the filesystem, physical device information, the number of 512-byte blocks used, and the number of i-nodes used. The second line of output displays the total number of 512-byte blocks in the partition and the total number of i-nodes. Figure 1 shows sample output of the command. Notice the difference in physical device information for remotely mounted filesystems such as /usr/spool/lp.

The program begins by declaring two functions, fsyssize and compare. The first function, fsyssize, uses the awk utility to extract information from the command df -t. Because both functions and awk scripts use curly braces, I have placed comments to the right of curly braces to distinguish their usage. Each of the lines output from the command df -t will have one of three possible formats. Although df -t outputs two lines of data for each filesystem, the first line differs for local and remote filesystems in that extra white spaces are used in the physical device information for remotely mounted filesystems. Because awk uses white space for a field delimiter, the remotely mounted filesystem will contain one extra field.

fsyssize uses the if conditional to determine the data in each field, then uses awk's split function to store each field in an array. Functions in awk should not be confused with shell functions such as fsyssize and compare. The split function requires three augments in parenthesis. The first augment is the string to be saved; the second is the name of the array where the string is to be stored; and the third is the field delimiter being used -- in this case, white space. The semicolon after the split function is used to separate two commands on the same line. The second command assigns an array element to a variable. Because the final if condition will be true for each filesystem checked, it performs the calculations and supplies the output. This output contains the name of each filesystem, the number of available blocks, the total number of blocks, the percentage of free blocks, and the number of megabytes available. The program will redirect the output to a file, where it will be stored till the next time cron executes the program. At that time the output will be redirected to a second file so that the results of the two files can be compared.

The compare function expects the name of a filesystem for an augment. The compare function will carry out different instructions depending on the filesystem it is checking and whether or not it finds any changes in size from the previous check. The function uses the case control statement to test for one of four conditions. The first test condition will send a message to the console and log changes in filesystem size for /, /usr, /usr2, and /informix. The second test condition simply checks to see if /tmp is 90 percent full. If so, the function performs the same actions as for the first condition as well as the following additional actions: the find command builds a list of files changed more than three days ago and mails the list to root, then deletes the files in an attempt to keep the system running. The third test condition, when true, does nothing: since this filesystem is remotely mounted, I have chosen not to intervene. Instead, the remote system will take action, since a copy of this program runs on all the machines. The fourth and final condition, when true, indicates that another filesystem has been mounted after the creation of this program. If this condition is met, you should modify the program to include the new filesystem to be checked and the appropriate action to be taken when changes are detected.

After the functions have been declared, the actual program -- which is short and easy to follow -- begins. The program uses variables to define where work files will be located. It checks for the existence of $PREVCHK because if this file does not exist, it must be created. If $PREVCHK does exist, then the file $CURRCHK is created and compared against $PREVCHK. Each filesystem is checked within the while loop. The last instruction moves $CURRCHK to $PREVCHK to prepare for the next time the program is executed.

Using the Results

If a filesystem suddenly increases dramatically in size, use grep to check the logfile and figure out how many blocks have suddenly been used. To check the activity in the /usr filesystem, use the following command:

grep "/usr " /usr/bill/prog/logfile

Notice that the space after /usr serves as an anchor. Without the space, grep would also match on the pattern /usr2. This command should yield several lines of output resembling the following:

"/usr has decreased by 2 % at Thu Oct 29 1992
17:50:43 EST 1992 leaving 25 MB."

If the creation of one large file caused the increase in filesystem size, you can use the find command to locate the culprit. To find a file created on October 28th with a size greater than 100 blocks in the /usr filesystem, execute the following command:

find /usr -size +100 -exec ls -l {}; | grep "Oct 28"

Conclusion

The Filesystem Activity Monitor is a tool for pro-active system administration, which is the philosophy I want to emphasize. A good system administrator can get out of trouble, but a better system administrator can prevent trouble.

About the Author

William Genosa is the Chief System Administrator for a leading systems intergrator. He may be reached at 186 Bryant Avenue, Floral Park, NY 11001.