WTMP: Another Step Toward Centralizing Event Management

Joe Aguiar

This article addresses yet another aspect of centralized event logging and management integrating data from the UNIX wtmp file into the the syslog text files. Because wtmp is not a text file, the typical shell commands are of limited use when examining its contents. I will discuss how to examine and integrate the data with the UNIX syslog utility for processing with familiar tools using a package written in C called logutmpd.

Although most of this discussion applies both to the utmp and wtmp files, I will focus primarily on wtmp because new records are appended to wtmp, whereas with utmp records are as likely to be deleted as appended. You might think of the info in utmp as indicating the current state of things, while wtmp contains historical data.

Events Reported in wtmp

Any process may write to the wtmp file providing that process has the required write privileges (user write, where user is root). The most common entries are made from the init process. The processes that init starts are listed in the /etc/inittab file. A wtmp entry may be made for each of the processes that init manages. Other programs that may make wtmp entries include telnetd, login, and xterm, or anything else that may connect to ttys, alter run level, or set the system date.

The type of information you might find in wtmp includes login/logout times for specific ttys, users, and sometimes remote hosts. Also, changes in run level and boot times are typical information to find in wtmp. Clearly this type of information can be of great interest to a systems administrator, particularly in regard to security management.

The amount and type of information found in wtmp can vary quite a bit from system to system. Some systems feature an extended wtmp file called wtmpx. This is true of Solaris and other SVR4-based systems. In fact, under Solaris, there is a wtmp file as well as a wtmpx file. The utmp struct that defines the records in both the utmp file and the wtmp file may be found in the file /usr/include/utmp.h, or perhaps /usr/include/utmpbits.h under Linux. This file, or actually the C struct statement within it, will describe what pieces of information are potentially available from each record of the wtmp file. Listing 1 shows an example of a typical utmp struct and the type of data to be had from it.

The only items that may not be obvious are ut_type and ut_exit. The ut_type describes the type of entry (e.g., boot time, run level, login process, etc.) These are described in the header files mentioned previously. The e_termination value is the signal number that caused termination of the child and e_exit is the exit code of the child. Your system man pages for utmp/x(5) as well as wait(2) should shed additional light on this.

Reading Data from wtmp

Because wtmp is a fixed-length record, binary style, data file, we cannot easily use normal text-based tools to look at its contents. Tools like last are extremely useful to examine wtmp data. However, it may not be easy or efficient to use last to examine wtmp in real time. Most UNIX systems provide the getutent(3), setutent(3), utmpname(3), and endutent(3) system calls to read and manipulate wtmp/utmp entries. Under Solaris, there are corresponding system calls for managing the wtmpx file.

The logutmpd daemon uses these, or alternatively, customized versions of some of these calls to continuously get at the latest entry in the wtmp file.

Using Logutmpd

The whole purpose of logutmpd is to allow the sys admin to efficiently and continuously monitor the wtmp file for new entries, convert them into the desired text-formatted message, and forward that message to the local syslog where the event may be easily viewed or acted upon.

To do this, logutmpd, upon startup, opens the system wtmp file. It then traverses the wtmp file one utmp struct record at a time until it reaches the end of the file. At this point, logutmpd waits for any new data to be written to wtmp. The interval that logutmpd waits (by default) is 1 second. The user may specify a different interval using the -us option to specify the number of microseconds, and -s to specify seconds. The timer used is based on the system select() call. When new data is detected it is read, reformatted, and written to the syslog at a priority of auth.notice.

By default, logutmpd runs as a daemon process and can be started at boot time using a normal UNIX startup script. The logutmpd distribution includes a sample Sys V-style init script. A command-line example of starting logutmpd might look like the following:

# logutmpd -us 500000 -f /var/log/wtmp

This would run logutmpd as a daemon process, reading from the file /var/log/wtmp, checking for new data every 500,000 micro-seconds (.5 seconds). Logutmpd may also be run in foreground or debug mode using the -d command-line switch. Using -d will prevent logutmpd from writing to the syslog. Instead, it will read all the data currently in wtmp, writing it out to stderr, and then write any new data to stdout. In this way wtmp data may be redirected to a text file or piped to another process, if desired. For instance, the following command would discard all the historical data and send any new data to the system console:

# logutmpd -d 2>/dev/null 1>/dev/console

The nice thing about having a daemon like this is to be able run this on all your remote hosts and let syslog do all the work of forwarding the data to your central log host. There is a tool called ulogger included in the logutmpd package that can be used to send messages to wtmp to test your installation (similar to using the logger command to send messages to syslog).

Customizing Logutmpd

Simple configuration changes to logutmpd may be accomplished at build time by defining certain macros using Makefile variables. More complex customization of logutmpd can be done by modifying the macros defined in the logutmpd.h file.

Let's examine what can be changed easily by modifying the Makefile. To change a configuration in the Makefile, we will need to change how the DEFINE variable is set. For instance, if our operating system is Digital UNIX, the line in the Makefile should read:

DEFINE = -DOSF

This has the effect of setting the OSF preprocessor macro at build time, which will ideally set things up for a proper build of the logutmpd binary on a Digital UNIX system. Other build options that are easily set up in the Makefile include the syslog priority that will be used for new messages and a flag to include the time data from the wtmp record.

Because syslog priorities have two components (a facility and a level), you must define the UT_FACILITY and UT_LEVEL macros to change the default logutmpd behavior. The default is LOG_AUTH and LOG_NOTICE, respectively. If you were to set the DEFINE variable in the Makefile, it might look like this:

DEFINE = -DUT_FACILITY=LOG_USER -DUT_LEVEL=LOG_WARN -DOSF

After rebuilding, each new message would now be sent to syslog at the user.warn priority. To investigate other possible settings for priority, check out the /usr/include/syslog.h file. syslog priorities are pretty standard, but there are subtle differences between systems. To include the time data from wtmp record in your syslog messages, simply append -DUT_TIME to the DEFINE variable definition in the Makefile.

Logutmpd is known to run on Linux, Solaris, and an ancient version of Digital UNIX. If you want to run this on a different platform, the easiest way may be to copy the set of macros for a system that comes close. Change the outermost #ifdef statement to reflect your OS, and customize the remaining macros as needed. The OLIST and UT_FORMAT macros are probably all that will be needed in the way of customization.

Suppose the utmp struct on your operating system only has two pieces of data: ut_user and ut_time. The OLIST might be changed from:

#define OLIST u->ut_user, u->ut_line,
u->ut_id, u->ut_pid, u->ut_host

to:

#define OLIST u->ut_user

and the corresponding UT_FORMAT from:

#define UT_FORMAT "%s, %s, %s, %d, %s\n"

to:

#define UT_FORMAT "%is\n"

The T_UT_FORMAT may require changing as well; the only difference is the time is placed at the front of the macros with the T_ at the beginning. The ut_time data will be included automatically if the macro discussed above has been set. Also, you have the option of building logutmpd using the systems *utent() calls or the custom ones supplied with logutmpd. Use whichever works best with your system. Don't forget to include -DYourOSName when setting DEFINE in the Makefile so your new macros will be used when building. That's about all that should be needed to get things working on your system.

Exploring Alternatives

There must be many alternative methods of centralizing all the wtmp data. One interesting approach is discussed in UNIX System V Network Programming by Stephen A. Rago, in which the author provides examples in the use of the ONC RPCs XDR to format the wtmp data before transferring it to a central host where it is imagined that the data can then be easily interpreted with another application since all the data, no matter what type of processor it came from, is in a uniform, XDR representation.

Another, perhaps simpler, approach is to write a Perl script that uses the unpack() function to read the data. One such script is shown in recipe 8.18 of Perl Cookbook, by Tom Christianson and Nathan Torkington. The unpack() argument format can be tricky to get right. Because it is based on the utmp struct, it may vary a bit between operating systems. You will need to "use Sys::Syslog;'' and add the openlog(), syslog(), closelog() calls. Also, to get it to run as a daemon, you need to add code similar to:

sub daemon_init {
    my $cpid;
    if ( !defined( $cpid = fork())) {
        perror( "Cannot fork a new process.");
        exit( -1);
    } elsif ( $cpid != 0) {
        exit( 0);
    }

    open( STDOUT, ''>/dev/null'');
    open( STDERR, ''>&STDOUT'');

    setsid();    # use POSIX;
    chdir( "/");
    umask( 0);

    return( 0);
}

Using this method may give you a slight performance hit compared to the C-based solution. On the other hand, if you have Perl on all of your hosts, this might be a way to go.

Conclusion

UNIX has a long history and exists in many forms. As a result, achieving consistent and complete logging across platforms can be a really interesting project. I hope the tools presented here will help you achieve an extra step toward that goal.

The latest version of the logutpmd package may be downloaded from http://home.inreach.com/jaguiar. Questions or comments should be addressed to jaguiar@inreach.com.

References

1. Rago, Stephen A. UNIX System V Network Programming, 1993, Addison-Wesley.

2. Christianson, Tom and Torkington, Nathan. Perl Cookbook, 1998, O'Reilly & Associates.

3. Stevens, W.R. Advanced Programming in the UNIX Environment, 1992, Addison-Wesley.

4. UNIX Manual pages for wait(2), utmp(5), init(8), getutxent(3C).

Joe Aguiar has worked as a Systems and Network Administrator and Software Engineer since 1986. He holds a BSEE degree from CSU, Fresno. He can be reached at jaguiar@inreach.com.