Detecting System Intruders
Jeffrey B. Davis While working on a project one afternoon, I noticed the load average of my machine starting to climb. I could tell the machine was also slowing down a bit because my compiles were taking longer than they had a few minutes before. Within a few minutes, things returned to normal, but I wanted to know what had happened. I decided to look around to see what had occurred, because I thought there might be an intruder on the system. Detecting system intruders usually just involves searching for an intruder's mess, and this article describes some of the things to check if you suspect an intruder. You should start by checking some of the more common logs (see Table 1) including lastlog, messages, and syslogs. By checking the lastlog, you might find some interesting login attempts. last is a utility that displays login information, and doing a last -10 shows who, where, when, and how long a connection was open. See display examples in Figure 1. I look for connections from locations that should not be connecting to the box, along with errors in the lastlog listing. Determining which machines and users have access will allow you to decide which, if any, log entries are from possible intruders. Using the sample lastlog listings, I would become suspicious upon seeing the ftp session from www.yourcompany.net, if I knew they were not allowed. I would get even more suspicious because the length of that connection was considerably longer compared to the rest of the ftp sessions. In the heavy use ftp site example shown in Figure 1, the only interesting access is the 4-hour access. This could just be a slow link; however, Mary (an example office employee) probably does not normally log into the system right after midnight, use it heavily until 4:18 a.m., and then show up to work the next morning. Sometimes, intruders attempt to cover their tracks by fixing the logs. An example of a fixed log might look like this:
ftp ftp dial04189.netviX Sun Aug 4 00:08 - 00:48 (00:39) ftp ftp XXX.170.136.134 Sun Aug 4 00:08 - 00:16 (00:08) ftp ftp ftp ftp XXX.171.140.188 Sun Aug 4 00:04 - 00:20 (00:16) john ftp XXX-152-156.ipt. Sun Aug 4 00:02 - 00:12 (00:10) wtmp begins Sun Aug 4 00:02
The above example would be cause for alarm because the wtmp file should continue to grow and should have just started out on the time reported. Also, it is strange to have an entry without the standard information (this one has white space instead). Sometimes, the log fixing utilities used by intruders do not do a good job of hiding their last login. I also check the messages file to make sure that nothing nonstandard appears. The messages logs usually reside in /var/adm. Depending on how your /etc/syslogd.conf file is constructed, different information will be in this file. I check for odd su root attempts from users who do not have su privileges, along with other unusual lines. Example messages lines from a Solaris 2.3 box:
Aug 4 11:25:04 strange su: 'su root' failed for smith on /dev/pts/3 Aug 4 11:25:24 strange su: 'su root' failed for smith on /dev/pts/3 Aug 4 11:25:44 strange su: 'su root' failed for smith on /dev/pts/3 Aug 1 19:16:47 strange sendmail[17472]: AA17470: SYSERR: net hang reading from duffel.net: Connection timed out during greeting wait wi Aug 2 15:42:38 strange /usr/local/sbin/sshd[2490]: Local: Session terminated on signal 1.
The sshd is great for doing secure TCP/IP connections in a noncommercial environment. (See http://www.uni-karl sruhe.de/~ig25/ssh-faq.) Also during my log search, I read some of the syslog. The syslog is another log file to which other system information gets written. The contents of these logs is definable by the /etc/syslog.conf files. in the syslog file. I check for unusual, incomplete, and corrupt entries. Figure 2 shows a sample syslog file. In that example, the ftp connection from the competitor's host would cause me to suspect an intrusion. If you see an entry from an unauthorized machine (or set of machines) while reading these log files, you should become suspicious. Also, if you are positive that these logs are being created, but find that they do not exist or have become corrupt, your should definitely suspect an intrusion. Next, you should take a look at some other system information. You should check for unauthorized users on the box. Doing standard w and who commands to look for unauthorized users might reveal a strange connection or user, but chances are it will not. Doing a netstat to see whether a foreign connection is open might help. You should also check the process table to see what is in ps -ef or ps -auxw, depending on your flavor of UNIX. These binaries are all good targets for modification once a hacker has gained access. Compare the information from the log files with the information you received from the previous commands to help narrow your search. The next check would be to use the find utility to look for any setuid files that you know should not be setuid.
find / -perm -4000 -user root -print
I also look for shells (ksh, zsh, tcsh, bash, etc.), old security bugs (time, uudecode, etc.), and other abnormalities. By now, you should have a pretty good idea of whether there is or was an intruder. The next step would be to comb through shell history files. I use bash, so I would look in ~jeff/.bash_history, whereas some users might have a .history. If you run system accounting, where all commands get logged, you would check there for commands that your suspicious userid executed. In the shell history file, you might see it linked to /dev/null (which should raise more suspicions) or possibly deleted. You might also read commands that users or possible intruders executed. If you think you have found an intruder, refer to your security policy on how to proceed. If you need to create a security policy, I suggest you start by looking at http://www.cs.purdue.edu/homes/spaf/hotlists/csec-top.html, which has numerous links to security pages and to some policy-creating pages. If you stumble upon an intruder's mess, such as corrupt log files, setuid binaries that should not exist, and history files containing privileged commands, those things should be backed up, but also noted. Remember, the intruder could be the boss's child, the boss, a future employee, or a criminal. For further help, there are some utilities that will do some security checking for you - these should be run consistently. Take a look at tiger, cops, and satan or some of the commercial products for up-to-date security hole detecting. Finding an intruder's mess is usually easier than cleaning it up. Some additional web sites that I have found useful are:
http://www.alw.nih.gov/Security/security.htmlhttp://www.alw.nih.gov /Security/security.html http://www.underground.org http://ciac.llnl.gov/ http://www.cs.purdue.edu/coast/coast.html http://nsi.org/compsec.html http://www.iss.net/vd/vd.html
About the author
Jeffrey Davis is currently a system and network administrator at NETCOM Interactive. When he is not installing web servers, his time gets divided between his wife and kids.
|