 Tools
Built to Sort Through Snort
Kristy Westphal
Snort, the lightweight network intrusion detection system, is
already a great tool for any security-conscious network (see "Snort
-- A Look Inside," Sys Admin, September 2000). The
latest version of Snort, 1.7, has many new features above and beyond
1.6, including dynamic rules (rules that can turn on other rules),
statistical anomaly detection preprocessor, improved IP defragmentation
preprocessor, and a TCP stream reassembly preprocessor. New features
and functions are continuously added.
This article will examine four add-ons for Snort that assist the
Snort user in making logging and reading Snort data easier. The
add-ons include:
- Logging alerts to a MySQL database
- Utilizing the Web GUI ACID as a live console for Snort data
- A Perl script that reports Snort data to Big Brother (a network
monitoring tool)
- SnortSnarf, another handy Perl script that converts your alerts
file to html pages.
Throughout the various tools mentioned, I am using the gcc as my compiler
on an HP-UX 10.20 server.
Logging Snort to a MySQL Database
The required ingredients for the MySQL/Snort recipe include the
latest version of MySQL (http://www.mysql.com), installed
before installing Snort. The version used in this article is 3.23.32.
There are security issues with versions of MySQL prior to 3.23.31.
If you have an older version, you may want to think about upgrading.
After I completed my installation, version 3.23.33 was released,
which fixed many of the security holes that existed in prior versions.
I recommend either upgrading or starting out with the latest version.
MySQL also has excellent documentation on their Web site that may
provide answers to issues not covered here.
You have a choice when installing MySQL -- either compile
it yourself or use a precompiled tarball available for various operating
systems. After much anguish of trying to compile on my HP-UX box,
I tried one of the binary tarballs for my OS. Before loading MySQL,
ensure that both a MySQL user and group exist on your system. Then,
install your MySQL distribution in your chosen method. In my case,
I took the precompiled binary and unzipped and untarred it into
a directory ($MYSQLDIR). Once complete, cd into the $MYSQLDIR/scripts
directory and run the mysql_install_db script. Run the following
commands to get MySQL owned by the proper person and running:
chown -R mysql $MYSQLDIR/mysql
chgrp -R mysql $MYSQLDIR/mysql
$MYSQLDIR/bin/safe_mysqld --user=mysql &
To ensure that your database is running correctly, try the following
commands:
$MYSQLDIR/bin/mysqladmin version
$MYSQLDIR/bin/mysqladmin variables
These commands will tell you whether you can communicate with the
database. If you run into any problems here, I suggest consulting
the documentation at:
http://www.mysql.com
Please note that at this stage, your root user for the database has
no password. You can remedy this situation by issuing the following
command:
$MYSQLDIR/bin/mysqladmin -u root -p password 'new-password'
When it prompts you for a password, hit enter (because your original
password is blank), and then your password will be set. There are
other instructions to secure your database on the MySQL homepage,
and I recommend following them before proceeding. A final note on
MySQL initial installation -- don't forget to add it to your
startup scripts. The script you want to use is:
$MYSQLDIR/support-files/mysql.server
In my case, I copied it into the /sbin/init.d directory, then
made a link to it in the /sbin/rc3.d directory. (As always,
check your permissions to make sure the script is executable.)
Now that MySQL is running, you are ready to set up the database
to log Snort alerts. We will be using Snort 1.7 in order to be compatible
with ACID and MySQL. Snort 1.7 will log the full packet payload
in addition to the headers, which previously was not possible. There
are guidelines to using databases with Snort at:
http://www.incident.org/snortdb/
a site maintained by Jed Pickel, as well as in the README.database
file in the Snort distribution. These instructions cover how to log
to Oracle, unixodbc, and postgresql databases as well, so MySQL is
not your only choice. Per the incident.org site's excellent
instructions, the first step is to create the Snort database:
echo "CREATE DATABASE snort;" | mysql -u root -p
Then create a user with INSERT and SELECT privileges (other than root)
in the system's /etc/passwd file. Give them this privilege
once you have fired up the MySQL client:
$MYSQLDIR/bin/mysql
grant INSERT, SELECT on snort.* to kristyw@localhost;
In this case, my user's name is "kristyw".
At this point you need to create some tables for Snort to log
to. The collaborators of Snort have made this easy by providing
a script under their distribution's contrib directory
called create_mysql. You can run it from the Snort source
directory:
$MYSQLDIR/bin/mysql -D snort -u root -p < ./contrib/create_mysql
We are ready to compile Snort! When you run the configure script,
in addition to any of the other options you want to use, make sure
you include -with-mysql=$DBHOME. Then the configure script
will know to test your installation and make sure that it works properly
with Snort, which you will see when it tests for the various databases,
and MySQL returns a "yes". Then proceed to make and
make install as normal.
Once this is done, you will need to configure your rules database
to include the database plugin. If you are upgrading, don't
forget to customize your ruleset to utilize your existing tweaks
and customizations. Download the latest ruleset from:
www.snort.org
If you are installing this for the first time, then you need to make
sure that you change the network options to fit your environment.
Specifically for this plugin, you will want to add an output rule
to your rules database. Generally, there are two ways to do this.
The first way is to update the snort.conf by unremarking and
customizing the output rule that specifies:
output database: log, mysql, user=kristyw dbname=snort host=localhost password=password encoding=hex detail=full
Where "log" is the type of output going to the database,
"Mysql" is the type of database, the "user" is
who will be inserting the data, name of the database, what host it
is on, password, "encoding" is what format to log in, and
"detail" is how much you would like to log.
The second way is to add this line to the single database file
that you have downloaded from the snort.org site. Then you
can start Snort and watch the logging take place! I started Snort
with the following options:
$SNORT_HOME/snort -D -c /$SNORT_HOME/rules.file -e -i /dev/diag/lan0
To check that data is actually getting into your database the way
you want it to, you can check the table contents with the following
MySQL commands. Enter the MySQL client:
$DBHOME/bin/mysql -u root -p -D snort
You will be prompted for a password.
Show the tables in the database:
mysql> show tables;
+-----------------+
| Tables_in_snort |
+-----------------+
| data |
| detail |
| encoding |
| event |
| icmphdr |
| iphdr |
| opt |
| sensor |
| tcphdr |
| udphdr |
+-----------------+
10 rows in set (0.00 sec)
Show the data in the selected table:
mysql> select * from event;
+-----+-----+-----------------------+---------------------+
| sid | cid | signature | timestamp |
+-----+-----+-----------------------+---------------------+
| 1 | 1 | IDS152 - Ping BSDtype | 2001-01-31 09:09:53 |
| 1 | 2 | IDS152 - Ping BSDtype | 2001-01-31 09:09:54 |
| 1 | 3 | IDS152 - Ping BSDtype | 2001-01-31 09:09:55 |
| 1 | 4 | IDS152 - Ping BSDtype | 2001-01-31 09:09:56 |
| 1 | 5 | IDS152 - Ping BSDtype | 2001-01-31 09:09:57 |
[snip]
| 1 | 30 | ICMP Unknown Type | 2001-01-31 09:18:19 |
| 1 | 31 | ICMP Unknown Type | 2001-01-31 09:28:19 |
| 1 | 32 | ICMP Unknown Type | 2001-01-31 09:38:19 |
| 1 | 33 | ICMP Unknown Type | 2001-01-31 09:48:19 |
Once you are satisfied with the setup of your database, you can either
leave it as is, or set up ACID, if you would like to have a front-end
console for Snort. You also have the option at this point to run the
snortdb-extra script, which adds some extra tables that the
original script does not include. The snortdb-extra script
can be found in the Snort contrib directory as a gzip file.
If you unzip the file and read it, it says to use it by running zcat.
This only works with compressed files, so you should first gunzip
it, then compress it, then run it by using the following:
zcat snortdb-extra | mysql -D snort -u root -p
I recommend adding these tables because they provide more depth in
the entries that you log to your database.
ACID: Analysis Console for Intrusion Databases
Now you have invested all this time and sweat into configuring
your Snort to log to a database, but without getting deeply imbedded
in SQL statements, how do you get any information out of it? That
is where ACID comes in. ACID is "a PHP-based analysis engine
to search and process a database of incidents generated by security-related
software such as IDSes and firewalls (e.g., Snort, ipchains --
see Resources at end). ACID is also great for logging several disparate
sensors into one console. The extra ingredients needed for this
setup (in addition to Snort 1.7 and MySQL) are the Apache Web server
with PHP 4:
http://www.apache.org
or:
http://www.php.net
ACID:
http://www.cert.org/acid/
or:
http://acidlab.sourceforge.net/
and ADODB v0.93+ if you are using ACID v.0.9.6b2+ (php.weblogs.com/ADODB).
For my installation, I am using Apache 1.3.17, php 4.0.4pl1, and ACID
0.9.6b1.
To get everything to work properly together, I found it best to
install these packages in the order suggested by the Apache README.configure
file (Apache, PHP, and ACID. You also need flex to complete this
install; it's also a requirement for Snort so if you are compiling
on the same server, then you should be set). There are other packages
required, depending on what you are doing overall with this distribution,
so be sure to read the README files closely. The steps to do this
are:
1. Untar the Apache distribution ($APACHEDIR).
2. Cd into this directory and run the configure script
(default installation goes to /usr/local/apache), using any
options you might need for your site, then make and make
install.
3. Untar the PHP distribution in a separate directory ($PHPDIR).
4. Configure PHP with ./configure -with-apache=path_to_apache
-with-mysql=/$MYSQLDIR; make; and make install.
5. For some reason, the correct lib file did not get copied
to my Apache directory during this process, so I manually copied
/$PHPDIR/libs/libphp4.a to /$APACHEDIR/src/modules/php4.
6. Then, go back to the Apache directory and run ./configure
-activate-module=src/modules/php4/libphp4.a.
7. Fire up your Apache server with /usr/local/apache/bin/ \
apachectl start and make sure it starts up.
One hitch that I ran into on HP-UX was that Apache would not start
right away. After some digging through the Apache FAQ a bit, I discovered
that HP will not allow you to run a server like this with nobody/nogroup
as owner and group. I needed to create a WWW user (or whatever you
prefer), and a WWW group with WWW as a member. I changed my httpd.conf
file accordingly, and everything came up fine. Also, make sure that
the PHP plugins are unremarked in your httpd.conf file before
starting up the Apache server. The two lines to look for are:
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps
To set up ACID, untar the distribution in a designated place. I set
up a virtual host through Apache in order to reach the PHP files for
ACID. To do this, I turned once again to the httpd.conf file
and added near the end of the file:
<VirtualHost Ipaddress>
ServerAdmin mymail@mycompany.com
DocumentRoot $ACIDDIR
ServerName acid
ErrorLog /var/log/httpd/error-log
</VirtualHost>
The ServerName can be whatever URL you would like to call it. I then
restarted the Apache server and went to my ACID URL, and up comes
the acid_main.php page. There is more configuration, however,
to properly point it to your database. Editing the acid_conf.php
file is in order, updating the following variables:
$alert_dbname: MySQL database name where the alerts are stored
$alert_host: host where the database is stored
$alert_port: port where the database is stored
$alert_user: username for the database
$alert_password: password for the username
See Figures 1 and 2 for snapshots of ACID.
After ACID is set up and running, you can use it for many things.
You can search for incidents based on specific criteria, or complex
combinations of criteria. ACID can also be used for alerting, statistical
analysis, and graphical reporting. Items like these (especially
reporting) help to show management why you have perimeter security
in place.
Snort and Big Brother
Say that you don't have the resources for the scenario that
we have just built utilizing a database, Snort, and an Apache/PHP
Web server. But you already have an Apache Web server running Big
Brother (a free, Web-based monitoring and reporting tool for networks
and servers) that runs all of your network monitoring. Snort to
Big Brother, written by Mattias Sandstrom, is the tool for you.
The Snort to Big Brother link is:
http://www.ffoff.org/~mattis/linux/snort2bb/
I am using snort2bb-000831.tar.gz for this article. Big Brother
can be found at:
http://bb4.com/
This section assumes that you already have Big Brother set up. (See
March 1997 and September 1998 issues of Sys Admin for articles
about Big Brother.)
Snort to Big Brother is a Perl script that parses your Snort log
and sends reports to Big Brother. Advantages to this include that
Big Brother can be set for a certain level of notifications and
then page you and update the central Big Brother console, which
gives a visual alert. The key ingredients to make this script work
include: your Snort machine, Perl v.500503 or later, and the Big
Brother client. For Perl, you will need the File::Tail module, which
needs the Time::HiRes module. Both modules can be downloaded at:
http://www.cpan.org/modules/index.html
The first issue to tackle is the Perl module installations. After
downloading the required modules (Time-HiRes-01.20.tar.gz and
File-Tail-0.98.tar.gz), gunzip and untar them in a secured
location:
Cd /tmp
gunzip Time-HiRes-01.20.tar.gz ; tar xvf ./Time-HiRes-01.20.tar
gunzip File-Tail-0.98.tar.gz ; tar xvf ./File-Tail-0.98.tar
cd Time-HiRes-01.20
perl ./Makefile.pl
make
make test
make install
cd into the File-Tail-0.98 directory and repeat the same process.
Install the Big Brother client next to get your Snort box to talk
to the Big Brother server. First, add a bb user to your Snort
server. To do the client installation, there are two possibilities.
If your Big Brother server is running on the same platform as your
Snort box, then you can go into the Big Brother install directory
and run the bbclient script with the host name of your Snort server
-- bbclient snortserver. This will create a tar
file of the files that you will need to run the Big Brother client
on your Snort server. If, however, your Snort server runs on a separate
platform, then you will need to compile the Big Brother code on
this platform. To do so, download the latest code from bb4.com,
untar the distribution, cd into the install directory and
run bbconfig $OSNAME (where $OSNAME is the name of
the OS on which you are installing).
After a running through a brief and intuitive installation script,
you will need to cd into the src directory:
cd ../src
make; make install
Then, change the ownership of the Big Brother directory to be owned
by the bb user. Double check that the bb-hosts file
in your etc directory has the correct BBDISPLAY variable for
your Big Brother server.
Now we are ready to fix up the snort2bb.pl file to read
Snort alert files and to send reports to the Big Brother server.
Untar the snort2bb-version.tar in a directory, then cd
into the snort2bb directory. You need to edit the snort2bb.pl
script, specifically the $bbhost and $localhost variables
to point to the Big Brother server and the hostname of your Snort
server. You also need to make sure you change the $snort_file
and $snort_file_type variables. The former is the name of
the file to which Snort is logging alerts, and the latter is either
the default (which looks to /var/log/secure) or you can change
it to a 1 to indicate that the log is in the "-A" fast
option for Snort, you need to log Snort this way or snort2bb
will not work! I also had to modify the path for Perl as I had it
installed in /opt/perl5/bin, not /usr/bin as the script
mentions.
There are other options that you may want to tweak, specifically
the $WARN and $PANIC variables, which will tell Big
Brother when to notify you (either by pager or to the console).
After tweaking, you are ready to run the snort2bb.pl script.
If you experience problems, there is a verbose mode (using -v)
so that you can see what is beginning.
Snortsnarf
There is also a tool called Snortsnarf written by Jim Hoagland
and Stuart Staniford of Silicon Defense that helps sort through
Snort logs with relative ease. You can find Snortsnarf at:
http://www.silicondefense.com/snortsnarf/
Snortsnarf is essentially a Perl script that parses through your Snort
alert file and creates a clean set of html files that you can review
to see what is going on with your Snort sensor. It also includes links
to look-up hosts that cause source traffic through ARIN, RIPE, APNIC,
or Geektools, as well as links to the whitehats.com signature
database to obtain more information on the type of alert you are seeing.
See Figures 3 and 4 for a more detailed look at Snortsnarf.
Another great feature of Snortsnarf is that it is easy and quick
to set up. You will need Perl on your system, then download the
latest version (which was SnortSnarf-011601.1.tar.gz when
I wrote this). Gunzip and untar the distribution:
gunzip SnortSnarf-011601.1.tar.gz; tar xvf SnortSnarf-011601.1.tar
then grab your snort.alert file (put it wherever you would
like to keep a sizable amount of data) and run:
$PATHTOSNARF/snortsnarf.pl snort.alert
This will generate the necessary html files. Depending on how fancy
you want to get, you can open a browser local to the snortserver and
then open up the index.html file and be on your way. I set
up a virtual server on an existing Apache Web server on the network.
I wrote a script to grab the previous day's alert file, delete
the old html files, run Snortsnarf, and then check my Web site. This
approach has made it easier to read the Snort alerts and understand
exactly what I am being hit with. It also helps weed out rules that
may be causing a lot of false positives, or that you don't necessarily
want to track, so you can remove them.
To summarize, the Snort project has really grown. Not only in
having more rich features itself, but also in additional projects
that can assist you in making sense of your data. There are numerous
tools that were not mentioned here, and the best source to find
something that would work for you is the snort.org site.
It is updated constantly with the latest accompaniments for Snort.
The other key source of information is the Snort users mailing list,
which can be joined on the snort.org site. This list is maintained
by Martin Roesch and the key Snort developers. The information gained
through this list is invaluable and often not documented anywhere
else.
Resources
http://acidlab.sourceforge.net/ -- ACID readme and
homepage
http://www.snort.org
http://www.incident.org/snortdb/
http://www.bb4.com
http://www.ffoff.org/~mattis/linux/snort2bb/
http://www.silicondefense.com/snortsnarf/
Kristy Westphal is a versatile network administrator, skilled
in troubleshooting and process analysis. She is knowledgeable in
UNIX and NT, as well as project management and security/disaster
recovery planning with more than 7 years of experience in the IS
field. She can be reached at: ckwestphal@home.com.
| 
