 SNIPS
Ron McCarty
SNIPS, or System and Network Integrated Polling Software, is Netplex
Technologies' revamp of Network Operations Center On-line (nocol),
which I wrote about in the August 2000 Net Admin column. SNIPS is
a network-monitoring tool that provides both a command-line and
Web interface to monitoring alarms. SNIPS provides alarm levels
that provide an escalation of conditions based on the number of
failures. This allows flexibility in reporting and prevents one-time
anomalies (such as a network engineer resetting an Ethernet port)
from creating alarms for network services that were temporarily
unreachable while the Ethernet port was resetting.
SNIPS architecture logically can be divided into the monitoring
function and alarming function. The monitoring function is responsible
for determining whether services are running and reporting the status,
which can be viewed in real-time by the systems administrator through
a terminal or Web interface. The alarming function (actually an
API from SNIPS that logs the change of status) can then be alarmed
or simply stored for historical purposes. (The logging portion can
actually run on another system.)
SNIPS comes with more than 25 monitors for monitoring both network
elements (etherload, ciscomon, bgpmon) and system service elements
(WWW, email, name services). The monitoring levels supported by
SNIPS are: info, warning, error, and critical. These alarms are
based on escalations of the previous level (i.e., warning is considered
worse than info, and critical is the highest level).
SNIPS Installation
SNIPS runs on most versions of UNIX. For this particular installation,
Red Hat 7.3 was used. SNIPS can be downloaded from http://www.netplextech.com/software/downloads/snips/.
Documentation for installation is included, but here is a short
overview.
As root, place the gzipped tar ball in /usr/local/src and
unzip it:
gunzip snips-1.1.tar.gz
Untar it and cd into the directory structure created from the
tar command:
tar xvf snips-1.1.tar
cd snips-1.1
Run configure and the make commands that will install
SNIPS into the /usr/local/snips directory, unless you tell
the configure script otherwise:
./Configure
make
make install
make root
Add the following entries to your /etc/services directory:
nipslog 5354/tcp # snipslogd with TCP
snipslog 5354/udp # snipslogd with UDP
hostmon 5355/tcp # hostmon uses TCP
Edit the SNIPS config file /usr/local/snips/etc/snips.conf
and enter the following (or remove the comments from the existing
lines from the standard snips.conf file):
LOGHOST localhost
DATADIR /usr/local/snips/data
CONFIGDIR /usr/local/snips/etc
PIDDIR /usr/local/snips/run
(If you keep your log files in /var, then you can point the
DATADIR to the appropriate location or link the /usr/local/snips/data
directory to the appropriate location.)
Create the snips user. The snips user can be used for a standard
login for operators to monitor the system through a terminal window.
The following command will create the user and add the snipstv
as the shell, which will start the snips text viewer (tv)
when the operator logs in. Use the following command to create the
user:
useradd -d /tmp -s /usr/local/snips/bin/snipstv snips
Change the password of user snips as appropriate. (Some administrators
choose not to give it any password, which allows operators to log
in and immediately be given the SNIPS terminal interface (snipstv)
since it has been set as snips default shell.)
Now a simple connectivity monitor using ippingmon can be created.
SNIP config files are stored in the etc subdirectory of the
application directory (/usr/local/snips/etc/). Create a file
called ippingmon-confg with the following entries. (Change
router1's IP address into an IP address on your network that
answers, and use a non-existing IP address for router2 to see SNIPS
in action when router2 does not respond:
POLLINTERVAL 300
PACKETSIZE 100
NPACKETS 5
PKT_THRESHOLDS 3 2 1
RTT_THRESHOLDS 100 500 1500
router 192.168.1.1
router2 192.168.1.2
These entries tell ippingmon how often to run and how to determine
the monitoring level alert if changes occur. The POLLINTERVAL
tells ippingmon to run its test every 5 minutes (300 seconds) and
to use an ICMP package size of 100 bytes. NPACKETS determines
the number of packets to send with the ICMP echo request (ping).
The PKT_THRESHOLDS and RTT_THRESHOLDS determines when
the ippingmon should change levels. The PKT_THRESHOLDS tells
ippingmon the number of packets (echo replies) that will cause a change
of level. In this case, if three packets were received, the level
would change from info to warning; but if only two packets were received,
the level would change to error.
The RTT_THRESHOLDS stands for round-trip time and is defined
in milliseconds (in this case, 100, 500 (1/2 second), and 1500 (1.5
seconds)). If a packet took more than 1.5 seconds to be received,
then the level would change to critical.
ippingmon allows each host to have specific settings of PKT_THRESHOLDS
and RTT_THRESHOLDS by placing values on the particular line:
router2 192.168.1.2 3 2 1 100 500 1500
The 3 2 1 is the PKT_THRESHOLDS entry for router2, and
the 100 500 1500 is the RTT_THRESHOLDS.
Once the /usr/local/snips/etc/ippingmon-confg is created,
the ippingmon can be started:
/usr/local/snips/bin/ippingmon &
The results of the ippingmon can be seen by telneting to the system
and logging in as snips (or su'ing to snips). The various
levels can be seen by typing "L", and then the level. For
example, pressing "L" and "4" will give a screen
similar to that shown in the Figure 1.
In addition to the basic connectivity shown with ICMP above, SNIPS
can be used to monitor many services. In this article, I'll
cover the ones that are commonly within the realm of UNIX sys admins.
DNS Monitoring
Domain name services are often implemented on UNIX systems, but
regardless of the OS providing the name services, the SNIPS DNS
monitor (nsmon) can monitor DNS servers.
The nsmon-confg file is straightforward, with a minimum of four
entries: a POLLINTERVAL, a DOMAINNAME, and a host
entry that is made up of a host name and IP address. For example,
the following entry will monitor the BIND daemon running on sunny.mcwrite.net:
POLLINTERVAL 600
DOMAINNAME mcwrite.net
sunny.mcwrite.net 192.168.1.252
The DNS monitor can then be started much like the ippingmon:
/usr/local/snips/bin/nsmon &
The results can also be seen the same as with ippingmon -- telnet
to the system as snips or use the Web, which will be covered later.
TCP Services
Services that use the TCP protocol can be monitored using the
portmon monitor. Its configuration is similar to both the ippingmon
and the nsmon, but the connect and disconnect strings must be provided
in the portmon-confg. For example, to monitor client email
(POP3), the following portmon-config can be used:
POLLINTERVAL 600
TIMEOUT 30
SIMULCONNECTS 64
HOST sunny.mcwrite.net 192.168.1.252 SMTPport 25 Critical HELO portmon.test
info 250
QUIT quit
Web Viewing
For the above examples, the snips user account was used for viewing
SNIPS reporting activity. This section covers the configuration
of SNIPS to use a Web server to display the results. SNIPS should
function with any Web server that supports CGI. If CGI is not preferred,
the script can be run as part of a cron job, and content created.
With the steps below, either method can be used.
There are two subdirectories in the /usr/local/snips/web
directory: cgi and html. The contents of the cgi
directory need to go to the webs cgi-bin directory, and the
files in the html directory need to be copied into the snips
subdirectory within the html document tree of the Web server.
The following steps will perform this task with Apache running under
Red Hat 7.3:
cd /usr/local/snips/web
cp -R cgi/* /var/www/cgi-bin/
mkdir /var/www/html/snips/
cp -R html/* /var/www/html/snips/
To see immediate results, the genweb.cgi can be run, which
will create html documents for each of the four levels (info, warning,
error, and critical):
/var/www/cgi-bin/genweb.cgi
The results can be seen by pointing a browser to the particular html
file in the Web server's snips subdirectory:
http://righty.mcwrite.net/snips/Info.html
http://righty.mcwrite.net/snips/Earning.html
http://righty.mcwrite.net/snips/Error.html
http://righty.mcwrite.net/snips/Critical.html
Figure 2 shows and example Critical.html output. These pages
are linked to each other for easier navigation, but an index.html
could also be created as a table of contents for the four pages.
As mentioned, the genweb.cgi can be run as part of cron.
To add it, run the following as root:
crontab -e
and then enter the following into the crontab:
0-59 * * * * /var/www/cgi-bin/genweb.cgi
The script will then run every minute.
Loose Ends
This article did not cover the automatic startup of the monitors.
The monitors can be started as part of the System V startup scripts.
Additionally, the keepalive_monitors.pl script in the /usr/local/snips/bin/
directory can also be run as part of snips crontab to ensure all
monitors are running. External alerts were not covered, but are
also supported by SNIPS.
SNIPS uses two aliases -- snips-ops and snips-admin --
that require aliases to be set up with your mail transport agent
(typically /etc/aliases or /etc/mail/aliases).
SNIPS Fit
SNIPS is an ideal monitoring tool in small and some mid-sized
operations, and it can probably be maintained by a sys admin or
net admin without creating too much additional work. SNIPS can also
be used in large organizations to monitor specific areas (such as
all Web servers); however, the standard monitoring tools do not
encompass enough services for it to be deployed as the standalone
network monitoring solution, "out-of-the-box". Larger
organizations require tools that are application- and system-centric
monitoring (e.g., email queues, database performance, and other
monitoring), which require more than knowing simply whether the
service is running.
Ronald McCarty received his bachelor's degree in Computer
and Information Systems at the University of Maryland's international
campus at Schwaebisch Gmuend, Germany. He works for Sonus Networks
as a senior systems engineer on a customer team responsible for
a major telecommunications carrier. Ron is the co-author of New
Rider's Linux Routing (http://www.linuxroutingbook.com/).
He spends his free time with his two best friends in the world:
his daughter, Janice, and his wife, Claudia. Ron can be reached
at: ronald.mccarty@gte.net.
| 
