Securing Solaris

Ido Dubrawsky

Activity on the Net has grown phenomenally over the past several years. More and more companies and individuals are connecting to the Internet every day. Today, it is essential for any company to have some sort of presence on the Net. Whether it be a Web page or simply an email server, this presence could mean the difference between success and failure.

Given that fact, more and more companies are setting up some sort of server to provide an Internet presence for themselves. These servers, however, are sometimes not installed with security in mind, thereby becoming tempting targets for system crackers. As evidenced recently with a sudden surge of distributed denial of service attacks (or DDOS), system crackers are finding these machines and using them for setting up their DDOS clients. Web sites, especially highly visible ones, are choice targets of attacks whether for profit or just plain vandalism.

The tools for identifying machines have certainly improved. One of the most well-known tools for scanning a network, NMAP, is well written, easy to use, and (for anyone who has root access to a box) provides a great deal of information about a potential target.

The best way to reduce the possibility of attack on a system is to provide the least amount of information about it. An attacker knowing that a system was running Solaris 2.6 or Solaris 7 back in December of 1999 might also know that rpc.sadmind could be running and is exploitable. By reducing the amount of information that can be gathered about a system, a systems administrator can reduce the likelihood of a successful attack. This article describes simple changes that can be made to make a Solaris system a harder target. While the techniques described here do not guarantee that an attacker will not find a way in, they do increase the difficulty for a successful attack.

TCP Wrapper

One of the first things to do after installing a Solaris, or any other UNIX, operating system, is to install Wietse Venema's TCP Wrapper. TCP Wrapper provides a measure of host-based access control. While not providing a great deal of extra security to a system, it provides better logging as well as some other customizable features such as reverse fingering of an unauthorized access attempt, or the displaying of banners in case of a failed access attempt. Take care when configuring some of the TCP wrapper capabilities. For example, using the reverse finger configuration, if the system also has TCP wrappers and is configured to perform a reverse finger, may cause the two systems to get into a loop that will eventually bog both systems down. Although a discussion of configuring and installing TCP Wrapper is beyond the scope of this article, the software is very much worth having and can be found at: ftp.porcupine.org:/pub/security/ \
tcp_wrappers_7.6.tar.gz.

RPCbind

Remote Procedure Call (RPC) services are used in many network services including NIS, NIS+, NFS, Kerberos, sadmind, and others. Software utilizing RPC uses the rpcbind program to convert RPC program numbers into universal addresses. When a client makes an RPC call to a server, the client first contacts the rpcbind program to find out the address on which the server software is listening. Solaris has many daemons and services that utilize RPC. These include:

• sadmind
• rquotad
• rusersd
• rsprayd
• rwalld
• rstatd
• rexd
• ufsd
• kcms_server
• rpc.cmsd
• gssd
• rpc.ttdbserverd
• rpc.bootparamd
• cachefsd

All of the above services, except for rpc.bootparamd, are started from inetd.conf. rpc.bootparamd is started from /etc/rc3.d/S15nfs.server and is only necessary if the system is also a boot server. Unless specifically required, the above services can all be commented out of inetd.conf to disable them.

A further step to take regarding rpcbind is to replace the stock Solaris rpcbind program with Wietse Venema's rpcbind 2.1 program. Venema's version is designed to be a drop-in replacement for the stock Solaris rpcbind. This new rpcbind is based on the transport-independent RPC source distribution from Sun. The replacement rpcbind program provides for host access control similiar to that provided by TCP Wrapper. Furthermore, all requests that are forwarded by the rpcbind process are done through an unprivileged port, and all remote requests sent to high-numbered UDP ports are rejected. Unlike TCP Wrapper, the host access control code looks only at the IP address of the source. Overall, rpcbind 2.1 provides another layer of protection beyond the stock Solaris system. Wietse Venema's rpcbind 2.1 program can be found at:

ftp.porcupine.org//pub/security/rpcbind
Reducing Solaris' Footprint

$ telnet 10.16.17.205
Trying 10.16.17.205...
Connected to 10.16.17.205.
Escape character is '^]'.

SunOS 5.7

login:

BANNER=""

$ telnet 10.16.17.205
Trying 10.16.17.205...
Connected to 10.16.17.205.
Escape character is '^]'.
login:

$ ftp -i 10.16.17.205
Connected to 10.16.17.205.
220 nostromo FTP server () ready.
Name (10.16.17.205:ido):

$ telnet 10.16.17.205 25
Trying 10.16.17.205...
Connected to 10.16.17.205.
Escape character is '^]'.
220 nostromo.dubrawsky.org ESMTP Sendmail \
   8.9.1b+Sun/8.9.1; Fri, 18 Feb 2000 \
   06:13:09 -0600 (CST)

# SMTP initial login message (old $e macro)
#O SmtpGreetingMessage=$j Sendmail $v/$Z; $b
O SmtpGreetingMessage=

$ telnet 10.16.17.205 25
Trying 10.16.17.205...
Connected to 10.16.17.205.
Escape character is '^]'.
220  ESMTP
inetd

/usr/sbin/inetd -s -t &

If the system is running Secure Shell, then the r-* programs (e.g., rsh, rlogin, rexec, etc.) should be disabled. Listing 1 provides a list of services that should be removed from inetd.conf (excluding telnet and ftp).

Unless an application is installed that requires the services of one of these inetd clients, it is best to disable them completely by either commenting them out of inetd.conf or deleting them and then sending the SIGNUP signal to the inetd process so that it will re-read its configuration file. telnet and ftp should also be disabled and replaced with secure shell even if the system is behind a firewall. All too often, the source of system compromise is not the hacker on the outside, but the hacker on the inside!

Improving Solaris TCP Stack Strength

Once unnecessary services have been disabled, the next step in hardening the Solaris host against attackers is to prevent them from using common scanning tools such as nmap to detect the operating system type. Generally, this is done by tweaking Solaris's TCP/IP stack so that nmap cannot identify it through TCP fingerprinting. Several kernel variables affect the ability of a scan's success. Consider a vanilla Solaris box, that has already been installed, but has had no changes made to the kernel parameters governing the network services. A sample scan of such a system is provided in Listing 2.

The services that nmap identifies can be removed either by shutting off the service entirely (as in the case of X11) or removing the service from /etc/inet/inetd.conf. The information to focus on is the “TCP Sequence Prediction” and “Remote OS Guesses” that nmap provides.

By identifying the target operating system, an attacker can narrow potential exploits. In the case of the system shown here, sadmind might be a good starting place to try an attack. An attacker might also wonder whether the secure shell daemon, sshd, was compiled with the rsaref2 library, recently shown to be vulnerable to a buffer overflow.

Nmap classified the TCP Sequence Prediction for the example host as “random positive increments” with a difficulty on the order of 105 (i.e., a worthy challenge). If the attacker knows Solaris well enough, this tells them the host has a setting of TCP_STRONG_ISS=1 (as found in /etc/default/inetinit). This host may be susceptible to a TCP sequence attack, especially since it permits rsh. Fortunately, by tuning TCP/IP kernel parameters, OS fingerprinting can be made much more difficult.

Solaris Kernel Tuning

Solaris 2.5.1 and above provides for the ability to change certain kernel tuning parameters through the use of the program /usr/sbin/ndd. ndd gets and sets selected configuration parameters in kernel drivers that implement the TCP/IP protocol family. Changes made by ndd can be made at any time while the system is running. However, these changes do not survive across a system reboot and must be applied every time the system reboots in order to be permanent. One of the main problems of using ndd and writing scripts that depend on it is that the kernel parameters are not readily available from Sun's documentation. Also, these kernel parameters may change without notice from one version of Solaris to another. For that reason, it is important to proceed with caution. Changing some of these parameters may cause problems; however, the parameters discussed in this article have been altered in systems installed in a production environment and do not appear to cause any harm. Furthermore, the parameter names that are discussed here and that are adjusted by the included program, secureip, have not changed in the Solaris operating systems versions 2.5.1, 2.6, and 7. The following sections describe some possible kernel tuning options. You must decide whether these options are appropriate for your environment.

TCP/IP Fingerprints

Before discussing changes to the default Solaris TCP/IP stack parameters, I will discuss how TCP fingerprinting works as it relates to nmap. nmap and other scanning tools rely on certain unique elements in a TCP/IP stack to identify the operating system. nmap is good enough that it can reliably distinguish between a host running Solaris 2.4 vs. Solaris 2.5-2.51, and Solaris 2.6. The program does this with several techniques:

1. FIN probe -- A FIN packet is sent to an open port, and the response is evaluated. The correct response, defined by RFC 793, is not to respond at all. However, many stack implementations do not adhere to this and return an RST response.

2. TCP Initial Sequence Number (ISN) -- Determining the TCP ISN is crucial if an attacker wants to hijack a session with the target host. ISN generation can be broken down into three categories: 64-K increments, random increments, and truly random increments (RFC 1948 compliant). By default, Solaris installs using “random increments”. Given sufficient time, an attacker can collate enough data to guess at the next ISN number that the system will use.

3. Don't Fragment (DF) bit -- Solaris sets this bit by default, regardless of whether it is needed.

4. TCP initial window -- Some operating systems, Solaris included, return a constant window size.

5. TCP Options -- These generally provide the most information. Since they are optional, not all hosts support them, however, if they do, then the host may echo back the options in the response. If the options are echoed back, the values set in the response can be used to identify the operating system.

With the above information, it is possible to change some of Solaris's default TCP/IP stack settings, without harming performance and gaining the benefit of hiding the operating system identity from such tools.

The first of these settings to consider is the TCP Maximum Segment Size (MSS). TCP MSS is used to set the largest amount of data that TCP will send to the receiving side. Whenever a connection is about to be established, the segment size used is set to the MTU of an outgoing interface or to the MSS announced by the peer. Each side announces its MSS and, if one end does not receive the other side's MSS option, it uses a default of 536. Both nmap and queso exploit this fact. The best solution is to change the default MSS value. Generally, the larger the MSS the better, until fragmentation occurs. To change the default MSS on Solaris:

# ndd -set /dev/tcp tcp_mss_def 546

# ndd -set /dev/ip ip_path_mtu_discovery 0

# ndd -set /dev/ip ip_ire_pathmtu_interval \
   <time>

To decrease the possibility of a TCP sequence attack (i.e., TCP hijacking), the kernel value tcp_strong_iss needs to be set to 2. This can be done either through ndd directly or by changing the value of TCP_STRONG_ISS in /etc/default/inetinit to 2. Doing so causes the TCP Initial Sequence Number generation by the kernel to be a random number rather than a predictable value.

ARP and ARP attacks

The Address Resolution Protocol (ARP) is used to map 32-bit IP addresses to Ethernet (or hardware addresses). Every network interface in a host has a unique hardware address. Sun SPARC systems have the capability to assign a single Ethernet address to all network interfaces. This capability is achieved by having the network interface acquire its address from the PROM.

ARP works by sending out an address request, usually to a subnet broadcast address, and then collecting the responses to dynamically create an Ethernet-to-IP address map. The ARP packet contains the IP address of the desired host. The response to the ARP packet, the ARP reply, contains the IP address of the desired host as well as its Ethernet address. This reply is then stored in one of two tables in the Solaris kernel: the ARP cache, maintained by the data-link layer, and a host address mapping table maintained by the network layer. When resolving hardware addresses, the ARP cache is queried first by the network layer. If the information is not available in the ARP cache, then an ARP request is broadcast.

There are two types of attacks possible with ARP: denial of service [DOS] and spoofing. A DOS attack prevents one system from exchanging packets with another on the same network, whereas spoofing allows one system to masquerade as another. Both of these attacks make use of ARP's dynamic nature. ARP information may be inserted by an attacker into the ARP cache either locally, through a root compromise, or by feeding invalid replies to a system making ARP requests. Since there is no authentication at this layer, all information provided to the ARP reply is inserted into the ARP cache. This type of attack is known as ARP cache poisoning.

The spoofing attack is similiar to an IP spoofing attack and relies on the “trusted host” model of security. In this case, the attacker disables the peer of the victim host, host B, and uses IP spoofing to insert the attacker's Ethernet address into host A's ARP cache, by having his system masquerade as trusted host B. Countering such an attack requires one of two procedures: 1) disable ARP resolution on host A and use only static ARP entries, or 2) flush the ARP cache as well as the IP address map often. The first option is manageable only in a small environment. Option 2 then becomes the most reasonable way to counter ARP cache poisoning. ndd can be used to change the ARP cache flush interval, as well as the IP routing table flush interval, as follows:

# ndd -set /dev/arp arp_cleanup_interval <time>
# ndd -set /dev/ip ip_ire_cleanup_interval <time>

Securing IP

IP is the protocol that provides connectionless and unreliable delivery of data from one machine to another. Solaris provides many kernel parameters that can be adjusted to provide a more secure system. The default Solaris installation permits IP forwarding whether the system has two or more interfaces or not. The following command can be used to disable this feature while the system is running:

# ndd -set /dev/ip ip_forwarding 0

Another problem with multihomed systems is that an attacker may try to spoof packets coming in from one interface so that they appear to have originated from a different interface. If the system is multihomed and non-forwarding, the attacker may try to send packets to an interface with a destination address in a network attached to a different interface. The following command prevents this problem by requiring all packets entering an interface to have originated from a network attached to that interface:

# ndd -set /dev/ip ip_strict_dst_multihoming 1

ipfilter

Although reducing the possibility that an attacker can use a TCP stack fingerprint to identify the operating system is a step in the right direction, another option to use is the freely available packer filter program ipfilter. ipfilter runs on Solaris 2.5.1, 2.6, 7, and 8. There is even a 64-bit version of ipfilter for those systems installed with 64-bit Solaris. Compiling ipfilter for Solaris is easy, and it installs as a Solaris package. To compile and install ipfilter:

#cd ip_fil3.4.3
#make solaris
#make package

PortSentry

Another useful tool to use with ipfilter (or on its own) is PortSentry. This program monitors probes to a defined subset of ports on a system. If enough probes occur to trigger a response, PortSentry blocks the source of the portscan by installing a static route into the system routing table, which sends all future packets from that source to a routing “dead end”.

If used in conjunction with ipfilter, PortSentry can add a blocking rule to the ipfilter ruleset. PortSentry is configurable to the extent that it can be made to work with various firewall systems including Gauntlet and Checkpoint. For PortSentry to work with ipfilter, a shell program is needed to add the new deny rule to the ruleset in the proper place. The source to the shell program is given in Listing 4. PortSentry can be downloaded from:

http://www.psionic.com/abncus/portsentry

An aspect to consider when securing a system is the logs produced by syslog. Unfortunately, by default Solaris tends to lump many things into /var/adm/messages and /var/log/syslog -- so much so that it becomes difficult, if not impossible, to easily determine what is going on in a system. The first step is to separate much of the information that is logged through syslog. One way to achieve this goal is to reconfigure syslog to log messages to separate files based on syslog facility. All kernel messages would be written to a logfile called /var/log/kernel.log; all daemon messages would be written to a file called /var/log/daemon.log, etc. Furthermore, when installing sudo or tcpd, the LOCAL[0-7] facilities can be used to categorize the logs from those programs as well. A simple, improved syslog.conf might look something like Listing 5.

By breaking up the logging information as much as possible, it becomes easier to discover potential problems sooner. However, even by breaking up the logs this way, the amount of information to be processed is still very great. To ease the management of the log system, the program Logcheck can be used. Logcheck parses through a user-defined set of system logs and maintains an accounting of where it last read the logs. This way, after the initial run, the systems administrator only gets a “differential” report since the last time the logs were inspected. Logcheck can be configured to ignore key phrases in a file so that certain messages are ignored. Also, it can be set to focus on various phrases such as expn or vrfy in logs from Sendmail. The main benefit of Logcheck is that it provides the systems administrator with differential reports from the various logfiles rather than long streams of log data. This makes it easier to identify a potential problem. Logcheck can be downloaded from:

http://www.psionic.com/abacus/logcheck

Every operating system has some unique parameters that can be used by a scanning tool, such as nmap or queso, to identify the operating system. By changing these parameters, it becomes harder for an attacker to use such scanners to identify potential targets. Furthermore, simple steps, such as removing the identity “tags” from telnetd, ftpd, and Sendmail, help reduce the amount of information provided. Without knowledge of what type of operating system is running on a target, an attacker may be less willing to try to break in and may move on to easier targets. While the solutions covered in this article are by no means exhaustive, they do provide a good starting point for securing Solaris Systems.

References

Noordergraaf, Alex and Keith Watson, “Solaris Operating Environment Security”, Sun Blueprints.

Noordergraaf, Alex and Keith Watson, “Solaris Operating Environment Network Settings for Security Security”, Sun Blueprints On-Line, December 1999.

Noordergraaf, Alex and Keith Watson, “Solaris Operating Environment Minimization for Security: A Simple, Reproducible and Secure Applicable Installation”, Sun Blueprints.

“Creating Login Banners”, CIAC Information Bulletin J-043f, June 19, 1999.

Spoon, Kelly, “What You Can do with tcpd”, Linux Gazette, Issue 15, March 1997.

About the Author

Ido Dubrawsky has been working with various UNIX systems since 1992, including SunOS, Solaris, HP-UX, AIX, Linux, OpenBSD, and Ultrix. He currently works as the team lead of UNIX and Network Systems at Globeset, Inc. When not working on UNIX systems, switches, or routers, he spends his spare time with his wife and 3-year-old son. He can be contacted at: ido@globeset.com.