Cover V08, I09
Table 1
Table 2


Hardening a Host

Dave D. Zwieback

The first postulate of computer security is that no computer system can be completely secure. While this statement makes the work of computer security specialists seem like that of Sisyphus, rolling the heavy rock up the hill only to have it roll back to the bottom in a never-ending cycle, careful design of a system can greatly reduce the potential risk of compromise. Security must be balanced with user requirements, however. For instance, one of the most effective ways to improve security is not to connect the system to the network (like the Internet) and to restrict all physical access to it. This, unfortunately, defeats the purpose of having a system in the first place.

In this article, I will focus on building a hardened host, which is to be connected to a network (like the Internet) and which will run one or more applications (e.g., a firewall, Web or mail servers, database servers). Any application adds its own distinct vulnerabilities to the mix; however, I will concentrate on making the underlying operating system as secure as possible as well as limiting the damage that compromised applications can cause. In my discussion, I use Solaris 2.6 as the default OS; however, the general concepts discussed here apply to most modern UNIX-based systems and beyond. Because of the narrow focus of this article, the discussion here is by no means all-inclusive. For a more complete approach to security, I refer the reader to the resources listed in the appendix.

A successful approach to building a hardened host can be summarized in three steps:

1. Disable all possible ways of accessing the system.

2. Allow only those ways that are necessary for the user.

3. Secure the allowed means of access.

This bottom-to-top approach makes sure that nothing is missed. That said, let's examine areas of system vulnerability before delving into how to harden a host.


Although the sheer number of potential vulnerabilities and exploits that have been documented to date can be staggering, most of these can be classified into only a few categories:

1. Denial of Service (DoS)

2. Buffer Overflows/Race Conditions

3. Trojan Horses/Viruses

4. Password Cracking

5. Improper permissions on admin files and programs

A typical DoS attack succeeds when the attacked machine or service is disabled or its performance is severely impacted. For instance, an unhardened Solaris 2.5.1 host is susceptible to SYN and ICMP flooding attacks, which will often slow the machine down to a halt. Another example of a DoS vulnerability is incorrect quotas and permissions that may allow malicious users to fill up a file system. While under a DoS attack, the system or data is rarely compromised. However, DoS attacks are often used as part of larger attacks, and their impact can be equally severe. Consider, for example, the cost of an automated stock trading system that was unavailable for an hour because of a DoS.

Perhaps the most dangerous category of vulnerabilities consists of buffer overflows and race conditions. Even though these types of vulnerabilities have been exploited for years, new ones are being constantly discovered in various parts of the OS and applications. In most cases, they arise due to unchecked user input (read: bad design). For instance, if a program (running as root) does no checking of user data input and normally expects a 4-bit integer as input, but instead receives a carefully constructed 4000-bit string, it may crash. In the process, it may overwrite the stack and execute an arbitrary command (e.g., /bin/sh) as root. Buffer overflows are especially dangerous because, in many cases, they may be performed remotely (as in the recent imapd exploit).

Race conditions are simply another variation on the “bad design” theme. In a typical scenario, a program writes to a temporary file in a directory that is publicly writeable (like /tmp). This file may be present for only a few seconds, but if a malicious user manages to replace it with a symbolic link to another file, the latter could be overwritten, possibly with improper permissions. This type of attack is commonly used to overwrite important system files (like /etc/password or /etc/shadow).

An equally dangerous type of exploit is the so-called “Trojan Horse”, a program that appears to be harmless but is not. For instance, a malicious user may compile a program called ls and place it in a public directory. When another user -- possibly root -- executes this program, it may act as the normal ls and show the contents of a directory, but it might also remove all the files in the directory, or change permissions for them. If a host has already been compromised, special sets of tools like RootKit and DaemonKit are often used to cover up the attack. These tools replace common binaries (like ls, ifconfig, and ps) with modified versions that hide the fact that the system has been broken into. Additionally, these tools can be used to bypass the standard log files or remove any trace of suspicious activity from them.

Passwords are perhaps the weakest link in security. Besides being easily guessed, in a shared network like the Internet, passwords can be “sniffed” in transit, because in most cases the user's input is passed in plaintext. On older or improperly configured systems, encrypted passwords can be read from the /etc/password or /etc/shadow files, and there are a number of brute force tools available to crack these passwords.

Finally, improperly set up systems often have important system files configured incorrectly, or perhaps have unnecessary setuid bits on programs that can be used to compromise the system's security. The good news is that careful system design will reduce the risk associated with most of the above mentioned vulnerabilities.

Presenting Vulnerabilities

The first step in designing a more secure system is to precisely define its purpose. Whether the system is going to be a Web server, an ftp server, a mail relay host, or a firewall will largely determine how it should be configured and what precautions should be taken to increase its security. In general, it is not recommended to run more than one application on a system since a vulnerability in one application can be used to compromise another.

It follows that, to reduce the risk of exploit, the system should be configured with a bare minimum number of options. For example, you might install only the Solaris Core and the Terminal Information clusters, along with the latest recommended patch cluster from Sun. The disks should be partitioned to allow enough space for the logs stored in /var, and with additional swap space to help deal with DoS attacks. As an added precaution, the system should not be connected to the network until it is completely set up and secure.

As much as it would be preferable to have all access to the system disabled by default, even this basic Solaris setup comes with a variety of potentially unsecure services enabled. Thus, the next step is to comb through the files in /etc/rc*.d directories (or /etc/init.d) and rename any that are associated with services that are not absolutely necessary. Perfect candidates for removal may be Sendmail, tftp, nfs, all of which have a rich history of vulnerabilities.

Let's turn our attention to permissions of files. Solaris ships with the Automated Security Enhancement Tool (ASET), which automatically tightens system security by adjusting permissions on system files (among other things). The public domain program “fix-modes” does much the same thing.

Ideally, there should not be any setuid programs on a secure system; however, Solaris comes with more than 70 setuid programs by default. It is advisable to go through all of them (find / -perm -4000 -print) and turn off the setuid bit unless it is required for the intended use of the system. To further decrease the risk of exploit, it makes sense to modify the /etc/vfstab to mount all file systems with the nosuid option. Taking this one step further, it is possible to mount the / and /usr file systems with the ro (read-only) option. Highly secure systems accomplish this by accessing these partitions from a CD-ROM or other physically read-only media.

Network Setup

Having a host connected to a network introduces a large set of vulnerabilities into the mix. For instance, by default, if more than one interface is enabled, Solaris will route between them, and this can be used to attack other systems. To disable routing, execute:

touch  /etc/notrouter

Additionally, instead of using the dynamic routing that is enabled in Solaris by default, use static routing: disable the in.routed and in.rdisc daemons, create /etc/defaultrouter; add /usr/bin/route commands to /etc/init.d/inetinit file if multiple routes need to be defined.

Since both DNS (bind) and NIS have been the subject of a number of well-publicized security breaches, it is best not to rely on these services for name resolution. Instead, where feasible, use the /etc/hosts file. DNS resolution should be used in all other cases, since NIS is extremely unsecure.

Solaris 2.6 network settings should also be tuned to prevent a number of vulnerabilities (see Table 1 for ndd commands that should be added to the end of /etc/init.d/inetinit).

/etc/inetd.conf is used by the inetd daemon for network services that are started on demand. Following the bottom-to-top philosophy, make a backup of inetd.conf and re-create it from scratch, including only those services that are necessary for the intended use of that specific system. Additionally, instead of running these services directly, install Wietse Venema's TCP Wrappers, which allow logging and limiting access. Table 2 lists additional replacement recommendations.

Additionally, if the system is not used for receiving email, run Sendmail periodically run from cron to process outgoing mail rather than as a service. This reduces resource utilization while improving the security of the system.

Preventing Buffer Overflows and Race Conditions

To prevent stack-based buffer overflows from being exploited, and to be notified when such an attempt is made, place the following two options in /etc/system:

set noexec_user_stack=1
set noexec_user_stack_log=1

Since legitimate programs are commonly written to execute code from the stack, they will not work properly when noexec_user_stack is set to 1. Unfortunately, there is no way to allow some programs to execute code from the stack, and not others. The noexec_user_stack setting is system-wide, so test the particular programs by using the noexec_user_stack_log option and watching the syslog before enabling this setting.

Preventing race conditions is best addressed by proper software design. However, if you have to use software that you didn't write and for which you have no source code, run truss on the program to see which files are being read or written. Once you identify the location of files that are being written with potentially dangerous permissions, change their location to a more secure area (if possible), or change their permissions by setting the proper umask value.

Preventing Trojan Horses and Viruses

Although viruses are rare on UNIX systems, Trojan horses abound. Once on the system, these programs are extremely difficult to deal with. There may not be an easy way to find them, because the tools that are used to locate them may be trojaned themselves! Thus the best way to deal with these malicious programs is proactively.

ASET can (among other things) create a database of checksums of important files on the system and will alert the administrator if these files are modified. Tripwire is a more popular tool that has the same functionality. Whatever your tool of choice, it is advisable to keep the created database of checksums on write-protected media, like CD-ROM, and periodically scan the system for changed files. You should also keep a clean copy of system binaries on hand, again preferably on read-only media. This can be invaluable in case of a security breach.

Preventing Password Cracking

Password cracking is an extremely effective way to gain unauthorized access to a system. The encrypted passwords can be sniffed from a shared network, or obtained from compromised /etc/passwd or /etc/shadow files. Passwords are usually cracked either via brute force method (e.g., trying all 10-character combinations of letters and digits) or the so-called dictionary attacks (when all words in a dictionary are used). With the increase of CPU speed, programs like crack are extremely effective.

A proactive method for ensuring good passwords on your system with Solaris 2.6 is to use its Pluggable Authentication Module (PAM) to ensure that easy-to-crack passwords are not used. S/Key (and other one-time password generation methods) is another popular way to provide an additional level of security against password cracking. At the very least, obtain the crack program and run it against your passwords to see if they can be easily cracked.

Log! Log! Log!

If a system has been compromised, nothing is more useful than a complete log of the attack. One of the reasons why TCP Wrappers and other recommended Solaris program replacements are so useful is that they provide additional logging with their functionality. For instance, wu-ftpd can be configured to generate copious logs of activity that regular ftpd cannot.

A log of logins and logouts can be enabled by touching the /var/adm/loginlog file (make sure it's owned by root:sys and has mode of 600). By modifying the /etc/syslog.conf, the syslog facility can be configured to generate additional information, and logs can be further separated by type into different files. Finally, at up to a 20% performance penalty, Solaris process accounting (acct) can be turned on.

Security-related logs are best stored in encrypted format (e.g., with PGP); all logs need to be periodically archived, preferably onto a very secure system (i.e., one connected to the system periodically via a serial or SCSI connection, but not connected to any network) or at least onto tape.

Finally, to ensure that the timestamps in the system log files are accurate, synchronize the system time with xntp.

Odds and Ends

Remove all but the necessary accounts on the system or lock them by putting “NP” in the password field of the /etc/shadow file. Verify that all users have a safe path and umask settings: . (current directory) should never be in the path due to risk of inadvertently running a trojaned program, and umask should be either 077 or 027. If possible, change the permissions on the .login/.profile/.cshrc files to prevent modification by the users.

Consider disallowing root logins via the network by enabling the “CONSOLE” line in /etc/default/login. This way, especially with loginlog enabled, there will be a record of who su-ed into root. When su-ing into root, make sure that you use /bin/su -, which loads the proper environment and also runs su from its proper location. Add users that do not need ftp access to /etc/ftpusers.

Review the permissions and the contents of the cron file for every system account in /var/spool/cron/crontabs. Delete unnecessary jobs or cron files, and enable cron logging by ensuring that CRONLOG=YES in /etc/default/cron.

Consider the use of proxies for commonly compromised services such as Sendmail and httpd. Such proxies can be effective in making sure that valid data are passed to these vulnerable programs. Some proxies simply check the validity of commands (including length, helping prevent buffer overflows), and several also add virus checking. As mentioned before, to reduce potential vulnerabilities, proxy should ideally run on a separate server.

Backing Up

After you've configured the system with security in mind, test it to make sure that it functions as you think it should. Visit security sites to learn of exploits used against your OS and applications and try them on your system (of course, with prior permission from management). Keep abreast of the latest developments by subscribing to security mailing lists, especially the full-disclosure ones (see bibliography).

Upon completion, make a complete backup -- it will be extremely handy in case you have to ever restore a compromised system. Store it in a safe place. In case the physical security of a site is compromised, you should have a set of known good backups offsite.

Wrapping Up

Although complete system security cannot be attained in our imperfect world, careful bottom-to-top design will lead to a system that is more resistant to attacks and that is able to recover more quickly. Many important topics that were not covered in this article include physical security, overall network security, application security, creating an effective security policy, and protecting against an extremely versatile hacking tool -- “social engineering”. For a more complete security solution, all these topics must be considered.


Galvin, Peter Baer. 1999. The Solaris Security FAQ --

Darmohray, Tina and Phil Cox. 1998. Computer Attacks: Trends and Countermeasures. The Fourth Annual UNIX and NT Network Security Conference Course Book.

Garfinkel, Simson and Gene Spafford. 1996. Practical UNIX and Internet Security. Second Edition. O'Reilly & Associates.

San Diego State University's NAC-Security Committee. 1998. Basic Solaris Security Setup --

Sun Microsystems -- and

Tools and Replacements

Wietse Venema's TCP Wrappers and rpcbind --

fix-modes --

xntp (for systems prior to Solaris 2.6) --

Secure Shell (ssh) -- and

Sendmail -- and

Crack --

wu-ftpd --

Tripwire --

Security Resources

L0pht Heavy Industries --

Bugtraq --

About the Author

Dave D. Zwieback is the Technical Director of inkcom (, a New Jersey-based consultancy specializing in UNIX, Networks, Security, and Internet/Intranet development. He can be reached at: