Cover V05, I11
Listing 1
Sidebar 1
Sidebar 2


Enhanced Security on Digital UNIX

Matthew Cheek

This article describes the benefits and management of Enhanced Security (or C2 Security) on Digital UNIX. Other UNIX vendors, notably SCO and HP, implement Enhanced Security similarly to Digital, and most of these concepts and strategies apply to their operating systems. However, I will use Digital UNIX v3.0 and above (including v4.0) to demonstrate how Enhanced Security can enhance your ability to manage Digital UNIX (DU) systems.

One of the first things I discovered after being introduced to DU (formerly OSF/1) was the lack of shadow passwords in the default security configuration. Having shadow passwords means moving the encrypted password from the world-readable /etc/passwd file to a root-only readable file. I consider shadow passwords a necessity given the general availability of cracking tools that compare the encrypted password string with a dictionary of encrypted words looking for a match.

After consulting the system and security administration manuals about this, I found that there are actually two levels of security under DU: Base Security and Enhanced (or C2) Security.

Base Security is the default security level with traditional UNIX passwords. Enhanced Security provides a rich set of Password and Login controls and extensive Auditing features. The Password controls include the shadow passwords feature, configurable password length (both minimum and maximum), and password usage history. The Login controls provide per terminal settings for delays between consecutive successful or failed login attempts, the ability to retire or lock accounts, and logging of the last successful login and unsuccessful login attempt. The Audit features include per user audit profiles, extensive site-definable event auditing, and the ability to send audit logs to a remote host.

Some DU systems have Base Security enabled because of the perceived complexity and limited understanding of the features of Enhanced Security; however, corporate security requirements for shadow passwords and password aging necessitated enabling Enhanced Security on my systems. Unfortunately, Enhanced Security is not well documented, and was met with some apprehension by the system administration staff and user communities. Because of these concerns, I decided to enable Enhanced Security on a test system, work through the implementation and management issues, and document the process.

Enabling Enhanced Security

Assume that you are setting up Enhanced Security on a running system with existing user accounts. This procedure would be similar if you were enabling Enhanced Security on a newly installed system with no user accounts.

You must first choose the global defaults for the password and login control pieces of Enhanced Security. These system-wide global defaults reside in /etc/auth/system/default and provide values for users and devices. Determine and record the settings that make the most sense for your environment. (See sidebar "System Default Database File Format" for field definitions and suggested values.)

Once you have selected values for the system defaults, you can begin enabling Enhanced Security. The steps are:

1. Log in as root

2. Install the Enhanced Security subsets if not already installed

3. Run the secsetup utility and select the ENHANCED security level

4. Reboot the system

5. Adjust System Default Database file to reflect desired global defaults

6. Remove hashed passwords from /etc/passwd file

7. Test your application(s)

Obviously, to enable Enhanced Security, you must have the ability and opportunity to reboot your system(s). Coordinate a time that is convenient with your users to avoid disrupting their work.

The following sections provide detailed instructions for enabling Enhanced Security.

1. Log in as root

This process can be done either while logged in directly as root from the system console or while logged in from a terminal or across the network as a regular, non-privileged user su'ed to root. Check that Enhanced Security is not already enabled with the following command:

# /usr/sbin/rcmgr get SECURITY

If the string "BASE" is returned, you are running Base Security. If, however, the string "ENHANCED" is returned, Enhanced Security is already enabled on this system, and you should jump to step 5 to adjust the global defaults.

2. Install the Enhanced Security Subsets

Before you can enable Enhanced Security, two Enhanced Security Software Subsets must be loaded. These subsets are named OSFC2SECxxx and OSFXC2SECxxx. (The "xxx" specifies the version of the subset.) To determine whether these subsets are loaded, you can check as follows:

# /usr/sbin/setld -i | grep -i c2sec
OSFC2SEC350  installed  C2-Security (System Administration)
OSFXC2SEC350  installed  C2-Security GUI (System Administration)

In this example, both subsets are installed. If you do not receive any output or if the "installed" keyword is absent, you must install both subsets from the master operating system install media using the setld command. For assistance, see the "Installation Guide" for your revision of the operating system.

3. Run the secsetup Utility

The secsetup utility is an interactive program with toggles between Base and Enhanced Security on DU systems. You must be prepared to answer the following questions in addition to selecting which System Security Level you desire:

  • Do you wish to disable segment sharing?

  • Do you wish to run the audit setup utility at this time?

    Typically, the answer to both of these questions is NO unless you have special requirements. See the "DEC OSF/1 Security" manual for more information on these options.

    The following shows an example secsetup session showing the security level being changed from Base to Enhanced:

    # /usr/sbin/secsetup
    Enter system security level(BASE ENHANCED ?)[ENHANCED]: <RETURN>
    ENHANCED security level will take effect on the next system reboot.
    Do you wish to disable segment sharing(yes no ?)[no]: <RETURN>
    Do you wish to run the audit setup utility at this time(yes no ?)[no]: <RETURN>
    Press return to continue: <RETURN>

    Note that default answers to each prompt are in square brackets and can be selected by simply pressing the Return key.

    4. Reboot the System to Enable Enhanced Security

    Before the new security level change takes effect, the system must be rebooted. A simple way to accomplish this is:

    # /sbin/shutdown -r +2 "System being
    rebooted to enable Enhanced Security."

    This shuts down and reboots the system with 2 minutes grace time and displays an informative message to any logged-in users.

    5. Adjust System Default Database File

    Once the system has successfully rebooted, log in as root and make the previously recorded adjustments to the System Default Database File:

    # cd /etc/auth/system
    # cp default default.orig
    # vi default (and apply desired changes)

    6. Remove Encrypted Password Strings from the /etc/passwd File

    The secsetup utility has copied the encrypted passwords from the password field of /etc/passwd into the individual user security databases under the /tcb/files/auth hierarchy. However, secsetup does not remove the encrypted passwords from /etc/passwd. When Enhanced Security is installed on your system, the password field should contain an asterisk (*), as the encrypted password is no longer stored in /etc/passwd. A small Korn shell script that safely removes the encrypted password string from the password field of /etc/passwd and replaces it with an asterisk is shown in Listing 1.

    7. Test your application(s)

    At this point, your system is running Enhanced Security and you should test your applications as appropriate before declaring success. Before your users attempt to log in and perhaps run afoul of the system, note that if any user's password under Base security was longer than eight characters, only the first eight characters will now be accepted as valid. For example, if a password was "toogood2be," the user must enter only "toogood2" to successfully log in. When the user changes his or her password under Enhanced Security, the global or user defaults, if different, will dictate the password length. Make sure your users are aware of this password issue before they log into the newly Enhanced Security-enabled system.

    Disabling Enhanced Security

    If you find it necessary to disable Enhanced Security and return to Base Security, the steps are:

    1. Log in as root

    2. Run the secsetup utility and select the BASE security level

    3. Reboot the system

    This will quickly revert the system back to Base Security and copy the encrypted passwords from the /tcb/files/auth hierarchy into the /etc/passwd file. Rebooting the system completes the process. This will leave all the Enhanced Security files (/etc/auth/* and /tcb/files/*) in place if you decide to re-enable Enhanced Security.

    Account Management

    A primary responsibility of a UNIX system administrator is user account management. This includes account creation, account modification, and account removal. Once Enhanced Security is enabled on a DU system, new utilities are provided and should be used whenever possible to manage user accounts. The two primary tools are XSysAdmin and Xisso, and both are X Window applications. Unfortunately, Digital has not provided a character cell interface to accomplish the same functionality. However, account management can also be performed without running these GUIs, which may occasionally be necessary. Additionally, both XSysAdmin and XIsso will be replaced after the v4.0 release of DU with the single program dxaccounts, the new Common Desktop Environment (CDE) account management tool. Since both XSysAdmin and XIsso still work in the 4.0 and all earlier releases of DU, I will use them for my examples.

    XSysAdmin - This program's role is to create new user accounts, create new groups, modify the new user account template, and retire user accounts.

    New accounts are created by filling out a dialog box with necessary information such as the account name, home directory, desired shell, etc. New accounts are created without passwords. Use the passwd command to set a password for the account after creating the account. New accounts are also created in a locked state. Use the XIsso command to unlock the accounts.

    Account creation can also still be accomplished with the Base Security command adduser. This command is mostly Enhanced Security-aware, with one exception: the string "Nologin" is inserted in the password field of the /etc/passwd file rather than an asterisk. This does not prevent the user account from being used, since the password field of the /etc/passwd file is completely ignored when the security level is Enhanced. However, I find this inconsistency confusing and manually edit the /etc/passwd file (via the vipw command) to correct it. You could also run the remove_passwd script (Listing 1) again after adding accounts with adduser to clean up this minor wrinkle.

    I strongly recommend that you create accounts on DU systems with either the XSysAdmin program or the adduser command unless you have a thorough understanding of all the details of the security database files.

    Creation of new groups is accomplished by specifying the group name and group ID in the XSysAdmin New Group dialog box. Optionally, the Base Security command addgroup can be used to create new groups or the /etc/group file can be manually edited by the superuser to add a new group. XSysAdmin is also used to edit the new user account template, which specifies the default account parameters used when a new account is created. These parameters include password length, account expiration date, time of day restrictions, etc. These parameters are only the defaults and can be overridden on a per account basis. The parameters are actually stored in the System Default Database File, /etc/auth/system/default, and can be manually edited by the superuser.

    Finally, the XSysAdmin program is used to retire user accounts. Retiring a user account, rather than simply deleting it, is a requirement of C2 Security to which DU with Enhanced Security conforms. Retiring an account permanently locks the account and prevents reuse of that account's user ID. Once an account is retired, that account can not be re-enabled.

    The Base Security command, removeuser, will completely remove all traces of an account. If the stringent account retirement requirements of C2 Security are not necessary, the removeuser command can continue to be used to remove user accounts.

    XIsso - This program is used to modify existing user accounts and to edit system security defaults.

    Existing account characteristics can be changed via the Modify User Accounts dialog box of the XIsso program. You can specify the groups the account belongs to, the login parameters, and the password parameters. These parameters are stored in the protected password database files that reside under /tcb/files/auth. This hierarchy has, underneath it, subdirectories with single letter names, each of which is an initial letter for account names. Each file in these subdirectories contains a protected password entry for a single user account, and the filenames are the same as the user account names. For example, the protected password file for root is /tcb/files/auth/r/root. These files can be manually edited by the superuser. See sidebar "Protected Password Authentication Database File Format" for format and field definitions.

    XIsso also allows the system administrator to set the global system parameters of Inactivity Timeout and Account Expiration Warning. The Inactivity Timeout is the number of minutes that a logged in account can remain idle before the session is closed. If a value of zero minutes is specified, no inactivity timeout is enabled and account sessions will not be terminated. The Account Expiration Warning parameter specifies the number of days prior to an account's expiration that warning messages will be shown to the user upon login. These parameters are universal and cannot be set on a per-user basis. Both of these parameters (d_inactivity_timeout and d_pw_expire_warning) are stored in the System Default Database File, /etc/auth/system/default, and can be manually added or edited by the superuser.

    Another aspect of account management under Enhanced Security is the Network Information Service (NIS). NIS is a distributed system used to centralize the management of a common set of system and network files such as the password, group, and host files. NIS allows the system administrator to manage these shared files from a single server. NIS supports both Enhanced and Base Security on Digital machines in addition to non-Digital machines. I do not manage any NIS systems, so I leave it as an exercise for the reader to determine the issues of implementing Enhanced Security in an NIS environment.

    Enhanced Security Logging

    One unique feature of DU is a consolidated security authentication mechanism called the Security Integration Architecture (SIA). This SIA layer isolates the security-related commands such as login, su, and passwd from the specific security mechanisms that include, in addition to Base and Enhanced Security, optional products such as the Distributed Computing Environment (DCE). You do not need to be concerned with this SIA layer except to take advantage of the centralized logging the SIA provides via the sialog file. This log will record all security events, including the success and failure results of login, password changes, and su. With this single logfile, I do not have to monitor multiple logs. To enable the sialog, simply create the logfile:

    # touch /var/adm/sialog

    and the log can be written to by the SIA. The recommended permissions for the sialog are 600. Typically, you would want to prevent non-privileged users from viewing the contents of security logs such as sialog, because there is the possibility of passwords appearing in the log. An excerpt from the sialog shows the types of events recorded:

    SIA:EVENT Wed Jun  5 05:22:08 1996
    Successful session authentication for mcheek on :0
    SIA:EVENT Wed Jun  5 05:22:08 1996
    Successful establishment of session
    SIA:ERROR Wed Jun  5 05:24:11 1996
    Failure on authentication for su from mcheek to root
    SIA:EVENT Wed Jun  5 05:24:40 1996
    Successful authentication for su from root to mcheek
    SIA:EVENT Wed Jun  5 05:25:46 1996
    Successful password change for mcheek

    This log will continue to grow without bounds, so it must be manually truncated periodically. To stop logging of SIA events, remove the logfile. Since the SIA is part of the DU operating system, the sialog can be used with either Base or Enhanced Security.

    Application Issues

    System administrators are typically responsible for installation and configuration of commercial (off the shelf) software. Before enabling Enhanced Security, you must ensure that any applications currently installed are Enhanced Security-aware. Relational Database Management Systems, such as Oracle or Informix, that can be configured to rely on the operating system for user authentication may be affected by Enhanced Security. If such an application were not aware of Enhanced Security, user authentication into the application would most likely fail. If the application vendor does not know whether Enhanced Security will affect its product, implement Enhanced Security slowly and test the application as much as possible before going to production.

    You may also have to support software development efforts for both in-house and commercial applications. Coordinate the implementation of Enhanced Security with any software developers you support on your system(s), and refer to the "DEC OSF/1 Security" manual's section called "Programmer's Guide to Security" for details.

    A final software issue is the freely available software typically obtained from the Internet. There are many security-related tools and utilities available that may or may not support Enhanced Security. I have successfully installed sudo (maintained at the University of Colorado), and wuarchive-ftpd (Washington University FTP Server) on DU Enhanced Security systems.

    sudo is a useful utility that allows the superuser to give certain users or groups of users the ability to run some or all commands as root while logging all commands and arguments. Sudo, version 1.4 and later, supports DU Enhanced Security without modification. wuarchive-ftpd is a replacement ftp server that provides more extensive logging of commands and file transfers than the stock ftp servers provided by UNIX vendors. wuarchive-ftpd version 2.4 supports DU Enhanced Security, but only after applying a patch to the wuarchive-ftpd source before compiling the program. This patch is approximately 150 lines and is available from in /pub/sysadmin.


    Successfully implementing Enhanced Security requires a thorough knowledge of your environment and requirements, careful planning, and ongoing administrative responsibility, both in account management and general system maintenance. However, I have found the benefits of Enhanced Security, comprehensive auditing of security events, and more robust identification and authentication features far outweigh the disadvantages of the Enhanced Security system. What began as a way to comply with corporate security standards significantly enhanced my ability to manage my systems.


    DEC OSF/1 Security, Maynard, MA: Digital Equipment Corp.

    DEC OSF/1 System Administration, Maynard, MA: Digital Equipment Corp.

    About the Author

    Matt Cheek has been a UNIX sys admin since 1988 and currently works for MCI Telecommunications in Colorado Springs, CO, where he supports a variety of mission-critical systems both in development and production. He can be reached at