A Holy Grail for the New Millennium:
No Plaintext Passwords

Victor Hazlewood

Today, approximately 56 million computers are connected to the Internet [1]. All have the potential to connect to any other computer on the Internet to perform authorized and unauthorized activities. This ability to connect to any other computer is the Internet's most useful feature, and arguably its biggest risk.

The continuing advancement of computer networking capabilities and lower costs brings networking technologies such as high-speed modems, Frame Relay, cable modems, ISDN, and DSL to more people. These technologies are changing the way people access computers for work, home, and recreational use. Because of this access by millions of people worldwide, new Internet security methodologies and implementations have emerged in the last few years to combat unauthorized access to computer systems.

One of the most useful methodologies that several sites have adopted in the recent past is the elimination of plaintext passwords for the most widely used TCP/IP services on their networks, including telnet, rlogin, rsh, and ftp. This methodology was adopted on September 1, 1998 by the San Diego Supercomputer Center (SDSC) -- the leading-edge site of the National Partnership for Advanced Computational Infrastructure (NPACI) -- and several of its partner sites, including the University of Texas' Texas Advanced Computing Center, and the University of Michigan's Center for Parallel Computing. This article chronicles the motivations, methodology and implementation, the user experience, and the lessons learned.

The elimination of all incoming plaintext passwords from the most widely used TCP/IP services is considered by Tom Perrine, the Security Officer of NPACI/SDSC, and by myself as the single most significant event in security implementation at SDSC. I also agree with security experts that the elimination of plaintext passwords for all incoming Internet Protocol packets from untrusted networks is a lofty goal for computer systems in the new millenium -- a Holy Grail for the New Millennium.

Risks and Motivations

To understand the risks that exist on the Internet today, and the motivation for SDSC and other sites to eliminate the use of plaintext passwords over the Internet, you have to know a little about where the Internet came from and how it has evolved.

One of the first computer networks was the ARPANET, which was developed in the 1970s. It was developed for the Department of Defense's Advanced Research Projects Agency (ARPA) by universities and corporations. As regional and campus-wide networks came into existence, they linked to the ARPANET in the 1980s. In the late 1980s, NSFNET, which was funded by the National Science Foundation (NSF is an agency of the United States federal government), became the backbone of the Internet. As the popularity of the Internet grew, the NSF decided to lift its ban on commercial traffic starting in 1991. Applications such as gopher and Mosaic began to appear on the Internet, and the first tools for the age of electronic commerce emerged. By 1992, more than one million hosts became part of the Internet [2]. Traffic on and access to the Internet exploded in the early 1990s, and it continues today.

Virtually all Internet applications depend on the Internet Protocol (IP), Transmission Control Protocol (TCP), and User Datagram Protocol (UDP) as developed by a collaboration between Robert Kahn and Vinton Cerf in the early 1970s [3]. These protocols were designed to transmit packets from one computer to another but not to provide security between computers. While these standards were being developed and implemented, security problems on the ARPANET were very rare as access to the computers on the network was very limited [4]. Today's Internet paints a very different picture with all sorts of security-related intrusions, viruses, and denial-of-service attacks in regional and national news on a weekly basis.

A variety of methods exist today to limit the risk of computer intrusions. Each systems administrator or security officer must weigh the costs and benefits of the methods that could be implemented. All risks cannot be eliminated and to attempt to limit all risks to near zero would, of course, be cost prohibitive. Identifying as many risks as possible and addressing those risks with the most manageable, cost-effective security plan is the goal of SDSC.

All SDSC systems administrators agreed around 1996 that one of the most effective security policies that SDSC could implement would be to address the problem of plaintext passwords being passed in packets delivered to SDSC computer systems. Many instances have been documented in the 1990s at SDSC and all over the world of intrusions that led to the eventual installation of a packet sniffer to listen to subnet packet traffic and capture usernames and passwords as they were being delivered to their destination. Packet sniffing has led to the spread of intrusions and the widening of the perpetrator's unauthorized activities. However, until the early to mid 1990s no easy to implement, easy to use, cost-effective method of protecting packets existed. Solutions did exist, but it was debatable whether they were easy to implement, easy to use, and cost-effective for all but the largest and most well-funded sites.

In 1995, a software product called Secure Shell was introduced to the Internet community as a possible solution to the packet sniffing problem. This implementation provided encryption and strong authentication options for communication between computers over the Internet. The author of Secure Shell, Tatu Ylonen of Finland, intended the software to be a replacement for rlogin, rsh, rcp, and rdist. In practice, it is also used as a replacement for telnet.

Top-Down Approach

As interest and experience with Secure Shell grew at SDSC, it was proposed around 1997 by Tom Perrine that SDSC and our partners begin the evaluation of adopting of a strict no plaintext password policy for access to SDSC and partner computers from untrusted networks. Many meetings were held to define the policy and address the concerns of users and management. The policy as it related to SDSC became the following:

The primary purpose and goal of computer and network security activities at for the Partnership is to:

• Protect the intellectual property of the authorized users of the Partnership from unauthorized access or modification

• Enable new modes of utilizing network-based resources by improving the security of network environments

• Prevent the interruption of services to authorized users

• Prevent unauthorized use or abuse of the services

• Protect the computing infrastructure itself from misuse by all users, authorized or not

The NPACI Security Working Group will issue Standards and Guidelines for all users and managers of NPACI computing resources. Compliance with these standards is required by all users having NPACI computing accounts, in order to protect the NPACI computing resources as a whole, as well as all other NPACI users. The Guidelines are recommendations, suggestions and information to assist users in protecting themselves and their own computing resources and data in a fashion consistent with their own tolerance for risk and computing needs.

Based on the above Policy, the following Standard was adopted by the NPACI Security Working Group:

Reusable plaintext passwords will no longer be accepted by SDSC's HPC and other UNIX computing platforms for interactive access from “untrusted” networks.

It was feared the no plaintext password standard could be seen as a radical step that might be unpopular with the users of SDSC computer systems. With this in mind, Tom Perrine decided that the support for such a standard must come from the very top, including NPACI's funding agency, the National Science Foundation (NFS). After communicating the relatively inexpensive cost and the huge potential benefits from the adoption and implementation of this standard, the executives at NPACI and NSF agreed that the standard was not radical, but very forward looking and of great benefit to the NPACI userbase. The no plaintext password standard and its implementation date of September 1, 1998 was adopted.

Implementation Framework

With the adoption of the no plaintext password standard in hand, the task of detailing a plan to implement the standard began. The adopted standard did not specify the services that were affected, so the implementation details could be left up to the systems administrators. Implementation questions or issues could be worked out with Tom Perrine as the Security Officer and Chair of the NPACI Security Working Group.

After several years of experience with Secure Shell, it was decided that the framework at SDSC would revolve around the implementation of Secure Shell and Kerberos to eliminating the incoming plaintext passwords. Other incoming services, such as rlogin, rsh, telnet, and ftp, which allowed plaintext passwords would not be allowed from untrusted networks. Untrusted networks were defined as any network outside of the SDSC security perimeter. Inside the security perimeter, use of plaintext password services would be allowed, but strongly discouraged. Use of ftp seemed to be the only obstacle to the successful removal of incoming (and potentially outgoing) plaintext passwords. Secure Shell does not implement an ftp equivalent interface to transferring files, though one does exist in the Secure Shell version 2 implementation that was not available at the time. The Kerberos package does implement an ftp equivalent interface to ftp, however, it was not expected that the Kerberos solution would be as widely accepted as Secure Shell. (For those target systems who do not also implement the no incoming plaintext password standard, the practice of using the Secure Shell client to connect to SDSC computers and then ftp back to the target host became, and remains, common practice.)

With the framework for implementation outlined, it was up to the systems administrators at SDSC to make sure that the server side of Secure Shell and Kerberos was ported to all SDSC systems in time to test and meet the September 1, 1998 deadline.

For Secure Shell, this turned out to be a simple task as almost all computer systems running at SDSC were already supported by Secure Shell or Secure Shell had already been ported to the unsupported systems. The unsupported systems included the Cray T90 and Cray T3E, which by late 1997 were already running Secure Shell. All desktop machines and other servers were supported by the version of Secure Shell that was available at that time. This included Dec Alpha UNIX, Sun Solaris, Sun SunOS, IBM AIX, and SGI IRIX computer systems.

It was expected that Kerberos would not be as popular a solution by SDSC users as Secure Shell. Kerberos, like Secure Shell, was already supported or ported to all systems at SDSC well before the September 1 deadline. (Support for Kerberos on the Cray systems has waned recently as use of Secure Shell has eclipsed that of Kerberos. Kerberos has not been adopted yet as an NPACI or SDSC authentication standard, so Kerberos support on the Cray's has not been an issue.)

The consultants at SDSC were charged with addressing the needs of the user community and making arrangements for the client side of Secure Shell and Kerberos to be available in some form. The typical user of an NPACI/SDSC system connects to the computer resources from either a Macintosh, WinTel-based PC, or a UNIX system.

Regarding Secure Shell, the UNIX version was available to the academic community for free by just downloading, compiling, configuring, and installing the software. This addressed the needs of the UNIX user community. The Macintosh and WinTel PC systems had clients available -- some were free and some were not. The most robust and feature-rich version of Secure Shell client for Windows and Macs, called F-Secure SSH Tunnel & Terminal, is available from Data Fellows and costs approximately $99 with 50% discounts available for educational use (http://www.datafellows.com). Free copies of the Windows versions of the software exists, but they have limited features and are sometimes are of limited use. For more details about what Secure Shell software is available for Macs and PCs, see:

http://www.npaci.edu/Security/npaci_security_software.html.

Regarding Kerberos, a freeware version exists for UNIX, PCs, and Macintosh computers. It is available at:

http://web.mit.edu/kerberos/www.

For those users who would not or could not use Secure Shell or Kerberos, a backup plan of providing a Digital Pathways SNK one time password device for a fee was put into place. This involved creating a server that ran the SNK software and providing a user with an SNK one-time password card, if necessary. The SNK server system was called snk.sdsc.edu. See Figure 1 for an example SNK session.

The last implementation detail that needed to be worked out involved the notification of the user community. This included the notification of users that access to SDSC systems was going to change via standard methods (message-of-the-day, U.S. mail, email, and Web site messages). It also included the notification of users that services were being removed by displaying a notification banner for the user when the service was requested. These service notices were implemented by using the banners capability of tcp wrappers.

As part of SDSC's security implementation, all systems employ tcp wrappers to control access and log activity on services managed by inetd, including telnet, rsh, rlogin, and ftp. Secure Shell even has an option to implement tcp wrappers when not run from inetd (see Secure Shell documentation) Tcp wrappers has a feature that allows a banner to be delivered before the user sees a login prompt. Listing 1 shows the tcp wrappers /etc/hosts.allow specification at SDSC, which implements banners for telnet, rlogin, rsh, and ftp. SDSC used this feature to notify users of the impending change in service access and continues to use this service today to notify users that plaintext password access via telnet, rlogin, rsh, and ftp is not allowed. Creating the messages is straightforward for all services except rlogin, which requires that the first character of the banner message be NULL (0x0). Figure 2 shows today's message when connecting to telnet from a network outside of SDSC's security perimeter.

Implementation Day Results

After the policy issues and implementation details were worked out, it was only a matter of notifing the user community and waiting for September 1, 1998 to arrive. Sometime around the end of July 1998, all the details fell into place and several SDSC administrators and consultants began to worry and speculate about the rash of complaints and problems that lay ahead. To the amazement of all involved, the implementation of the no plaintext password standard was essentially a non-event at SDSC. September 1, 1998 came and went with only a handful of users contacting SDSC consultants for issues relating to the implementation of the new standard.

Looking back at statistics from our problem ticket database, only five incidents were logged regarding the change:

• One clarification of the use of ftp after Sept. 1

• One kerberos client implementation issue

• Three telnet-related issues from users who were unaware of the policy change

Lesson Learned

The success of the implementation of the new standard as measured by the number of user problems was, I believe, directly related to the advanced planning, testing of implementation, and execution of a well-conceived plan. This included, as with any significant change in policies and standards, commitment by the highest levels of management at NSF, NPACI, and SDSC to the new standards to prevent unfair or inconsistent enforcement. Adequate notification by all means possible (not just in the message-of-the-day) contributed significantly to the success of the project. Finally, the receptive attitude of the user community to the goal of securing and more adequately protecting their intellectual property from the risks of today's Internet cannot be underestimated.

Encouragement to discontinue the use of plaintext password applications continues at SDSC and at other sites around the world. Additional work continues to provide and implement easy to use, secure replacements for services, such as ftp. Secure Shell version 2 may make this possible as it becomes more pervasive and is ported to more computer systems. As implementation of non-plaintext password solutions become standard, tracking intrusions will become more complex as backdoors to unraveling the encryption of sessions must exist and be available for systems administrator monitoring and auditing of suspected intrusions. But that is the subject of a whole other article.

Won't you contribute to the systems administration Holy Grail for the New Millennium and eliminate plaintext passwords from your network? It's never too late!

Acknowledgements

All brand names and product names are trademarks or registered trademarks of their respective holders.

References

1. Robert H'obbes' Zakon, Hobbes' Internet Timeline, http://www.isoc.org/guest/zakon/Internet/History/HIT.html, 15 Aug 1999.

2. Public Broadcasting Service, PBS Life on the Internet Timeline, http://www.pbs.org/internet/timeline/index.html, 11 Sep 1999.

3. B.M. Leiner, V.G. Cerf, D.D. Clark, R.E. Kahn, L. Kleinrock, D.C. Lynch, J. Postel, L.G. Roberts, S. Wolff, A Brief History of the Internet”, http://www.isoc.org/internet/history/brief.html, 20 Feb 98.

4. Garfinkel, Simson and Gene Spafford. 1996. Practical UNIX and Internet Security, 2nd Edition, O'Reilly & Associates, Inc.

About the Author

Victor Hazlewood is a UNIX administrator and manager of the HPC Systems group at the San Diego Supercomputer Center. Victor has worked with UNIX accounting for more than 10 years on a variety of platforms for resource reporting and auditing purposes. He plans to work closely with UNIX vendors and standards organizations to improve the capabilities and robustness of UNIX accounting.