Administering Linux IPSec Virtual Private Networks

Duncan Napier

In my article "Introducing FreeS/WAN and IPSec" in the November 2000 issue of Sys Admin magazine, I discussed the basics of setting up IPSec for Linux using the FreeS/WAN package (http://www.samag.com/documents/s=1159/sam0011i/). This article will discuss some of the more advanced features of FreeS/WAN that you can leverage to implement flexible and reliable IPSec VPNs. The ultimate source of information on FreeS/WAN is the official FreeS/WAN Web site (http://www.freeswan.org). The Web site has links to virtually all the tools and information that you will need to implement IPSec on Linux.

An Overview of IPSec

IPSec is an extension to the Internet Protocol (IP) that provides not just encryption but also authentication at the transport layer (layer 3 of the OSI Reference Model). The next generation of IP, IP version 6 (IPv6), supports IPSec natively, since IPSec is a requirement of the IETF's specification for IPv6.

IPSec is a collection of protocols. Three protocols are used to handle encapsulation, encryption, and authentication -- the AH (Authentication Header), the ESP (Encapsulating Security Payload), and the IKE (Internet Key Exchange). IPSec is typically transparent to end users. Applications do not need to be rewritten nor do users need to be retrained to use IPSec-based networks. End users need not even be aware that they are using IPSec to tunnel data through an insecure network.

The AH and ESP handle encryption and authentication. The AH is added after the IP header but before the data (payload); see Figure 1. The AH carries authentication information, typically an MD5 (Message Digest Algorithm) or SHA (Secure Hash Algorithm) generated key. The AH is purely for authentication and used to verify that the senders are who they say they are. The AH does not perform encryption.

ESP provides for one or both of encryption and authentication. It may be used with or without AH. While it is possible to set up tunnels with only one of authentication or encryption deployed, this method leaves communications open to numerous forms of attack. Encryption is carried out using a block cipher (a symmetric or shared-key cipher operating on fixed-size blocks of plaintext), with 3DES being commonly used. Figure 2 illustrates the encapsulation process. RFCs for the deployment of other encryption algorithms (e.g., IDEA, Blowfish, and RC4) have been published and are actively supported by vendors. Encryption keys are shared using IKE.

IKE negotiates the connection parameters, including initialization, handling, and renewal of encryption keys. Authentication is carried out using privately shared secrets (e.g., a secret phrase) or RSA cryptographic keys that guarantee the identities of both parties. IKE is based on the Diffie-Hellman method for exchanging authentication tokens. Bulk encryption algorithms, such as triple DES or Data Encryption Standard, are used to encrypt data. Hash algorithms such as MD5 and SHA provide authentication of each packet. Connections are "re-keyed" (i.e., a new authentication key is negotiated) at frequent time intervals for added security.

IPSec inserts header and encrypted payload data into an existing IP (often non-IPSec) packet. This allows IPSec data to traverse any IP network. A primary reason for the widespread adoption of IPSec (in contrast to IPv6) is that all IP networks (including IP version 4) can pass IPSec traffic with no modification to the underlying network. IPSec has been implemented both in hardware and software. All major vendors of network hardware and software applications support IPSec networking. Virtually all networked operating systems now support IPSec. IPSec also enjoys considerable support in the Open Source community. Although virtually all IPSec implementations are compliant with the existing RFCs, they may not necessarily interoperate. IPSec implementations from different vendors should be tested to ensure that they are fully compatible.

FreeS/WAN -- An Open Source IPSec Implementation for Linux

FreeS/WAN is composed of two distinct components. The first is KLIPS (KerneL IP Security), a collection of modifications to the standard Linux kernel. For information on compiling KLIPs into a standard Linux kernel, see my previous article or the FreeS/WAN documentation. The second component is the standalone Pluto daemon, which is also installed and configured in the FreeS/WAN install process. Pluto handles IKE protocol authentication requests and interacts with the KLIPS components of the kernel, which handle encapsulation and encryption.

FreeS/WAN IPSec is configured through the ipsec.conf file. Once FreeS/WAN is installed on the system, virtually all configuration changes involve editing the ipsec.conf configuration file. Another file, ipsec.secrets, holds authenticators (i.e., cryptographic key sets, shared secrets).

The ipsec application (not to be confused with the IPSec protocol -- see the manpage for ipsec) is a set of utilities that control and invoke the IPSec implementation. The ipsec application also contains, among other utilities, a troubleshooting aid, utilities to bring IPSec tunnels up and down, as well as querying the status of each IPSec tunnel. All we need to start building VPNs is to edit the ipsec.conf and ipsec.secrets files and then invoke IPSec through the ipsec application.

Introduction to ipsec.secrets and ipsec.conf Files

The ipsec.conf file contains most of the configuration and control information for FreeS/WAN and is located by default in the /etc directory. The man page for ipsec.conf gives full details.

The ipsec.conf file is a text file and comprises one or more sections. Anything following a "#" is a comment. A section typically begins on an un-indented new line with two space-separated strings. For example, in Listing 1(a), the section starts with the line:

config setup

Lines within a section are whitespace-indented from the left-hand margin and of the form:

klipsdebug=none

In all ipsec.conf files, the first section "config setup" contains a section that deals specifically with FreeS/WAN settings, that is, interfaces and connections to negotiate when IPSec is enabled. The following line configures the default gateway on the machine as the interface for all the VPN tunnels:

interfaces=%defaultroute

klipsdebug=none
plutodebug=none

plutoload=%search
plutostart=%search

The section:

conn %default

keyingtries=0

The default authentication method for FreeS/WAN is a publicly shared key (PSK) set with:

authby=secret

In Listing 1(a), all sections after the "conn %default" section define a series of tunnels that I will explore in more detail. Connection specifications are written in terms of "left" and "right" hosts, rather than in terms of local and remote. Which participant is considered left or right is arbitrary; FreeS/WAN determines this based on internal information (e.g., the local IP address). This permits use of identical connection specifications on both systems.

IPSec can be used to support secure and transparent host-to-host, subnet-to-subnet, or subnet-to-host tunnels. FreeS/WAN's power and flexibility can be demonstrated with some real-world VPN scenarios.

Example Configurations

A Transport Mode (Host-to-Host) IPSec VPN -- Listing 1(b)

IPSec has two modes -- transport mode and tunnel mode. Transport mode provides source-to-destination protection of the datagrams only. It authenticates, encapsulates, and encrypts the IP data only, but leaves the transport headers (IP information) intact (see Figures 1(a) and 2(b)). As a result, transport mode is typically used to build authenticated host-to-host encrypted tunnels. In Listing 1(b), the machine IP 1.2.3.4 peers through an encrypted tunnel with another machine, IP 5.6.7.8. Figure 3 illustrates this scenario. The line "auto=start" signals that the tunnel connection be brought up when invoked from IPSec.

A Tunnel Mode (Subnet-to-Subnet) IPSec VPN -- Listing 1(c)

In tunnel mode, a new IP header is created and encapsulates the entire original IP datagram, effectively hiding information about the original sender (Figures 1 and 2(c)). Encapsulation enables complete packets from one network to "tunnel" through other networks. One application is the use of IPSec gateways that connect trusted private networks through an untrusted public one (e.g., the Internet). Figure 4 shows such a configuration.

Tunnel Mode IPSec VPN with RSA Authentication -- Listing 1(d) and (e)

The RSA algorithm patent expired in September 2000, and RSA-based authentication is now freely available for FreeS/WAN. IPSec uses RSA keys for authentication, not encryption. Symmetric bulk ciphers are used at the data encryption stage, and keys for these ciphers are distributed using the RSA keys. RSA-based authentication allows network administrators to distribute public RSA keys to all who wish to communicate via IPSec. Possession or theft of the public key does not endanger security because only the bearer of the secret private key (presumably the legitimate owner of the key pair) can complete the authentication process. In Listing 1(d), the rightid and leftid lines ensure that authentication is restricted to specific IP addresses.

The ipsec utility run with rsasigkey option generates a public-private key pair in the required format for the ipsec.secets file (see the manpage for ipsec_rsasigkey for further information). Public keys generated by the rsasigkey can then be cut-and-pasted into the ipsec.conf and assigned to leftrsasigkey and rightrsasigkey as in Listings 1(d)-(f). Listing 2(b) shows the ipsec.secrets entry accompanying Listing 1(d) with the RSA key output from generated rsasigkey.

FreeS/WAN can also implement X.509 certificates (commonly used for the Secure Sockets Layer -- SSL), which are created and distributed by a trusted third party, called the certification Authority (CA). The X.509 implementation was developed independently from the FreeS/WAN project and is freely available from by StrongSec.com (see http://www.strongsec.com/freeswan/). Once the certificate has been stored locally on the FreeS/WAN host in a specified location, entries leftrsasigkey=%cert and leftrsasigkey=%cert specify use of the certificates. Third-party tools for authentication based on PGPNet are also available (http://www.zengl.net/freeswan/download/).

The PSK method uses a single, mutually shared secret as a primary authenticator and the IP addresses as a secondary authenticator. RSA authentication only requires appropriate knowledge of public and private keys of the communicating parties. Therefore, for FreeS/WAN installations that use RSA key pairs, fixed IP addresses no longer serve as identifiers. As a result, RSA authentication can be used to authenticate users having dynamically assigned IP addresses. Listing 1(e) shows such a setup where the local IP address of a gateway with a dynamically assigned IP (defined by the line left=%defaultroute) is loaded automatically.

The rightid and leftid lines are used as an extra verification step. These lines can either contain an IP address (Listing 1(d)), a fully qualified domain name (at the risk of bringing down the tunnel should DNS lookups fail), and a non-DNS resolved name preceded by an "@" (as in Listing 1(f)). The last option is basically an arbitrary pair of shared keys. However, unlike the PSK authentication case, security is not endangered for RSA-authenticated installations by theft of this information alone.

Road Warrior IPSec VPN Configuration with RSA Authentication -- Listing 1(f)

This setup is virtually the same as that for Listing 1(e) except that, instead of a subnet-to-subnet connection, we have implemented a subnet-to-host VPN (Figure 5). Thus, there is only one subnet defined.

Note the line auto=add, as opposed to auto=start. Instead of bringing up the tunnel, the tunnel is merely "added" onto the static-IP side in a listening state. The tunnel lies dormant and "wakes up" when the host having auto=add set is contacted by the dynamic IP side (the laptop in Figure 5), which has its conn section set to auto=start.

FreeS/WAN and Firewalls

In the example listings following 1(a), one or both of the FreeS/WAN gateways is also a router. In many scenarios, these gateways would also be firewalls. Under many circumstances, these firewalls would do Network Address Translation (NAT).

Running a FreeS/WAN IPSec tunnel through any device with NAT (Network Address Translation) or IP masquerading will have unpredictable results and is not recommended. If the IPSec gateway is not itself the platform that performs NAT/masquerading (i.e., the tunnel terminates at the NAT gateway), then it is strongly recommended that the IPSec gateway lie in front of the NAT/masquerading platform. Essentially, IPSec must verify the packets before they are rewritten for NAT. The reason for this is that when NAT rewrites the IP headers of packets, the alteration causes the packets to fail IPSec's integrity checks (Figures 1 and 2). Some attempts are being made to overcome this problem, including at least one IETF Internet Draft (http://search.ietf.org/internet- \ drafts/draft-aboba-nat-ipsec-04.txt) and RFC 2709.

FreeS/WAN runs on Linux firewalls using the standard distribution of ipchains. FreeS/WAN has also been tested with the Linux 2.4 Kernel's iptables firewalling tools. Iptables has numerous new features that add many enhancements to the firewalling features of ipchains.

For Pluto's automatic keying to work, port 500 must be accessible to UDP packets on each of the IPSec gateway hosts. ESP uses protocol 50; AH uses protocol 51 and, if either ESP and AH are used, the firewall rules must be set up accordingly. Sample firewall scripts containing filtering rules for FreeS/WAN are available from the FreeS/WAN Web site. Many canned firewalling scripts, such as those from David Ranch's TrinityOS or the Seattle Firewall Project (SeaWall), contain specific options for configuring firewalls that are IPSec gateways. Generic IPSec settings work for FreeS/WAN.

You can further secure IPSec VPNs in addition to the authentication features within FreeS/WAN by restricting connections at the IP level through firewall IP filtering. If, for example, you need to modify the firewall configuration as the status of a connection changes, you can use FreeS/WAN's updown feature in the ipsec.conf file to run modifications to the ipchains script and auto-configure routing tables.

Opportunistic Encryption

For large systems of interconnected gateways and hosts, keeping track of authentication keys quickly becomes an unmanageable task. Each VPN tunnel has to be laboriously preconfigured with appropriate key and peering information in conn sections at both ends of each tunnel. Consider a complex full-meshed VPN topology with many tunnels, where every node connects once to every other node. The task of manually adding and accounting for all possible peers on each machine increases dramatically as we increase the number of remote gateways. The present situation of connection sections scales poorly in the absence of a key management infrastructure.

The latest version of FreeS/WAN (Version 1.9.1 at time of writing) attempts to alleviate some of these issues with opportunistic encryption. Opportunistic encryption does not require prearranged connection, nor does it require the administrators of IPSec gateways to communicate connection section information to each other. In essence, outgoing packets are examined at the IPSec gateway to see whether an encrypted connection to the destination is allowed. If an encrypted connection is permitted, the packet is encrypted. The conn section may look like Listing 1(g), which only specifies the current subnet.

The destinations of all outgoing packets are checked for a public key through a DNS record that publishes public keys. In conventional implementations of DNS using BIND, the public key is merely a string placed in the domains TXT records for that host. For opportunistic encryption to work, control of the DNS reverse maps is required.

Because connections to DNS servers are based only on very loose trusts and generally not authenticated, this system is not considered completely secure. The IETF's SDNS security working group is currently drafting the full specification of a secure DNS system. Among the enhancements is a secure KEY resource record that addresses the issue of storing cryptographic public keys. Refer to documentation for FreeS/WAN 1.9.1 and later for this interesting and potentially useful enhancement.

Conclusion

The recent rise of ubiquitous access to networking and explosive growth of network capacity has pushed interconnectivity to unprecedented levels in recent years. Distributed computing, increasingly mobile users, and growth in the numbers and capabilities of mobile communication/networking devices are driving us toward a more networked future. I hope that this article has demonstrated that the FreeS/WAN Project has risen to meet the challenges of secure and scalable open standards for VPN networking.

Duncan Napier is an avid mountain biker. When not out riding trails, he may be found running Napier Systems Research, an IT networking consultancy based in North Vancouver, B.C., Canada. He can be contacted at: napier@computer.org.