Using Linux as a Router
Linux can be a terrific "poor man's" router.
It takes a little more
startup configuration than a typical hardware router,
but once it is
going, very little will stop it, at least in our experience.
using the router setup detailed below in production
for six months now
(a few months more worth of pilot-testing), and the
only faults we've
encountered were either service-provider related or
due to water-soaked
cables -- unbelievable, but true, and a really long
story I won't get
Since I'm about to describe how to roll your own router,
it will help if
you understand how most TCP/IP routing works. Any router,
Proteon, or Linux, is based upon the premise that packets
need to be
forwarded. Why? Because, presumably, the packets that
come in on one
interface are not local to the other interfaces in the
Therefore, the router in question must be able to take
a packet, look at
its destination, and forward it to the appropriate interface.
How does the router map network destinations to interfaces?
as every router needs to be able to forward packets,
so too it needs the
ability to consult, build, and update a lookup table,
called a routing
table, that maps destination networks to interfaces.
If a packet comes in for network X, the routing table
is consulted and
the packet is dumped to the appropriate interface --
a local node that is
either another gateway or the packet's final destination.
If network X
does not exist in the routing table, the packet is dumped
to the default
destination, also denoted by IP address 0.0.0.0. If
you have not
specified a default destination, the packet is dropped,
and an ICMP
"Destination Unreachable" message is sent
back to the originating
How does the router maintain the routing table? In two
and dynamically. Static routes are added by you, in
a known, and
sometimes tedious, fashion. Typically, default routes
are static routes.
Dynamic routes are "learned" routes, learned
by some sort of routing
protocol. For our installation, we were only interested
in one routing
protocol, RIP (Routing Information Protocol). It is
simple, and widely
When a RIP router sees a RIP packet (which is a broadcast
packet -- good
to know for troubleshooting purposes), it updates the
routing table so
that future packets will follow the learned route to
By the same token, the router "advertises"
when it has a destination
that other machines should know about. Obviously, routers
that need to
talk to each other need to speak the same protocol,
or all is confusion!
Although RIP is okay for small networks with non-variable
you will want to check out OSPF and BGP if you are about
to embark on a
complex networking plan. First of all, RIP is a broadcast
which means your network is subject to broadcast storms.
RIPgives no "weight" to any particular route
-- one route is considered
as good as another. For example, RIP would consider
your 9.6KBps backup
link to a site to be just as good as a your primary
Our needs were extremely simple: a routing system that
would connect one
remote site to a shared Internet router that spoke RIP
on our end. The
site had one LAN to start with, but would probably be
adding a few more.
We decided to use Linux because it was inexpensive enough
for a pilot
project: it would cost us two PCs plus labor. Also,
we would be able to
have one machine handle routing services, mail services,
services, and name services. This saved us the startup
costs of a
dedicated router plus a UNIX or dedicated DOS box to
manage DNS and POP.
Our goal, and current implementation, is shown in Figure
We built the first machine out of a surplus (and highly
motherboard with 8Mb of RAM and a 120Mb IDE hard drive.
We purchased and
installed the Slackware distribution of Linux off a
borrowed NEC CD-ROM,
making sure to install kernel source code, the PPP daemon,
various user-level services we wanted to run.
The first thing to do was to modify the kernel to act
as a packet
forwarder. Slackware came with many excellent precompiled
IP forwarding was not an option. Apparently, some random
that IP forwarding is not to be turned on by default.
makes sense: why spend the processor cycles dealing
with it if you don't
need to? So, compiling a kernel ourselves was our only
it's not the only option you have: if you want a precompiled
kernel with IP forwarding, ethernet, and token-ring
support, get it from
At first, the notion of compiling a kernel seemed daunting.
we'd only re-linked OEM UNIX kernels. It turned out
to be much easier
than expected; the processor and hard drive were the
only things that
had to work hard.
The directory /usr/src/linux is, by convention, a symbolic
link to the
current production version of Linux running on your
machine. Finding it
is easy: The README file contained therein is fairly
short and to the
point, and describes how to configure the kernel and
To make a kernel, we typed:
# cd /usr/src/linux
# make config
* Networking options
TCP/IP networking (CONFIG_INET) [y]
IP forwarding/gatewaying (CONFIG_IP_FORWARD) [n] y
# make dep ; make clean
# make zImage
Then, before installing the kernel, in addition to backing
current kernel, we also made a boot disk, just in case.
After putting a
floppy in the A: drive (for you DOS types), we typed:
# dd if=/vmlinuz of=/dev/fd0
# cp /vmlinuz /vmlinuz.safe
# mv /usr/src/linux/arch/i386/boot/zImage /vmlinuz
# sync ; reboot
A Few Caveats
If you have played with LILO (the LInux LOader), /vmlinuz
may very well
not be your system kernel. Check /etc/lilo.conf if you're
Make sure you use networking tools that match your
kernel version -- all sorts of strange things can happen
with a mismatch!
Specifically, watch out for arp, route, ifconfig, and
gated. A good rule
of thumb is to stick to the versions that come on the
distribution. If you're getting your files from the
Internet, pay close
attention to READMEs.
Stick to the defaults when configuring by hitting ENTER
at the prompts.
Be sure you don't configure a device that you don't
have -- if you do,
you risk lockups and erratic system behavior.
In our case, we also had to scour the earth for a Token
Ring driver for
Linux, and compile it into our kernel, but that's another
long story. If
you are looking for Token Ring support, and are willing
to live without
busmastering and TI chipset support, check out Peter
Wonderful Token Ring driver:
See Terry Dawson's comprehensive NET2-HOWTO for current
available networking technology support. Look in:
Token Ring may well be included in many distributions
by the time you
read this. I will mention, however, that we had problems
earlier than 1.2.
Once the kernel was recompiled and ready to forward
packets according to
its routing table, we needed to give it routing "smarts,"
so we started
up routed, the ancient routing daemon. This was a mistake;
obsolete and flakey, and should not be used here. Instead,
use gated. It
will speak every routing protocol you're ever likely
to need, and it is
easy to configure for RIP (see Figure 2). If you need
to use something
other than RIP, say, OSPF, you can rest easy knowing
that the gated
distribution has quite a few sample configuration files
for you to copy.
Any router needs more than one interface to be useful,
and our Linux
router was no exception. We chose to use PPP (Point-to-Point
in conjunction with ISDN terminal adaptors, because
ISDN service is
extremely inexpensive here. (ISDN was not supported
with most Linux
distributions at the time, but it may be by the time
you read this.)
Also, pppd, the point-to-point daemon, supports "proxy
arp," which lets
you avoid wasting a network on the point-to-point link.
arp makes your remote machine's point-to-point interface
appear as if it
is on your local network, in our case, the 167.195.160
The pppd support for a Token-Ring frame type was not
complete at the
time of this writing. The version of pppd that we used
proxy-arp, but with a frame type of Ethernet, not Token
Ring. The files
/etc/ppp/ip-down and /etc/ppp/ip-up (Figure 3) were
necessary in order
to kludge the proxy arp on the local end. You don't
have to do this if
you're using Ethernet, or if this has been fixed in
your version of the
We chose terminal adaptors that would turn a synchronous
into an asynchronous 57.6Kbps channel, so that we could
use a standard
16550 UART serial port, /dev/ttyS0 on both machines.
Note that you must
use a 16550 or other buffered or intelligent serial
port to run at this
speed (or 128K synch transformed into 115K asynch).
If you do not, you
will experience data overruns and thus network errors.
Also note that when using Linux, if you want to use
serial speeds above
38,400Bps, you must use the setserial program to make
transform into either 57,600Bps (spd_hi) or 115,000Bps
Figure 4 for details).
Testing and Troubleshooting
Once the first system was built, we tested the PPP connection
connecting a DOS-based PC to it, with FTP Software's
PC/TCP PPP stack.
Everything looked fine; a ping from the PC to the Linux
which meant that PPP was happening. Then we pinged the
successfully, which meant that the IP forwarding also
At this point, it was time to "clone" the
first machine to create the
remote router. A quick modification of the Slackware
root diskette was
in order. We deleted tar and replaced it with cpio,
so that we could
restore a backup of the entire filesystem, including
device files. We
backed up the first system over the network to the unixhost:
cd / ; find . -print | cpio -ocv | rsh unixhost -l username
Then, we booted the second machine, a Dell 486/33SL
with 8Mb and a 250Mb
hard drive. We started with the Slackware boot diskette,
the modified root diskette when prompted for the root
We used the convenient Slackware setup program to format
and setup the
root and swap partitions identically to the first machine.
we made the root partition and the swap partition the
letter as the first machine, /dev/hda3 and /dev/hda2
ensured that the root filesystem and swap partition
would be the
"cloned" kernel expected them to be. Then
we exited setup, started the
network, mounted the new root filesystem, and restored
from the unixhost tape drive.
#ifconfig tr0 220.127.116.11 netmask 255.255.255.0 broadcast 18.104.22.168
#route add -net 167.195.160 netmask 255.255.255.0
#mount /dev/hda3 /mnt
#cd /mnt ; rsh unixhost -l username "dd if=/dev/rmt1" | cpio -icv
We shut down the clone, removed it from the LAN, and
rebooted. We then
changed the IP number and network on the clone machine
as shown in
Figure 1 (full startup files for both machines are detailed
in Figure 4).
We set the clone up three feet away from the first machine,
directly connected null-modem cable, in order to test
the PPP without
the complications of ISDN or geographic distance. We
connected a hub and
a PC to the cloned machine to simulate conditions at
the remote site.
This was where routed caused everything to fall apart.
As soon as we
replaced routed with gated, all was well.
We moved the clone to the remote site, connected the
adaptors, auto-dialed between them, then plugged them
into both of our
new Linux routers. Of course, one of the ISDN modems
turned out to be
configured for the wrong type of channel. As soon as
that was corrected,
everything worked perfectly.
Since Linux is extremely well documented, and supports
a wide variety of
protocols and networking technologies (including Novell's
hard to see any reason to stop using it. During the
project, we were
consistently amazed at how many well-written and detailed
there were. Some of them are truly better than their
One day we may have to replace the Linux routers with
routers -- but it's hard to say when and if this will
happen. If the
system doesn't break, we won't fix it.
A measurement tool to tell us what kind of routing volume
and with what kind of latency would be nice. Right now,
our only way of
measuring traffic is to check ifconfig output and do
an average per
sample period. We could just wait until the users complain
network's too slow, but this is probably a method to
be avoided if at
all possible. We're looking into coding a program to
automate the router
We've managed to save ourselves the startup costs associated
hardware routers. And, due to the nonproprietary nature
of Linux, we've
left ourselves "open" to a very flexible future.
About the Author
Jonathan Feldman works with UNIX and NetWare at the
Government in Savannah, Georgia. He likes to keep things
simple so that
even he can understand them. When he is not chasing
around with his
18-month-old son, he likes to write, grow roses with
his lovely wife,
and play guitar with his bare feet. He is reachable
via email at