of Service: Types and Effects
Peter H. Salus
Because of its very nature, it is difficult to destroy a packet-switching
network, much less the network of networks that is the Internet.
However, a number of events have disrupted Internet service --
some large, some limited in extent. This article is an attempt at
a typology and a spur to discussion.
We all know that in a packet-switching network, there are multiple
routes between any two points. As a consequence, such networks are
more robust when under attack than direct-line connections or circuit-switching
networks (like the telephone system).
But disruptive events have visible effects on the Internet. Over
the years, many of these events have been featured in the press.
But the press has never looked at the disruptions as "natural
phenomena." Nor have important distinctions been drawn.
I talk about disruptive events, because these things are quite
different from virus attacks or denial of service attacks. On the
one hand, virus attacks afflict individual PCs (sometimes hundreds
of thousands of them), but have little visible effect on the Internet.
On the other hand, denial-of-service attacks may put an individual
site out of commission, but only influence the Net because such
attacks add to the flow of bits in the Net as a whole.
Since 1990, Matrix.Net (aka Matrix Information and Directory Services)
has been surveying traffic and mapping the Internet and other networks.
We thus have a solid corpus of data to look at. Here are some instances.
The 1994 Northridge Earthquake
On 17 January 1994, at 4:31 AM PST (12:31 PM GMT), in the Northridge
suburb 20 miles northwest of Los Angeles, a magnitude 6.7 earthquake
struck. In 15 seconds it led to the deaths of 15 people and injuries
to more than 9,000. This was a major disaster. What effect did it
have on the Internet?
The Internet Weather Report (IWR, http://www.mids.org/weather/)
was already running back then. It ran scans at 2AM and 6AM PST,
thus taking before and after snapshots of the Internet. At that
time, we noted:
Comparing the two series of scans, effects of the earthquake can
be seen as far north as Lawrence Livermore Laboratories northeast
of San Francisco (which shows higher latencies immediately after
the earthquake) and as far south as San Diego (which shows more
traffic several hours later). Ensenada in Baja California, Mexico,
disappears just after the quake, but that is probably coincidence,
since the host we are pinging in Ensenada sometimes doesn't
Here are some Matrix.Net graphs of the IWR data from the Northridge
Earthquake event. You can see the initial earthquake event in both
packet loss and on the Internet at large. The effect is shortlived,
however, and by the next scan, most of it is gone. Apparently the
earthquake crashed quite a few computers briefly, but many of them
were soon back up (see Figures 1-3).
This quake did tens to hundreds of billions of dollars of property
damage, taking the San Fernando Valley off the Internet for more
than a day. Most of the affected areas were back on line completely
after a week. The Internet as a whole was not affected beyond the
immediate geographical vicinity: there was no overall effect on
packet loss; there was no visible difference in latency or packet
loss of the Internet overall.
The main reason that much of the local Internet was down was simple:
computers don't run without electricity. The only long-term
Internet infrastructure damage was one router that fell over. All
the others came back up when power was restored.
But let's move up a few years.
Hurricane Floyd threatened the Caribbean islands and the eastern
coast of the United States for the first two weeks of September
1999. It achieved Category 4 status before it made landfall. It
missed Hispaniola and Cuba completely, passed over the Bahamas,
missed Florida, and Georgia, grazed South Carolina, and went ashore
in North Carolina. By then it had tapered off to tropical storm
classification, still generating high winds and much rain, but instigating
far less damage than anticipated.
Overall Internet Effects
The expected overall effects on the Internet were negligible.
These graphs represent average Internet performance. The darker
line represents the Internet as a whole. The lighter one represents
the WWW, a subset of the Internet.
Looking at the Internet Average is informative. The Matrix.Net
Internet Average is a high-level summary of performance data measured
from hosts all around the world. It provides one baseline against
which more specialized Internet performance data might be compared,
serving a similar role as the Dow Jones Industrial Average does
in the financial world.
However, negligible isn't the same as indetectible. There
was, in fact, a surge in both latency and packet loss on Monday
13 September 1999, starting around 8AM EDT and continuing the entire
working day (see Figure 4).
The Internet "pinglist" shows a pointed latency spike
about noon EDT. The peak of that spike is at 186.4 ms, which is
about 16% higher than the usual 160 ms for a Monday afternoon. The
WWW pinglist shows a smaller increase, to about 169 ms over the
usual 158 ms of a Monday afternoon, or about 7%. There's a
smaller spike on both pinglists about 5PM Monday.
What we don't see is much effect on Wednesday or Thursday,
when Floyd actually made landfall. If anything, there may be a decrease
in latency on Wednesday.
Looking more closely at the underlying data, and looking only
at nodes in Florida as an example, we can see that there was a drop
in reachability at noon EDT Monday, and another at 5PM. Reachability
rachets back up by the same amounts at 7AM and 9AM Wednesday. That
looks as though a number of people turned off their machines as
they were being evacuated, and turned them back on when they returned.
What might have been expected would be brief spikes in ISP performance
either when Floyd passed over an area or later when batteries were
drained. Most ISPs these days have pretty good backup power, so
this would seem the most likely scenario. For big ISPs, the battery
would only be on for about five minutes, and then emergency generators
would take over. Then it's just a question of availability
of fuel for the generators. Smaller ISPs might have bigger problems,
as they have fewer resources for USPs and generators.
Or, if damage were sufficiently severe and direct to actually
take routers off the Net, we might see an outage of a day or more,
with a sharp beginning and a gradual end as ISPs dug out and replaced
broken equipment. This would be similar to the effects of the Northridge
Earthquake, except probably more severe because the earthquake effect
was caused more by power outages than by direct damage.
What we see instead is a lot of small latency and packet loss
events on Monday, before Floyd hit anything, and negligible effects
thereafter. The only big ISP that seems to have significant later
effects was Global-One.
Friday 17 September 1999
This hurricane's bark was worse than its bite, where the
Internet was concerned. Evacuation and fear of the hurricane had
much more effect on the Internet on Monday than the hurricane itself.
However, there was finally a brief but noticeable event on Friday
(see Figures 5 and 6).
ISPs these days are quite hardened to problems with electricity
and telephone service, and even flooding. Certain kinds of problems
still affect the Internet, however, such as hysteria, seen here;
cable cuts, which are likely the cause of the Friday event; and
configuration problems inside the ISPs themselves, which were not
observed in this particular week.
Earthquakes and hurricanes are genuinely natural events; but there
are other types of physical disruption.
Fiber Cut of 29 September 1999
About noon EDT on 29 September 1999 there was a massive fiber
cut in Ohio, which took more than four hours to fix. This infrastructure
outage was noticeable across the entire Internet, as illustrated
by the Matrix.Net Internet Average.
Not much damage is visible in latency, although interestingly
enough most of what is visible is in the curve for Top Level Domain
(TLD) Domain Name System (DNS) servers.
Packet loss clearly shows the event in all three curves (Figure
7). Latency shows the most dramatic effects (Figure 8).
We also examined the top 30 ISPs one by one, and found that only
a few of them (AboveNet, GTE Internet, and PSINet) were noticeably
affected. AboveNet got it worst, and took a day to completely recover.
All the other ISPs were essentially unaffected.
Outage of October 7, 1999
About 3AM EDT (8AM GMT) on October 7, 1999 there was a massive
Internet outage, bigger than the one caused by the fiber cut of
September 29, 1999. This infrastructure outage was noticeable for
the entire Internet. Packet loss shows the outage clearly. And reachability
shows it even more clearly (Figures 9-11).
Denial of Service attacks
On Monday and Tuesday, February 7 and 8, 2000, a large number
of major sites across the US were assaulted by "Denial of Service"
(DoS) attacks. These attacks were the result of millions of messages
flooding a particular host or gateway, overwhelming the resources
and backing up traffic in a domino fashion (Figure 12).
In the days since, Attorney General Janet Reno has said that the
FBI would track and punish the miscreants. The President has called
another meeting. All the "usual suspects"
were rounded up.
There have been security warnings, as well. In general, the newspapers
and the TV reporters appear clueless on the differences between
cracking a site and blocking a site.
"Since smurf attacks originate from a variety of sources
at unpredictable times, an Internet-wide system analysis is necessary,''
said John Quarterman, CEO of Matrix.Net, in Austin, TX. He continued,
"Just look at the 'Internet Average' for 8
February. It's completely clear that the entire Internet had
higher packet loss and far lower reachability for several hours.
It's like a shark took a bite out of the Net."
There have been other interesting events: the Victoria's
Secret lingerie show slowed down AOL, but had no real effect on
the Internet; the release of the Starr Report caused several spikes
in latency, at the release time and at about 5PM EDT and thereafter.
Presumably, folks wanted to see the report when it went up, and
then read parts of it when working hours were over.
Even more recently, on September 25, 2000, the ISP Applied Theory
showed the a wild fluctuation in reachability. This was caused by
a "router flap" at Sprint. Rather than the absolute drop
shown in the cases of DoS attacks, or the massive outages shown
by fiber cuts, the up-down seesaw here reflects the router trying
to come up and succumbing again. When it was swapped out, everything
returned to normal.
The important thing is to recognize just how limited the effects
of all of these events have been, as well as how different from
each other natural disasters, fiber cuts, blown routers, and denial
of service attacks are. We are currently datamining our records
to ascertain exactly what the early warning signs are.
Knowledge is power: it may be that with early recognition, the
ISPs will be able to route around disruptions and thus ameliorate
Peter H. Salus, the author of A Quarter Century of UNIX
and Casting the Net, is Chief Knowledge Officer at Matrix.Net.
He can be reached at: email@example.com.