Introduction to IBM's Network Dispatcher
The Internet has brought us many new technologies as well as new
requirements for the Web-based application. One urgent need is for
high availability and rapid scalabilty of the infrastructure supporting
our Web-based application. To meet this need, I am going to discuss
"client spraying". In a basic setup, an HTTP client will
connect to one HTTP server. However, if this server is down or is
busy, there is not a backup server for the client to connect to.
Although there are other ways to correct this problem, the method
used by IBM's Network Dispatcher is called "client spraying".
The idea behind client spraying is to create a pool of HTTP servers
and allow a client to connect to any one of the available servers.
Because the Network Dispatcher only allows connections to available
servers, high availability can be achieved. By adding more servers
to the pool and then using advisors (which I will address later),
the Network Dispatcher can load balance among the available servers.
The ease of adding servers combined with the load balancing provides
a powerful architecture to scale our application's server resources.
My goal in this article is to introduce the technology and then,
via an example, elaborate on how the Network Dispatcher works. I
will start with a discussion of how load balancing and high availability
of the servers is achieved using Network Dispatcher's advisors
and metrics. I will then discuss how high availability is built
into the Network Dispatcher, so that the Network Dispatcher is not
a single point of failure.
I'll begin with a simple example of one machine running Network
Dispatcher (ND1) and two servers (HTTP1 and HTTP2) running our application.
See Figure 1. Our Network Dispatcher cluster address is 188.8.131.52
(www.mike.com), and we are spraying port 80. From the client's
perspective, the client makes a request to a specific IP address
and port and gets a response for a server, in this example HTTP2.
There are two critical concepts here: one is that the client does
not know that Network Dispatcher is involved, and the second is
that the response from the server (in our example, HTTP 2) is sent
directly back to the client and bypasses ND1. The second point is
subtle, but important. From a performance standpoint, we do not
want all the traffic to have to go from the server back through
ND1; and if we consider the typical HTTP-based application flow,
the client will send a small amount of data to ND1 (e.g., a URL
and page request). The HTTP server can send large amounts of data
back to the client (e.g., to send an HTML page with many pictures).
Network Dispatcher Advisors
The next time the client makes a request, it may get connected
either to HTTP1 or HTTP2, depending upon what ND1 chooses. To determine
the availability of a server, ND1 will be running what are called
Network Dispatcher Advisors. These advisors are Java programs that
run on ND1. Advisors that come with the product include: ping,
socket, SSL, telnet, ftp, NNTP, SMTP, POP3,
IMAP, and HTTP). In this example, I will use an HTTP advisor on
ND1, which sends an HTTP "Get request" to each server,
and then time how long each server takes to get a response back
to ND1. The server with the shortest response time is marked the
best, while the next shortest response time is marked second, and
so on until all the servers have been ordered. In this example,
only two servers are involved, so either HTTP1 or HTTP2 will be
marked as the "best."
If a server does not respond in specific period of time (e.g.,
if the server is down), then the HTTP advisor marks the server down
and ND1 will not dispatch any more clients to that node. If the
advisor code determines that the HTTP Get response was unsatisfactory
(e.g., if "Error 404" occurred in response to the HTTP
Get), then the HTTP advisor could mark the server down as well.
Another key feature of Network Dispatcher is that it handles server
failure by routing around the failed server, and we have a very
powerful tool to use to decide when a server is actually down. The
default HTTP Advisor in ND will only mark a server down if the server
response times out, but you can easily create your own advisor.
The appendix of the Network Dispatcher User's Guide contains
the source code for an HTTP Advisor, and you can easily modify the
advisor to look for the actual response from the HTTP Get.
This approach to determining whether a server is up or down has
been described as the "client's eye view of the application",
because we're looking for a response from the server that indicates
that the application is available for useful work from the client's
perspective. The client may have to traverse many layers to get
to usable data. For example, the client may have to connect to the
HTTP server first and then on to the application server, which will
then make a call to a database. If any of these layers has a problem
on the server being tested, then the advisor should mark the server
as down. We then have the ability to dispatch around a "bad"
server with either the supplied advisor or our own custom "client's
Which Is the Best?
I will now discuss how Network Dispatcher selects the "best"
server for the client to connect to. If we can select the best server
on a regular basis, we can then "load balance" across
our servers. There are actually four components, I will call them
metrics, used to determine the "best" server. These metrics
1. The number of active connections to the server
2. The number of connections sent to the server recently
3. The advisor's response
4. The Interactive Session Support (ISS) agent input
These four metrics can be weighted in any fashion. For example,
30%, 30%, 30%, and 10 % would indicate that ISS input is not as
heavily weighted as the other three metrics. The first metric is
simply a count of the connections, while the second metric is a
count of the number of connections during the last window. The window
is a value that can be changed, but changing the window length is
generally not required. The third metric was discussed above, while
the fourth metric is a new topic. ISS agents provide statistics
based on system load as derived from CPU load or free memory, or
you can write a custom value. Typically, the best weighting of the
metrics is 30% or greater for active connections and 30% for recent
connections, with the remaining 40% falling on ISS and advisor input.
Network Dispatcher is composed of two main programs: the executor
and the ndserver. The executor is responsible for reading the TCP
header from the client, doing a lookup in a table to find the correct
server to spray to, and then dispatching the client packet to a
server. Because every client packet is sent to the executor, the
executor is a key part in determining the Network Dispatcher's
performance. The "in memory connection" table in combination
with the executor running in kernel space allows the Network Dispatcher
to be very efficient in "spraying" a client request. With
the ndserver handling all the other activities (particularly the
important task of updating the "in-memory connection"
table), the executor is free to handle only the few tasks mentioned
The last issue involves what happens with Network Dispatcher software
crashes. The answer lies in using Network Dispatcher's High
Availability option, which comes free with the product. With the
HA option, you run Network Dispatcher code on two separate machines.
One machine acts as the primary (ND1-primary) and the other machine
acts as a backup (ND1-backup). See Figure 2. The cluster status
is communicated between the machines via heartbeats. Then, when
the primary machine dies, the backup Network Dispatcher takes over.
One of the keys to the success of the failover is the passing
of the "in-memory" connection table from the primary Network
Dispatcher to the backup Network Dispatcher. The table is actually
passed via the heartbeats. With the "in-memory" connection
information on the backup node, items like the stickiness setting
are maintained after failover. Mutual high availability is supported
with the latest version of Network Dispatcher (version 3). Mutual
high availability allows you to have one ND as the primary ND for
one cluster address and as the backup ND for a second cluster address.
Therefore, you utilize both machines for dispatching. By sending
out ping-like requests to a third machine, typically a high-profile
router, the network dispatcher can also detect a network card failure.
Thus, in the event that a network adapter fails on the primary (ND1-primary),
the "reach machine" will not be accessible, and a failover
to the backup network dispatcher (ND1-backup) will occur.
Network Dispatcher also supports "co-location", which
allows you to run your application on the same machine as the Network
Dispatcher. Thus, in the previous example, we could run the Network
Dispatcher (ND1) on the server node HTTP1. If we run the backup
Network Dispatcher on HTTP2, we have reduced the setup to only two
nodes. However, we have a highly available application, because
any one node could crash and we would survive. We also have load
balanced our client connections and, if we need more servers, we
simply add them to our Network Dispatcher cluster.
I have provided a brief introduction into the features of the
Network Dispatcher. In addition to the key points of high availability,
load balancing, and scaling, remember that the Network Dispatcher
will dispatch any port, not just port 80 (the example port). With
this in mind, many applications can use the Network Dispatcher.
For example, Oracle's financial application (Oracle 10 or 11),
routers, firewalls, and so forth. Some barriers may come to mind
as you think through the potential uses for Network Dispatcher,
but I have only discussed a few of the features here. In the future,
I may address additional features such as content-based routing,
wide-area dispatching, dynamic DNS, IP-rules, and Web traffic express.
Network Dispatcher can run on Windows NT and 2000, AIX, Solaris,
and Linux. Good luck, and start the sprayers.
Mike Wilcox has worked with AIX systems and TCP networking
for the past 11 years. The majority of this time has been as a consultant
to fortune 500 companies, building the infrastructure for applications
ranging from ERP applications to "Web browser-enabled applications".
Mike is a senior member of IEEE and has been certified in HACMP,
SP, TSM on AIX (a.k.a. ADSM), and multiple AIX certifications. His
most recent project was to build a highly available e-business infrastructure
composed of multiple backend platforms using WebSphere Application
Server, WebSphere Performance Pack, IBM's content manager,
MQ, DB2, and AIX. Mike can be contacted at: firstname.lastname@example.org.