Load Balancing Cluster
A cluster is a group of computers that have been interconnected
to share processing. Clustering technology, which coordinates the
computers as they work in parallel on computing tasks, comes in
a variety of flavors. Clusters can be built from custom hardware
and software that implement a fast interconnect and some kind of
shared memory or cache. Clustering technology can also be completely
implemented in software, making use of the power and low prices
of today's PCs, inexpensive high-speed networking like Fast
Ethernet, and a robust open operating system like Linux. An example
of this kind of cluster is the Linux Beowulf system, which consists
of a server node connected to multiple client nodes via some type
of private high-speed network.
In this article, I describe the Java Application Server Pseudo-cluster
(JASPer), a simple cluster built with commodity hardware and free
software from the Apache Foundation. The master server runs the
Apache HTTP daemon and the cluster nodes are running the Apache
JServ Java application server. Using this combination of open software
and inexpensive equipment, my team has been able to reap the benefits
of clustering while avoiding some of the disadvantages.
Advantages and Disadvantages of Clusters
There are three primary benefits of clustering, depending on the
computing tasks to be shared by a cluster. First is the ability
to scale capacity. Additional computers can be easily added to a
properly constructed cluster to gain additional capacity for processing.
This works only until any shared resources in the cluster (the master
node and cluster interconnect) begin to become saturated. Second
is fault tolerance. Assuming that the cluster is smart enough to
stop trying to use any nodes that are no longer available, it will
continue to run as long as the master node and at least one client
node can communicate. Finally, higher performance is often achieved
with a cluster by distributing work across multiple servers, although
the shared resources in a cluster will again place an upper limit
on how far this benefit can extend.
There are, however, some problems with clusters, which have limited
their use. The difficulty of setting up and administering a cluster
has prevented some from taking advantage of them. IT shops without
the appropriate expertise find the administration of a cluster very
complex and, depending on the type of clustering technology used,
support services can be expensive or unavailable.
Another limitation is the kind of applications that can benefit
from a clustered platform. To take advantage of the performance
of a cluster, computing tasks must be partitioned in some way to
distribute parts of the tasks across multiple processors. In parallel
computing, this partitioning and distribution is usually performed
in the application program, inserted by intelligent optimizing compilers
or based on directives included by the programmer. For example,
on a Beowulf system, the programmer decides which parts of the program
can be run concurrently on different client nodes and uses a messaging
software API or remote shell commands to run those procedures on
Load Balancing Clusters
However, certain computing tasks are inherently concurrent and
can be partitioned on a cluster without special programming. A high-volume
transaction processing system, in which multiple transaction requests
are submitted concurrently, can be naturally partitioned on the
transactions. A common example of such a system is a Web server,
and a simple and popular cluster technology is Web redirection,
where a central switch (the master node) redirects HTTP requests
to one of a set of Web servers based on server load or geographic
proximity to the client (or both).
Load balancing software can be used in a similar way to create
effective clusters for Web applications. My team has done this with
Apache JServ, which provides load-balancing across a set of Java
servlet engines. Methods similar to those described here can be
used to cluster any Web application server that supports load balancing.
This technique can minimize the primary disadvantages seen with
other kinds of clusters. By using your existing application server
software, administration of the cluster is only incrementally more
complex than administering a single Web server. And, the transaction-oriented
nature of Web applications means that no special programming is
required to partition the computing tasks.
On the other hand, a significant limitation of this kind of cluster
is that it only runs applications written for the application server.
It is not a general purpose cluster and that is one reason we named
our implementation a "pseudo-cluster" (although the main
reason was that it sounded better than "Cluster" in the
In "Building a Web-Based Java Application Server with Apache JServ"
(Sys Admin, February 2000) my coauthor and I described how
to install, configure, and administer JServ as part of a distributed
application architecture. As described in that article, load balancing
is performed by the Apache HTTP server module mod_jserv.
When an initial request is received by mod_jserv on the master
node, it randomly selects a JServ cluster node from the defined
set of servers. The request is passed to that cluster node using
the Apache JServ Protocol (AJP), a network protocol for communicating
between the master and cluster nodes. mod_jserv adds a server
id to the JServ session id so that subsequent requests for that
session are recognized and routed to the same cluster node.
A weight can be added to each defined JServ server to have more
powerful cluster nodes process more of the requests. If a JServ
server goes down (i.e., fails to respond to mod_jserv), then
its requests will be routed to another server. In that case, a new
session needs to be started unless your application has implemented
some kind of persistent session (e.g., by saving session information
to a database accessible by all JServ servers). The previous Sys
Admin article describes the routing of requests in more detail.
There is also more information about JServ configuration and its
load balancing features on the Java Apache Project Web site (http://java.apache.org/).
In practice, we have found the JServ load balancing mechanism
to be quite effective. The load has been very closely balanced as
measured by the number of sessions on each server. During moderately
busy times, each JServ node in our cluster handles between 65 and
75 sessions. Actual load may not always correspond to the number
of sessions being handled, but in our portal application, it is
close because there is not a wide discrepancy in the processing
required to handle different kinds of requests.
The three parts of a load balancing cluster comprise the master
node, which performs the load balancing; the cluster nodes, which
process the transactions; and the cluster interconnect, which allows
the master and cluster nodes to communicate. The components used
for JASPer are illustrated in Figure 1.
The master node can be the weak link in a cluster since all transactions
must go through it. A master node failure will bring down the whole
cluster, so the platform must be very robust. To enhance availability
and performance, a Web redirecting switch could be used to route
HTTP requests to redundant master nodes, which are configured to
use the same set of cluster nodes in a many-to-many cluster configuration.
However, there is still a potential single point of failure. It
has just been moved from the master node to the (presumably more
robust) Web redirecting switch.
For our cluster, we used a Sun Enterprise 4000 with six 250-Mhz
processors and 3.5-GB RAM as the master node. This is more processing
power than needed to receive HTTP requests and route them to the
cluster nodes, but this server hosts several Web sites and an online
library catalog application besides being the JASPer master node.
This platform was selected for its reliability more than its performance.
Cluster nodes can be any combination of servers that run Apache
JServ, including Linux, Solaris, and Windows NT platforms. If a
combination of different hardware or software platforms is used,
you should make some guesses about their relative performance so
you can weight them in the load balancing set defined for mod_jserv.
Benchmarking of the individual nodes, as described below, can be
used to make better estimates of their relative capacity. For example,
if you have one new high-speed node that can handle three times
as many requests per second as an old slower node, you might ask
mod_jserv to route three quarters of the requests to the
faster node with these directives:
ApJServBalance setname FASTNODE 3
ApJServBalance setname SLOWNODE 1
Since JASPer is supporting our primary enterprise Web portal, we
were able to get funding to buy four new Pentium III 600 Mhz Linux
servers to serve as the JServ cluster nodes. Memory requirements
for the cluster nodes depend on the Java applications that run on
them. In our case, we use lightweight sessions, saving the minimum
information about each session that we need, but load a lot of configuration
information in each servlet at startup to avoid having to read the
configuration database and files for each request. 256 MB RAM appears
to be sufficient to avoid swapping. Memory requirements of the Java
Virtual Machine can be determined at runtime by a Java monitoring
agent class that is presented below.
We also purchased a dedicated 100-Mbs network switch with a Gigabit
Ethernet upload module to connect to the master node. We initially
purchased more bandwidth than necessary to make sure we could scale
the cluster as usage increased. Like the master node, the cluster
interconnect is a shared resource and potential single point of
failure. For performance and fault tolerance, redundant interconnect
networks could be implemented. But in our experience, network switches
are more reliable than servers, so this redundancy is usually not
necessary. Although virtually all the interconnect traffic is between
the master node and a cluster node, this switch is also connected
to external networks via a router to support occasional connections
required for services such as DNS lookup and remote administration
with telnet. Alternatively, we could have had our master
node act as a router for the cluster's external network traffic.
The total cost for the four cluster nodes, network switch, and
Gigabit Ethernet adapters for JASPer was about $10,000. We can scale
up to 10 cluster nodes for just the cost of additional nodes before
reaching the capacity of our cluster interconnect. The cost of the
cluster could have been reduced significantly if we had used our
existing LAN for the interconnect and recycled old PCs as the cluster
nodes. However, we think the cost is justified when you factor in
the increased reliability of the new equipment along with the performance.
JASPer Administration Web Page
Maintaining a cluster complicates systems administration. Besides
having more servers to take care of, the administrator must combine
information from multiple sources to get an overall picture of the
status of a cluster. Fortunately, the Apache servers provide some
status information through dynamic Web pages. Additional information
about the state of a servlet engine can be obtained from a monitor
servlet. We have collected this information in a multi-frame Web
page to provide a quick picture of our cluster performance and status
(see Figure 2).
The top frame contains the Apache HTTP server status page. This
page lists some vital statistics for the HTTP server, such as uptime,
total traffic and throughput, and current requests. It also lists
each child server process and its most recent request processed.
To enable this page, turn on ExtendedStatus and define the
server-status handler in the httpd.conf file:
Deny from all
Allow from 127.0.0.1
The server status page is then available at: http://localhost/server-status.
The example directives only grant server status access to the server
host itself. As with all the status pages described here, you should
restrict access to the status page to prevent untrusted users from
accessing information that should remain secret. In our environment,
we allow access to a small LAN protected by a firewall that we can
dial into for remote administration and troubleshooting.
JServ provides a set of status pages displayed on the right side
of our frameset status page. The top-level page is created by mod_jserv
and includes links to the master node and each cluster node. Clicking
on the master node under "Configured Hosts" displays the
mod_jserv parameters and each servlet zone that has been
mounted. If a servlet zone is defined as part of a load balancing
set, then each cluster node in the set is listed. A "test"
button is provided to run the servlet engine on that node (JServ
is actually a servlet and can execute itself). This provides access
to the configuration parameters for each individual JServ server.
You enable the JServ status pages in the httpd configuration,
just as we did for the HTTP server status handler above (in this
example, we again grant access only to the HTTP server host):
Deny from all
Allow from 127.0.0.1
To let each JServ execute itself to display its configuration parameters,
you must also specify "security.selfservlet=true"
in the jserv.properties files.
On the left side of the frameset, we display a graph of the Web
server traffic over the past 30 hours. This graph is generated every
15 minutes by MRTG (http://ee-staff.ethz.ch/~oetiker/ \
webtools/mrtg/mrtg.html). MRTG was designed as a tool for graphing
traffic on network equipment, but can easily be extended to graph
one or two values representing any kind of resource usage by providing
an external program in the Target directive. When run, the external
program should return four lines containing, in order, the first
value to graph, the second value, a string specifying the uptime
of the server, and a string telling the name of the server.
Our external program for JASPer status is a simple script that
counts occurrences of GET and POST requests in the
HTTP server access log. One graph line plots total requests, and
the other plots portal requests for access to remote resources.
MRTG expects, by default, continuously incrementing counters, so
our script can count all the requests without worrying about how
many it counted last time. Still, it is not a very efficient process,
so we increased the interval from the default 5 minutes to 15 minutes.
We also roll the access log every night, so no more than a day's
worth of requests are counted each time the script is run.
The bottom frames of our combined status page contain brief information
that is extracted by a special servlet, MonitorAgent, running on
each cluster node. If a cluster node is down, then the corresponding
frame will not be displayed -- providing a quick way to see
whether a server has died. Java source for the MonitorAgent class
is shown in Listing 1.
The MonitorAgent information includes the version number of JServ
and the servlets to help ensure that each cluster node's configuration
is identical to the others. The version of each servlet is available
because our local coding standards specify that every servlet return
its version number from the getServletInfo() method. The
names of each servlet must be specified in the servlet zone properties
file, with each name separated by a special character:
The MonitorAgent class also displays the total number of connections
and the total and free memory available to the Java Virtual Machine.
The class can be extended to display application-specific session
information. For our portal application we count sessions for each
client institution and further break it down by whether the user
is accessing the portal from on or off campus.
After we built our cluster, we wanted to find out how well it performed.
In particular, we were interested in seeing how much additional
capacity we gained from the clustering. We also wanted to develop
a benchmark for determining when we might need to add cluster nodes
as usage increased, and to be able to recognize when JASPer began
to reach the point where additional nodes did not increase capacity.
There are several tools available that throw a bunch of requests
at a Web server and come back with a measure of how many requests
the server can handle. We chose ab, the Apache HTTP server
benchmarking tool, because it was easy to use and came free with
the Apache source distribution.
It's important to realize, however, that the resulting number
of requests per second (or similar measure) is artificial, and almost
certainly doesn't correspond to any real-life limit or load.
Most of these tools do not allow you to create a realistic sample
of URLs to simulate real user sessions, particularly when custom
authentication is used to create and maintain the sessions. Also,
different tools will give you different results for the same URL
samples because they have different ways of simulating multiple
users, none of which will correspond exactly to real loads. However,
the numbers are still useful for relative comparisons of different
configurations. For example, by running repeated tests after tweaking
the httpd configuration parameters, you can get a good idea
of what parameter values work best for certain kinds of loads.
Loads are defined in ab by specifying a concurrency level
(corresponding to a number of users who might be making requests
at the same time), a total number of requests, and a single URL
to be used for the requests. The URL can be requested using either
the GET or POST method, and a file can be specified
for the data to be POSTed with the request. ab also
supports HTTP basic authentication and the passing of cookies. However,
there is no way to receive a cookie from the first set of concurrent
requests to be used by subsequent requests to simulate a set of
user sessions that log in for the first request but not for subsequent
Thus, the URL you use with ab must be carefully chosen
to provide the most useful benchmarking data. For JASPer we wanted
to use a URL that would fully exercise the cluster, so we selected
one that did more work than average requests. We chose a request
that authenticated the benchmark program's IP address and created
a new user session each time, before building and returning a custom
menu page. Also, to minimize other factors that affect response
time, but are not directly related to cluster performance, we ran
ab on the master node while the system was not doing any
We ran that benchmark test with two load configurations. To simulate
light loads, we specified 10 requests with a concurrency level of
two as if two concurrent users were making 5 requests each. To simulate
a higher load, we specified 500 requests and concurrency level 50,
or 50 users making 10 requests each. These tests were run with JASPer
configured with one, two, three, and four cluster nodes.
The results are illustrated in Figure 3. Under the light load,
performance increased relatively little as the number of cluster
nodes increased. Two nodes could process about 30% more requests
per second than one, and the addition of the third (25% more requests/second)
and fourth (8%) nodes provided even less additional capacity. However,
under the higher load, JASPer capacity scaled very well as nodes
were added to the cluster.
In fact, the addition of each node in the higher load tests increased
capacity by about 110% of the requests per second that a single
node could handle. This "better than linear" scaling is
due to an unexpected benefit of clustering that was revealed by
these tests: When ab simulated 50 users making concurrent
requests, one- and two-node configurations could not handle all
the requests without errors. 62 out of 500 requests failed when
JASPer had one node, creating an abnormally low requests-per-second
rate compared to the other configurations. With two nodes, two out
of 500 requests failed and there were no failures with the three-
and four-node configurations. When the one-node baseline was normalized
to ignore errors, (6.4 requests per second instead of 5.61), scaling
with additional nodes was about 90%.
These benchmark tests demonstrate that adding additional cluster
nodes to JASPer results in higher capacity and throughput, particularly
under higher Web server loads. Additionally, they indicate that
the current configuration of four nodes has plenty of capacity for
the normal loads our portal application sees. And, as that usage
grows, we should be able to increase capacity by adding additional
nodes. This scalability and performance, along with the fault tolerance
that the JServ load balancing mechanism provides, are the key benefits
of clustering. With JASPer, we have been able to realize these benefits
without investing a lot of money and effort into setting up and
administering the cluster.
Don Gourley works at the Washington Research Library Consortium
where he develops and administers Web-based information systems
for academic libraries. Don can be reached at: email@example.com.