Cover V11, I02

FIgure 1


Web Publishing with Perl Objects

Reinhard Voglmaier

The Internet has grown exponentially in the past ten years, and along with technical improvements have come changes to the way we work. First, a few people may have worked on a whole Web site, but now sometimes a whole team of Web editors may work on only a little part of an enterprise portal.

A standard life cycle for Web pages has evolved in our company. Web pages initially live only on a developer's desktop and are eventually added to a test site. After the updated pages are approved, they're put into production and the whole site is then in a constant state of change. This cycle of Web page development -- from test to production -- continues as long as the site is on the Internet.

In our environment, many Web editors produce material for our Web sites, and it would be unwise to give them all full access to our Apache servers. Furthermore, our internal review process for Web pages requires an official (a publisher) to approve every page before it goes online. The sheer volume of material flowing to the site requires some degree of automation so that the few full-access Web managers do not have to manually post every new page.

We have developed the following system to help meet the requirements of our process:

1. Web editors post their completed pages to a test Web site.

2. Supervisory Web publishers review and edit the completed pages at the test site. When the page meets the publisher's approval, he or she executes a script that places the page in the queue for the next production update.

3. An automated script with full access to the Apache environment posts the approved pages to a production Web server at regular intervals.

I described this environment in a previous article ("Simplifying Web Production", Sys Admin magazine, May 2000: Since then, the network has become much more complicated, and our original approach proved too simplistic. Some of the changes we've made to system are:

1. The sites hosted on one of the test servers can be transferred to different production servers.

2. The server receiving the site does not have to be a Web server -- it could be a staging server.

3. A number of different test servers can put their pages on one production server.

4. The system can work across firewalls.

In this article, I will describe these changes to our original system, particularly the framework for moving Web sites from a test environment to a production server. Note that our site has grown considerably since we originally envisioned the system. We now have several different test or staging servers on an intranet and also on the Internet. One requirement of the system is that it now must be able to move production pages to any server inside or outside the firewall.

The Basic Idea

The centerpiece of this system is the code that transfers Web content from a directory on a testing server to a directory on one of the production servers. The idea behind the content transfer between the server directories is to create an agreement between the receiving and the sending server that contains what exactly is going to be transferred, how this data has been produced, and where it has to go. It models the same process our Webmaster previously did by hand -- the supplier of the Web site sends the pages he produces and the Webmaster installs them. The server producing the data to be installed on the partner server sends this information along with the data to be installed. With this information at hand, the receiving server knows the site for which the data is intended, where he has to install it, and how to install it.

I initially had only a test and production environment in mind, but I am now also using this solution for other applications (e.g., replication servers or staging servers designed to collect parts of Web sites produced from different suppliers).

After we send data and instructions from server A to server B, we can also send instructions for after the page installation (e.g., instructions to launch the indexing process of the search engine or other type of Web site maintenance tools).

The Design

The system consists of three parts:

  • A GUI
  • A database
  • Procedures to let the GUI "speak" with the database

The GUI is the interface to the user, so it is the visible part of the application. It offers users a list of pages and let's them choose which site to put into production (Figure 1). The GUI also offers administration options including the ability to add users, modify user configuration, and add new sites or servers.

The database holds all data about users, sites, and permissions and the procedures execute the various requests made by the GUI. The procedures ensure that the data in the database is updated as needed. See the sidebar "MVC Paradigm".

In our case, the separation developed naturally:

  • The presentation happens on the client browser produced by a number of CGI scripts on the server side.
  • The data structure is fixed in a database.
  • We needed a way to access the database from the CGI scripts, and this happens via a number of methods offered from a library of objects.

Perl offers object-oriented features as well as a large number of libraries ranging from CGI to database management modules. However, you can rebuild the whole framework in any other language. It would be very interesting, for example, to rewrite the whole thing in Java or Python.


The first approach, as described in my previous article, used DBM as a repository. DBM exists on almost every platform (including Win32 systems). DBM is available for free from the Free Software Foundation ( or Sleepycat ( An RDBMS would be useful if you handle multiple sites with thousands of publishers and editors. In the approach described in this article, I used an XML-based database to hold all the necessary information. I chose XML because it is ASCII and, as such, human readable. You can also record new information without using the administrator interface or, if your database gets corrupted, you can quickly identify and correct damaged data.

Here is our Web site information:

Information about our users, their roles, and where they work:

<User UserId=webuser1>

Information about the Web sites:

<WebSite Name=/medical/projects</WebSite>
We also need tools to access this data. I wrote a public interface describing the necessary functionality and then the private access functions. The application doesn't know about the private methods, so they can be changed without changing the application.

Here is the user object:

1     sub new  {
2               my ($class,%args) = @_ ;
3               my $self = {
4                                   UserId      => $args{UserId},
5                                   CName       => $args {CName},
6                                   SurName     => $args{SurName},
7                                   Roles       => $args{Roles},
8                                   WebSites    => $args{WebSites}
9               };
10              bless($self,$class);
11              return($class);
12    }
Line 2 puts the first argument, the class name, in the class variable and the rest of the arguments in a hash table called %args. The hash table is used from line 3 to line 9 to initialize the object's data. The variable $self is a pointer to a hash table containing the object's data. We then create the object with the pointer attached (function bless) and turn it back to the calling procedure. Look at the public interface:

1    sub save {
2             my $this = shift;
3             my $User = _save();
4             #   ....     Some housekeeping can go on here anyway
5    return($User);
6    }
To create a new user:

1    $user = new User(    UserId             => "Admin001",
2                                   CName    => "Reinhard Erich",
3                                   SurName  => "Voglmaier",
4                                   Roles    => "Administrator",
5                                   WebSites => "",
6                                  ) ;
7    $user->save();
GUI -- The Cool Part

Remember when you wrote code for Motif or Visual C++ with hundreds of lines of code for one single widget? The combination of HTML and Perl gives you the same power, but with a few lines of code plus the ability to run anywhere:

1  #!/usr/bin/perl -w
2  use WSP;
3  use CGI;
4  use Template;
6  my $Provider = new WSP();
7  my $Widget = new CGI();
8  my $Template = newTemplate("/d01/web/WSP/templates/Hello.html");
10  my $UserId  = $ENV{'REMOTE_USER'};
11  my $User = $WSP->User($UserId);
13  $Template->display();
14  if ( ! init_Template($User) ) {
15            die "Could not load application";
16  }
19  sub init_Template {
20            for my $Sites ( $User->webSites() ) {
21            # Generates Java Script to put the User Data into the form
22            . . . .
We set up some variables until line 12 and then initialize our objects -- Provider (the main object containing all configuration data), Widget (holding the CGI object), and Template at the end (a facility to read a template HTML file). Line 10 gets the UserId and line 11 the User Object via this UsrId. Line 13 displays the empty Template and function init_Template generates a JavaScript that fills the User Data into the HTML file. There's a submit button in the HTML file that calls a procedure that posts the request (more about this later).

This is simplified -- the true application allows you to choose how to proceed with the site prior to production. You could put into production only the files changed since the last site update, or choose a method to update the database of a search engine, and so on. There are also scripts for administration of users, sites, and requests; the logic is still the same. We create the HTML parts via CGI, we display the template, and we put the data into the template using JavaScript.

The Glue -- The Application Logic

The systems knows of the existence of four directories:

  • Accepted
  • Scheduled
  • Executed
  • Data
When a publisher decides that it's time to update the site, the system creates a file in "accepted". This file contains enough information to permit the system to begin the update process (i.e., the directory to be put into production, the site the directory is living on, the person requesting the update, the type of update, and the date of the request). Because the Web server is running normally at the lowest privilege level (let's assume as "nobody"), the directory "accepted" must be writable for the user "nobody".

There's a process that runs under the UserId of the Web site owner that looks into the accepted directory for instructions. When it finds a file, it writes the file into "scheduled" and puts the data produced into "data" when it's finished. The process then notifies the person who requested the update; additional site updates can be done without putting it into production. It starts a cron job that collects all scheduled procedures and executes them at predefined times.

Controller -- Three Objects

The first object is called "WSP" and not only holds the configuration files but also administers the other two objects (Server and Job). It knows where the previously mentioned directories are living and makes this information available via method calls. As with the database, the information held in the objects is made persistent via XML files.

To see the information held in the WSP file:

<WSP Name=MainWebSite>
All servers are held in the ServerDir and look like this:

<SERVER Name="gsk">
These two objects can be considered static, because the information seldom changes. The SendDataProcedure entry is the name of the procedure called to transfer the data from one server to another.

The third object is dynamic because it gets generated when a Web site goes into production:

<Job Name=user1_13>
<Date>Mon Jul 16 09:11:55 2001</Date>
Here again all data is self-explanatory, thanks to XML. The user (UserId, SurName, and ChristianName) who wishes to put the site into production, the date of the request, and the requirements to put the site into production (in this case, to execute the procedure putInProduction()), are declared in the object's jobs.

Controller -- The Behavior

The three objects are the heart of WSP inasmuch as they determine the behavior of the whole application. When the user decides to transfer the site, he chooses the site to be transferred from a list of files (Figure 1):

1  my $User = new User($UserId);
2  my @SiteList = $User->SiteList();
3       # code to get the data in the HTML Template as described above
4  my $WSP = new WSP();
5  my $Job = $WSP->createJob();
Let's create a new User object from the UserId. This object contains all the User Data. We need this list to display the sites to be transferred by this particular user. In line 4, the controller springs into life and reads the configuration files to obtain the list of directories it should know about and the servers available. With this knowledge at hand, the WSP objects create a new job object and call the accepted method of the job. The accepted method creates a new file in the "Accepted Directory". Because these actions (caused by the call createJob (in line 5) are executed on behalf the UserId, the Web server is running (normally "nobody"), and the "Accepted Directory" has been writable by this user ("nobody"). This is the point at which security must be discussed.

The rest can be executed as a higher privileged user, normally the owner of the system. Once the job has been accepted, the scheduler does his job:

1  my $WSP = new WSP();
2  while ( $Job = $WSP->getNextAcceptedJob() ) {
3            $Job->scheduledJob();
4            if ( my $Server = $Job->transfer() ) {
5                      # we transfer the data to $Server
6                      ...........
This code creates new WSP Object to get all information about the system. Line 2 gets the first job from the accepted directory and executes the schedule procedure requested from the user. The Job Object knows what schedule procedures exist and how to invoke them. Furthermore, it knows to which server the data should be transferred, opens the corresponding server object, and uses the transfer function defined to the appropriate server. The information of the scheduling process is kept in the "scheduled" Directory to permit a control process to understand the state of the job.

Everything is now finished for the sending server. However, the site is still not in production. The same framework is now installed on the receiving server. The receiving server could forward the data on another server, or keep it and work as a staging server. Look at the "final" receiving server that updates the Web site:

1  my $WSP = new WSP();
2  while ( my $Server = $WSP->sourceServers()) {
3            $Data = $WSP->transferData();
4            $Instructions = $WSP->transferInstructions();
5            if ( $Instructions->execute($Data) ) {
6                      # eventually we could execute Control Functions
7                      # or Security Controls here
8              . . .   we're done !!!
It looks very simple because we reused what we had constructed for the sending server. The WSP object knows which servers could have sent data and how to get the data. This could be a simple opening of a directory containing the data; however, the data could be on a staging server or on the sending server (e.g., if the administrator of the sending server has decided not to transfer the data and to let the receiving server get the data instead). After data is transferred, it will be installed using the instructions from the sending server. The instruction object is created again, or transferred, or only reading a file just transferred. The instructions are then executed using the data objects previously created.

Current Development

The framework actually is installed and working as described above. Two important features are now going to be installed: security and control. All the objects described in this article will have two additional methods: security() and notification(). These methods are called at certain times of the life of an object -- when the object is created and when an important action is executed on behalf of the objects (e.g., when the job is transferred or a new job is created). This will permit callbacks to be executed in certain situations and a well-defined context.

When the job object is created, security is called with the job object and the reason as arguments. You can define a callback procedure that is called using these two arguments after this notification is called, again with the two parameters as above. You can register a notification callback that could, for example, send an email to the user who made the request. If the user contests the request, you can invoke a method on the job to cancel the job. The job cancel event can launch a notification to the administrator for permission for the action. Updates of these new features will be uploaded to my home page (


This article described a framework to take Web sites from a test environment to the production site. It is not limited to this particular situation, but can be used for a wide range of requirements. Furthermore, it does not impose a determined server architecture and it could even be used if the server is based on a RDBMS (such as Oracle's WebPortal). Although you have to export and to re-import the database to transport data from one site to another, this works and you maintain only one framework. The framework can fit all security requirements because you determine which security features you will implement in the system.


Berkeley DB --


Lincoln Stein CGI --

Reinhard E. Voglmaier HomePage --

Samba --

Sys Admin magazine, "Simplifying Web Production" by Reinhard Voglmaier, May 2000 --

WebDav --

Reinhard Voglmaier studied physics at the University of Munich in Germany and graduated from Max Planck Institute for Astrophysics and Extraterrestrial Physics in Munich. After working in the IT department at the German University of the Army in the field of computer architecture, he was employed as a Specialist for Automation in Honeywell and then as a UNIX Systems Specialist for performance questions in database/network installations in Siemens Nixdorf. Currently, he is the Internet and Intranet Manager at GlaxoWellcome, Italy. He can be reached at: