GNATS Revisited

Luca Salvadori

As soon as I read Dave Plonka's article on GNATS Problem Management System (Sys Admin 6(2): 51-56), a bright horizon opened for me. As head of the Information Systems Dept. at my company, one of my tasks is managing users' problems, both hardware and software, along with a handful of co-workers who are usually overbooked and burdened with help calls and whose skill varies in depth and scope.

Although our Sysadmin group's performance (faced with some 300 systems ranging from PC to legacy systems through several UNIX flavors with different esoteric software packages) is fairly good in my opinion, most users notice a lack of information both about solutions to past problems and progress of current help calls.

Up to now, a simple Web-based "helpdesk" system was in use. The user filled out a form containing information about the problem, and a mail message was automatically sent to a helpdesk administrator for subsequent action. Although this system gave us significant help in structuring our work and prevented administrators from being tackled by users in the corridors, no feedback was available nor information coherently stored for future use. Several improved prototypes never reached production status, mainly because of time starvation.

GNATS and its Web-based GUI seemed an obvious solution for my problem. Nevertheless, while installing the package as Dave wrote, I noticed some improved versions developed by CERN that I considered worth exploring. These versions had a number of glitches and some things that needed further improvement, but I worked around the limitations and produced a "production" version of WWWGNATS that I'll discuss in this article.

CERN's WWWGNATS

While sniffing around GNATS Web resources, I found that CERN had done some work on GNATS itself and on WWWGNATS, largely improving the capabilities of both. One major change to WWWGNATS was the ability to divide Problem Reports (PRs) into different "domains," each of which has it own environment of problem categories, maintainers, and responsibles (see http://wwwcn1.cern.ch/~lovell/prms.html). This is useful for large organizations, since the environment can be targeted to allow multiple support teams, perhaps in different locations, to manage their own tasks without multiplying GNATS servers. Since GNATS allows a single database, this capability is achieved by tricking it, using different environment files (e.g., "categories," "responsibles," "editors," and "submitters") for each domain, and copying them to "standard" ones as needed.

Another useful addition was the ability to have email discussions off-line with a simple click on relevant fields of PR description. This is useful for both support personnel and users, allowing them to get unstructured discussions without burdening PR descriptions and other fields. However, after downloading the package from CERN and doing a few experiments, I discovered that something still needed to be done in order to have a really usable package. CERN had made improvements, but the distribution still contained some minor flaws. Also, everything worked on CERN httpd, heavily utilizing httpd configuration tricks, while I was running NCSA httpd.

Porting CERN's WWWGNATS to NCSA httpd

As I experienced, CERN used many CERN httpd configuration features to address most functions of the package; mainly it was a matter of "aliasing" scripts and directories. This can also be done with the NCSA httpd, but some features are notably missing. Thus, I decided to avoid the problem by eliminating aliasing and addressing everything directly. Another issue was that all forms must be declared with a suitable "METHOD" statement to work properly. Because parameters and subroutines are handled by setting PATH_INFO and QUERY_STRING environment variables, it is vital to set method as "GET" in any <FORM> statement.

Some polishing of HTML code was performed as well. It was not strictly necessary, but the package showed clearly that many hands had worked on it, and some standardization was necessary.

Managing Domains

Domains were the most exciting (and tricky) part of the new package. I am responsible for computer operations at both plants of my company, which are located in Milan and Florence (Italy), and where two distinct support groups operate. Management of both groups is centralized under my responsibility, thus I need to keep an eye on both without burdening the Florence guys (who manage a smaller user pool) with the Milan plant's PRs. In CERN's distribution, domain management was applied to some but not all functions, notably the reporting ones. This caused reports to put together all PRs, regardless of domain, and to generate a lot of warnings because domain scope was not set.

I resolved this problem by adding two simple subroutines, called set_gnats_env and unset_gnats_env, to libgnats.pl. The first routine copies "domainized" configuration files to "standard," undomainized ones, and creates an empty file to lock the environment. The second routine does the opposite, deleting undomainized files and the lockfile. All WWWGNATS components now look for lockfile and loop waiting for it to be deleted: a simple, yet effective way to avoid confusion.

This way, any time a "domainized" call is made, the proper environment is set and only information relevant to the selected domain is managed. To allow management of unclear cases, PRs filed in the "pending" category are shown for all domains, so that any support people can take it and manage it. This is a desirable behavior in my opinion, since "pending" calls have neither scope nor responsibles defined so far.

General Improvements

I found some forms lacking information (i.e., not displaying in a simple but effective way what happened). Also, problem descriptions and other editable fields were not displayed for editing, but only changed as a whole. I modified the code to allow direct editing of all fields, and also added an ">Unformatted:" field to manageable ones, as Dave Plonka cleverly did. Now everything looks better. Finally, I revisited all functions related to mail management and reminders. CERN WWWGNATS used shell scripts for both, and they contained some CERN-isms I needed to address.

Off-line mail was handled by sendmailto.pl perl script, a simple script calling mailto.sh as form action. This caused no problems except for mailto.sh, which used cgiparse. cgiparse is a fine program used to parse cgi variables in scripts, and is part of CERN httpd distribution. Many other utilities do the same, and I wrote some code fragments in sh as well. However, I was tired, and since I had a fresh CERN httpd distribution, I simply compiled it and avoided changing too much. I suggest all friends do the same. Reminder messages were handled by a simple shell script, at-pr, to be run by cron. This script required so many arguments (PR number, submitter, responsible, etc.) that its use was quite difficult, because all necessary information had to be gathered elsewhere. I decided to rewrite the code totally, using Perl, and created at-pr.pl.

The basic function is the following: open index file, where most PR information is stored (notably PR's filename, submitter, responsible, and status), get info, and loop over all PRs with relevant status, sending proper email messages to domains' responsibles.

Message changes upon status: at the moment, "open," "assigned," "suspended," and "feedback" status are managed. Messages are sent only after a timeout (defined for each managed status) is expired. To avoid lots of complaints, I suggest running this script only once a day, Monday through Friday, via crontab. A "domainized" version of GNATS queue manager (called queue-pr.sh) was my final effort. This utility files incoming PRs into the database and is designed to be run every few minutes (10 in my case) by crontab. This standard utility is designed, as all, to work on a single database/domain. To allow proper characterization of incoming PRs in a domainized environment, it is necessary to let queue-pr work only on relevant incoming PRs, which have to be properly identified.

To do this, I introduced a dummy field in PR's email header, "X-gnats-domain: domainname," at creation time. Parsing this field allows queue-pr.sh to move all pending entries to a scratch directory, select domain, domainize environment, copy relevant entries back to queue, and finally run the standard utility. The process is repeated for all listed domains.

I thereby avoided any change in the basic GNATS files (written in C), allowing for any future change or improvement. This also allowed WWWGNATS to run on the top of GNATS without any reciprocal disturbance. Finally, I added some missing fields to the on-line HTML help file and inserted some links to it in forms to offer a comfortable tool even to occasional users.

Configuration Tricks

Some minor additions were necessary to standardize the installation/configuration procedures described in the relevant package's README file. First, since at-pr.pl works on all domains and some extra work would be needed to get email addresses from configuration files, I suggest putting all responsibles' addresses into mail aliases file (e.g., /usr/lib/aliases).

Second, I found a strange bug while testing queue-pr.sh. When I put the scratch directory into queue's directory (usually $GNATS_ADM/gnats-queue), I got two extra, empty PRs in the "pending" category for each new, real PR. I did not investigate it too much. I just put the scratch directory elsewhere by editing the initialization section.

Conclusion

WWWGNATS is a powerful tool to ease work of any support engineer and is also useful to track actions and solutions. CERN's additions make it suitable for large, structured organizations and multiple-site information systems. I just polished and trimmed the package a bit to allow it to work as specified and added some nice features that I hope will be useful.

Source code for this article can be found on our ftp site at:

ftp.mfi.com

in

/pub/sysadmin

About the Author

Luca Salvadori is head of Information Systems Dept. at LABEN S.p.A., a leading European space electronics supplier. His experience spans PDP-11 (the manuals for which taught him English) to Linux, Windows, PCs, and networking. He manages the Company's Web site (http://www.laben.it) as well as (in his free time with a bunch of friends) another Linux box (http://aeroweb.lucia.it) dedicated to aviation. As a hobby, he flies sport aerobatics. Luca can be reached at: salvadori@batman.laben.it.