Cover V03, I04
Figure 1
Listing 1
Sidebar 1
Table 1


Sharing Local Software on a Network

John Lees

The Department of Computer Science at Michigan State University operates several hundred UNIX workstations and servers on a campus-wide Ethernet network. We have a large collection of local (non-operating system) software we need to make available on most machines. This software currently occupies about two gigabytes of disk space, far too much to actually install on each machine.

Part of our solution is to use Sun Microsystems' Network File System (NFS) to make local software available to multiple machines. This solves the problem of making software that physically resides on one machine visible and usable on other machines, but does not solve the problem of how users can easily include in the appropriate search paths the executables, libraries, and man pages from up to 100 different software packages.

Our environment is very dynamic. Hardly a week goes by in which we do not add a package or install a new version of a package. In a university environment, most of our users are by definition not experts and do not work on the same projects for long periods of time. We want all changes to be immediately visible to our 1000 users with no need for them to fiddle with their paths, and no need to ask for help from a system administrator.

Some four years ago a scheme for making all local software visible through the /usr/local tree was developed (see the paper by Shing and Ni). We have since expanded and revised this scheme to use the /opt tree and to coexist with the package management tools provided with the Solaris 2 operating system (see the sidebar, "UNIX System V Release 4 Software Packages").

The specifics of the scheme are described in the following paragraphs.

  • Each package is contained in a single directory on an arbitrary filesystem. The setup script (Listing 1) makes the symbolic links that allow the package to appear to exist as part of the /opt tree on each machine on which it is set up. Users need place only the /opt/bin, /opt/lib, /opt/man, . . . directories in their search paths to use any package that has been set up. Experienced users know to always go through the /opt tree to find a package; they need never know where the package physically resides.

  • The symbolic links in the /opt tree exist on each machine. They are resolved locally, not over NFS. This reduces network traffic.

  • Because users access all packages via the /opt tree, the actual location of the package can be moved at will. If a disk containing packages goes down, we have only to change one symbolic link for each package in the /opt tree to point it at a different disk. Even if the NFS mount for the crashed disk or server is "hung," we can make the package usable again immediately by changing the /opt/PACKAGE link to point to a copy of the package on another disk or server.

    Forcing all packages to have the same organization saves considerable system administration time and confusion. The machines in the department are organized in a number of separate labs or domains (see my article "A Host Health Probe," for more details). Because all local software is organized in the same way, it is easy to share software among the labs.

    Organizing a Package for Use with the Setup Script

    A package consists of a directory containing at the very minimum a Readme.local file that describes the package. Optional subdirectories are:


    The bin directory contains the executable files; binaries and scripts. Each file in the bin directory is linked through /opt/bin.


    The doc directory contains any documentation for the package other than the man pages and texinfo files. A link is made from this directory to /opt/doc/PACKAGE.


    If this directory exists, it is linked to /etc/opt/PACKAGE. If it does not exist, there will be no /etc/opt directory for this package.


    Files in the include directory are linked through /opt/include. If possible, you should organize the package so that a directory under the include directory has the name of the package, e.g., PACKAGE/include/PACKAGE/*.h, to avoid name collisions with include files from other packages.


    Files in GNU Texinfo format are placed here and linked through /opt/info. The master directory file used by emacs and the info command must be edited by hand to include the files in /opt/info in its menu.


    Files in the lib directory are linked through /opt/lib. As with the include directory, you should use a subdirectory having the package name, if possible.


    Under the man directory are the traditional UNIX man and cat directories. These are linked through /opt/man. The setup script warns you if catman has not been run on the package.


    If there is a var directory, it is linked to /var/opt/PACKAGE. If there is no var directory, /var/opt/PACKAGE is created as an empty directory.

    All other files and directories in the package directory are ignored by the setup script. The setup script never modifies anything in the package directory.

    Figure 1 shows an example list of the links created for a small package. Table 1 shows the size of the /opt tree for two of our domains.

    Using the Setup Script

    The setup script has a large number of options. The usual way to add a new package to a machine is:

    setup -lDISK PACKAGE

    where DISK is the NFS-mounted filesystem on which the package resides and PACKAGE is the name of the directory containing the package. PACKAGE may specify more than one package. For example, to setup all packages on a filesystem:

    cd /home/server/l00
    setup -fl. *

    The "f" option is for fast -- it suppresses the unsetup normally done for each package. This is safe to do only when installing a package for the first time.

    The complete set of options is:

    -c Check the /opt tree for UFOs. Should be done occasionally on the server from which the /opt tree is distributed. Finds non-links that should not be in the /opt tree.

    -f Fast! Suppresses unlinking the old pointers for a package. It is safe to do this only when you are installing a package for the first time, or when you are installing a new version and you are certain that nothing needs to be removed from the previous version.

    -h Brief online help.

    -l The directory containing the package, e.g., "-lserver/l00", "-lserver/l42/gnu", "-l..". Usually a package is the top-level directory of an NFS-mounted filesystem, but this need not be the case.

    -p Make the top-level package links only (override existing links). This is used to switch a package from one filesystem to another, in the event of a disk crash, for example.

    -q Quiet, do not display links made.

    -r Check for UFOs and remove them. Same as "-cr".

    -s Skip any package that is already setup (for which the top-level link exists). A quick way to specify adding new packages is "-fsqlserver/l00 /home/server/l00/*".

    -u Unsetup the packages specified. (If you want to unsetup all the packages, just remove the /opt tree.)

    -v Verify. Look at all the packages already setup for collisions and correctness. Should be done on the server from which the /opt tree is distributed.

    -x Do not actually do the setup or unsetup (useful for finding collisions when installing a new package).

    What You May Need to Modify in the Setup Script

    Listing 1 shows the setup script, which was written in perl for Sun Solaris 2. (An earlier version of the script for SunOS 4.1 is available from the author.)

    You pretty well have to have the bin, include, lib, and man subdirectories in a package. But you may choose not to use the doc and info directories, or to add different ones. The setup script does not require any of these directories. There is no problem and no warning if they do not exist in a package.

    You may use a different naming convention for local software directories than the /home/server/lNN one we use. This would require a very minor change, as the setup script assumes the /home/ part of the path.

    Possible Problems

    The scheme discussed here results in one flat address space for executables (/opt/bin). While this sounds as though it would be a problem, in practice for us it has not been. But this is something to watch out for (the setup script warns you of collisions), particularly if you install a lot of commercial software for which you do not have source.

    There are other schemes available that let users choose the packages (modules) to which they have access. We decided on our scheme because we wanted all software available at all times, with no action on the part of the users. This is important in an instructional environment, where people are often not familiar with how the system works.

    Collisions are also possible in the other /opt directories, though in this case it is usually possible to rename files or put them under subdirectories. For example, if you have many files under the include or lib directories in a package, place them under a subdirectory:


    which will be linked to:


    See Figure 1 for a complete listing of the links made by the setup script.

    Sometimes there are unfortunate interactions between the package management tools, which install packages in the /opt tree, and the /opt tree as maintained by the setup script. Most of these problems are solved by installing and patching software only on a few selected servers and then using the setup script to make the resulting packages available in the /opt tree. We have had enough problems installing patches on workstations, though, that we are now modifying the installpatch script to coexist with our method of managing the /opt tree. (The sidebar on UNIX System V software packages has more information on this.)

    Some of our domains are large enough that we have more than one copy of the local software disks on which packages reside. It can be difficult to ensure that these disks are all up-to-date with the same versions of the packages. A scheme such as that discussed by Clive King in his article "A Distributed File Location and Properties Checker" could be useful for this.


    King, Clive. "A Distributed File Location and Properties Checker." Sys Admin, 3:2 (March/April 1994), pp. 32-48.

    Lees, John. "A Host Health Probe." Sys Admin, 3:2 (March/April 1994), pp. 17-30.

    Shing, Honda, and Ni, Lionel M. "Efficient Management of Multi-Server Network Systems." Proceedings of the 1990 Sun User Group Conference, December 1990, pp. 195-208. Available as technical report MSU-CPS-ACS-23 on in /pub/prip/lees/sysadmin/setup/

    About the Author

    John Lees has an M.S. in computer science and has worked during the past twenty years about equally as a teacher, technical writer, programmer, and system administrator. His computer experience began in the days of front panels and paper tape, and he doesn't have enough fingers and toes to count the operating systems he has used. His love/hate relationship with UNIX dates to early 1985. Currently Mr. Lees is a systems analyst with the Department of Computer Science, and manager of the Pattern Recognition and Image Processing Laboratory, at Michigan State University. He is a member of ACM, Computer Professionals for Social Responsibility, the League for Programming Freedom, the Society for Technical Communication, and the TeX Users Group.