Depot to Create a Centralized, Flexible, and Manageable Software
Do you wish you had a centralized software repository on one server
for all flavors of UNIX? And that you could associate each
file with the utility it belongs with? And have multiple
versions of the same software on the same system? And thatyour
users could install their own software versions without using disk
space? For free? These features and more are possible using a utility
called depot (http://ftp.andrew.cmu.edu/pub/depot/), which
has not changed since version 5.13 in 1995.
Depot alone is not enough, however. In this article, I will outline
how to combine depot with the use of NFS, wrappers, and a chrooted
environment to easily implement a very flexible and manageable UNIX
software repository. Within a day, it's possible to complete
the setup and start populating the repository with your favorite
Depot is central to the success of the software repository. It
is used to create a virtual directory tree of symbolic links that
point to the actual locations of the software packages that make
up your repository. The real repository can be centralized on one
file server or distributed across several servers. I describe the
one server approach here. Each version of each package for each
OS gets its own directory on the server. Each client then has a
depot.pref configuration file that instructs depot how to create
the symbolic links in its virtual directory tree. Depot superimposes
the links for all the software packages into one directory tree.
The result is a single entry in the PATH variable, yet each package
is in its own "space" on one server making package management
Figure 1 shows the scalable directory structure of a depot server
and the relationship of the virtual repository on a depot client
to the depot server. All directories and symbolic links within the
virtual repository are managed by the depot utility through entries
made in the depot.pref file on the client. In the sample depot.pref
file of Figure 1, the third column specifies the top-level directory
trees that are mapped into the top level of the virtual repository.
To have version 5.8 of vim available for use on Solaris 8 hosts,
you first populate the repository on the depot server using one
of the methods described in the section "Four Methods to Add
a Utility to the Repository" below. If you used method 4, the
resulting vim install tree would then get moved to /export/disk1/software/vim/5.8/sol8.
Then, on each Solaris 8 depot client, add the entry path vim
/depot/depot_server/depot/vim/5.8/sol8 to the /abc/depot/depot/depot.pref
file and run the depot utility. All the symlinks will be automatically
created and vim will then be available in the /abc/depot/bin
I am currently working for a startup that is almost exclusively
Solaris except for a few Linux hosts. Therefore, the details described
below are mostly for the Solaris operating system. There will be
minor differences with other flavors of UNIX. Since my company does
have a few Linux hosts, and plans to add more, I have verified this
technique on Linux, though I have not yet rolled it out to the development
environment. As long as depot can be compiled, and a chroot environment
created, this technique should port to any flavor of UNIX.
The required file server disk space depends on many factors. You
should plan on reserving at least 2 GB per OS version. Keep in mind
that you want your file server to hold the original downloaded packages
and tarballs, the installed directory trees, as well as a chroot
directory tree for each version of the OS. My chroot directory trees
alone are about 500 MB each for Solaris 2.5, 2.6, 7, and 8. My Linux
chroot directory tree is 830 MB, but could be significantly reduced
if disk space was more of a concern than the time required to optimize
the chroot environment.
On the repository file server, create a software directory with
the top-level directory structure shown in Figure 1. For this article,
I assume the software directory is a subdirectory of /export/disk1.
Export this directory to your network. Here is a sample dfstab
entry for a Solaris host:
share -F nfs -o \
Note that you will need to allow root access to one host for each
OS version you plan on supporting in the repository. These "build
hosts" need to chroot into directories that are created later.
On each client, create the directories /abc/depot/depot
and /depot/depot_server where abc is a short abbreviation
for your company or organization. Here, /abc/depot will be
the top of the virtual directory tree of symlinks created by depot,
and /depot/depot_server will be the automount point to the
real repository on the file server.
After reading this article, you may be tempted to use /usr/local
for the top of the virtual directory tree. I urge you not to
use /usr/local or any other path that is a default for precompiled
packages. If you do, then you will not be able to install any precompiled
Solaris packages independently of the depot.
Now, make the automount entries. Add the following line to /etc/auto_master:
Then create /etc/auto_depot and add the line:
Since these mounts might be in constant use, you could make them permanent
instead by adding equivalent entries to the vfstab file.
Also, on each build host, create a directory /software.
Note that it is possible for the file server to also be a client
and a build host.
Creating a chroot Environment
Unfortunately, there are some packages that need to be compiled
for and installed into their final resting places. They will not
work if they are later relocated. Other packages have environment
variables that are supposed to allow relocation but occasionally
fail to work properly if moved. I had such problems with emacs.
For these types of packages, you need to supply the prefix=/abc/depot
option to the configure utility. However, since your build host
is most likely a client as well, its /abc/depot directory
is where all of the depot's symlinks are. Executing make
install will superimpose the package's real files into
depot's virtual tree.
Unless you can afford the luxury of a dedicated build host for
each OS version so the build host does not need the /abc/depot
virtual directory tree, you will have to create a chroot environment
so you can have an empty /abc/depot directory. I recommend
placing these chroot environments on the file server since it probably
has plenty of available disk space and any host can become a build
host simply by modifying the root access list of the exported share
on the depot server.
Unlike a minimal chroot environment for Internet services described
in "Jailed Internet Services" by Liam Widdowson (Sys
Admin magazine, August 2001), this chroot environment needs
to be very robust to compile the variety of apps that are available.
Simple scripts are provided below, which create Solaris and Linux
chroot environments that are practically a copy of the real OS.
You will first need to mount the /export/disk1/software directory
on the filer to /software on the build host. The build host
will need root privileges on the mount point. The script runs on
the build host:
# Make a Solaris 8 chroot environment.
OSVERSION=sol8 ; export OSVERSION
CHROOTHOME=/software/chroot/$OSVERSION ; export CHROOTHOME
DOWNLOADHOME=/software/downloads ; export DOWNLOADHOME
mkdir -p $CHROOTHOME
mkdir -p usr/dt usr/local var/tmp tmp builds abc/depot
ln -s ./usr/bin bin
ln -s ./usr/lib lib
tar cf - sbin etc platform kernel devices | ( cd $CHROOTHOME ; tar xf - )
ufsdump 0ubf 126 - /dev | ( cd $CHROOTHOME ; ufsrestore rbf 126 - )
tar cf - bin lib ccs openwin ucb sbin share include | ( cd $CHROOTHOME/usr ; tar xf - )
tar cf - lib include share | ( cd $CHROOTHOME/usr/dt ; tar xf - )
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/make-3.79.1-$OSVERSION-sparc-local
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/gcc-2.95.3-$OSVERSION-sparc-local
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/perl-5.6.1-$OSVERSION-sparc-local
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/bash-2.05-$OSVERSION-sparc-local
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/gzip-1.3-$OSVERSION-sparc-local
pkgadd -n -R $CHROOTHOME -d $DOWNLOADHOME/zlib-1.1.3-$OSVERSION-sparc-local
# Make a Linux 2.4 kernel chroot environment
OSVERSION=lnx24 ; export OSVERSION
CHROOTHOME=/software/chroot/$OSVERSION ; export CHROOTHOME
mkdir -p $CHROOTHOME
mkdir -p usr/X11R6 usr/bin usr/share usr/lib usr/libexec usr/sbin usr/include \
sbin bin lib var/tmp tmp builds abc/depot
tar cf - sbin bin lib etc var | ( cd $CHROOTHOME ; tar xf - )
dump 0bf 64 - /dev | ( cd $CHROOTHOME ; restore rbf 64 - )
tar cf - X11R6 bin lib libexec sbin share include | ( cd $CHROOTHOME/usr ; tar xf - )
You may need to add additional packages and utilities to the chroot
environment to be able to compile every package you want. For example,
I found I later needed to install tk and tcl. I also found that some
packages required the existence of X header files. For Linux, I needed
to install byacc in order to compile depot. Just be sure NOT to install
these permanent packages into the /abc/depot directory. The
sole purpose of this chroot environment is to have a clean /abc/depot
directory, so you can make install a package into it and
then relocate it into the real software repository outside the chroot
If your OS does not have prebuilt packages available for GNU tools,
you will need to find some other way to get them into your chroot
environment. Copying /usr/local or whatever directory tree
your tools are currently in is probably the easiest method. Unfortunately,
this directory tree may be huge and contain gigabytes of unneeded
packages. This is the exact problem depot will solve for you, once
it is set up. For now, you may have to build them from scratch into
the chroot environment.
Adding the Depot Utility to the Software Depot
The first utility that is added to the software depot is depot
itself. Depot is not available as a package for Solaris, so it needs
to be compiled. It can be compiled in the chroot environment using
method 3 as described below. The GNU make utility and a compiler
are required to install depot. Also, gzip is required when running
the depot command to update the virtual repository. Because
these three utilities are able to be relocated, it is possible to
install each into the real repository using one of the four methods
below. Each utility's bin directory then needs to temporarily
be added to the PATH variable. After depot is installed and the
virtual directory updated, entries for the three utilities can be
added to the depot.pref file and another update performed. After
this update, links for gzip, depot, make, and gcc should exist in
/abc/depot/bin so the temporary PATH entries can be replaced
Four Methods to Add a Utility to the Repository
There are four basic methods to add a utility to the repository.
Before discussing the details, you must make a decision in each
of the four methods: Should the utility be added once for an older
OS version and then, if possible, used by later OS versions? Or
should the utility be added for each version of the OS? Since disk
space is not a concern for me, but repository consistency and organization
are, I always choose to add a separate instance of the utility even
if, for example, the Solaris 2.5 binaries will work on Solaris 8.
The only exception is if there is a precompiled version for Solaris
2.5 and not any later version.
The easiest and quickest method to install a utility is by using
the package-management utilities of the OS on precompiled binary
packages. The requirements of this method are:
- The package management utility needs to have relocation options
so that the directory where the package will be installed can
be specified. Solaris's pkgadd utility achieves this with
the -R option. Note that Red Hat Linux's rpm utility
has a similar option but I found that every rpm I tried to install
in an alternate directory reported that the package could not
- Package management utilities have a directory or database in
which package information is stored. This must be able to be relocated
to allow for installation of multiple versions of the same package
for multiple OS versions. Solaris's packadd utility meets
this requirement by placing the package information in the var
directory under the root directory specified by the -R
- The utility itself must be able to be relocated. Many prepackaged
utilities are available at http://www.sunfreeware.com.
By default, they are compiled and packaged to run under /usr/local.
Even though pkgadd can relocate the utility, it may not run from
/abc/depot. If this is the case, the utility must be compiled
in the chroot environment as described in method 4 below.
To add the precompiled Solaris 2.5 tk package to the repository,
execute the following on a Solaris build host. Note that the build
host for this method can be any version of Solaris since the downloads
directory is located on the file server and is common to all build
mkdir -p /software/depot/tk/8.3.3/sol25
pkgadd -R /software/depot/tk/8.3.3/sol25 -d /software/downloads/tk-8.3.3-sol25-sparc-local
Now add the following line to the /abc/depot/depot/depot.pref
file on any Solaris 2.5 client that needs tk:
path tk /depot/depot_server/tk/8.3.3/sol25/usr/local
Update the virtual repository on each Solaris 2.5 client by executing:
/abc/depot/bin/depot -u -T /abc/depot # unlock the repository
/abc/depot/bin/depot -T /abc/depot # Do the update
Document the install in a /software/notes/tk.8.3.3.install
The next easiest method to install a utility is simply an extension
to method 1. Basically, some precompiled packages can be relocated
only if environment variables are set that override the default
location for which the package was previously compiled. In this
case, a wrapper script must be created for the executable.
As in method 1, install the package into the repository. Here
I use the less utility as an example for Solaris 7. The precompiled
less package expects the terminfo directory to be located in /usr/local/share/terminfo.
However, if the ncurses package is installed using method 1, its
virtual location will be in /abc/depot/share/terminfo. Here
are the steps:
mkdir -p /software/depot/less/358/sol7
pkgadd -R /software/depot/less/358/sol7 -d /software/downloads/less-358-sol7-sparc-local
Rename the less executable and add a symlink to the wrapper:
mv less less.pkg
ln -s /abc/depot/wrappers/less.wrapper less
Create the wrapper:
chmod 755 wrapper
exec /abc/depot/bin/less.pkg "$@"
Add the following line to the /abc/depot/depot/depot.pref file
on any Solaris 7 client that needs less:
path less /depot/depot_server/less/358/sol7/usr/local
Note that for the first wrapper script, you also must add the following
line to the depot.pref file. This same entry will automatically update
any additional scripts added to the wrappers directory during future
path wrappers /depot/depot_server/wrappers
Update the virtual repository on each Solaris 7 client as described
in method 1 and document the install in /software/notes/less-358.install.
The third method to install a utility is used when the utility
can be relocated but is not available in a precompiled package format.
I seldom use this method because installation is nearly as much
work as for method 4; and method 4 will always work whereas there
is a chance this method will not. If it turns out the utility cannot
be relocated, you will need to repeat the installation using method
The one utility for which I do use this method is depot itself.
Because the depot binary can be relocated, it can be installed directly
into the real repository, even though it will be run from the virtual
This method must be performed on a build host with the same OS
version as the destination repository. In this example, the build
host must be Solaris 2.6:
mkdir /builds ; cd /builds
tar xf /software/downloads/depot-5.13.tar
./configure --prefix=/software/depot/depot/5.13/sol26 --exec-prefix=/software/depot/depot/5.13/sol26
Note that yacc and ar are needed to compile depot. For Red Hat Linux
7.2, I needed to install the byacc rpm to have yacc available.
Add the following line to the /abc/depot/depot/depot.pref
file on any Solaris 2.6 client that will be using the depot repository:
path depot /depot/depot_server/depot/5.13/sol26
Update the virtual repository on each Solaris 2.6 client as described
in method 1. However, in this example, depot is the first utility
to be added to the virtual repository. /abc/depot/bin does
not yet exist. Therefore, depot's absolute path must be used
just this once. Also, there is no depot database to unlock so running
depot with the -u option can be skipped:
/software/depot/depot/5.13/sol26/bin/depot -T /abc/depot
Document the install.
The fourth possible method to install a utility is the most universal
and also the most time consuming. However, if there is no precompiled
binary package, or the utility can't be relocated, or you simply
want to be in control of the parameters you compile with, this is
the method to use. For some flavors of UNIX, this may be the only
method that can be used for all utilities. Keep in mind, though,
that after you compile and install once, all clients will have access
to the utility just by making the appropriate entry to the depot.pref
file. A little work up front will have a big payoff later.
The basic concept for this method is to compile and install the
utility in a chroot environment where /abc/depot is an empty
directory. Then the install tree is relocated to the real repository
outside the chroot environment. When the depot is updated, symlinks
will be created under /abc/depot to the real repository.
This allows the utility to believe it is running from the location
for which it was compiled.
Because the packaged version of cvs on http://www.sunfreeware.com
is two-years old, I decided to install the latest version using
cp /software/downloads/cvs-1.11.1p1.tar.gz /software/chroot/sol8/builds
Now change to the chroot environment on the build host, compile, and
chroot /software/chroot/sol8 /bin/bash
tar xf cvs-1.11.1p1.tar
./configure --prefix=/abc/depot --without-gssapi
At this point, the make install should have created a directory
tree under the chrooted /abc/depot. Exit the chroot environment
and relocate the entire cvs install tree to the cvs V1.11.1p1 repository:
mkdir -p /software/depot/cvs/1.11.1p1/sol8
mv /software/chroot/sol8/abc/depot/* /software/depot/cvs/1.11.1p1/sol8
Now add the following line to the /abc/depot/depot/depot.pref
file on any Solaris 8 client that will be using cvs:
path cvs /depot/depot_server/cvs/1.11.1p1/sol8
Update the virtual repository on each Solaris 8 client as described
in method 1, and don't forget to clean up the chroot environment
Although brief, this section is as important as all the other
sections combined. For every hour you save in administration time
by using the software repository, you will save another hour by
taking a few minutes to document each and every utility you install.
Almost every utility has its installation quirks, whether or not
depot is used as described in this article. Even if an install goes
cleanly, it is worthwhile to document that fact.
Even though many people like to document by installing within
a script session, I like to work in three to four open windows simultaneously.
I find it easiest to cut and paste my commands into an editor as
I proceed. When I finish installing a utility for one version of
Solaris, I simply cut and paste out of my install notes to install
the other Solaris versions of the utility, making slight modifications
where necessary. Whatever your documentation style is, please create
a file "pkgname.version.install" in the /software/notes
directory and place your commands in it.
A common problem when updating your repository's symlinks
with the depot command involves errors about unresolved conflicts.
These are due to identically named files in two repositories trying
to exist in the same virtual directory space. For example, many
make installs will place a "dir" file in the info
directory. Rename all but the emacs "dir" file to dir.utility,
then copy the pertinent lines from "dir.utility" to the
emacs "dir". Similarly, some installs put their documentation
directly into the doc directory. Create a subdirectory of doc named
"utility", then move all the documentation files to it.
Some utilities are replacements for standard OS utilities, but
usually more efficient and powerful. If they are placed in the /abc/depot
repository and /abc/depot/bin is first in your PATH, some
problems may occur by using the replacement version instead of the
standard OS version. In this case, create two virtual repositories
-- one contains symlinks to utilities needed at the front of
the PATH, while the other virtual repository contains symlinks needed
at the end of the PATH variable.
Taking the last comment a step further, note that many virtual
repositories can be created on the same client using almost no disk
space while having only a single real repository on the file server.
Each virtual repository might contain different combinations or
versions of utilities for use by different user groups on the same
client. Also, users can use the depot command to create virtual
repositories within their home directories, again using very little
space. As the administrator, you can populate the real repository
with as many different versions of a utility as you want, then let
the users modify their depot.pref file(s) to suit their needs. This
only works, however, for utilities that are location-independent
or able to be relocated through the use of environment variables.
Documenting the contents of a depot.pref file for a particular
build can help with configuration management. It is very easy to
return a client to a previously known state simply by restoring
the depot.pref file to its previous state, which assumes you leave
old versions of utilities in the real repository.
Because of security and availability concerns, it may not be desirable
or possible to have NFS mounts within a production environment.
However, because of the modular nature of the development repository
and the depot utility, a very specific subset of the real depot
repository can simply be copied to each production server, then
the depot.pref file can be configured to point to the local repository
instead of the remote one. To ensure the utilities work properly,
keep the virtual repository in the same location as in the development
environment, namely /abc/depot.
On a similar note, the entire repository can be replicated to
various sites. Each site can pick which versions of which utilities
to use simply by maintaining the depot.pref files on the clients.
To centralize the management of the depot.pref file, you can write
a script that concatenates or merges network, group, and host-specific
pieces of depot.pref together and then pushes them out to the depot
For utilities that have config files within the repository, replace
the config file in the real repository with a symlink to a local
location, then add the real config file to that location on each
In the rare event that a utility needs a regular file and not
a symlink to exist in the virtual directory, make an entry in the
depot.pref file such as:
Note that I do not recommend placing system-critical utilities into
a repository unless the repository is local to the host.
To switch between versions of a utility on a host, simply change
the entry in the depot.pref and rebuild the virtual repository with
the depot command.
I have set up an email account at email@example.com.
Depending on the mail volume and time commitment, I will try to
respond to questions concerning this architecture. There are many
details I simply cannot cover here.
Combining the depot utility with a chroot build environment for
each OS allows an administrator to create a single software repository
that is extremely flexible, scalable, portable, and manageable.
This power is achieved because each utility is self-contained within
its own directory tree. Basically, any version of a utility only
needs to be built once on the file server to have it available for
every host of that OS type within an organization.
Bob Jobsky has been working as a UNIX systems administrator
since 1993. He holds Chemical and Electrical Engineering degrees
and currently works at SupplyEdge Inc. in Pasadena, CA.