Hence, ShrinkWrap was born.

Shrink Wrap

The above requirements drove me to write the ShrinkWrap utility. ShrinkWrap is nothing more than an automated backup and restore utility. The contents of the backup are defined by a template file which describes the disks and slices to be captured. The restore procedure involves booting a miniroot and restoring the captured items. There is no real input required from the installer during the ShrinkWrap load. When the load is complete, the target machine will essentially become a duplicate of the source machine. As a result, the installer will need to change the target machine's name and IP address so there are no duplicates on the network.

ShrinkWrap is written as a cooperative set of C programs. The version provided with this article is for NCR MP-RAS UNIX, but seems to be relatively flexible and can be ported easily. It could have been written in shell script or perl, but I am more comfortable as a programmer with C. As a compiled language, C has the additional benefit of security; we maintain control of the source code.

ShrinkWrap was written to enforce software standards, but in doing so it must rely upon hardware standards. The most important requirement is that the SCSI addresses of captured disks must match on the source and target machines. This is non-negotiable; ShrinkWrap performs none of the complex translations that would be required to support loading an image on disks with different SCSI addresses.

Often I am asked what else must be the same between the source and target machines. Technically, it depends upon the version of UNIX you are using and how cleanly you would like the machine to boot after the restore. The NCR UNIX kernel, for example, is aware of things such as card types, locations, and counts. Changes are detected during the boot cycle and the user must make decisions to fix the discrepancies. (Some versions of UNIX such as Sequent's Dynix will invariably require licensing information after a restore.) Typically, some experimentation will be required to iron out some of these details and document the installation procedure for your particular platform. Be sure to test image captures and loads to two completely different machines to get an accurate representation of the load process.

Template File

The template file used by ShrinkWrap is the key to most ShrinkWrap operations. It must have the extension .snd which stands for slices 'n disks. The format is fairly straightforward; each line is a comment, a disk description, or a slice description. The following is an example of a template file:

#
# Slicing information for savesys/loadsys programs.

#
# This configuration is for a gold-installed 4100 with 2 2-gig
# drives using Gold 3.0 slicing.  This file is "4100Gold30.snd"
#
c0t6d0s1                ufs     /
c0t6d0s4                ufs     /var
c0t6d0sa                bfs     /stand
c1t5d0s1                ufs     /home
c1t5d0s3                ufs     /usr
c1t5d0s4                ufs     /tmp
c0t6d0s7                rawc
c0t6d0s0
c1t5d0s0

Comment lines begin with a # character and are ignored. In this example, the first line after the comments is a disk slice description. It begins with a disk slice name, in this example c0t6d0s1, followed by the file system type and the mount point. Typically each slice description has these three components, with one exception. Note the last slice description for device c0t6d0s7. In this example that slice has no file system but should be captured anyway. The type is then designated as rawc, short for raw compressed, and the mount point is irrelevant. ShrinkWrap will use the cpio command to capture slices with file systems and the dd command to capture the rest.

The last two lines of the sample template are device names of disks. (In NCR UNIX disks are referenced as slice 0.) ShrinkWrap will capture the vtocs from these disks and restore them on the target machine. Naturally, you would want to capture the vtocs from any disks with captured file systems. Otherwise, the image may not restore correctly.

There is only one rule for ordering the contents of the template file. File systems mounted in other file systems must have their parent file systems listed first. In this example, every file system is mounted in root, so root is listed first. If /usr/local were a separate file system it would need to be listed after /usr, etc.

ShrinkWrap Programs

ShrinkWrap consists of several programs to create and manipulate system images. A list of the programs and their uses are:

The code is available at www.samag.com. As you can see, savesys can only capture ShrinkWrap images to tape media. savesys can load images from a file system or tape. storesys and cut_tape are used to convert images between file systems and tapes. Because ShrinkWrap can read images from file systems, CD-ROM burning and network load operations are possible. To cut a CD-ROM image, use savesys to capture the image to tape, and then use storesys to copy the tape image to a machine capable of burning CD-ROMs. The CD-ROM content should include all the files created from storesys.

The instructions for using savesys are:

1. Insert the ShrinkWrap Savesys diskette into the floppy drive.

2. Log in as root and enter the command mount -F s5 /dev/dsk/f0 /mnt.

3. Enter the command cd /tmp.

4. Enter the command PATH=/mnt:$PATH.

5. Enter the command export PATH.

6. To save a Phase I ShrinkWrap image, insert the target media into the tape drive and wait for the tape drive to stop flashing. Enter the command savesys $SND /dev/rmt/$TAPE, where $SND is your .snd file and $TAPE is the tape drive you wish to save the image to. Remove the target media upon completion.

7. Enter the command umount /mnt and remove the Savesys diskette from the floppy drive.

As implied by the previous list, we maintain a set of Savesys disks which contain a selection of template files and the savesys and removeb programs. You could alternatively use NFS to network mount these files instead of distributing a floppy disk.

savesys can be run on a system without shutting it down, but it should be idle. Be particularly careful to shut down databases using raw slices prior to capture.

The loadsys program needs to be launched from a miniroot loaded in a location which will not interfere with the load procedure. Many PC-based UNIX systems supply bootable floppy disks with a miniroot kernel which gets loaded into RAM. We modify these disks to automatically run the loadsys program so there is no input required from the installer; all they need to do is boot the disks and the process begins. On other systems we load the miniroot into the swap area of the hard disk so it will not be overwritten during the load process. You will also need to make sure that all the commands that ShrinkWrap uses are available on your miniroot.

Additionally, if the ShrinkWrap image is being loaded from tape, the header section of the image must be loaded into the current working directory before loadsys is started. This is easily accomplished with the command cpio -icvdumB -I $TAPE where $TAPE is the non-rewinding tape device containing the ShrinkWrap media. The header section is very small so you won't need much file system space. File system images already have the header section unarchived so this step isn't necessary.

The command-line syntax is as follows: loadsys template-file [-n]. The -n option is useful to avoid destructive partitioning on systems with diagnostic partitions, but usually it is not needed. It expects to run out of an /install directory and will in turn place its log there. A file can exist in the /install directory called tape which contains a single line of text defining a tape device from which to install. The absence of that file indicates that ShrinkWrap should load a file system image from the current working directory.

Once an image has been captured to tape, it may be necessary to convert it to a file system image for purposes of network loads, burning CD-ROMS, or archival purposes. This is the function of storesys, whose command-line syntax is storesys [tape-device]. It will read a ShrinkWrap image tape and copy it into the current working directory. File system images consist of multiple files and a unique directory is required. The tape-device parameter is optional but you will need to modify the default device based on your hardware standards.

If storesys has been used for archival purposes, it may be necessary to create another tape media. cut_tape fills this role. Its command line syntax is identical to storesys, but its operation is reversed. cut_tape copies the file system image from the current working directory to the specified tape device.

How the Code Works

All of the ShrinkWrap programs start off by reading the image template file into a set of global variables with the function readinfo(). Disk information is stored in the array disk and slicing information is stored in an array of structures named fs. The variables disk_count and fs_count indicate how many array entries are used of each.

The operation of the storesys and cut_tape programs are fairly straight forward after this. They simply copy image sections using the dd command. The header information is used to determine how many image sections to copy, but the actual contents of each section are disregarded. It may not be necessary to modify either of these programs when porting ShrinkWrap to other UNIX platforms. However, you may want to change the default tape devices.

The operations of the loadsys program are defined in the operating system's documentation. There should be a section describing how to recover a root disk from a backup; this is the procedure which loadsys automates. This article and the accompanying code reflect NCR's procedure, but other operating systems will have analogous operations.

The first step during the restore is to run a batch file called makenodes. Stored in the ShrinkWrap header and generated by the savesys program, makenodes creates the device nodes in /dev/dsk needed to access the system drives. Other flavors of UNIX such as Dynix have built-in commands to automate this process (i.e., devbuild).

Now that the disks are accessible they are checked and formatted as needed. NCR uses the dklayout command to perform these operations. It may not be necessary to perform this operation if your vendor supplies only formatted disks. On NCR this operation also writes the boot loader to the hard disk. Notice that operations on disks are performed in loops based on the disk and disk_count variables. Similarly, file system operations will be carried out in loops based on the fs and fs_count variables.

Once the disks are formatted, they need to be partitioned. Some more traditional flavors of UNIX don't require disk partitioning. Partitioning is usually required on UNIX systems which run on PC-based platforms. Each disk entered in the template file is partitioned using the fdisk command. Notice that the -n option leaves a small amount of room on the hard disk for a diagnostic partition; otherwise it is destructively partitioned to 100 percent UNIX.

Next the vtoc tables need to be applied to each disk. I have seen several commands for this operation typically edvtoc or mkvtoc. These commands apply a vtoc file created by the savesys program. savesys uses a program such as prtvtoc to create the file which is stored in the header section of the ShrinkWrap image.

At this point the disk restructuring is complete. Now ShrinkWrap needs to rebuild and populate the file systems and raw slices. For each file system it runs a batch file called $SLICE.fs where $SLICE is the name of the slice where the file system lives. This batch file was created by the savesys program using the mkfs -m command. Unfortunately, NCR UNIX is not very consistent with the output of this command and savesys needs to massage it before it builds the final $SLICE.fs files (using removeb). On most versions of UNIX, file systems can be created just by knowing the slice and the type. ShrinkWrap uses these $SLICE.fs files to ensure an exact reproduction, however. When porting ShrinkWrap to other environments this feature may not be important and it might be easier to just build file systems on the fly without capturing anything about the originals.

Now that the final systems are created, their contents are restored. For each file system specified in the template file, a cpio is performed to restore the contents. For raw file systems, dd is used to restore the contents. This section shouldn't need any changes when porting to other platforms.

Finally, NCR UNIX requires that the boot kernel be restored using another dklayout command. You will need to consult your system documentation when porting ShrinkWrap to see if there is an analogous command required for your version of UNIX.

The remainder of the restore process is primarily cleanup operations. The file systems are checked and certain key files are restored. (These files are extracted for future versions of ShrinkWrap.) The machine should be rebooted upon completion.

One of the files restored is /etc/rc2.d/S01Shrink2. This can be supplied with the miniroot to perform additional cleanup operations during the boot cycle. For example, our version reinitializes the swap and dump areas. It then disables itself so it is never run again.

Limitations and Enhancements

ShrinkWrap has several limitations. First, the images are limited in size to the tape media being used. In other words, it does not have facilities to prompt for tape switching. (This is really a limitation on the part of the dd command used for raw file systems.)

ShrinkWrap does not handle software-defined virtual volumes well. It can save and restore them as RAW file systems, but the template needs to include all of the pieces that comprise the virtual volume.

Aside from repairing the limitations described above, there are numerous enhancements which could be made. System audits could be used to interactively present capturable items to the user and automatically generate the template files. It is possible to support image deltas. In other words, images could be created which capture changes since the last capture or load. The code could be more intelligent about problems; for example, it might try alternative approaches to a task and diagnose errors with probable causes. Furthermore, a fancier display with progress bars would give the user better feedback. Indeed, all of these features are planned for our next release of ShrinkWrap.

Another interesting idea would be to improve performance through multithreading. To prevent disk thrashing (and an overall decrease in performance), threads would need to be synchronized to work on different disks.

Conclusions

ShrinkWrap has proven to be invaluable in our department. It can cut down a server installation from a day to under an hour. Our certified standards are propagated internationally without flaw. Indeed, it has been targeted for use in disaster recovery situations and has been used repeatedly in failure recovery situations.

About the Author

Rick is System Engineer for Intel Corporation. He manages a central support group in a department which is responsible for NT and UNIX infrastructure design and integration with enterprise applications.