Backup Scripts from UnixReview.com

Edited by Ed Schaefer

I host UnixReview.com's Shell Corner column. Each month I choose a winning script submitted to me by a UNIX professional. UnixReview.com publishes the script and pays the lucky submitter $100.00. By far, the most popular submission topic is UNIX backup scripts. For this article, I have selected three previously unpublished UnixReview.com submissions. (All Sys Admin magazine listings are available for download at: http://www.sysadminmag.com/code/.):

nfs_backup -- by Robert Sciuk (rob@ControlQ.com). This Bourne shell script backs up files to a Network File Server. This script replaces traditional tape and CD media with backing up to another network server. Sciuk mounts an NFS line to the backup (or destination) server, and then copies files to this link. See Listing 1.

mirrorcopy -- by Chris Matta (chris.matta@sun.com). This Solaris Korn shell script uses the UNIX dd command to perform an image copy of the root file system to another unused partition. Matta uses the Solaris Print Volume Table of Contents command, prtvtoc, to verify that the destination partition is viable before executing the dd command. After dd completes, the fsck command individually checks each partition for corruption. See Listing 2.

backup.ss -- by John Spurgeon (john.p.spurgeon@intel.com). Newer versions of tar provide an exclude option, -X, and an include option, -I. Spurgeon supports his software distribution process by using these tar options with backup.ss, a Korn shell script. See Listing 3.

Backing Up a Network File Server

by Rob Sciuk

How many times have you unintentionally deleted a directory or file only to realize that there was some wheat amongst the discarded chaff? Wouldn't it be nice to be able to immediately grab last night's backup from a network repository? I will demonstrate NFS_backup (Listing 1), a simple Bourne shell script that performs a sophisticated and selective nightly backup of a BSD-based network server.

Near Line Backup/Recovery

Why a network repository? With the steep drop in the price of IDE drives and the commensurate increase in drive densities, the newer commodity disk drives offer an unparalleled bargain in the dollar-per-megabyte for storage. So much so, that an old, redundant computer might be redeployed running BSD or Linux to serve as a backup server.

Configuration

Configure NFS_backup using the /usr/local/etc/backup.conf file. backup.conf contains the name of a given archive and the path to the files separated by a colon. Each archive is on a separate line:

<Archive Name 1>:<Local Path to Files>
<Archive Name 2>:<Local Path to Files>

For example, I mounted an NFS link to my file server "hippo" as /mnt/hippo. Within that directory, I maintain two important but logically separate projects. "Repository" is where I keep globally used software, and "GoodStuff" is where I keep source code for projects currently under development. Finally, I back up the NFS_backup script and the backup.conf configuration file. My three-archive backup.conf file looks like:

GoodStuff:/mnt/hippo/share/GoodStuff
Repository:/mnt/hippo/share/Repository
Root:/usr/local/etc

The Target Area

The archive target area, /ul/Backup, living on my big, cheap IDE drive contains the following directory structure:

/u1/Backup
/u1/Backup/usr/local/etc
/u1/Backup/GoodStuff
/u1/Backup/Repository

Each of the above directories corresponds to the archive name in the backup.conf file. If necessary, the script creates a directory corresponding to each archive in the configuration file. However, the script does not delete archives removed from the script. Furthermore, within each directory are ten archive files (configurable by RETAIN=10), with a naming convention as follows:

<archive>.<YYYYMMDD>.tar.gz

For example, the files contained within the Repository archive in my target directory would be:

Repository.20010301.tar.gz
Repository.20010302.tar.gz
.
.
Repository.20010310.tar.gz

Excluded Files

By using the tar -exclude-from option, the files I want to exclude from nightly archiving are listed within a hidden file, .excludes, in the actual target directory. Typically, I exclude publicly accessible software available via the Internet. Thus, my /mnt/hippo/Repository/.exclude file might contain:

/mnt/hippo/Repository/netscape
/mnt/hippo/Repository/samba
/mnt/hippo/Repository/postgres

The Crontab Entry

Because I'm using older hardware and I don't want speed to be a critical issue, the script executes from cron during off hours. I execute this script daily at 0259 on my FreeBSD 4.2 backup box:

#minute hour mday month wday who  command
59      2    *    *     *    root /usr/local/etc/NFS_backup|mail rob@schizo

Owing to the network-intensive nature of NFS transfer, this script is definitely an unattended after-hours process. It chews up close to all my bandwidth on a 10-BT/10-B2 Ethernet. I recycled an old Pentium 120-Mhz box into a FreeBSD-based guardian angel for my network, and you can too!

Backing Up the Root File System with mirrorcopy

by Chris Matta

I periodically back up my root file system to an unused drive by using mirrorcopy (Listing 2), a Korn shell script. Here's my logic:

1. Find the root file system from the UNIX df -k command.

2. Using the Solaris Print Volume Table of Contents command, prtvtoc, determine the sizes of the root and destination drives. The prtvtoc command determines the geometry and partitioning of a disk.

Assuming the root drive, executing prtvtoc /dev/dsk/c1t0d0s0 could produce:

*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      2    00       2568    559824    562391   /
       1      3    01     562392    821760   1384151
       2      5    00          0  17773128  17773127

Under Solaris, slice two is known as the "overlap" slice. It contains the whole drive's cylinders. I use it here to determine the size of the drives. In the script, I eliminate comment lines beginning with an asterisk with grep, and the awk script delivers the partition size (field 5) if the first field is 2, the second slice.

3. If the destination device doesn't exist, exit with an error.

4. If the root size is larger than the destination size, exit with an error.

5. If the destination drive doesn't respond to prtvtoc, exit with an error.

6. Use slice 2 to dd copy the entire root drive, table of contents and all, ending up with an exact copy of the root file system.

7. Perform a system check, fsck, on each object identified by prtvtoc. This ensures that there's no corruption.

In conclusion, be aware of these issues:

1. Since this is potentially a destructive script, the destination drive is hard-coded and is not a command-line option.

2. This script must be run by root, but mirrorcopy contains no "check for root section". Root permissions are required to execute prtvtoc and to perform the copy, so non-root users receive "permission denied".

Including and Excluding Files Using tar

by John Spurgeon

Many systems administrators use the tar command to back up files. However, it might be news to some admins that the tar command bundled with certain flavors of UNIX, such as Solaris 7, lets you include and exclude specific files. After giving some examples of why you might want to use these options, I present a simple shell script (Listing 3) that was developed to back up an application directory prior to performing an upgrade.

Why You Might Want to Include or Exclude Files

As release manager for a software development group, I am responsible for writing the upgrade instructions used by local systems administrators to install new releases of our application. Most of our application's files are stored in a directory called /usr/app/bin. Before overwriting the files in this directory, administrators back up the existing files in case the new release needs to be uninstalled. I developed a backup script that uses tar with the include and exclude options to solve three problems.

First, some of our application's programs create named pipes in the /usr/app/bin directory. If the named pipes are included in the set of files to be backed up, the tar command hangs. I use the exclude option to ensure that the tar command runs successfully by excluding named pipes from the backup.

Second, the subdirectory /usr/app/bin/new shouldn't be backed up because it typically contains large files that are unrelated to rolling back the upgrade. I use the exclude option to eliminate this subdirectory from the backup.

Third, it's convenient to keep copies of site-specific configuration files located in /usr/app/bin separate from the complete backup. In the event that the files are accidentally overwritten, it's a little easier to recover these configuration files if they are stored in a separate backup file. I use the include option to back up specific configuration files.

A Simple Backup Script Listing 3, backup.sh, backs up the /usr/app/bin directory. All files except for named pipes and the /usr/app/bin/new directory are backed up to the file $backup_name. Only the files listed in $include_file are backed up to the file $recover_name.

Ed Schaefer is a frequent contributor to Sys Admin. He is a software developer and DBA for Intel's Factory Integrated Information Systems, FIIS, in Aloha, Oregon. Ed also hosts the UnixReview.com monthly Shell Corner column. He can be reached at: olded@ix.netcom.com.