Manual Mirroring

Peter Baer Galvin

This month's column includes a handy script for performing a "manual mirroring" of your root disk drive. Also included is a way to recover a failed disk, a book review, a security hint, and some follow-up information about last month's column.

In last month's column "Reliable and Practical Root Disk Mirroring", a complex solution to "perfect disk mirroring" was presented. That solution included a combination of Solstice Disk Suite and Veritas Volume Manager. Disksuite was used to mirror the root disk, and Volume Manager was given a small slice of root for rootdg to keep it happy and left to manage the remaining disks.

There are times when a simple disk copy is more appropriate than a RAID-1 set. There are also times when both, together, are the best solution. For example, a workstation might need its second disk for scratch or work space, but may on occasion need its root disk copied (for example, before an operating system upgrade, as a backup copy). On the other extreme, a "must never fail" server of an extra-paranoid systems administrator can provide the most uptime by having RAID-1 of the root disk, and a nightly static copy of the same disk.

This static copy can protect you when RAID-1 cannot. For example, a new systems administrator may make a mistake on the root disk, deleting files or making inappropriate edits to system files. RAID-1 will happily, immediately, replicate the mistake to the mirror copy. With a static nightly copy, undoing the mistake is fairly trivial. Likewise, a security incident that effected the root disk could be undone by booting from the previous night's copy.

There are many, many scripts floating around that help perform this manual mirror. The script presented below is the simplest and most effective that I have found. It performs the manual root disk copy that was mentioned last month. By periodically firing it off from cron, it can be used as a fail safe, even in cases where other methods are used as well. It can also be used ad hoc when a snapshot of the current root disk is needed. For example, you could run the script before attempting a serious patch installation or even operating system upgrade. Should you be unhappy with the result of the system changes, you could reverse the process to reset the root disk to pre-change state.

Do not use this script without customizing it or it could have a negative impact on your system's functionality. Be sure to test it in your environment and on your systems before counting on it. Watch the script the first time it runs on each system to be sure the execution is clean. For example, start it with sh -xf to watch each line execute. And before first use, be sure to back up the target system.

This script assumes that it is called /opt/local/etc/rootcopy. Fundamentally, it performs the following steps (words in capital letters are variables in the script):

• It makes MOUNTDIR, a mount point for the duplication, if it doesn't already exist.
• It duplicates the SRC disk's partitions onto DEST disk.
• It makes DEST bootable.
• It does a newfs on each DEST partition.
• It does a ufsdump pipe to a ufsrestore for each partition.
• On the root DEST partition, it modifies /etc/vfstab to change each mention of SRC to DEST.
• On the root DEST partition, it renames itself so it will not run accidentally if you boot off of the DEST disk.
• It echoes some state information, for logging purposes.

Without further ado, the script:

#!/bin/sh
# Original Script written by Constantin Ionescu
# Modified by Carlo Cosolo
# Modified by Peter Baer Galvin
# Modified by John West
# Use and distribute freely

# Define variables for use in the script
# ! Important, these must be set correctly !

# The root disk to duplicate (leave off slice numbers and path)
SRC=c0t0d0

# The empty disk to duplicate it to (leave off slice numbers and path)
DEST=c0t1d0

# The directory to mount destination partitions on while duplicating
MOUNTDIR=/dup_0

# The file name of this script, to rename it on the destination to avoid execution
SCRIPT=/opt/local/etc/rootcopy

# The slices that should be copied
SLICES="s0 s3 s4 s6 s7"

echo ====================================
echo Disk Copy script started `date`
echo

# Make sure the mount point for duplicate partitions exists
if [ ! -d $MOUNTDIR ]; then
  mkdir $MOUNTDIR
  chmod 700 $MOUNTDIR
fi

# Partition the duplicate disk, make filesystems, make it bootable
prtvtoc /dev/rdsk/${SRC}s2 > /tmp/vtoc
fmthard -s /tmp/vtoc /dev/rdsk/${DEST}s2
installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/${DEST}s0

# Modify the following loop to handle any special cases
for fs in $SLICES
do
  newfs /dev/dsk/${DEST}${fs} < /dev/null; mount /dev/dsk/${DEST}${fs} ${MOUNTDIR};
  ufsdump 0f - /dev/dsk/${SRC}${fs}|(cd ${MOUNTDIR}; ufsrestore rf -);
  if [ $fs = "s0" ]; then
    sed 's/${SRC}/${DEST}/g' /etc/vfstab > ${MOUNTDIR}/etc/vfstab;
    mv ${MOUNTDIR}/${SCRIPT} ${MOUNTDIR}/${SCRIPT}.DONTRUN;
  fi
  umount ${MOUNTDIR}
done

echo
echo Disk Copy script ended `date`
echo ====================================
echo

Failed Disk Blues

But what if a disk fails, you need its contents, and you have not created a mirror copy or backed it up to tape recently? Such a thing can happen to even the most experienced systems administrators. Long-time friend Carole Fennelly had a disk in her system go bad. Unfortunately, it was somehow missed in her backup rotation. Rather than gnashing teeth and pulling out hair, she sent the disk off to Ontrack. Within a few days, they were able to restore 99% of the data from the failed drive.

A Book for Your Manager

Secrets & Lies (Bruce Schneier, John Wiley & Sons) is a very good book. It's the type of book that you read even if you already know most of the points being made. Bruce Schneier is an entertaining, effective, and informative writer, and he out does himself with his latest book. His points, for people intimate with computer security, are fairly basic. Security is a process; there are lots of bad guys; all they need is a small opening to get into your systems, and so on. But Schneier makes the points authoritatively, memorably, and reasonably. Even if you are a computer security maven, Secrets & Lies is worth picking up. Give it a read, and then pass it along to your boss, or someone else you want to scare into increasing security budgets or allocating more time to security efforts.

DiskSuite and Security

Last month, I recommended the use of DiskSuite for root disk mirroring. Carl Marino pointed out that the Yassp security tool documentation (http://www.yassp.org/after.html#RPC) says that RPC is a security hole, (which it is) and that DiskSuite uses RPC. As far as I can determine (again with help from my friends), RPC is not used by the core DiskSuite disk management software. It is probably used by the GUI, and possibly by the login daemon. In sites where RPC is going to be disabled (any machine on which it is not a necessity) simply use the command-line interface and avoid installation of the logging daemon, and DiskSuite will be functional and secure.

Fix to and Comment about Previous Columns

Also from last month, there was a typo in one of the commands. Rather than vcdctl add disk c0t0d0s7 type=simple It should read vxdctl add disk c0t0d0s7 type=simple Sorry for any confusion that might have caused.

In another section last month, I reference /etc/opt/SUNWmd. Note that as of DiskSuite version 4.2.1, the installation directory moved to /etc/lvm, so all references to the former directory should be changed to the latter.

Thanks for the feedback from Selwyn Schultz, who reported I saw your article root disk mirroring on the Web today. I have in fact experienced the problem with the root disk failing when it was mirrored with Volume Manager and the system coming to a screeching halt. To get the system booted I had to boot off the secondary disk from the prom level (that did work). Replacing the disk was quite complicated involving unencapsulating the root disk, copying the OS back to the root disk, installing the boot sector and then encapsulating the root drive again.

My experience in talking with many systems administrators, as well as administering many systems, is that DiskSuite is more reliable than Volume Manager for root disk mirroring.

Netra X1 and Its New Network Device

For Sun machines to work properly at 100-Mb full duplex networking, I always recommend that these settings be "hard wired" in /etc/system or via ndd commands in the startup files. Too many times, a slow system has resulted from an improper negotiation between a Sun and its networking devices if they are left to their own, well, devices.

As it turns out, the new Netra X1 uses a new chip set for its built-in networking. This new "Davicom" device has a different way of hardwiring its settings. This fact is not yet well documented, so I thought I'd mention it here. Thanks to Manny Korkodilos for pointing this out. The details are part of Sun service order #62507881 on Sunsolve. The commands you will need to set the interface to 100-Mb full duplex include:

ndd -get /dev/dmfe0 link_status
( 0 = link down, 1 = link up )
ndd -get /dev/dmfe0 link_speed
( 100 = 100 megabit, 10 = 10 megabit, 0 = link down )
ndd -get /dev/dmfe0 link_mode
( 0 = half duplex, 1 = full duplex )

ndd -set /dev/dmfe0 adv_100fdx_cap 1  (enable 100 megabit full duplex)
ndd -set /dev/dmfe0 adv_100hdx_cap 0    (disable 100 megabit half duplex)
ndd -set /dev/dmfe0 adv_10fdx_cap 0     (disable 10 megabit full duplex)
ndd -set /dev/dmfe0 adv_10hdx_cap 0    (disable 10 megabit half duplex)
ndd -set /dev/dmfe0 adv_autoneg_cap 0   (turn autonegotiation off)

Next Month

Thanks for tuning in this month. Next month, I'll take a look at the new RBAC tools built in to Solaris 8.

If you have feedback about this column, or feel that you have important information to share with your fellow Solaris administrators, please get in touch (mailto:pbg@petergalvin.org). We are looking for articles, book reviews, helpful tips and tricks, conference reports, and of course useful resource pointers. Fame and fortunate (well, at least a little of each) are yours for the asking.

Peter Baer Galvin is the Chief Technologist for Corporate Technologies, a premier systems integrator and VAR. Before that, Peter was the systems manager for Brown University's Computer Science Department. He has written articles for Byte and other magazines, and previously wrote Pete's Wicked World, the security column, and Pete's Super Systems, the systems management column for Unix Insider. Peter is coauthor of the Operating Systems Concepts and Applied Operating Systems Concepts textbooks. As a consultant and trainer, Peter has taught tutorials and given talks on security and system administration worldwide.