Hierarchical Storage Management

Jeffrey R. Thomas

ierarchical Storage Management, or HSM, has been around almost as long as mainframes. HSM is advertised as the way to migrate older, less frequently used data from expensive magnetic disks to lower cost media such as tape. The theory is that as data ages, the likelihood of user access diminishes at some predictable rate, and the data can be moved to less expensive media with minimal user inconvenience. Unfortunately, the philosophy of this sales tactic is as old as HSM and needs to be ushered into the 21st century. Disks are becoming very inexpensive, Network Attached Storage (NAS) has broadened the availability of data, Storage Area Networks (SANs) are on the doorstep, and managing large tape subsystems can be a daunting task.

HSM, SANs, and NAS are all about data retention and accessibility. HSM is a concept that has been packaged in a variety of ways by various vendors. Vendors, of course, are trying to increase their profits by selling additional hardware and software to provide a complete solution. The bottom line is that HSM is a concept; period. HSM can be implemented on an endless array of hardware provided you have the software to support it.

HSM works by keeping track of a managed filesystem. A managed filesystem usually means that the HSM code is somehow keeping track of and mucking about with the inode information for the managed area. Once HSM has this information, it can migrate files from point A to point B without the OS knowing that the data isn't really where it thinks it is. The actual process used to track and modify inodes is as individual as the various vendors and is beyond the scope of this article.

At some point in every system administrator's career there will be a disk space crunch. HSM is one way to manage disk space. By having a system online that appears to be an infinite sink, system administrators can keep the critical space areas under control. "All of our data is critical!" How many times have these words been uttered? HSM, or any data management scheme will only work as well as the policies that are applied to it. Policies will be discussed a little later on, so for now assume that the users are cooperating and that data can be segregated into critical and non-critical pieces. In this environment, an HSM methodology can provide benefits.

Policies? Tapes? Robots? What a pain, let's just go to the local computer store and buy some 48-GB disk drives for next to nothing and then the problem goes away, right? Enter SANs. A SAN is a mechanism for attaching large numbers of disk drives together with multiple access points for clients. Clients can be computers or tape devices that need high-speed access to the data. SANs take advantage of current RAID technology to insulate from single drive failures. Mirroring, clustering, and snapshots are a few other technologies being offered by vendors to provide fault tolerant data access. NAS vendors are clamoring just as loudly that they have the better solution for data access. SAN or NAS? Good question. Most vendors are still struggling with what is NAS and what is SAN and what is a hybrid. The lines are fuzzy at best, and it will be very interesting to watch the standards evolve and see who emerge as the victors.

A Bigger Bucket

SAN and NAS are analogous to making the bucket of disk space bigger. If there's no way to drain the bucket at the same rate at which it's filling, it's going to overflow. HSM allows you to never (almost) throw anything away. If the user community is one of the rare ones that actually deletes unneeded data, then it may be possible to create a big enough bucket to handle the flow and never have to manage the data. Not likely, but not impossible. The philosophy of HSM compliments whatever storage technology is being used. In fact NAS or SAN components could be used in an HSM implementation.

To continue the analogy, HSM adds a drain to the bucket. How many drains do we need? Since HSM policies usually apply to an entire managed area, we'll need one managed area (drain) per policy. Policies tell HSM under what conditions to migrate data from point A to point B. Points A and B are often referred to as staging levels or methods depending on the vendor. Multiple staging levels can add performance and will always add complexity. For example, a classical HSM approach would have a managed area on high-speed magnetic disk (NAS or SAN). The first staging area would be on optical disk, and the second staging area would be on tape. As data ages or becomes less important based on some algorithm, it begins its journey to tape with a layover on optical disk. Newer tape technologies make it feasible to stage directly to tape in some cases. The number and type of staging levels is solely based on cost and required performance levels. Multiple magnetic disk areas could even be used. For example, 10,000-rpm disks migrating to 5400-rpm disks migrating to....

So, how does the system decide to empty the bucket? Most HSM systems have a user-tweakable algorithm for determining which files are eligible for migration. Once a threshold is crossed, such as percent disk full, migration of files occurs according to this algorithm. Be very careful here. I heard of one experience where an admin applied HSM to a large file server without fully understanding the migration scheme. Over a weekend the HSM system migrated more than 100 GB of data that it thought was non-critical. The design teams using the data didn't agree.

There are two basic options here: 1) User-controlled managed area(s) and 2) Admin-controlled or automatically controlled area(s). In the first case, it is up to the end users to copy non-critical data to a designated area that HSM will sweep up periodically. To the end user, it looks like an infinitely large directory. The second case is where the system scans all of the data on a file server and moves whatever meets the criteria. Both schemes have their places but both need to be fully understood before implementation.

Migration policies define the criteria used for moving data. The age and size of a file are two very common elements used in this calculation. The threshold at which migration takes place is also defined here. This threshold must be set judiciously. If the staging area moves data at 1 GB/hr and the threshold is set to 1 GB free space, then the assumption is that the user community will never add more that 1 GB/hr to the managed space. A managed area with no free space is a bad thing. Policies should also define how long the data is retained online and offline. Most HSM solutions provide a mechanism for allowing very old media to be removed from the stores and then brought back if needed. The expected access time for fully migrated data is also important to the user community. Invariably something will get migrated to HSM that is immediately needed back. For HSM to work well, you must set policies and manage expectations.

Data management is usually a cost-based function. Very few of us still sell disk space to our customers for a profit. HSM systems can have significant startup costs depending on the amount of space they hold. The cost per MB is still lower than for a disk-based counterpart. One cost advantage for systems administrators is that most HSM vendors are also backup vendors and the same hardware can serve both functions. There are some issues with doing this for the long term, but it can serve to determine how well HSM works in a particular situation without a lot of funding.

Solutions

Let's look at several options for storage and compare the costs. Figure 1 shows the cost in $/MB of storage for four different solutions - two that use HSM and two that use online disks only. The first solution is the "Small UNIX" solution. This solution is the simplest, smallest, and has the lowest startup cost. The small UNIX solution consists of a low-end UNIX workstation and a single chain of low-cost SCSI disk drives. This solution provides 100 GB of space for about $0.14/MB with a startup cost under $20K. The next solution is "Large UNIX" solution. This solution involves purchasing a large file server from one of the major NAS vendors such as Auspex, EMC, or Network Appliance. This solution will provide up to 6 TB of disk space for around $0.35-$0.40/MB. All three vendors offer excellent high-availability solutions and, depending on the size, the startup cost can be in the $1M range.

The next solution is the "Small HSM" solution. This involves a low-end workstation; a 50-GB managed space and a 500-GB tape library something like an Exabyte 10I or ATL47000 stacker. This solution provides 550 GB for about $0.09/MB with a startup cost in the $50K range. The final solution is a "Medium HSM" system. It uses a mid-range workstation/server, the same 50-GB managed space and a 4-TB tape library such as an ATL4/52-7000. This drives the cost down to $0.035/MB, with a startup cost of about $125K. All of these prices are for estimation only and will vary depending on vendor and location. The small solutions don't scale well, and the administrative costs increase if multiple small systems are used to create a larger solution. Experience shows that the large UNIX and medium HSM systems can be run in a "lights out" fashion requiring very little administration once they are set up.

Magnetic disk, optical disk, and tape are all players in the data management game. Each of these technologies can be used to implement HSM, and each has cost and performance considerations. For example, one approach to HSM would move data from magnetic disk to optical disk, and finally to tape. The introduction of the optical media as an intermediate data staging area keeps the data directly accessible, but on slower, less-expensive media. Once files have been migrated to tape, providing user access again is a process of the HSM software restoring the individual files, much like restoring from a backup tape. Obviously, to restore data in this manner, you must maintain sufficient free space on the magnetic disks to accommodate the process.

As with any solution that will house production data, some very important considerations need to be addressed. Unlike the UFS data stored on disk, most HSM solutions use a proprietary method for storing the data on tape. Single tar streams and the like don't offer the performance that is required, so enhanced methods are often employed. The fact that the data is now in a proprietary format means that the data is now very dependent on the longevity of one vendor. HSM systems that offer standard file formats may be better choices if portability of data is a high concern. Multiple parallel writes should also be used to make sure that the data is not susceptible to single device failures. If the data is to be kept for scores of years, then we must be sure that we have chosen an optical or tape technology that will be supportable far into the future. Although it's not something we'd want to do very often, the ability to migrate all of the HSM data from one system to another should be investigated with the vendors as well.

Reliability is paramount. Whether we buy a solution that costs 2 cents per megabyte or 20 cents per megabyte, if we can't access the data, we're dead. Storage vendors have invested significant time and effort in preventing a disk drive failure from becoming a data access problem. We all know that a lower purchase price doesn't mean a lower cost solution. An 18-GB disk drive from the local computer store may be great for a PC or other non-critical application but may not make sense for a high-availability production data system. Many disk drive manufacturers publish MTBF numbers for their products. Most drives today boast MTBF numbers of 400,000 hours or better. Understanding the differences between MTBF and actual useful life can be very confusing, but must be considered when deciding on what solutions to implement. Even if a drive has a 3-year warranty, if the failure of that drive causes a major business hiccup, that one drive just became very costly. A good whitepaper on MTBF can be found at:

http://www.quantum.com/src/whitepapers/mtbf/qntmtbf4.htm

SAN, NAS, HSM, and JABOD (just a bunch of disks) methods are all tools that systems administrators have at their disposal. These tools provide the means for creating a data management system. To say that any one tool is better than any other would be rash. All of these tools have their places, and all have advantages and disadvantages. Organizations must define their needs and then apply the correct tool(s) to meet those needs. Single disk drives in the 18-36 GB ranges are relatively inexpensive and can provide archive and even backup solutions. Awareness of these tools is the best ally the systems administrator can have. In this way, the changing needs of the business can be addressed quickly and efficiently.

Conclusion

In general, HSM is still cheaper, $/MB, than low-cost disk drives. The migration threshold (how much data must be kept immediately available on disk before being migrated to tape or other media) and the startup costs for the desired HSM system are what will determine whether HSM makes sense in a particular environment. As SAN technologies emerge, the need for HSM will continue and, it is hoped that SAN vendors will recognize the need to incorporate HSM into the high-speed switching solutions that will make up the SANs of the future. NAS vendors will continue to drive the cost of storage down, and the amount of data that we all keep on line will continue to grow. All of these seem to ensure that HSM will continue to be a viable solution for the terabytes, and eventually petabytes, of data that we'll all be dealing with in the not too distant future. n

About the Author

Jeff is the site IT manager for Texas Instruments, Inc. in Houston, Texas. He has been a systems administrator since 1984. The UNIX environment consists of approximately 450 workstations, 8 TB of active data and an HSM system to provide archive services to the user community. Jeff may be reached at: jr-thomas@ti.com.