DR planning begins with commitment

It's a lot of effort, it's an ongoing plan, and you have to constantly update it. Christine Forbes vice president, technology, The Daniel and Henry Co.

At least, that's the advice from IT managers who have had to rely on their disaster plans.

Christine Forbes, vice president for technology at St. Louis-based insurance company The Daniel and Henry Co., turned to her DR plan after a series of summer storms knocked out much of the electrical service in the city in 2006.

"Make sure you have a plan in place, and that you revisit it frequently. We did have a plan, and we realized it wasn't enough," Forbes said. "We were fortunate that we actually had revisited it before the power went out. So when it happened we had a much better plan in place." The DR plan allowed the agency to keep processing storm-related claims.

While the wrath of Mother Nature -- best exemplified by earthquakes and hurricanes such as Katrina -- has drawn attention to disaster planning, plenty of companies still fail to develop and test plans.

"People have started to think more about disaster recovery. But the struggle in SMBs has been how to do it in terms of manpower and money," said Gary Chen, a senior analyst at Yankee Group Research Inc. in Boston. "Disasters are more prevalent than people think. But it's usually the smaller stuff, things like doing construction and cutting power lines or plumbing lines."

Server virtualization brings new life to DR for SMBs Server virtualization has led to some of the biggest improvements on the technology side of disaster recovery (DR) and business continuity planning. "Virtualization has really changed how an infrastructure is run, for the better. Not only does it have general implications in terms of power, cooling, consolidation and management, it has huge implications for business continuity and disaster recovery. It makes it much more affordable for the midrange folks," said Gary Chen, senior analyst at Yankee Group Research Inc. in Boston. With server virtualization, an IT group can run its own backup data center or hot site at a safe distance from the main data center without having to precisely match the original hardware configuration. Virtualization has also allowed partnering. "One company will find another one that is similar to it, and they will be the other's DR site," said Mike Karp, senior analyst at Enterprise Management Associates in Boulder, Colo. He cited as an example Bowdoin College in Brunswick, Maine, pairing with Loyola Marymount University in Los Angeles. The schools back up each other's data centers through virtualization technology from Palo Alto, Calif.-based VMware Inc. DR planning is often based on more established technologies such as rudimentary backup software and services, Chen noted. He said additional functionality, such as true replication, is working its way down from the enterprise to small and medium-sized businesses (SMBs), but such migrations take time. Chen said SMBs are likely to set up their own DR sites, but new services will target smaller companies. "I think you are going to see more sophisticated storage services. So you will be able to do things like SAN replication, and you will see more virtualized services, so you can buy virtual machines in the cloud," he said. Service-based disaster recovery should appeal to SMBs because of reduced hardware and software costs, but advanced virtualized services for SMBs are two years away for storage and five years for servers, Chen said. Emerging tools that test the entire plan, such as those offered by Continuity Software Inc. in New York, should help SMBs, Karp noted. "The real issue with disaster planning is being able to test the plan. Even at the enterprise level almost nobody tests, and when they do test it's under conditions that are nothing like the ones they will have to deal with," he said. -- James M. Connolly

Get business buy-in. Companies that aren't planning for the worst run the risk of losing customers that require vendors to have a proven DR plan, being fined for lack of regulatory compliance, or even facing bankruptcy. Knowing the risks can help a CIO get executive and departmental backing for a plan.

"There is plenty of evidence that shows that if companies go offline for 10 days or more they have a better than 60% chance of going out of business," said Mike Karp, a senior analyst at Enterprise Management Associates in Boulder, Colo. "The CIO has to tell the CEO what the cost is of not doing it. If we spend a half million dollars on this, how much is it going to cost us if we don't do it? You start to lose customers, employees and customer satisfaction levels. All of these things start to pile up on the debit side of the ledger."

First planning steps

Forbes and others offered their advice on how to start preparing for a DR plan:

Make the commitment. "It's a lot of effort, it's an ongoing plan, and you have to constantly update it," Forbes said.

Have good backups. Swamy Gundar, director of IT at Sentry Credit Inc. in Seattle, advised, "Make sure you have backups of everything, most importantly base images of all your servers. If you do have to rebuild, you don't have to spend hours and hours searching for CDs or whatever."

Like Forbes, Gundar is a client of Agility Recovery Solutions Inc., a Charlotte, N.C.-based company that provides mobile data centers. Gundar also suggested IT professionals shouldn't trust tape. He backs up crucial data on at least two types of media in at least two locations.

Know what you need. One key to Gundar's plan is tracking which servers he has, the services they support and what licenses and access methods he needs in an emergency. "There is so much equipment that you may have and you don't realize you have that you don't realize are critical to a system until they go down," said Gundar, whose list of equipment and applications fills a 10-page spreadsheet.

Test. Forbes advised that companies go through the phone trees that will help to notify employees; practice evacuation of buildings; and test data restoration to make sure data is backed up properly. Gundar added that some of his clients require Sentry to not only have a recovery plan, but also to test it on a regular basis.

Identify which services must be restored immediately. "We have 15 servers here. We had to know what our critical services were. Was our BlackBerry server critical? Probably not, but email was," Forbes said.

Know what you will need if you move to a hot site or a mobile data center. Forbes and Gundar have done that with Agility Recovery. In testing his plan last year, Gundar discovered that he didn't have all of the CDs, licenses and access codes he needed. Forbes discovered that the satellite data services provided through a mobile center required a clear southern exposure.

Leverage relationships. Some of the insurance carriers that Forbes' agency works with had local office space where agency employees could utilize PCs, network access and fax machines. Other employees could work from home offices as long as the core servers were up.

Know who has to be involved in a recovery. Gundar said a plan must identify which employees should be in the building or backup data center, who is responsible for tasks such as rerouting phone service, and who should back up those people.

Learn from those who went before. Gundar noted that some organizations, in particular universities, may have posted their DR plans on the Internet. He said he encourages IT managers to use those plans as free guides for building or modifying their own plans.

James M. Connolly is a freelance writer based in Norwood, Mass. He can be reached at jamesmconnolly@verizon.net.