Natural disasters have recently been very active in North America! It started with the F3 Tornado that tore apart Goderich, Ontario, Canada. A couple days later, the 5.9 magnitude quake shook up North Eastern USA and the South Eastern provinces of Canada. Finally Hurricane (now Tropical Depression) Irene passed directly over the Eastern Seaboard flooding low-lying areas and causing billions in damage. Such devastation brings up a common question; does your company have a Disaster Recovery Plan?
On Friday, August 26, 2011, I was watching Hurricane Irene on radar. Taking screen shots from datacentermap.com and tornadovideos.net into Photoshop, I realized that Irene was going to pass directly over 200+ data centers. Thinking about the Data Center Managers that were in for a long stressful weekend, I turned to the web and ask members of several Data Center groups, What do Data Centre Managers need to do to prepare for such an intense storm such as a Hurricane?
John Wallerich, Principal Architect, Office of the CTO at Netapp Inc in San Francisco replied that, “It requires the one thing most IT operations don't have, a functional and frequently tested DR and BCP plan.” He cautioned that the most difficult part is getting funding for it. Wallerich noted that “some executives see this as akin to snapping your fingers to keep the alligators away.” Wallerich’s “preferred solution, in lack of a multi-facility cloud environment is to turn the DR center into an active production site, and load balance workloads. In the event of a failure, it can then seamlessly (if designed right) take on the full load without loss of service.”
Steven Hegg, a Senior Data Center Systems Engineer in the Greater New York City Area noted that “DR and BCP plans are only as good as the last time they were tested and validated. With larger enterprises, testing can become a cyclical if not continuous process.” Hegg suggested that a risk matrix be prepared to show break even points for “labor/cost budgeting of DR/BCP planning versus probability/cost of the service outage.”
So how does a company develop a Disaster Recovery (DR) Plan? In our next edition, we will explore how to develop a DR Plan and provide resources to develop your own.
Special thanks to John Wallerich and Steven Hegg for their response and great insight.