How to Avoid Business Downtime: Four Lessons from a Formula One Pit Crew

You’re zipping around Monza at the Italian Grand Prix, slowing to the corner of the first chicane when your carbon-carbon disc brakes overheat and explode along with your rear tire. Up until then you were leading the pack by half a lap with 10 to go. What next?

In an ideal world, this happens: Instantly, your pit crew appears with a fresh car—completely identical down to the PSI on the Pirelli tires—and three seconds later you’re back in the cockpit and not out of the race.

Winning a Grand Prix usually comes down to a few seconds of precise team coordination. When Lewis Hamilton is right on your rear wing, you don’t have time to pull into pit lane and replace a shredded tire.  Although I’m not a professional driver, I have pushed my own Lotus Elise around numerous race tracks at high speed. I can tell you first hand that any kind of failure at speed is both scary and very time-consuming.

As the Formula One circuit ratchets up around the world, it’s clear to me that businesses can draw several parallels with car racing and their disaster recovery (DR) strategy. Using four lessons from Formula One, it’s possible to have a real-time support crew on hand the moment something goes wrong in the data center.  And, just seconds after a failure, the support crew solves the problem and the business gets back to what it does best: winning.

Start the recovery engine with a plan

The first, most obvious point is achieving 100 percent uptime. No driver or business can afford the luxury of downtime when they have their shredded tire moment—for a business most often in the form of a failed production server.

As any glance at the news will tell you, enterprises of all sizes are at the risk of data loss due to disasters, but the problem is, like exploding brakes and blown tires, data disasters are unplanned and unexpected—and usually aren’t caused by tornadoes. Although advance planning cannot eliminate or prevent all unexpected events, it can provide an edge in overcoming any long-term consequences like lost sales information and damaged internal records. To take the checkered flag, you have to plan ahead and consider fuel stops, tire wear and car weight in the race strategy.

Similarly, a DR plan must be built for speed, agility and endurance. It should be designed to protect all of your files and records, including the physical and virtual servers themselves. The plan should provide a framework with the capability to quickly retrieve information and virtually replicate your business. This will allow your operations to continue at a new location, if necessary. However, the infrastructure for DR often sits underutilized, so in these times of tight budgets and staffing it’s critical to get more value out of your DR strategy even when you’re not experiencing downtime.

Use purpose-built technology – from onsite recovery to DR clouds

To achieve these goals, the organization must have technology in place that will be the support crew in the event of a disaster. Formula One cars use bleeding-edge components from Kevlar tires to carbon brakes and advanced energy recovery systems to help drivers achieve top speed while staying in control under stressful conditions. Likewise, to keep their businesses viable and running, companies need to implement solutions designed specifically for disaster recovery and continuity, not bolt onto existing backup tools.

For instance, advances in virtualization and cloud technology today are making hybrid DR clouds and DR-as-a-service (DRaaS) an effective choice today. Because DRaaS doesn’t have the physical infrastructure and synchronization requirements associated with traditional disaster recovery, it’s a flexible option. A hybrid cloud-based solution combines on-premise hardware, public cloud and SaaS (News - Alert) automation software to make continuity planning easier than ever. The DR cloud provides companies with data backup, fail-over of servers and the ability to have a secondary data center at a different site to allow for regional disaster recovery.

Put recovery capabilities to the test

DRaaS solutions give IT computing capacity on standby to recover applications if there is a disaster. This can be easily tested without impacting the production servers or unsettling the daily business routine.  A so-called ‘sandbox’ copy is created in the cloud, which is only accessible by the system administrator. These copies are created on demand, paid for while being used and deleted once the test is complete.  This makes testing simple, cost effective and does not disrupt the business. After all, you wouldn’t think of starting a Grand Prix season without first testing every component, would you?

There’s a method to conducting cornering tests and live DR tests. You can test DR, software updates and other configuration changes to your business servers every day without missing a beat. Test cases can be performed against the recovery systems in as little as 15 minutes depending on the application, often with no incremental costs. Applications and services are immediately available for other uses, enabling businesses to efficiently adopt cloud infrastructure or speed time to production for new applications and initiatives.

Think outside the box: create value beyond DR

Beyond the operational advantages, there are financial benefits to cloud-based testing. Service providers regularly offer sliding scales for DR testing. Disaster recovery on demand allows businesses to only pay for what you need, when you need it. Putting your DR solution in the cloud also means there isn’t a redundant in-house infrastructure that is sitting unused most of the time. You can prioritize recovery based on the level of protection you require for each server without wasting time and money on recovering systems that aren’t critical for competing with the pack.

Another challenging part of a DR plan is to get employees to know what to do if an outage occurs. Pit crews and employees learn by repetition, so just like in the pit lane we have to create practice drills, which are critical to a DR plan. Companies who don’t do these regularly should not be shocked if their employees don’t respond appropriately and panic when a server goes down.  But you will still find more companies with self-hosting based DR services hoping for the best.

Finishing first in Formula One requires a strategy for managing corners, straightaways, pit stops and the drivers themselves. To win in a competitive market, the smallest to the largest enterprises must plan holistically for recovery using onsite and cloud techniques. It’s clear that most CFOs and IT leaders understand the multi-faceted need for disaster preparedness but have previously found it difficult to formulate a DR plan. The main barriers to implementation are now broken down by DRaaS. It not only addresses recovery plan goals, it also supports regular testing without the traditional overhead costs and logistical nightmares. You can protect your company against data loss and have peace of mind that whenever you need to implement a new business process or application, it’ll work the first time, every time. But you must plan well, keeping your operations running smoothly until you need to recover from a disaster, without getting boxed in by traditional backup and recovery methods that just don’t cut it anymore. Be smart with your DR plan, and you’ll have a clear shot at taking the checkered flag.

About the Author: Dr. Marc H. Goroff has over 20 years of experience in technology and academia. Prior to cofounding Quorum, Marc was the head of Software Development at Sandpiper Networks where he led the team that created the internet's first Content Delivery Network. Through mergers and acquisitions, Marc continued to lead software development at Digital Island, Exodus, and Cable & Wireless (News - Alert).