How a DRBC plan can save your business
According to statistics found on Statista (www.statista.com) the cost of a data loss for American businesses is upwards of $7.35 million – per occurrence. That cost doubled in a 10-year period ending in 2017, because of digital dependence.
When we asked business analysts, managers, and CEO’s to estimate the value of their data, they all gave a singular answer. Priceless.
Disaster recovery and business continuity – aka DRBC — is the way an organization can prepare for and aid in disaster recovery. It is an arrangement agreed upon in advance by management and key personnel of the steps that will be taken to help the organization recover should any type of disaster occur.
Data loss occurs when least expected, for reasons never thought of. That’s why DRBC plans must be put in place and practiced. After all, you only get one shot at getting it right.
When the backup doesn’t get you back up
ATC Trailers manufactures 5,000 trailers of various shapes and sizes annually. Nestled in Nappanee, it’s the mom-n-pop shop that grew to 270 employees and three manufacturing plants all within a few miles.
Ten years ago, ATC was 40 percent automated and 60 percent paper driven, according to Dave Kiefer, an IT analyst who joined the company in 2007. “The company was slowly migrating to automation, both in administration and manufacturing,” he says. “We progressed at a steady state but didn’t have a need of urgency.”
Until five years ago, when one of their servers crashed. Kiefer took out the failed hard drive and loaded in the backup. But the server didn’t come back up as expected.
“It took us 36 hours to get back to where we were when the original hard drive failed,” he says. “We were able to recover everything, but for those 36 hours we were very nervous.”
Management decided to build a comprehensive DRBC plan that would copy all data and streamline the process of recovery. “Things like data recovery are outside the scope of production,” Kiefer says. “But I would advise anyone who asked me that you should treat it like your number one asset. Because it really is.”
Easy as 3-2-1
Jim Peterson is the CEO and principal consultant at Kinetic IT Solutions located in South Bend. As an IT guru, he preaches the value of DRBC daily. To help convince his clients of the process he professes, he uses it himself for the critical Kinetic data.
“How can I suggest a client use a process that I’m not using?” he asks rhetorically. “I practice what I preach and use the same process that I always recommend to clients.”
The process Peterson described is software independent. “The 3-2-1 rule is a best practice for backup and recovery,” he says. “It means that when you build out your backup and recovery strategy you should include the following steps.”
- Keep at least three copies of your data, the original and two backups.
- Keep the backed-up data on two different storage types. The chances of having two failures of completely different storage types is small.
- Keep at least one copy of the data offsite. A local disaster could wipe out everything onsite, so keep a copy in an offsite location.
“The 3-2-1 backup rule is a best practice because it ensures that you’ll have a copy of your data no matter what happens,” Peterson says. “Multiple copies prevent you from losing the only copy of your data, and multiple locations ensure that there is no single point of failure.”
An electrifying disaster
Kurt Kruggel is partner and CPA at Kruggel Lawton CPA firm. Their original office is in historic downtown South Bend. With historic buildings sometimes comes historic infrastructure.
“We had a pretty severe power outage in that whole part of downtown,” he says. “As a result, we couldn’t access our data – backup or current – because they were both affected by the outage.
Kruggel Lawton didn’t lose data, but they lost the next most expensive resource, time. “It took us a while to get back up and running,” he says. “It was a nervous time until we knew our data was safe and complete.”
Kruggel Lawton is a full- service CPA firm. They handle personal and commercial tax returns for hundreds of clients of all sizes. The data collected is not only critical, it’s sensitive and confidential. “We decided that an offsite DRBC solution would provide us with security from power outages as well as hardware failures. Now that we have three locations, it’s important to be able to access the data from any location.”
A hybrid solution that includes onsite and offsite data storage was the best way to go.
The benefits of hybrid
Chris Kotul is the division manager at Chester Inc. in Northwest Indiana. He’s a major proponent of hybrid cloud solutions as a DRBC strategy.
“There are numerous advantages to a hybrid cloud solution,” Kotul says. “Here are a few reasons worth noting.”
On-premise appliance costs
“Your local equipment can be used to store the most recent and critical data, so it can be a much smaller backup and cost less than traditional backup appliances used with an onsite only solution,” he says.
Cloud storage costs
“Cloud storage costs are at their lowest prices ever,” he says. “Keeping your long-term data on affordable cloud storage and your most critical data on a local appliance gives you the best of both worlds.”
Recovery time
“Your most critical data is likely to be within your most recent backup,” he says. “Since your most recent backups are stored locally in the on-premise appliance, the most common types of data recoveries can happen over LAN speeds and are pulled from high speed storage.”
Local and Cloud Virtualization
“Having a hybrid system allows us to perform a virtual boot of backed up systems on the local appliance or in the cloud,” he says.
Practice makes perfect
The only way to know if a procedure works is to put it in play. Steve Massa, CEO and president of Golden Technologies in Valparaiso, puts mockups into play on a regular basis. “Mock disaster recovery tests are a critical part of any disaster recovery scenario and at Golden Tech we practice them regularly,” Massa says. “These tests can be as simple as restoring a small group of files to something as large as a full scale spin up of cloud resources that simulate business continuity processes and procedures.”
Massa adds that the frequency of these events can differ based on the accompanying infrastructure technologies that support and prevent outages. “No matter the infrastructure, the importance of these mock scenarios cannot be understated,” he says. “Knowing that your disaster recovery solution and procedures are effective not only provides peace of mind in an emergency but can limit the impact of the disaster by having trained professionals who are prepared to handle the situation.”
After all, no one wants to be figuring out how to recover or if they even can recover from a disaster while in the middle of it.
A sticky situation
REV Renegade manufactures high-end RV’s at their plant in Bristol. The detail involved creates sophisticated blueprints used in production. Those drawings have become digital over the years.
Mike Sullivan a cost account at REV, shared a lesson learned about data recovery. “We had a fire in one of the buildings that stored finished RV’s,” he says. “The sprinkler system kicked in and contained the fire to a singular building. But, as we later learned, the smoke went everywhere.”
The plastics and glues used to build the RV created a thick, black smoke that wormed through the ventilation system to adjacent buildings. That included the server room that housed the backup system.
“The smoke was incredibly sticky,” Mike says. “Even when it dissipated, it left behind this sticky residue. It worked its way into desktop computers and ruined the motherboards.”
Sullivan was able to move the backup system to a room that was on a different ventilation system. REV didn’t lose any data, although they lost several desktops.
“We have employee, production, accounting, and HR data, like any company,” he says. “The fire made us realize the importance of a solid recovery strategy. Part of that strategy included the use of cloud technology.”
Testing the process for function and viruses
It’s also important to test backups quarterly or bi-annually. “I have had clients run into recovery issues,” says Joe Grossbauer, security analyst at GGNet Technologies. “The most common issue is the backup stopped working and no one noticed. It’s important to have monitoring set up to automatically alert if there is a problem with the backups.”
Most businesses rely on their internal data to continue running smoothly. For businesses that maintain client files, a simple hard drive failure or corrupt partition could put them out of action. “That’s why we have both offsite and onsite backup,” says Grossbauer. “Onsite backups can be lost simultaneously as the server or computer being backed up. Fire, theft, or even a sprinkler system can damage a server and its on-site backup at once, causing significant downtime.”
Grossbauer also stresses knowing that your backups are protected against ransomware. “If a computer is infected and ransomware encrypts the files, it will see the onsite backup drive or network and encrypt that too, spreading the virus.” Grossbauer says that spreading ransomware is preventable with an advanced configuration as well as an offsite configuration.
Disaster strikes in every way imaginable. Data recovery means business continuity. But data can only be recovered from reliable backup procedures that are tested and updated as needed. The examples shared in this article were near misses. But everyone was close to true disaster. It’s no coincidence that every near miss resulted in a comprehensive overview and revisions.
Click to read more from the Jun-Jul 2018 issue of Northwest Indiana Business Magazine.
