IT Disaster Recovery Planning

It’s not just hurricanes or earthquakes that can be a disaster for your business. Smaller catastrophes like a burst pipe or smoke damage from a fire in an adjacent office can be just as debilitating.
Whatever the cause or extent, the impact of network downtime or data loss could mean loss of revenue, loss of future business, reduced credibility and customer trust, increased expenses to re-capture lost customers, lower profits, exposure to legal fees and more.

The smart business will therefore have an IT disaster recovery plan in place to minimize financial impact, reduce downtime and speed the return to normal operations. Having an IT DR plan in place also helps eliminate panic. It provides a roadmap of actionable steps written with a calm hand in the event of an unexpected outage or data loss and becomes even more critical when management is absent or incapacitated for any reason.

• Data Backup is the number one best practice. Perform periodic back up of critical files in an easy to access off-site storage medium to maintain current records.
• Distribute a list of key management’s business and personal contact information to select employees in the event of an emergency.
• Assemble a list of emergency contacts for all IT vendors, utility and telecommunication partners.
• Maintain secure employee access privileges to the data center at all times.
• Install an Uninterruptable Power Supply (UPS) generator. Install a second generator or a battery backup system in the event the first UPS fails.
• Contract with multiple ISPs. Have a secondary network connection and internet service provider ready to take over for the primary carrier if disaster strikes their network.
• Position fire suppression equipment appropriately throughout the premises. Check and test periodically and make sure employees know how to use it.
• Install elevated flooring in IT rooms if located on ground level to guard against water damage.
• Enter into a reciprocal agreement with a similar business or organization to share space and access to IT and telecommunications in the event of a disaster.

Step 1: Determine the cause. Is the event a widespread natural disaster, localized fire, human error or hardware failure?

Step 2: Assess damage. What systems are affected? Are other areas of the business still operational? Will data be permanently lost? Are people and property at risk?

Step 3: Estimate length of potential outage. Will the network be down for minutes, hours or days?

Step 4: Determine if the severity of the event, its impact and estimated downtime warrants the initiation of disaster recovery procedures. If so, implement the IT DR plan.

Step 5: Make all internal stakeholders aware of the incident and keep them updated on progress toward recovery. Alert customers, partners and vendors with a potential timeframe for return to normal operations.

Data Center Disaster Recovery. This refers to the traditional on-premises IT department with servers, switches and hard drives physically located in the same facility as the primary business operation. Internal IT personnel must be prepared to handle any event that disrupts business. They have total control over their IT infrastructure, but also must have the manpower, equipment and expertise on site to restore power, replace hardware and troubleshoot telecommunications and ERP systems to keep the business running.

Cloud-Based Disaster Recovery. As computing technologies shift to the cloud, many SMBs and small enterprises are switching to cloud-based service providers for their data storage and real-time data management needs. Businesses employing this strategy need only to maintain a secure, stable connection to the service provider. In the event of a disaster at the business site, all vital records are safely stored in the cloud. 

The trade-off is that the business has no control over cloud performance or disaster recovery capabilities of the provider should the cloud fail. One possible solution for SMBs is to use their on-premises data center as the primary IT infrastructure and enter into a contract with a cloud-based DRaaS provider for disaster recovery/backup purposes.

Virtualization Disaster Recovery. The issues facing companies employing a virtual computer network environment are akin to cloud-based disaster recovery. In this IT model, the entire network including the operating system, applications, patches, updates and all data are stored and run on a software-based server. This means there are fewer physical devices to track and repair, but companies must have a plan in place to backup data and replicate the virtual server to access the network from an off-site location in the event the primary location becomes inaccessible. Then, all that is required in the event of a disaster or network outage is a process to flip the failover switch to a new virtual host.

• Begin with a Business Impact Analysis (BIA). This is a risk assessment exercise that identifies all time sensitive and mission critical functions and processes in each business unit and the resources required to support them. 

• Create an emergency response team (ERT). Develop a contact list of the most critical employees who will be called in to work first during a crisis. ERT members should have the expertise to implement the processes to restart/restore operations in their respective department.

• Set a disaster threshold. Define the level of event severity that qualifies as a disaster to trigger implementation of the plan. Pinpoint a recovery time objective (RTO).

• Draft the plan. Use the information from the BIA to document the processes, milestones and/or manual workarounds needed to recover data and restore IT operations in order of the identified system priority.

• Publish the plan. Make sure each ERT member has hardcopy of the plan at the office and at home. Mid-level employees should know where to find the plan if needed. Put a copy of the plan online on a separate digital platform.

• Establish damage assessment and recovery/restart roles and responsibilities. Ensure that each individual has the managerial and/or financial authority to initiate restart processes or replace/repair damaged equipment as outlined in the plan.

• Transition back to normal operations. Outline the minimum requirements needed to function before transferring operations back to the primary business site.

• Updating and periodic testing. Regularly review and update the plan to reflect any changes to software applications, operating system versions, hardware and personnel. Test the plan periodically through drills and event simulations.