CIOREVIEW >> Disaster Recovery >>

Reasons behind Disaster Recovery Test Failures

By CIOReview | Wednesday, August 10, 2016

Disaster Recovery (DR) brings in a set of procedures and policies to enable the continuation or recovery of vital technological infrastructures and systems after any natural or human-induced disaster. At the time hurricane Sandy hit the Atlantic coast in October 2012, the New Jersey data center of Allied Building Products was flooded with four feet of water within a couple of minutes. All the systems and infrastructure were severely affected; but the company had an effective and well-tested disaster recovery plan in place. Allied instantly switched their operations to a disaster recovery facility by SunGard AS at Philadelphia. All the applications and servers were brought back online and company’s IT infrastructure began operating from SunGard AS’s Philadelphia center until Allied switched to new data center. This episode illustrates that having strong IT disaster recovery plans is the need of the hour for all companies.

According to a report by the National Federation of Independent Business (NFIB), human errors affect 10 percent of small businesses while natural disasters cause 30 percent of total data loss. A simple power failure can put the whole company at risk. A research done by the University of Texas shows that only 6 percent of total disaster hit companies survived from catastrophic data loss while 51 percent close within two years and 43 percent never reopen.

Over 90 percent of the midsized to large companies have business continuity and disaster recovery plans in place, but this is not enough to fully overcome IT disasters because about 60 percent of the plans are ill-protected. Ending up with multiple DR plans can only result in a documentation exercise and will never rise above regular business priorities. The DR testing phases generally come across problems that need to be properly handled to fulfill the norms of testing processes.

Finding the Scope of Failure while Making Recovery Plans

One of the biggest challenges while performing DR testing is to determine the scope of disasters. Is the company testing the right software versions, data integrity, data storage, hardware platform compatibility, and communications? While planning for bigger issues like natural disasters or hardware failures, there is a possibility to miss some of the minor ones, like errors by employees. While making DR plans, every possibility of disaster needs to be covered along with its impact on the company.

The best possible way to resolve the scope issue is by performing a comprehensive analysis that will identify the area of failure. Companies can start by setting up a database, where different teams can share data in real time. After finding a fully fledged list of failure points, the company can develop different strategies to deal with the possible ways of failure testing. A biannual test will never provide the coverage that an organization needs so the test must be augmented with daily tasks to assure that company is ready to face disasters. The selected DR test needs to be integrated into the company’s daily operations.

Plan Fails to Fulfill Prime Motive of Data Restoration

IT managers tend to focus on system restoration tasks like application installs, checking images, and restoring servers, whereas executing a disaster recovery test and remaining backup functions are taken for granted. Periodic tests are sufficient for the data backup tapes. However, it is necessary to have the right data on the tape otherwise the test result will return false positives because the process will not be able to fulfill the prime motive of system restoration tasks.

A DR plan is equally important as the business continuity plan. DR testing needs to have a clear nexus to the business supported data, and testing demands more frequent execution than an annual exercise. Businesses grow at very fast pace and it results in storing data at multiple places and the aftereffects of data loss are significant. Therefore, data backup function needs to execute on regular basis and complete DR testing needs to be incorporated with operations to ensure timely backup of the right data.

Errors Help in Making Strong DR Plans

The infrastructure of a company changes rapidly because of its business extension, and it makes them wide open to multiple environmental risks, from natural disasters to employee turnover. The errors and problems occurred during DR testing phase will help in designing of a strong disaster recovery plan.

The main motive of testing is to identify the points of failure in a safe environment, and the same goes for DR plans. It is better to fail in practice tests instead of the real scenario. For example, during the execution of DR tests, a company gets to know about unavailability of offsite storage facilities which are crucial during an unfortunate incident. Testing is performed to find such shortcomings and resolve it well beforehand.

Benefits of a Solid DR Plan

A company can provide uninterrupted services to customers and save costs by developing a solid disaster recovery plan. Every business is vulnerable to disasters, but a well-crafted recovery plan can provide a speedy recovery. Companies should learn to embrace failures and use the experience to achieve the aim of a DR plan—being prepared for the unexpected.