Week 7 Case Study 2: Disaster Recovery (DR) Lessons Learned September 11th

CIS359: Disaster Recovery Management

Disaster Recovery (DR) Lessons Learned Sept 11th
Introduction
Prior to the 9/11 terror attacks most companies never believed their entire organization much less the computer systems, data, and employees could be completely wiped out by a single event. Most companies thought systems could be maintained and replicated on site and then restored with minimal downtime, but in the wake of 9/11 it became evident the disaster recovery models had to change by incorporating IT and people management into the plans (Gross, 2011).
DR Affects After 9/11 IT became of major component of disaster recovery (DR) because of the lessons learned from the attack. Merrill Lynch, which is one of the largest financial firms in the world, lost one of their primary data centers for over 6 weeks from the attack, and most businesses cannot survive a loss of critical data of this magnitude (Merian, 2011). Companies became more aware of the dangers of having onsite data storage without offsite replication, which would have limited the data losses at Merrill Lynch and made recovery of systems much easier and faster. The ability to lose an entire facility including all critical business data through a single event became a true reality after 9/11 and businesses began exploring offsite data storage, which led to further recognition that these data recovery centers should not be in close proximity to the main business location, because again a single event such as a hurricane or mass power outage could affect both locations. It became important to choose a data recovery site had different risk and risks not associated with the main business location(s) to limit the exposure of the company during major events. As stated earlier prior to 9/11 this was not a major consideration for most businesses that did not reside in areas prone to natural disasters. The second major affect from 9/11 related to DR was the need for people management because it also became clear a company could lose a major portion of their workforce in a single event through loss of life as was the case with Cantor Fitzgerald LP, who lost 658 employees during 9/11, or through employee displacement as a result of the event, which is a major concern during many natural disasters (Merian, 2011). Either way care from employees and their families as well as providing alternative ways for employees to continue working and supporting the company and their families is very important. Through the use of virtual private networks (VPN’s) employees can access the corporate network from anywhere with an internet connection. I am personally a beneficiary of this technology as I am a remote employee, who works from home and manages projects all over the country through my VPN connection. These connections are critical to getting employees back to work to earn a living and to get the business back to normal operations as quickly as possible.
Social Media and DR Social media and other emergency mass notification systems provide companies with the ability to provide real-time notifications and updates to employees, customers, and the general public. These communications are essential for employee safety, work updates, recovery updates, and status of the business as a whole. Most of these systems are internet based, can be activated by a single user, and are integrated with current phone and PC technologies already possessed by the business and employees. Employees can provide personal cell phone numbers or link to company social media accounts to receive these critical updates anytime and anywhere as long as they have internet or phone service. As of 2011 social media was the 4th most popular method for emergency notifications because social media sites are integrated into society and today’s mobile workforces and social media sites can be linked easily to national, state, and local emergency management sites to provide relief information, travel warnings, and other advisories (Lindsay, 2011). Having this flexibility and ease of use for employees and customers during an emergency is critical to business continuity, employee safety, and customer relations.
Geographic Considerations for DR The ability of a single event such as a hurricane, earthquake, or power outage to affect a large scale area is the primary reason for having greater geographic distance between onsite and offsite DR storage or recovery locations. If a company were to design, build, and outfit an offsite DR location that was affected by the same disaster event that knocked the main business location offline, then not only was the investment in the offsite location a waste of money but the company is now at further risk of failure because the planning and expenses for disaster recovery have failed so the business has no path to recovery. Offsite data recovery and storage locations, whether proprietary or co-location sites, should be on separate power grids, have separate network providers, and far enough away where the same disaster risks do not exist for each location. For example, I live in Houston, which is in a known hurricane zone, and many of the companies I have dealt with put the disaster recovery locations for the main Houston businesses further inland in Waco, Austin, Dallas, or College Station out of the hurricane zone. These are big cities within Texas with interstate access or major thoroughfare access, separate power grids, and separate network providers or separate circuits. These locations are convenient to service or to build a support staff in these locations because of the size of the city, education level of the residents, and ease of access to the location by driving or 1hr or less flight times. Recovery time objective (RTO) and recovery point objective (RPO) are important factors in disaster recovery because the shorter these time frames the more expensive disaster recovery will be for the organization (Lindsay, 2011). RTO is the interval between a failure of the critical system and the critical impact on the business resulting in unfavorable consequences. If the RTO is 20 hours then the recovery systems must be back online by 20 hours to prevent further losses to the business. The disaster recovery location must be built to accommodate this need as determined by the business and the shorter the time frame typically the more robust the DR systems must be to meet these times. RPO is the amount of data the company can afford to lose from between the disaster event and recovery. Again this has to be determined by the business and the smaller the amount of data the more real-time the disaster recovery site will have to be in regards to data backups, which is much more expensive than periodic or daily backups because the recovery location is basically a redundant operation of the main site. If this is the case a larger network bandwidth, a larger recovery site, and more equipment and monitoring will be needed to operate the DR site. Data lag between the main location and the DR site may also be a consideration if acceptable data loss is very low.
Cloud Services Virtual servers, which include the OS, software, patches, and data, are complete software bundles that can be easily copied or backed up to a DR site and then brought online on a virtual host very quickly. This simplicity of use and ease of replication make cloud recovery very affordable and enticing to small and medium sized businesses that have limited IT resources and budgets. With all of this information residing in the cloud there is no need for a company to have a redundant data center, additional IT resources, and additional IT infrastructure all of which are very costly to the business (Gsoedl, 2014). The downside of cloud storage is control for the company. The company no longer controls the data and cannot know the data is safe, truly does exist, and cannot control authentication of access to the data. With cloud based services the company is at the mercy of the service provider to protect the data and provide all necessary security measures to do so. Of course the cloud DR provider is liable, but that is would be little consolation to a business owner, who just lost everything because of a failure by the DR services provider. Bandwidth access is also a consideration because the bandwidth needed to push data for storage is much different than the bandwidth needed to access the data in the event of an emergency. If the entire workforce is supposed to access the remote site in an emergency the DR services provider would need to be prepared for this influx of traffic and the company would need to be prepared to pay for the extra bandwidth usage. I do not feel DR cloud storage is right for every company because many companies have the resources and IT skill to build and manage their own datacenters, which service multiple locations company wide. I do feel the benefits of DR cloud services from small and medium sized businesses do out way the risk and as cloud services continue to develop these services will continue to get better. The uses of cloud services allows small to medium businesses to get enterprise level services as a fraction of the cost, which allows the businesses to continue to invest in growth while adequately protecting the companies data.

References

Gross, G. (2011, September 8). 9/11: Attacks Changed the way Companies View IT. Retrieved from PCWorld: http://www.pcworld.com/article/239693/911_attacks_changed_the_way_companies_view_it.html.

Gsoedl, J. (2014, unknown unknown). Disaster Recovery in the Cloud Explained. Retrieved from techtarget.com: http://searchdisasterrecovery.techtarget.com/feature/Disaster-recovery-in-the-cloud-explained.

Lindsay, B. R. (2011, September 6). Social Media and Disasters: Current Uses, Future Options, and Policy Considerations. Retrieved from Congressional Research Service: http://www.infopuntveiligheid.nl/Infopuntdocumenten/R41987.pdf.

Merian, L. (2011, September 9). 9/11: Top Lessons Learned for Disaster Recovery. Retrieved from Computerwold: http://www.computerworld.com/article/2510996/disaster-recovery/9-11--top-lessons-learned-for-disaster-recovery.html.

.