Network Strategy and Design Final assignment disaster rec

IT19 (Network Strategy and Design)


Level 9,815 George street Sydney NSW 2000 Australia

Department of InfrastructureDISASTER RECOVERY PLAN REPORT

Dec 2009

ã Department of Infrastructure (AICL)

All rights reserved. No part of this publication may be reprinted, reproduced, stored in a retrieval system or transmitted, in any form or by any means, without the prior permission in writing from the AICL Department of Infrastructure.


Contents

Introduction:-....................................................................................................................4Overview..........................................................................................................................4Need for Disaster Recovery (Purpose)............................................................................5Scope:-.............................................................................................................................6Roles and Responsibilty of Disaster Recovery Team......................................................6Risks:...............................................................................................................................8Analysis............................................................................................................................9Establish Role Departments:-........................................................................................10Solution..........................................................................................................................10Policies and procedures.................................................................................................11Preparing to Handle Disaster.........................................................................................12DR plan procedures:-.......................................................................................................9Testing the disaster recovery plan.................................................................................13Conclusion:-..................................................................................................16 Glossary of Terms:.........................................................................................................16....................................................................................................................... Reference:-.......................................................................................................................................16


Introduction:

In 2004, AICL formed a partnership with TAFE NSW - Sydney Institute to sequentially deliver the Advanced Diploma of Hospitality Management. AICL started delivery in February 2005 at TAFE Loftus campus, but the partnership became so successful within 12 months it delivered this course on four campuses:

- AICL - Loftus TAFE - Mount Druitt TAFE - Padstow TAFE

As hospitality is very important industry where Australia is suffering from a skills shortage, AICL have actively participated in industry to give concrete support. Students have been placed in industry working part-time in some very prestigious establishments to consolidate the skills they are acquiring whilst studying on campus.

AICl has won three consecutive awards since 2006. AICL was awarded the Tourism Training Australia "National Training Legend Award" for outstanding achievement in delivery and positive contribution to the hospitality industry.

In 2007 AICL won the Australian TAFE Marketing Association (ATMA) Award for Business Development - Innovative Commercial Service Category. An award titled The Minister’s Student Achiever Award (for Tourism and Hospitality) was won by AICL in 2008.

Overview

Planning for the business continuity of an organization in the aftermath of a disaster is a complex task. Preparation for, response to, and recovery from a disaster affecting the administrative functions of the organization requires the cooperative efforts of many support departments in partnership with the functional areas supporting the "business" of DOI.

This document proposes disaster recovery plans to address various types of possible disaster scenarios. The plans reflect the analysis and determination of appropriate responses as agreed in discussions with representatives from Corporate IT and other departments.

This document is intended to provide a framework, with some possible solutions, of the backup and disaster recovery plans for the DOI project. As with all disaster / recovery situations, not all variations can be documented.


Why Disaster Recovery?

Planning for the business continuity of Disaster Action Team (DAT)/DOI in the aftermath of a disaster is a complex task. Preparation for, response to, and recovery from a disaster affecting the administrative functions of the organization requires the cooperative efforts of many divisions in partnership with the functional areas supporting the "business" of DOI.

The objectives of a disaster recovery plan for information services are to make sufficient preparations, and to establish a sufficient set of agreed upon procedures, for responding to a disaster or emergency, in order to minimize the effect upon the operation of the business.

Need for a Disaster Recovery Plan

Three areas need to be reviewed: legal responsibility, financial loss and business service interruptions.

Legal Responsibility : Management has a legal responsibility to protect its corporate resources and information.

Financial Loss : Because of the efficiency, accuracy, speed and control of information services methods, organizations are more dependent on their information services in normal business operations. If the information systems services break down, a great financial loss to the company could develop, or even destroy the business if proper disaster planning has not been done.

Business Service Interruption : This can be very damaging to future relationships with customers. It can also affect the public image of the organization. The costs of not taking precautions could be much more damaging and costly than modest preparation for disaster recovery.

Purpose:The purpose of disaster recovery/business resumption planning is to assure continuity of computing and telecommunications operations needed to support critical agency functions. The business resumption plan should aim at achieving a systematic and orderly resumption of all agency computing and telecommunications services. The plan should provide for restoring service as soon as possible. Those functions that are most critical to achieving the agency mission must remain in operation during the recovery period.

Scope:

These guidelines apply to all executive and judicial branch agencies and educational institutions, as provided by law, that operate, manage, or use IT services or equipment to support critical state business functions.


Guidelines:

Emergency response/problem escalation procedures prescribe how to respond to two kinds of situation:

Disaster events : Fires, floods, earthquakes, and bombings are examples of disaster events. They often take the form of unforeseen events that cause damage or lengthy disruption or threaten to do so. One can more readily recognize the situation is a disaster during this type of occurrence.

Problem : A disaster may evolve from a problem that disrupts normal operations and then worsens or continues so long that disruption becomes critical.

Roles of Team Organization:

Planning for the business continuity of DOI in the aftermath of a disaster is a complex task. Preparation for, response to, and recovery from a disaster affecting the administrative functions of the organization requires the cooperative efforts of many divisions in partnership with the functional areas supporting the "business" of DOI.

The following personnel are required to be present during pre and post recovery process.

Responsibilities of Roles

IT System Administrator(Roshan B)

Responsible for the verification and operational maintenance of the system at

the Server level

Shutdown of the Workgroup(TRIM) and Master services

Shutdown of the Fulcrum Indexer, PDF Generator and Encapsulator services

Reconcile Fulcrum and TRIM database with the execution of the maintenance tool

Perform store check

Perform shakeout testing

Execute SQL query to determine missing records at the SQL Server database level with assistance from the SQL Server DBA


Execute SQL query to remove unwanted record information at the SQL Server database level with assistance from the SQL Server DBA

Identifying and recovering missing files from backup or workgroup server cache with assistance from the SQL Server DBA

Records Manager / System Administrator

Responsible for the verification and operational maintenance of the system at

the business level

Notify all users of the DR procedures, advising them to log off and verify the process

Identifying records to be recreated

Verification and maintenance of the records at the TRIM level

Identification and removal of information from the system after the database restoration process for records supposed to have been expunged or purged with the assistance of the SQL Server DBA

SQL Server DBA

Responsible for the operational maintenance, backup and restoration of the

SQL server database.

Daily full backup of the SQL database

Hourly backup of the SQL log dump

Backup of the corrupted database

Backup of the SQL log files

Restore last SQL backup from tape

Application of SQL logs at SQL Server level

Daily full backup of File System information with assistance from the IT System Administrator and NT Administrator

NT Administrator

Responsible for the maintenance of the system hardware, communications, security

and network operation

Execution of standard hardware maintenance

Maintenance of the Server hardware environment including communication, network, etc


Risks:

There are many natural and Human made threats to service areas which could cause business interruption. Potential threats to consider include personnel, physical environment, hardware/software systems, telecommunications, applications, and operations.

Disaster Recovery Team proposed to develop a schedule back up within a week and help to maintain the Faculty as well as student database of the organization. The major challenge faced by the IT department was to be able to restore the original applications and database without having to go through the whole process of installation which would take much longer in the event of a failure. Specifically, they needed to identify and tackle a large number of system issues such as which processes to stop, which and who’s files to modify and which steps to automate or perform manually at the time of the recovery.

Threats affecting contingency planning.

Natural hazards:

Earthquake

Tornado

Flooding

Landslide

Volcanic eruption

Lightning

Smoke, dirt, dust

Sandstorm or blowing dust

Windstorm

Snow/ice storm

Accidents:

Disclosure of confidential information

Electrical disturbance

Electrical interruption

Spill of toxic chemical

Environmental failure:

Water damage

Structural failure

Fire

Hardware failure

Liquid leakage


Operator/user error

Software error

Telecommunications interruption

Intentional acts:

Alteration of data

Alteration of software

Computer virus

Bomb threat

Disclosure of confidential information

Employee sabotage

External sabotage

Terrorist activity

Fraud

Riot/civil disturbance

Strike

Theft

Unauthorized use

Vandalism

ANALYSIS

Technology and telecommunications systems are becoming increasingly important for businesses to perform their most basic business functions. Disruptive events, such as natural disasters, intentional or unintentional errors in human judgment and vulnerabilities in computing hardware or software, can be so disastrous and debilitating to a business that it renders itself inoperable.

With a rising number of threats and an intense competitive business landscape, it is becoming increasingly important that your organization have the ability to withstand a disaster. Certain organizations also have implied legislative requirements to fulfill various disaster recovery and business continuity obligations often resulting in additional costs and complexities.

Research shows that of those businesses that spend less than 5% of their IT budget on disaster recovery strategies, less than 50% are likely to reopen or remain in business within 2 years of experiencing a major loss of business information and less than 6% will survive long term.


As such, the need to implement a Disaster Recovery Plan to protect business information and core technology platforms are becoming an increasing priority for many businesses that want to operate in the unfortunate event that a disaster occurs

Establish Role Departments:

Disaster Recovery is all about planning and having a sound strategy for data protection. Our Disaster Recovery plan is relatively simple. Your data is stored within our Data Centres and we adhere to industry best practices for Disaster Recovery using the latest and most reliable technology, systems and procedures.

Emergency ResponseThe strategies selected must provide a sufficient base upon which procedures can be devised which afford all personnel the immediate capability to effectively respond to emergency situations where life and property have been, or may be, threatened or harmed.Backup OperationsMost backup sites will not have sufficient equipment, personnel, supplies, etc., to sustain the complete operational requirements or another facility. In this case, a more detailed backup strategy must be developed.

Server Farm - Load Balanced Infrastructure Data Centers provide High Availability through the logical allocation of Hardware resources to different users, minimizing the impact of a hardware failure or disaster. Multiple servers are grouped together. Applications can be configured to be delivered from all servers in the farm or any subset easily from the application publishing allowing for simple load balancing across the available platforms.

Virtualization Infrastructure: Traditional disaster recovery plans require many manual, complex steps to allocate recovery resources, perform bare metal recovery, perform data recovery, and validate that systems are ready for use. Our Infrastructure service eliminates many steps in the process and simplifies the recovery process.

Virtualization removes the need to have the correct hardware configuration, patches and firmware in place and eliminates the challenges associated with recovering the operating system and installing applications on different hardware.

Virtualization converts a traditional Disaster Recovery Process and environment, including hardware configuration, firmware, operating system install and application install, into data stored in just a few files on disk. Protecting a complete system is just a matter of protecting a few files using backup and replication software. The files that comprise a virtual machine can be recovered to any hardware without requiring any


changes because virtual machines are hardware-independent. Servers can be reprovisioned in minutes not days in the event of a physical layer failure.

Policies and procedures:

The disaster recovery policy must be reviewed at least annually to assure its relevance. Just as in the development of such a policy a planning team that consists of upper management & personal from information security, information technology, human resources or other operations should be assembled to review the disaster policy.

Roles and responsibilities of the planning team should be as follows: Perform an initial risk assessment to determine current information systems

vulnerabilities. Perform an initial business impact analysis to document and understand the

interdependencies among business processes and determine how the business would be affected by and information systems outage.

Take an inventory of information systems assets such as computer hardware, software, applications and data.

Identify single points of failure within the information systems infrastructure. Identify critical applications, systems and data. Prioritize key business functions.

Project Planning

Get preliminary management commitment.Gets agreement from senior management on the need for disaster recovery/business resumption planning.

Designate a disaster recovery/business resumption manager.Designate a person to manage the agency's recovery from a disaster. The designated individual must have sufficient knowledge of information management and information technology (IT) within the agency in order to work effectively with IT hardware and software, the data centres, and service providers in re-establishing information processing and telecommunications services after a disaster has occurred.

Organize a disaster recovery/business resumption planning team.Organize a team that will be responsible for the detailed technical analysis and planning functions needed for a recovery plan.

Identify individuals from management, data processing, telecommunications, business operating units, and consultants to participate in preparing the disaster recovery/business resumption plan.

Audit current recovery preparedness.Determine what security/disaster recovery/business resumption plans are in place. Identify what planning remains to be done.


Develop the project schedule.Estimate task durations, identify responsibilities, assign resources, and document the schedule for plan development.

Preparing to Handle Disaster:

IT ThreatsBreach of Personal Information• All data owners must report any suspected or confirmed breach of personal information on individuals to the Chief Security Officer (CSO) immediately upon discovery.• Location managers are responsible for ensuring all employees in their unit are aware of policies and procedures for protecting personal information.• Informs the Legal Department and the Chief Privacy Officer that a possible privacy breach has been reported and provides them an overview of the situation.• Contacts the individual who reported the problem.• Reviews the preliminary details with the Legal Department and the Chief Privacy Office.

Denial of Service / Distributed Denial of Service• Inform relevant IT security personnel.• Ensure all communication links are up.• Ensure data integrity.

.• Provide alternate solutions in case primary communication channels are down.

Virus Outbreak• Isolate system, devices, servers.

, etc from the network and switch over to backup equipments.• Report the situation to the Network Security Officer.

Fire/Smoke• Activate the nearest fire alarm. You may find one at development lobby, server room, kitchen and corridor.• Call 16 and report location and source of fire, if known.• If it is possible and safe, turn off all electrical equipments.• Evacuate the building.• After reaching a safe location, contact other responsible departments.

Bomb or other terrorist threat • Call (92-21) 2416626, Bomb Disposal Squad.• If the threat has been received via phone call then keep talking to the caller and try


to get as much information as possible.• Evacuate the building; depending on the instructions provided by the disposal squad.

Power / Connectivity Failure• Switch off all electrical equipment• Determine extent of blackout by calling relevant department engineer.• Make sure that all doors will remain closed before evacuation. Consult the relevant person if any door found unlocked.• Initiate the telephone tree to ensure that everyone arrives at home safely.

Heavy Rain• Relocate the equipment and records to a dry location which may be affected.• Determine extent of the rain and its severity for the building.• Inform the management to take immediate actions for releasing the employees earlier.• Evacuate the basements in case some rain water poured in.

DR Plan/Procedures:

Key Components

An operational Disaster Recovery facility consists of three key components:

Facilities and Infrastructure – the underlying IT infrastructure and data must be structured to be recoverable – this involves physical infrastructure and software

Processes and Procedures – Business Continuity/Disaster Recovery must be incorporated into standard processes and procedures

Operational Business Continuity/Disaster Recovery Plan – there must be an operational and tested plan to recover


A fundamental requirement of an operational Business Continuity/Disaster Recovery plan is a High Availability IT infrastructure. This both tolerates some outages and minimizes the impacts of serious events by providing easy and fast recovery. A well-designed infrastructure will is a key enabler to achieving effective Business Continuity/Disaster..Recovery.

Increased data dependency between systems and applications mean than an increasing volume of data is critical for effective recovery.


Testing the disaster recovery plan:

The Recovery Planning Process

There are nine major phases in the recovery planning process:

1. Project Planning: Define the project scope, organize the project, and identify the resources needed.

2. Critical Business Requirements: Identify the business functions most important to protect, and the means to protect them. Analyse risks, threats, and vulnerabilities.

3. Recovery Strategies: Arrange for alternate processing facilities to use during a disaster. Make sure to store copies of computer files, work-in-process, software, and documentation in a safe place.

4. Emergency Response/Problem Escalation: Specify exactly how to respond to emergencies and how to tell when a "problem" has become a potential "disaster."

5. Plan Activation: Determine procedures for informing the right people, assessing the impact on operations, and starting the recovery efforts.

6. Recovery Operations: Develop the specific steps for reducing the risks of an outage and restoring operations should an outage occur.

7. Training: Make sure everyone understands the recovery plan and can carry it out efficiently.

8. Testing: Make sure the plan works effectively.

9. Plan Maintenance: Make changes and additions to keep the plan current.

Conclusion:-

Changes to organizations occur all the time. Products and services change as do their method of delivery.

The increase in technological based processes over the past ten years & particularly within the last few years, have significantly increased the level of dependency upon the availability of systems and information for the business to function effectively. These changes are likely to continue, and it is likely that the only certainty is that the pace of change will continue to increase.

It is necessary for the disaster recovery plan to keep pace with these changes in order for it to be of use in the event of a disruptive emergency. To ensure this, the disaster recovery plan update process must be properly structured and controlled. Further, whenever changes are made to the plan they are to be fully tested and appropriate amendments should be made to the training materials. This will involve the use of formalized change control procedures under the control of the plan's owner.


In short, update of the plan should not only be a formal process in its own right, but must be part of business as usual.

Glossary of Terms:

Term Description

API Applications Programming Interface

DR Data Recovery

Encapsulator Process that converts objects

GUI Graphic User Interface

HTML Hyper Text Markup Language

IE5 Internet Explorer version 5

PDF Portable Document Format

TRIM Tower Records and Information Management Software

URL Uniform Resource Locator eg. Web address

XML extensible Markup Language

References:

Disaster Recovery, viewed on 20dec 2009

http://searchenterprisewan.techtarget.com/sDefinition/0,,sid200_gci752089,00.html

Disaster recovery, Disaster recovery, viewed on 21dec2009, http://en.wikipedia.org/wiki/Disaster_recovery

Disaster Recovery Plan, viewed on 2dec 2009, http://www.anbg.gov.au/cpbr/disaster-plan/

Risk Analysis, viewed on 20dec , 2009http://www.devx.com/security/Article/16390


DISASTER RECOVERY PLANNING FROM A-Z viewed on viewed on 25 Nov 2009http://www.disaster-recovery-guide.com/