Quick Intro:

1 ISACA 2007, Jeffrey Blackmon

ISACA December 13th 2007

Auditing the Disaster Recovery Plan

What should be in a plan, and what should not

By:Jeffrey Blackmon CBCP, CISSP


Quick Intro: Jeff Blackmon, CBCP, CISSP

Started BC/DR planning in mid 80s Financial Petroleum Foreign Military Pharmaceutical

L3 Communications, Titan Group Support of Federal Government Contracts

(Kansas City and DC)


Format:

A little free format style

Open Discussion

Ask Questions


This may be somewhat a little different from the regular presentations

Usually have auditors speaking to auditors

Usually have computer people speaking to computer people

But not in this case


Computer person / business person speaking to the auditors

So expect a little different perspective


Computer Staff


The Auditors


Reason for some of the past relationships between Auditors and the Computer people


Why is BC and DR so difficult?

May not be well defined Big project Expensive Very difficult to take that 1st step


Topics

1. Goals and Reasons for doing Business Continuity and Disaster Recovery

2. What are BC and DR3. RTO/RPO 4. Good DR Plans5. Not so Good DR Plans6. Closing information


Goals and Reasons for BC and DR


Principle Goals

Provide for the safety of all employees

Minimize business downtime


Reasons for Doing BC and DR

Business Best Practices

FEMA Best Practices

Audit Requirements


Reasons for Doing BC and DR

Private Sector FSLIC √ HIPAA OCC √ GLBA Sarbanes Oxley √ NASD 3510

Government Sector FPC 65 √ NIST 800-34 A-123 Audit


Financial Reasons

Company Loss of $84,000 to $90,000 per hour of downtime

90% of companies that experience 1 week of data center down time go out of business within 12 months

(CIO INSIGHT, IDC)


More Financial Reasons‘The cost of being unprepared’

By Jim EllisEnergy $2,817,846Telecom $2,066,245Manufacturing $1,610,654Finance/Brokerage $1,495,134IT $1,344,461Insurance $1,202,444Retail $1,107,274Pharmaceuticals $1,082,252Banking $996,802Food processing $804,192Consumer $785,719Chemicals $704,101Average / hour $1,010,536


Costs(R. Witty, DRJ Fall 2006)


High Startup Costs


What are BC and DR?



DR Plan, what is it? IT Related

Major disruption has occurred that is not part of day to day SOP

Hardware / Software requirements Step by step directions for full

system recovery Very detailed documents required


DR Plan #1 Easy to use

Recovery of all major Computer systems based on Pre- determined priority (RTO)

Details, details, details

(Hardware, software, configurations, communications, disk storage, SAN connections……. )


BC Plan

#1 Easy to use

Recovery of all major business processes

People related Probably many manual processes

to be used for the short term



Plain and Simple

BC/DR are Risk Mitigation

No way to eliminate all risks

Proper planning will reduce the risks to an acceptable level


RTO and RPO


Recovery Time Objective (RTO)

The max allowable time that a business system, application or resource is allowed to be down or offline

RTO is determined by business owners, not IT department


Recovery Point Objective (RPO)

The amount of data that is acceptable to lose since the last successful backup was completed

RPO is determined by business owners, not IT department


Recovery Point Objective Recovery Time Objective

BackupTape Made

BackupTape Made

MidnightMondayNoon

MidnightTuesday

MidnightWednesday

NoonNoon

BackupTape Made

DISASTER

RPO (12 hours)

RTO (24 hours)Standard TapeBackup Recovery


Recovery Point Objective Recovery Time Objective

BackupTape Made

BackupTape Made

MidnightMondayNoon

MidnightTuesday

MidnightWednesday

NoonNoon

BackupTape Made

DISASTER

RPO (2 minutes)

RTO (12 hours, rebuild system)Replicated DataBackup Recovery

$$$ $

Real time replication


Find the Cost Effective Solution

Cost Effective Solution

Time

Costs

Business Interuption Cost Recovery Costs


RPO / RTO Example Major financial institutions on mission

critical systems RPO = 0 hours, on some applications RTO = 2 hours, on some applications

After 96 Hours, major financial institutions will probably not recover

By Jay Ranade, CISSP, CISA, CBCP, CISMPresident, Jay Ranade Consultants, Inc.


RPO / RTO Example Major breakfast cereal producer

RPO = 7 days RTO = 7 days

Put it all into perspective Very regular shipments to distributors by

boxcar Only breakfast cereal, if problems occur, then

re-ship

By DRII Classmate, 1999


RPO / RTO Expectations

‘Usually’ a large gap in management expectations as compared to actual recovery abilities

Talk with technical staff


What a plan should look like


Good DR plans

Be sure you keep in mind that DR plans are to recover computer and network systems


NIST 800-53, Recommended Security Controls for Federal Information System

FAMILY: CONTINGENCY PLANNING CP-1 CONTINGENCY PLANNING

POLICY AND PROCEDURES CP-2 CONTINGENCY PLAN CP-3 CONTINGENCY TRAINING CP-4 CONTINGENCY PLAN

TESTING CP-5 CONTINGENCY PLAN

UPDATE


NIST 800-53, Recommended Security Controls for Federal Information System

FAMILY: CONTINGENCY PLANNING CP-6 ALTERNATE STORAGE SITES CP-7 ALTERNATE PROCESSING SITES CP-8 TELECOMMUNICATIONS

SERVICES CP-9 INFORMATION SYSTEM

BACKUP CP-10 INFORMATION SYSTEM

RECOVERY AND RECONSTITUTION


Good DR plans

Disaster definition Who can activate the DR plan? Critical computer applications Escalation Plans / Decision Plans


Good DR plans

List of Recovery Team Members and contact info

Vendor Contact Information Communications Vendor Contact

Information Hotsite contact information Offsite storage contact information


Good DR plans Hardware / Software recovery for

each and every critical system based on RPO/RTO

Network recovery information

Detailed configuration information


Good DR plans

Up to date Information on last time this DR

plan was tested (Minimum is annually)

Change Log to the plan Returning to normal operations


Not so Good DR Plans


Not so Good DR plans No Executive Sponsor Unrealistic Budget

(< 2% of Data Center total budget) Unrealistic recovery strategy Not Exercised / Tested

Testing only partial of a system No training

No Priority on recovery of systems


Not so Good DR plans Copied from another site with no

updates General in nature 3 inch binder Overabundance of color charts and

slides High on fluff Short on useful information


Not so Good DR plans PURPOSE OBJECTIVES SCOPE AUTHORITIES REFERENCES MANAGEMENT RESPONSIBILITIES ORGANIZATION OF THE PLAN DEFINITIONS CANCELLATION DISTRIBUTION OVERVIEW POLICY ASSUMPTIONS CONCEPT OF ACTIVATION DEPLOYMENT CONDITIONS


With Logic like this


They may be trying to Bamboozal you!


Remember Review the plan at a high level Recovery of Systems and

Communications, that is key Who needs to be contacted? Where do we go? Acquire equipment Restore Operating Systems, applications

and data Restore Communication


Remember

Stick to the key points and don’t get distracted by all of the rest

Do not get bogged down in the fine detail


Closing


Front end security vs back end BC/DR

BC / DR activation are last resort efforts

Risk levels go high

Spend the time, effort & money to develop a very strong front end security program to avoid a disastrous event


Thank You for Attending!

Documents

Quick Intro: