Embed Size (px)
DESCRIPTION
Presentation for VMworld 2008 on designing choices and decisions for business continuous Computing Infrastructure
Citation preview
Designing and Implementing a Business Continuity Architecture
Breakout Session #2543
Christopher JanochSr. Network Engineer / Architect, Powell Goldstein, LLPSeptember 18, 2008
A Case Study of:
Powell Goldstein, LLP
Designing and Implementing a Business Continuity Architecture
Christopher Janoch
Senior Network Engineer / Infrastructure Architect
VMware Certified Professional
DoubleTake Certified Engineer
Zantaz Certified Engineer
ITIL Certified
Experience in designing DR and BCP infrastructure in
Legal Industries
Financial Industries
Construction Industries
Powell Goldstein, LLP
Business View
200 Lawyer Firm (600 Users)
Offices in Atlanta, Washington DC,
Dallas, and Charlotte
Technical View
Primarily Microsoft-based Technology
200+ Servers
High Availability (HA)
A system that can provide a continuous service by detecting
hardware, node or application failures and automatically
reconfiguring the system appropriately.
Fault-Tolerant Disk Array
Redundant Power Source / UPS
Redundant Network Connections
Multiple Endpoint Service Clusters
Failover Clusters
Disaster Recovery (DR)
A system to aid in the process of restoring operations critical
to the resumption of business (communications, data,
workspace) after a natural or man-made disaster.
Backup / Restore
4-Hr Replacement Service Contracts
Alternate Staging Site for Servers & Workstations
Rebuild and Reinstall Affected Systems
Business Continuity Process (BCP)
A system aimed at allowing an organization to continue
functioning after (and ideally, during) a disaster, rather than
simply being able to recover after a disaster.
Protection Strategies are Insurance
“Fast, Best, or Cheap – Choose any two!”
$$$ vs. SPEED vs. RISK – Choose any two!
You get what you pay for, But don’t pay too much!
YOU need to be the one with a complete PLAN
No one solution or vendor will adequately cover all needs.
No one methodology will cover all situations.
A Team-Oriented Approach is Needed
Too many people involved and nothing gets done.
Committee Effect
A Team-Oriented Approach is Needed
Not enough people and nothing gets done thoroughly.
Limited Focus and Few Opportunities
A Team-Oriented Approach is Needed
Department Management
Project Management
System Engineers
Support Teams
Business Representatives
Vendors / Consultants
A Team-Oriented Approach is Needed
Department Management
Project Management
System Engineers
Support Teams
Business Representatives
Vendors / Consultants
BCP
DESIGN
TEAM
Ideas & Solutions are Directed to a Core Design Team
Contributions are added by users that know the
Applications and Business Functions the best.
Application ManagerUser Support
Staff Users
Business Function
Representatives
Application
Engineers
BCP Design Team
BCP Compass Where are We?
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
User Access
Data
Application Services
Infrastructure Services
Communications
Environment
BCP MAP Where Are We Going?
User Access
Data
Application Services
Infrastructure Services
Communications
Environment
BCP MAP Where Are We Going?
BCP MAP Where Are We Going?
Most Design Work
Most
Troublesome
User Access
Data
Application Services
Infrastructure Services
Communications
Environment
Determine your Recovery Objectives
In some cases, the SERVICE is top priority
In others the DATA is more critical
Don’t Rely on IT’s judgment alone!
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
BCP RULER Recovery Solutions
Backup/R
esto
re
Serv
er R
ebuild
Contin
uous B
ackup
Backup/R
esto
re
Contin
uous B
ackup
BCP RULER Recovery Solutions
Backup/R
esto
re
Serv
er R
ebuild
Cold
Sta
ndby S
erv
er
Backup/R
esto
re
Contin
uous B
ackup
Contin
uous B
ackup
BCP RULER Recovery Solutions
Backup/R
esto
re
Asyn
chro
nous R
eplic
atio
n
Syn
chro
nous R
eplic
atio
n
Serv
er R
ebuild
Cold
Sta
ndby S
erv
er
Backup/R
esto
re
Asyn
chro
nous R
eplic
atio
n
Syn
chro
nous R
eplic
atio
n
Contin
uous B
ackup
Contin
uous B
ackup
BCP RULER Recovery Solutions
Backup/R
esto
re
Asyn
chro
nous R
eplic
atio
n
Syn
chro
nous R
eplic
atio
n
Serv
er R
ebuild
Redundant S
yste
ms
Cold
Sta
ndby S
erv
er
Backup/R
esto
re
Asyn
chro
nous R
eplic
atio
n
Syn
chro
nous R
eplic
atio
n
Contin
uous B
ackup
Contin
uous B
ackup
BCP RULER Application Measurement
Litig
atio
n S
upport F
iles
Litig
atio
n S
upport F
iles
BCP RULER Application Measurement
Litig
atio
n F
iles (A
RC
HIV
E)
Litig
atio
n F
iles (A
CT
IVE
)
Litig
atio
n F
iles (A
CT
IVE
)
Litig
atio
n F
iles (A
RC
HIV
E)
Protection Strategy Decisions
Centralized Services vs. Autonomous Offices
Centralizing proved to be more affordable, easier to design, and
much easier to maintain
Automation vs. Manual Processes
Automation simplifies crisis management, but adds new risks
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
Costs must be Contained, Predicted, & Controlled
Remember to account for Passive Infrastructure for *every*
Replicated System.
Beware the cost of adding too much redundancy.
The Solution MUST Survive in the Real World
“The more they over think the plumbing, the easier it is
to stop up the sink” – Scotty (Star Trek)
Understand Service & Application Interdependencies
Test each system before declaring it PROTECTED
Understand Service & Application Interdependencies
Test each system before declaring it PROTECTED
Understand Service & Application Interdependencies
Test each system before declaring it PROTECTED
Understand Service & Application Interdependencies
Test each system before declaring it PROTECTED
Understand Service & Application Interdependencies
Test each system before declaring it PROTECTED
Document Dependencies for Future Growth and Design Changes
User Access
Data
Application Services
Infrastructure Services
Communications
Environment
BCP MAP Powell Goldstein’s Map
Geographically Separate
Datacenters
Alternate Service Providers with
Diverse Paths
Redundant Servers
Virtualization
Data Replication
Survivable Remote Access
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
MaintenanceTesting & Vendor Selection
Manufacturer designs may not apply in your environment
“Your System” will *always* be an exception!
Testing & Vendor Selection
Products that look similar may not perform the same way.
Don’t be afraid of the “little guy”
Support Statements to cover your implementation and environment
are the Key to reducing future problems.
Take the time to compare alternative solutions!
You can’t learn “Everything about Everything”
Vendor implementation and “Health Checks”
Experienced Consultants
Continuing Support Contracts
Recognize when you need to hire assistance!
Organizational Acceptance
A Business Continuity Plan is an Investment
A Business Continuity Plan is a Marketable Commodity
When Projects are embraced at the top of an organization,
they are more easily accepted at the bottom.
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
The Slowest Adaptors may be those closest to you!!
The IT Department:
Non BCP-Compliant Projects
Hesitancy to trust the System
“Reasonable” Conversion Delays
The “Local” Pilot Group
The Blur between Development & Production
Implementation
Work your Map from both the Top and Bottom.
Starting with the Root Dependencies, protect each service
completely before moving to the next.
Your next Outage will not wait for you to be ready!
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
Take advantage of Redundant Systems
Buy New
The new servers become your test lab and
allow you to isolate the systems during implementation.
Avoid the “Re-wiring the House Live” syndrome
The System, Design, and Plan will continue to Change
New services will be added to the system.
Assumed RTO’s will be proved Incorrect.
Technologies will be updated.
Designed Solutions won’t work as Planned.
Don’t upgrade BCP Key Components without Testing!
Implement Change Management
NOTHING changes without knowledge and approval
EVERYTHING that changes gets documented
Identify who will be responsible for the
Implementation and Testing of which systems.
Standardize Quality Control Checks and
Officially Scheduled Tests.
Clear Processes for Updates, Changes, and Re-Builds must be
Documented and easily available.
Ongoing Maintenance
A New Mindset:
Business Continuity: Compliant or Not?
Regular Testing
Don’t just TEST….. USE!
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
BCP Navigational Tools
User Access
Data
Application Services
Infrastructure Services
Communications
Environment
MONTHS DAYS HOURS MINUTES SECONDS SECONDS MINUTES HOURS DAYS MONTHS
!!!! OUTAGE !!!!
RPO RTO
Analysis
Solution Design
Vendor
Selection
Organizational
Acceptance
Implementation
Maintenance
Q&A
Breakout Session #2543
Christopher JanochSr. Network Engineer / Architect, Powell Goldstein, LLPSeptember 18, 2008
