Business Continuity: Architecture Design Guide - · PDF fileBusiness Continuity: Architecture Design Guide ... QR, QNet, GridStor, Vault ... During the recovery the CommServe DR GUI

A CommVault White Paper:Business Continuity:Architecture Design Guide

CommVault Corporate Headquarters2 Crescent Place

Oceanport, New Jersey 07757-0900 USATelephone: 888.746.3849 or 732.870.4000

©2007 CommVault Systems, Inc. All rights reserved. CommVault, CommVault and logo, the “CV” logo, CommVault Systems, Solving Forward, SIM, Singular Information Management, CommVault Galaxy, Unified Data Management,QiNetix, Quick Recovery, QR, QNet, GridStor, Vault Tracker, Quick Snap, QSnap, Recovery Director, CommServe, and CommCell, are trademarks or registered trademarks of CommVault Systems, Inc. All other third party brands,

products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.

CDC #DR__SG_R2

Version History

CommVault Professional Services - Advanced Consulting Services DR Architecture Design Guide – April 12, 2007 – Page 2 of 13

Version Number

Revision Date (DD/MM/YYYY)

Contributor’s Name

Revision Description

R0 12/09/2004 John Crescenti Document creation R1 06/29/2006 Robert Brower Document update R2 04/12/2007 John Crescenti Updated to include 6.1 features

CDC #DR__SG_R2

Table of Contents

1 Executive Summary ....................................................................................................................................... 4 1.1 Objectives ................................................................................................................................................. 5 1.2 Core Platform Capabilities ........................................................................................................................ 5

2 Example Disaster Recovery Scenarios........................................................................................................ 6 2.1 Scenario 1 - Small to Medium Size Business with a Single Location ...................................................... 6

2.1.1 Solution 1a – SMB Single-Site Architecture ...................................................................................... 6 2.1.2 Solution 1b – SMB Dual-Site Architecture......................................................................................... 7 2.1.3 Solution 1c – SMB Dual Site with CDR for Selected Applications .................................................... 8 2.1.4 Solution 1 - Technical Background Information ................................................................................ 9 2.1.5 Solution 1 - Requirements ................................................................................................................. 9 2.1.6 Solution 1 - Benefits .......................................................................................................................... 9 2.1.7 Solution 1 - Risks............................................................................................................................... 9

2.2 Scenario 2 – Medium to Large Business with Multiple Sites in a Single CommCell................................ 9 2.2.1 Solution 2a – Multiple Site DR Architecture with Scheduled CommServe Failover.......................... 9 2.2.2 Solution 2b – Multiple Site DR Architecture with Scheduled CommServe Failover and Continuous Data Replication............................................................................................................................................. 11 2.2.3 Scenario 2 - Technical Background ................................................................................................ 11 2.2.4 Solution 2 - Requirements ............................................................................................................... 11 2.2.5 Solution 2 - Benefits ........................................................................................................................ 11 2.2.6 Solution 2 - Risks............................................................................................................................. 11

2.3 Scenario 3 - Large Business with Multiple Sites and Multiple CommCells ............................................ 11 2.3.1 Solution 3 – Large Business Architecture with Multiple Sites and Alternate CommCells ............... 12 2.3.2 Solution 3 - Technical Background.................................................................................................. 12 2.3.3 Solution 3 - Requirements ............................................................................................................... 12 2.3.4 Solution 3 - Benefits ........................................................................................................................ 12 2.3.5 Solution 3 - Risks............................................................................................................................. 13

3 Conclusion.................................................................................................................................................... 13


CDC #DR__SG_R2

1 Executive Summary Current circumstances in the global and business communities require businesses to plan for disaster scenarios of all sorts. The degree of dependence on computer based operations and data have made the IT infrastructure critical to business survival. As a result businesses are planning, architecting and deploying solutions which provide for business continuance in the event of natural or man-made disasters. These solutions range from simple backup with offsite storage of media to real time replication between redundant sites. The implementation of a Disaster Recovery / Continuation of Operations (CoOp) configuration as deployed by a given Enterprise is based on a number of data points including size and physical diversity of the infrastructure, projected revenue loss per unit of downtime and the cost of the Disaster Recovery (DR) solution. Faster restoration or recovery of data (Recovery Time Objective) and the reduction in the amount of data lost between the start of the disaster event and restoration of operations (Recovery Point Objective) drive the costs of the DR / CoOP configuration up. Determining where an enterprise must lie on the DR continuum is a calculation based on the cost of operations necessary to facilitate effective DR / CoOP versus the loss of revenue / functionality associated with that outage. For some enterprises, loss of access of a week’s data is not a significant impact while for other enterprises, outages greater than a few hours will result in loss of the business, physical resources, or lives. The diagram below summarizes the relationship between iterations of data availability and the increases in complexity associated with maximizing that availability.


CDC #DR__SG_R2

1.1 Objectives The CommVault platform provides a wide range of data protection alternatives which can meet the disaster recovery needs of businesses. This solution guide will outline the best practices for deployment of the platform based on some common scenarios. These recommendations can be easily modified to meet the specific needs of a given implementation through professional engineering services. Note that high availability solutions are not considered in this document as they are considered to provide a degree of functionality beyond that of disaster recovery.

1.2 Core Platform Capabilities There are a number of platform capabilities which facilitate the deployment of a DR solution. Table 1-1, below, provides a list of the most important of these features.

Table 1-1. QiNetix Core Capabilities for Disaster Recovery FEATURE DESCRIPTION

CommServe DR This mechanism provides a point in time copy of the CommServe Database which can be used to recover the CommServe

Continuous Data Replication (CDR) CDR provides for near real time replication of file system or application data from a live production server to a target server which can be either local or remote and is fully integrated with backup operations

Auxiliary Copy Aux Copy provides the ability to make replica copies of managed data which can be placed on any type of device in local and/or remote locations

DR CommServe Licensing Support This feature allows the CommServe to be activated on a DR host with a different IP address without requiring a license update

Restore From Anywhere This feature allows any Media Agent to restore data backed up through any Media Agent in the configuration which facilitates restoration at a DR site

Cross Server Restore Allows data from one server to be restored to another server Use Exact Index Restore Option This option used during restore allows for a previous index to be used

during the restore. This facilitates restoring data at the DR site from the tape based indexes which are on the local backup tapes.

VaultTracker Provides for the tracking and management reports of tape media outside the tape library.

1-Touch Mechanism for rebuilding an entire server with a minimum number of steps which utilize a generic boot CD or the network and automated recovery of data backed up through CommVault Galaxy Backup & Recovery.

Media Explorer A stand alone utility for examining and restoring data directly from tapes which were created by CommVault QiNetix.


CDC #DR__SG_R2

CommVault Professional Services - Advanced Consulting Services

2 Example Disaster Recovery Scenarios Using the Core Platform Capabilities, listed in table 1-1, in combination offers a great degree of DR implementation flexibility. The scenarios which follow will describe several alternative configurations for providing DR which range across various levels of automation and cost. With all scenarios a DR site is assumed. The DR site may be provided by a third party or may be another data center within the implementing company.

2.1 Scenario 1 - Small to Medium Size Business with a Single Location In this scenario we have a single site business with a modest number of systems and applications requiring a DR solution which will allow the recovery of business financial application data built on SQL-server, a web server and email within their infrastructure. In the case where the data center is lost through a disaster the business can survive a 1 week loss of data and the critical applications must be operational at the DR site within 48 hours. The business rents DR data center space from a third party provider in which it places cold standby servers which are equipped to support the critical applications that require DR coverage and a tape library which has compatible drives with their primary data center.

Note: The DR library shall use the same barcode format as the primary to facilitate tape management.

2.1.1 Solution 1a – SMB Single-Site Architecture This environment will be handled using a low end mostly manual solution. It will consist of using selective auxiliary copy on a weekly basis to make duplicate copies of full backups of the critical servers and the CommServeDR. These tapes will be tracked using VaultTracker and will be shipped to the DR facility weekly. After four weeks at the DR site they will be rotated back to the primary data center for reuse. Figure 2.1-1, below, presents a high level view of the configuration.

Figure 2.1-1. SMB Single Site with Leased DR Location and Standby Servers.

CommServe

Media Agent

Financial App

Server

Web Server

Email Server Tape

Library

CommServe

Media Agent

Financial App

Server

Web Server

Email Server

Primary Data Center

Tape Library

DR Architecture Design Guide – April 12, 2007 – Page 6 of 13

DR Site

CDC #DR__SG_R2


Operationally the critical servers are installed but dormant at the DR site. It should be noted that the machines located at the DR site could be built from scratch if necessary using standard installation procedures, however, this would add additional time to the recovery operation. Each of the servers will have the same name and IP address as the server that they are replacing in the primary data center. This approach provides the least automation but is also the least expensive since network bandwidth is not required between the primary and disaster recovery sites. During the recovery the CommServe DR GUI is used to recover the CommServe database. Once recovered the critical servers are recovered using standard restore procedures as if the operations were occurring at the primary data center. Depending on the expected duration of the outage at the primary data center servers for less important applications can be purchased/leased and recovered using the available backup data which can be streamlined by the use of 1-Touch for supported server platforms. Storage Policy and Job report output should also be sent to the DR site with the physical media so that all data can be located which is particularly important for the CommServe DR tapes. Platform features used by this solution are:

• CommServe DR • Auxiliary Copy • VaultTracker (Optional. If tape rotation is tracked manually) • 1-Touch (Optional. If used in the environment)

2.1.2 Solution 1b – SMB Dual-Site Architecture A more automated environment that can be implemented is to provide a network connection between the primary data center and the DR site. The bandwidth required is dependent on the amount of data that is being protected and the frequency of update. The connection will serve to provide a network for remote Auxiliary Copy to be utilized to automate the transport of managed data to the DR site. This implementation requires the use of one or more active Media Agents at the DR site. The cost trade-off between physical media movement and management and recovery time versus the cost of the network connection will dictate whether this approach makes sense. Figure 2.1-2 presents a high level view of the configuration.

Figure 2.1-2. SMB Single-Site with Leased DR Location with Standby Servers & Active MediaAgent.

Primary Data Center DR Data Center

CommServe

Media Agent

Financial App

Server

Web Server

Email Server Tape

Library

Financial App

ServerCommServe

Media Agent Web

Server

Email Server Tape

Library


WAN

CDC #DR__SG_R2


In this solution the high speed WAN is utilized to create Auxiliary Copies at the partner site for the primary servers and applications at the local site. Specifically in this example, primary backups of the Financial Application Server in the Primary Data Center which are directed to the tape library in Primary Data Center are Auxiliary copied to the tape library in the DR Data Center. In addition, stage 1 backups of the CommServe in the Primary Data Center are directed to the standby CommServe in the DR Data Center. Using this strategy for all data provides geographically dispersed data redundancy without the need to physically move media. To minimize recovery time the CommServerDR GUI can be utilized to recover the CommServeDR data to the alternate CommServe at the DR Data Center on a regular basis. This would have the alternate CommServe primed for activity and eliminate this step in an actual DR situation. Depending on the anticipated length of the outage at a Primary Data Center additional servers can be obtained for less critical applications and these can be recovered through the use of standard recovery procedures which include 1-Touch for supported platforms. Platform features used by this solution are:

• CommServe DR • Auxiliary Copy (Selective or Synchronous) • DR CommServe Licensing Support • 1-Touch (As required)

2.1.3 Solution 1c – SMB Dual Site with CDR for Selected Applications If a shorter recovery time is required for some applications then CDR can be added to the configuration to provide a near real time replication of critical data. In this example the Financial App Server data will be replicated to a target server in the DR site. The target server can have a dormant application instance installed on it so that down time can be minimized if it is necessary to fail over to the DR site. In this example the DR Financial Server will be running but the Financial Application will be offline. Data is replicated to the target machine as it changes on the production server so that the restore operation is not necessary prior to activating the Financial Application on the DR Financial Server. In addition to eliminating the restore step (assuming crash consistent data is recoverable by the application) it spreads the load on the network since the data is passed incrementally to the DR site.

Figure 2.1-3. SMB Single-Site with Leased DR Location with Standby Servers & Active MediaAgent


CommServe

Media Agent

Financial App

Server

Web Server

Email Server Tape

Library

CommServe

Media Agent

Financial App

Server

Web Server

Email Server Tape

Library

Primary Data Center DR Data Center

WAN

CDC #DR__SG_R2


In this solution CDR replicates financial application data to the DR financial server which is a member of the CommCell. At scheduled intervals Recovery Points can be taken to provide consistent point in time snapshots of the application data. These snapshots can be backed up locally at the DR site in conjunction with their creation for complete coordination of these activities. In addition, less important applications can utilize the Auxiliary Copy mechanism described earlier to provide for DR of those applications. Platform features used by this solution are:

• CommServe DR • Continuous Data Replication (CDR) • Auxiliary Copy (Selective or Synchronous) • DR CommServe Licensing Support • 1-Touch (As required)

2.1.4 Solution 1 - Technical Background Information The basic trade-offs to be made will include Recovery Time Objective (RTO), Recovery Point Objective (RPO), cost of downtime (COD) and cost of the implementation (COI). These four attributes will dictate the chosen solution.

2.1.5 Solution 1 - Requirements In solutions 1a, 1b or 1c an appropriate DR site is required with sufficient hardware for key servers, a CommServe, Media Agent and a compatible tape device for support of tape recovery operations. Solutions 1b and 1c require sufficient bandwidth between sites to move the required amount of data in the appropriate time period.

2.1.6 Solution 1 - Benefits The major benefit of these solutions is that they provide the most economical solution for DR. Solutions 1b and 1c provide reduced recovery time since the CommServe database is available on disk which can be recovered in very short order and in the case of 1c the data for critical applications is already primed for use. Also, no media transport is required which saves transportation costs and mitigates the potential of shipping damage on the media which can be traded off against the cost of the WAN connection.

2.1.7 Solution 1 - Risks The major risk with solution 1a is that it relies on manual manipulation of media. Care must be taken to insure that the correct media is sent to the offsite location along with supporting reports to be able to locate the appropriate media for restore purposes. Also, media is susceptible to damage in transit which may render it unusable during a restore operation.

2.2 Scenario 2 – Medium to Large Business with Multiple Sites in a Single CommCell

In this scenario the business being modeled has multiple data centers which are geographically disperse. As a result the DR equipment can reside at company sites which can be assigned the task of recovering partner sites. For the purpose of this discussion we will assume 2 locations connected via a high speed WAN link with one CommCell. Downtime needs to be minimized for critical applications in the event of a site failure.

2.2.1 Solution 2a – Multiple Site DR Architecture with Scheduled CommServe Failover


Several DR options exist for this scenario. In order to provide continued data protection operations in the event the primary CommServe is not operational an alternate server will be available at the second site. As in solution 1.2 this server will be loaded and running but the CommVault services will be left off-line. Also, critical servers will have alternates at the partner site. Figure 3 provides a high level view of the configuration.

CDC #DR__SG_R2


Figure 2.2-1. Medium to Large Business with 2 Active Locations each Providing DR Failover Capability in a

single CommCell . In this solution the high speed WAN is utilized to create Auxiliary Copies at the partner site for the primary servers and applications at the local site. Specifically in this example, primary backups of the Financial Application Server in Data Center 2 which are directed to the tape library in Data Center 2 are Auxiliary copied to the tape library in Data Center 1. In addition, stage 1 backups of the CommServe in Data Center 1 are directed to the standby CommServe in Data Center 2. Using this strategy for all data provides geographically dispersed data redundancy without the need to physically move media. To minimize recovery time the CommServerDR GUI can be utilized to recover the CommServeDR data to the alternate CommServe at Data Center 2 on a regular basis. This would have the alternate CommServe primed for activity and eliminate this step in an actual DR situation. Depending on the anticipated length of the outage at a Data Center additional servers can be obtained for less critical applications and these can be recovered through the use of standard recovery procedures which include 1-Touch for supported platforms. Platform features used by this solution are:

• CommServe DR • Auxiliary Copy (Selective or Synchronous) • DR CommServe Licensing Support • 1-Touch (As required)

Data Center 1 Data Center 2

CommServe

Media Agent

Financial App

Server

Web Server

Email Server Tape

Library

Financial App

Server CommServe

Media Agent

Web Server

Email Server Tape

Library


WAN

CDC #DR__SG_R2


2.2.2 Solution 2b – Multiple Site DR Architecture with Scheduled CommServe Failover and Continuous Data Replication

Solution 2.2 builds on solution 2.1. In this case standby critical application servers are replicated using CDR which reduces the recovery time using the live copy of the data or if necessary a copy-back of a recovery point. Platform features used by this solution are:

• CommServe DR • Continuous Data Recovery (CDR) • Auxiliary Copy (Selective or Synchronous) • DR CommServe Licensing Support • Cross Server Restore • 1-Touch (As required)

2.2.3 Scenario 2 - Technical Background The basic trade-offs to be made will include Recovery Time Objective (RTO), Recovery Point Objective (RPO), cost of downtime (COD) and cost of the implementation (COI). These four attributes will dictate the chosen solution.

2.2.4 Solution 2 - Requirements These solutions require a WAN connection and duplicate equipment at the alternate data center. Bandwidth and library storage engineering must be performed to assure that the required data can be transferred in the appropriate amount of time and that media shuffling can be minimized.

2.2.5 Solution 2 - Benefits The benefits of these solutions are the reduced time to recover especially with respect to replicated applications, the sharing of normal operational load and DR across multiple data centers.

2.2.6 Solution 2 - Risks A robust WAN environment is required for successful operation in this scenario.

2.3 Scenario 3 - Large Business with Multiple Sites and Multiple CommCells


In this scenario the business being modeled has multiple data centers which are geographically disperse with more than one CommCell deployed. This configuration is common in global enterprises or those in which organizational requirements dictate that more than one CommCell be deployed. In these cases the problem can be mapped back into Scenario 2 with the understanding that each of the CommCells will have an off-line alternate server available at the partner location along with one or more Media Agents and storage devices for auxiliary copy purposes in addition to standby application servers for critical applications.

CDC #DR__SG_R2


2.3.1 Solution 3 – Large Business Architecture with Multiple Sites and Alternate CommCells

Several DR options exist for this scenario. In order to provide continued data protection operations in the event the primary CommServe is not operational an alternate server will be available at the second site. As in solution 1.2 this server will be loaded and running but the CommVault services will be left off-line. Also, critical servers will have alternates at the partner site. Figure 2.3-1, on the following page, provides a simplified view of this Multi-Cell arrangement showing Cells A and B. This solution can utilize the same operational procedures as solution 2.2.1 or 2.2.2. Platform features used by this solution are:

• CommServe DR • Continuous Data Replication (CDR) for replicating critical application data • Auxiliary Copy (Selective or Synchronous) • DR CommServe Licensing Support • 1-Touch (For additional application servers as needed)

2.3.2 Solution 3 - Technical Background This solution can be treated like two instances of solution 2. It requires a proportionally larger investment in redundant equipment and network links to account for the larger infrastructure supported.

2.3.3 Solution 3 - Requirements This solution requires sufficient redundant equipment and WAN bandwidth to meet the RTO and RPO of the organization. Bandwidth and library storage engineering must be performed to assure that the required data can be transferred in the appropriate amount of time and media shuffling can be minimized.

2.3.4 Solution 3 - Benefits This solution provides good to excellent RTO and RPO.


CDC #DR__SG_R2


Figure 2.3-1. Large Dual-Site Business with Active-Active Locations for DR Failover Capability across Multiple

CommCells.

2.3.5 Solution 3 - Risks As with the other solutions a robust WAN is required. 3 Conclusion Given the number of possible customer deployments and wide range of business requirements surrounding DR no paper can be expected to cover them all. What this document attempts to do is describe a number of possible DR implementations which can be viewed as models for engineering actual customer environments. There was also a conscious effort to avoid using any non-Platform features such as remote clustering and SQL-Server replication. While using tools of this type are possible detailed analysis into their implementation is required before they could be incorporated into a recommended solution. The CommVault platform provides a rich set of features to facilitate DR. In particular, Continuous Data Replication provides hardware independent near real time replication, application aware recovery points and integrated backup. In the end every installation will need to be customized to the business needs and resources of the customer.

CommServe A

Media AgentA

Financial App

Server

Web Server

Email Server Tape

Library

Data Center 1

CommServe A (alternate)

Media Agent A

Financial App

Server

Web Server

Email Server

Data Center 2

Tape Library

WAN

CommServe B

Media Agent B

Web Server

Email Server

Financial App

Server Financial

App Server

CommServe B (alternate)

Web Server Media Agent

B

Email Server Tape

Library


CDC #DR__SG_R2

©2007CommVault Systems, Inc. All rights reserved. CommVault, CommVault Systems and logo, QiNetix, CommVault Galaxy, DataMigrator, DataArchiver, Continuous Data Replicator, Quick Recovery, StorageManager and QNet are trademarks and in some jurisdictions may be registered trademarks of CommVault Systems, Inc. Microsoft, Windows and Windows NT are trademarks of Microsoft Corporation. Other trademarks appearing in this document are the property of their respective owners.


DR Architecture Design Guide – April 12, 2007

Documents

Business Continuity: Architecture Design Guide - · PDF fileBusiness Continuity: Architecture Design Guide ... QR, QNet, GridStor, Vault ... During the recovery the CommServe DR GUI