Using HyperReplication of OceanStor V3 and OceanStor … Hu… · Using HyperReplication of OceanStor V3 and OceanStor SRA with VMware SRM to build multi-sites hybrid cloud ... Oracle

Using HyperReplication of OceanStor V3 and OceanStor SRA with VMware SRM to build multi-sites hybrid cloud

This document introduces a solution of VMware hybrid cloud across three sites, one of which resides on a public cloud, the

other two resides on two data centers locate at two cities far from each other, and VMware SRM with array-based replication

is established between the two sites. The replication is based on HyperReplication/A feature of OceanStor V3, and

OceanStor SRA 2.0 is installed on VMware vCenter to accept the storage synchronize and snapshot commands from

VMware SRM.

The solution test result shows that customer could benefit from the architecture and the features of OceanStor V3 to avoid

data losing and business interruption, with the RPO lower than 10 seconds and RTO lower to 5 minutes.

Author: Wang Yaohui

IT Storage Solutions, Huawei Enterprise BG

February 7, 2015 Version 1.0

Huawei Technologies Co., Ltd.

mailto:[email protected]

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2

2

Using OceanStor V3 HyperReplication and OceanStor SRA with VMware SRM to build

multi-site hybrid cloud

Contents

1 About This Document .................................................................................................................. 5

1.1 Overview ...................................................................................................................................................................... 5

1.2 Purpose ......................................................................................................................................................................... 5

1.3 Intended Audience ........................................................................................................................................................ 5

1.4 Business case ................................................................................................................................................................ 6

1.5 Customer Benefits ........................................................................................................................................................ 7

1.6 Key Components .......................................................................................................................................................... 7

1.7 Workload Models .......................................................................................................................................................... 8

1.7.1 Infrastructure.............................................................................................................................................................. 8

1.7.2 ERP application ......................................................................................................................................................... 8

1.7.3 CRM application ........................................................................................................................................................ 8

2 Products and technologies ......................................................................................................... 10

2.1 Huawei OceanStor V3 converged storage .................................................................................................................. 10

2.1.2 OceanStor OS .......................................................................................................................................................... 11

2.1.3 HyperReplication ..................................................................................................................................................... 13

2.1.4 OceanStor SRA ........................................................................................................................................................ 19

2.2 VMware vSphere ........................................................................................................................................................ 19

2.2.1 VMware ESXi and vCenter ..................................................................................................................................... 19

2.2.2 VMware Site Recovery Manager ............................................................................................................................. 24

3 Multi-site hybrid cloud solution .............................................................................................. 26

3.1 Business Challenges ................................................................................................................................................... 26

3.2 Solution ....................................................................................................................................................................... 26

3.2.1 Solution Architecture ............................................................................................................................................... 26

3.2.2 Solution Configuration ............................................................................................................................................ 28

3.3 Planning ...................................................................................................................................................................... 31

3.3.1 Solution build-up ..................................................................................................................................................... 31

3.3.2 Planned migration test ............................................................................................................................................. 35

3.3.3 Disaster recovery test ............................................................................................................................................... 36

3.4 Infrastructure build-up ................................................................................................................................................ 37

3.4.1 Prepare physical enviorment .................................................................................................................................... 37

3.4.2 Configure active directory and domain service ....................................................................................................... 37

3.4.3 Configure file share service ..................................................................................................................................... 37


3



3.4.4 Configure database for SRM ................................................................................................................................... 37

3.4.5 Configure vCenter and ESXi ................................................................................................................................... 37

3.4.6 Configure storage and datastore............................................................................................................................... 38

3.5 Virtual machine build-up ............................................................................................................................................ 45

3.5.1 Template VMs build up............................................................................................................................................ 45

3.5.2 CRM application build up........................................................................................................................................ 46

3.5.3 ERP application build up ......................................................................................................................................... 47

3.5.4 Applicaion build up for Site2 ................................................................................................................................... 49

3.6 Configure storage replication ...................................................................................................................................... 50

3.6.1 Add remote device on 5800V3-Site1 ....................................................................................................................... 50

3.6.2 Create remote replication on 5800V3-Site1 ............................................................................................................. 53

3.6.3 Create remote replication on 5300V3-Site2 ............................................................................................................. 58

3.7 Configure VMware SRM ............................................................................................................................................ 60

3.7.1 Install SRM and configure resource ........................................................................................................................ 60

3.7.2 Configure array manager ......................................................................................................................................... 60

3.7.3 Configure protection groups .................................................................................................................................... 64

3.7.4 Configure recover plans ........................................................................................................................................... 66

3.8 Run planned migration from Site1 to Site2 ................................................................................................................ 75

3.8.1 Start application and RPO timer .............................................................................................................................. 75

3.8.2 Migrate VMs to Site2 .............................................................................................................................................. 75

3.8.3 Re-protect VMs after migration ............................................................................................................................... 78

3.8.4 Check RPO timer log ............................................................................................................................................... 80

3.8.5 Re-start application and RPO timer ......................................................................................................................... 81

3.8.6 Migrate VMs back to Site1 ...................................................................................................................................... 81

3.8.7 Re-protect VMs after migration back ...................................................................................................................... 82


3.8.9 Collect test result ..................................................................................................................................................... 83

3.9 Run disaster recovery from Site1 to Site2 .................................................................................................................. 84

3.9.1 Start application and RPO timer .............................................................................................................................. 84

3.9.2 Simulate disaster ...................................................................................................................................................... 84

3.9.3 Recover VMs to Site2 .............................................................................................................................................. 85


3.9.5 Simulate disaster failback ........................................................................................................................................ 88

3.9.6 Run recovery plan again .......................................................................................................................................... 89

3.9.7 Reprotect the VMs fail to Site2 ............................................................................................................................... 90

3.9.8 Re-start application and RPO timer ......................................................................................................................... 91

3.9.9 Migrate VMs back to Site1 ...................................................................................................................................... 91

3.9.10 Re-protect VMs after migration back .................................................................................................................... 91

3.9.11 Check RPO timer log ............................................................................................................................................. 91

3.9.12 Collect test result ................................................................................................................................................... 91

3.10 Conclusion ................................................................................................................................................................ 92

3.10.1 Functionbility ......................................................................................................................................................... 92


4



3.10.2 Performance ........................................................................................................................................................... 93

3.10.3 Management .......................................................................................................................................................... 93

4 Appendix ...................................................................................................................................... 94

4.1 Reference Documents ................................................................................................................................................. 94

4.2 Terminology ................................................................................................................................................................ 94


5



1 About This Document

1.1 Overview

This document introduces a solution of VMware hybrid cloud across three sites, one of which

resides on a public cloud, the other two resides on two data center locate at two cities far from

each other, and VMware SRM with array-based replication is established between the two

sites. The replication is based on HyperReplication/A (asynchronous replication) feature of

OceanStor V3, and OceanStor SRA 2.0 is installed on VMware vCenter to accept the storage

synchronize and snapshot commands from VMware SRM.

This document covers the following contents:

OceanStor V3 technologies and features

Principle of HyperReplication/A

Principle of VMware SRM and OceanStor SRA

Introduction to the solution

Implementation steps of the solution

Test steps and results of disaster recovery

1.2 Purpose

This document aims to provide reference for Huawei's partners and customers who want to

use the HyperReplication/A feature of Huawei OceanStor V3 series converged storage (V3

storage for short) and OceanStor SRA with VMware SRM to build multi-site hybrid cloud.

Reading this document helps mitigating risks in IT solution design, operation, and

maintenance.

1.3 Intended Audience

The audiences of this document include Huawei's partners, customers, and employees. In

particular, this document is well-suited for storage and VMware administrators who plan to

use the HyperReplication/A feature of V3 storage and OceanStor SRA with VMware SRM, to

build multi-side hybrid cloud.


6



It’s assumed that the readers are familiar with the following products and technologies.

OceanStor V3 converged storage

OceanStor OS and HyperReplication feature

OceanStor SRA 2.0

OceanStor UltraPath for VMware

VMware vSphere 5.5, SRM 5.5

Microsoft Windows 2012

Microsoft Active Directory

Microsoft SQL Server 2014 and TPC-E Toolkit

Red Hat Enterprise Linux 6

Oracle 12c Database and SwingBench Test Tool

1.4 Business case

Data security and business continuity is always the key point of IT system. In nowadays, the

data becomes the core of business, which is much more important than fixed tangible assets

for enterprises, it is difficult simple measured in terms of money for data loss and business

interruption. Events like the Sichuan earthquake and 911 showed that natural disasters (fire,

flood, earthquake, etc) and human-made disasters (misuse, viruses, etc) are unable to avoid,

which bring heavy losses to business. Enterprises must be able to efficiently protect and

quickly recover data if a disaster, mis-operation, or application fault occurs. To achieve these

objectives, enterprises must not only back up data locally, but also transfer the data to remote

sites for storage. If no effective data protection or remote data transfer methods are available,

enterprises may suffer heavy losses.

How to protect data, and ensure that the system is stable and reliable continuous, as well as

fast and reliable access, became the most important challenges in the construction of

information systems. Disaster recovery technologies perfectly tackle the previous challenges.

As the core of a disaster recovery system, storage array-based remote replication is commonly

used.

Server virtualization brings lots of benefits to IT systems, and hybrid cloud is also a hot key

word in the last few years, which brings agile management to IT systems. How to build a

hybrid cloud with disaster recovery architecture becomes an issue for information system

builders.

This solution utilize VMware SRM based on Huawei OceanStor V3 to implement a hybrid

cloud across three sites, with management services (active directory, domain services, shared

file server, management database and vCenter server) reside on public cloud, production VMs

(ERP, CRM, and etc.) reside on private site1, and testing VMs reside on private site2.

VMware SRM disaster recovery architecture is implemented between private site1 and site2.


7



1.5 Customer Benefits

The solution test result shows that customer could benefit from the architecture to avoid data

losing and business interruption; with the disaster recovery RPO (recover point objective, or

data lose time) lower than 10 seconds and RTO (recover time objective, or business interrupt

time) lower to 5 minutes.

You can benefit from Huawei OceanStor V3 storage, OceanStor OS platform, SmartThin,

HyperReplication/A feature, HyperSnap feature, OceanStor SRA plug-in and OceanStor

UltraPath plug-in with the value of lower performance sacrifice, lower RPO & RTO, flexible

management and etc.

Table 1-1 Key benefits

Category Benefits

Disaster recovery RPO <10 seconds

Disaster recovery RTO Lower to 5 minutes

Performance overhead from replication no

Management Visualization DR management

One-click migration test

Integrated storage management

1.6 Key Components

The proven solution is based on the following hardware and software:

OceanStor V3 converged storage V300R001C10

OceanStor OS with HyperRelication, HyperSnap and SmartThin features

OceanStor UltraPath for VMware: 8.01.023

OceanStor SRA 2.0 V1.3.30.100

VMware vSphere 5.5.0-2183112-20140901-update02

VMware SRM 5.5.1-1647061

Microsoft 2012 with Active Directory feature

Microsoft SQL Server 2014 and TPC-E Toolkit 1.12

Red Hat Enterprise Linux 6.3

Oracle Database 12.1.0.2 and SwingBench 2.5.0.928


8



1.7 Workload Models

1.7.1 Infrastructure

The solution is based on VMware SRM with OceanStor SRA plug-in. When the recovery plan

testing or planned site migration are performed, SRM will issue storage commands like

storage replication synchronize, replication split, snapshot, and LUN mapping. The

commands are sent to OceanStor V3 storage via OceanStor SRA without human operating.

1.7.2 ERP application

ERP application is one of the workload simulated in the solution test, which is simulated

using SwingBench Order Entry 2.0 workload model on Oracle Database 12C. The model

is a widely used OLTP type workload with the similar character with ERP application, in

which lots of online users doing transaction processing, with and I/O character of small block

random read and write with an ratio of 5:5.

The Order Entry model defines an online order service and simulates a scenario where a

number of users are querying products, placing orders, processing orders, and viewing orders

online. Those operations are the most common operations in transaction systems. In this

workload model, there are two main performance indicators: TPS (transactions per second)

and transaction response time. The TPS indicates the number of transactions processed per

second. A higher TPS indicates higher productivity. The transaction response time directly

impacts the speed of user operations. Shorter response time indicates better user experience.

Order Entry 2.0 model defines 10 tables, storing information about products, customers,

orders, warehouses, and login. During the load test, 50% of operations are allocated to

SELECT, 30% to INSERT, and 20% to UPDATE, and none to DELETE operations. From the

perspective of I/O layer, the workload model is one of the most typical OLTP workload

models, where small data blocks are accessed at random and the ratio between reads and

writes is 5:5.

1.7.3 CRM application

CRM application is another workload simulated in the solution test, which is simulated using

TPC-E Like workload model on Microsoft SQL Server 2014. The model is a widely used

OLTP type workload with the similar character with CRM application, in which lots of

online users doing transaction processing, with and I/O character of small block random

read and write with an ratio of 7:3.

The workload is composed of a set of transactional operations designed to exercise system

functionalities in a manner representative of complex OLTP application environments.

These transactional operations have been given a life-like context, portraying the activity of a

brokerage firm, to help users relate intuitively to the components of the benchmark. The

workload is centered on the activity of processing brokerage trades and uses a schema, which

is logically divided in four sets of tables.

The workload models the activity of brokerage firm that must manage customer accounts,

execute customer trade orders, and be responsible for the interactions of customers with

financial markets. The following diagram illustrates the transaction flow of the business

model portrayed in the benchmark:


9



Figure 1-1 Business model of TPC-E like workload

The Company portrayed by the benchmark is a brokerage firm with customers who generate

transactions related to trades, account inquiries, and market research. The brokerage firm in

turn interacts with financial markets to execute orders on behalf of the customers and updates

relevant account information. The number of customers defined for the brokerage firm can be

varied to represent the workloads of different size businesses.

This benchmark is composed of a set of transactions that are executed against three sets of

database tables that represent market data, customer data, and broker data. A fourth set of

tables contains generic dimension data such as zip codes. The following diagram illustrates

the key components of the environment:

Figure 1-2 Components of TPC-E like workload

Customer

Brokerage

Market

Customer

Initiated

Transactions

Market

Triggered

Transactions

Customer

Brokerage

Market

Customer

Initiated

Transactions

Market

Triggered

Transactions

Customers Brokers Market

READ-WRITE

•Market-Feed

•Trade-Order

•Trade-Result

•Trade-Update

•Security-Detail

•Trade-Lookup

•Trade-Status

READ-ONLY

•Broker-Volume

•Customer-Position

•Market-Watch

Invoke the following transactions …

… against the following data

Customer Data Brokerage Data Market Data

Customers Brokers Market

READ-WRITE

•Market-Feed

•Trade-Order

•Trade-Result

•Trade-Update

•Security-Detail

•Trade-Lookup

•Trade-Status

READ-ONLY

•Broker-Volume


•Market-Watch

READ-WRITE

•Market-Feed

•Trade-Order

•Trade-Result

•Trade-Update

•Security-Detail

•Trade-Lookup

•Trade-Status

READ-ONLY

•Broker-Volume


•Market-Watch

Invoke the following transactions …

… against the following data

Customer Data Brokerage Data Market Data


10



2 Products and technologies

2.1 Huawei OceanStor V3 converged storage

Huawei OceanStor V3 mid-range converged storage systems (V3 storages for short) offer a

cloud architecture oriented operating system, high-performance hardware platform, and a

complete suite of smart management software. V3 storages are on-demand converged storage

systems with secure, reliable, efficient features for multiple applications, heterogeneous

device integration, data disaster recovery, fast data growth.

Enables convergence in five areas: SAN and NAS, heterogeneous devices, entry-level to

high-end storage, HDD and SSD, and storage and backup

Provides industry-leading specifications: Up to 8 controllers, 1 TB cache, 5 PB storage

capacity, and various interface types, including 16 Gbit/s FC, 56 Gbit/s InfiniBand,PCIe

3.0, 12 Gbit/s SAS, and smart I/O card

Simplest management platform handles multiple product models with a graphical

interface and is available for Windows, iOS, and Android versions

Protects customer initial investments and reduces TCO for multiple applications, various

product models, and fast business growth

For more information about V3 storages, refer to the follow link:

http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range

Figure 2-1 OceanStor V3 Converged Storage

http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range


11



2.1.2 OceanStor OS

OceanStor OS is a cloud oriented storage platform used in V3 storage, with wealth of

software features and convergence of storage technologies, provides on-demand, simple,

efficient, secure and reliable storage service.

Figure 2-2 OceanStor OS

Convergence of SAN and NAS – On-demand service deployment

Figure 2-3 Convergence of SAN and NAS

Block- and file-level data storage is unified, requiring no additional file engines,

simplifying deployment, and reducing purchase cost

The efficiency of databases and file sharing services is notably improved


12



Convergence of SSDs and HDDs – On-demand performance and capacity

Figure 2-4 Convergence of HDDs and SSDs

All-flash arrays are optimized to put the high performance and low latency advantages

of SSDs into full play

HDDs and SSDs are converged to meet the performance requirements of complex

services

Convergence of primary and backup storage – On-demand data protection

Figure 2-5 Convergence of primary and backup storage

Backup storage is perfectly integrated into primary storage, requiring no additional

backup software and reducing your purchase cost

Primary storage and backup storage are managed in a unified manner, simplifying the

O&M of backup solutions


13



Convergence of high-end, mid-range, and entry-level storage systems – On-demand data value

Figure 2-6 Convergence of high-end, mid-range and entry-level storage

High-end, mid-range, and entry-level storage systems interwork, enabling free data flow

High-end, mid-range, and entry-level storage systems are managed in a unified manner,

multiple times the efficiency

Convergence of heterogeneous storage systems – On-demand storage resource allocation

Figure 2-7 Convergence of heterogeneous storage

Legacy storage systems are reused, protecting the original investment

Pools of third-party storage resources simplify storage management and achieve

cloud-based storage

2.1.3 HyperReplication

HyperReplication is a remote replication feature provided by V3 storage to implement

synchronous and asynchronous data replication, supporting intra-city and remote disaster

recovery solutions.


14



HyperReplication requires at least two V3 storages. The two storages can be placed in the

same equipment room, same city, or two cities 1000 km apart. The storage that supports data

access by production services is called the primary storage, whereas the storage system that

provides data backup is called the secondary storage.

HyperReplication is divided into the following types:

HyperReplication/S for LUN: Data on the primary LUN is synchronized to the

secondary LUN in real time. No data is lost if a disaster occurs. However, production

service performance is affected by the data transfer latency.

HyperReplication/A for LUN: Data on the primary LUN is periodically synchronized

to the secondary LUN. Production service performance is not affected by the data

transfer latency. However, some data may lose if a disaster occurs.

HyperReplication/A for File System: Data on the primary file system is periodically

synchronized to the secondary file system. Production service performance is not

affected by the data transfer latency. However, some data may lose if a disaster occurs.

HyperReplication provides the storage array–based consistency group function for

synchronous and asynchronous remote replications between LUNs to ensure the consistency

of across-LUN applications in disaster recovery replication. The consistency group function

protects the dependency of host write I/Os across multiple LUNs, ensuring data consistency

between secondary LUNs.

The following terminologies is used in the next section to describe the principle of

HyperReplication/A for LUN.

Table 2-1 HyperReplication terminologies

Term Description

RPO Recovery point object (RPO) defines the maximum tolerable amount of lost

data if a disaster occurs. It is expressed in a period of time.

RTO Recovery time objective (RTO) defines the maximum tolerable downtime if a

disaster occurs.

CG

A consistency group (CG) is a collection of remote replication pairs managed

as a whole. Write I/Os of the primary LUNs are dependent, and the

dependency is protected during replication to ensure the consistency between

the secondary LUNs. CGs can be used for service takeover against disasters.

Primary LUN Primary LUNs store the data accessed by production services. Data is

replicated from primary LUNs to secondary LUNs.

Secondary

LUN

Secondary LUNs store backup data. Data is replicated from primary LUNs to

secondary LUNs.

Principle of HyperReplication/A for LUN

HyperReplication/A for LUN supports the long-distance data disaster recovery of LUNs. It

applies to scenarios where a remote disaster recovery center is used and the impact on production service performance must be reduced.


15



The feature employs the multi-time-segment caching technology to periodically synchronize

data between primary and secondary LUNs. All data changes to the primary LUN since last

synchronization will be synchronized to the secondary LUN. The working principle of the

technology is as follows:

1. After an asynchronous remote replication relationship is set up between a primary LUN

at the primary site and a secondary LUN at the secondary site, an initial synchronization

is implemented to fully copy data from the primary LUN to the secondary LUN.

2. When the initial synchronization is complete, the secondary LUN data status becomes

Consistent (data on the secondary LUN is a copy of data on the primary LUN at a

certain past point in time). Then the I/O process shown in the following figure starts:

Figure 2-8 I/O process of HyperReplication/A for LUN

Incremental data is automatically synchronized from the primary site to the secondary site based on the

user-defined synchronization period that ranges from 3 seconds to 1440 minutes. If the synchronization

type is Manual, a user needs to manually trigger the synchronization. When a replication period starts,

new time segments (TPN+1 and TPX+1) are respectively generated in the caches of the primary LUN

(LUN A) and the secondary LUN (LUN B).

The primary site receives a write request from a production host.

The primary site writes data of the write request to cache time segment TPN+1 and sends a write success

response to the host immediately.

During data synchronization, the storage system reads data in cache time segment TPN of the primary

LUN in the previous synchronization period, transmits the data to the secondary site, and writes the data

to cache time segment TPX+1 of the secondary LUN. When the write cache of the primary site reaches

the high watermark, data in the cache is automatically flushed to disks. In this case, a snapshot is

generated for data of time segment TPN. During synchronization, such data is read from the snapshot and

copied to the secondary LUN.

When the synchronization is complete, the storage system flushes data of time segments TPN and TPX+1

in the caches of the primary and secondary LUNs onto disks (the corresponding snapshots are deleted

automatically), and waits for the next synchronization period.

Functions and features of HyperReplication/A for LUN

Second-Level RPO

HyperReplication employs the innovative multi-time-segment caching technology to reduce

the RPO of asynchronous remote replication between LUNs to seconds.


16



Thanks to the multi-time-segment caching technology, data in cache and I/Os interacts with

the cache that carry time information. During replication, the storage system directly reads

data of corresponding time segments from the primary LUN cache and copies the data to the

secondary LUN. A data write result will be returned once the data is written to the secondary

LUN cache. In traditional asynchronous replication, snapshots must be created and deleted in

every replication period. Data must be read from the snapshots, and snapshots must be created

for written data. Compared with traditional replication technologies, HyperReplication greatly

shortens the replication period and delivers second-level RPO.

Quick Response to Host I/Os

All I/Os generated during LUN asynchronous remote replication are processed in the

background. A write success acknowledgement is returned immediately after host data is

written to the cache. Therefore, host I/Os can be quickly responded.

Split Mode and Quick Disaster Recovery

HyperReplication/A for LUN allows users to split and resume replication pairs.

A split asynchronous remote replication session will not be periodically synchronized. Users

can manually start synchronization. Then the session is synchronized based on a preset

synchronization policy (manual or automatic).

HyperReplication/A provides three data synchronization types:

Manual: Users need to manually synchronize data from a primary LUN to a secondary

LUN. In manual synchronization, users can update data to the secondary LUN as desired.

That is, users can determine that the data of the secondary LUN is the copy of the

primary LUN at a desired time point.

Timed wait when synchronization begins: When a data synchronization process starts,

the system starts timing. After one synchronization period, the system starts

synchronization and timing again. After a specified period of time since the start of the

latest synchronization process, the system automatically copies data from the primary

LUN to the secondary LUN.

Timed wait when synchronization ends: The system starts timing for the next

synchronization session after the last synchronization session ends. In this mode, when a

data synchronization session ends, the system waits for the duration preset by users.

When the duration elapses, the system automatically synchronizes data from the primary

LUN to the secondary LUN again.

Full Protection for Data on the Secondary LUN

HyperReplication/A provides full protection for data on the secondary LUN. At the secondary

site, hosts' permission to read and write the secondary LUN is under control. When a

synchronization process is interrupted or data on the secondary LUN becomes unavailable,

data of the previous period TPX can be recovered to the secondary LUN to overwrite data of

the current period TPX+1. Then the secondary LUN stores available data of the point in time

before the latest synchronization process.

Writable Secondary LUN

The writable secondary LUN function enables the secondary LUN to provide services when

the primary LUN fails. HyperReplication supports this function. That is, production hosts can directly access data on the secondary LUN. This function is used in the following scenarios:


17



Users want to use data on the secondary LUN for data analysis and mining without

affecting services supported by the primary LUN.

The production storage system at the primary site is faulty and the disaster recovery

storage system at the secondary site needs to take over services from the production

storage system. However, a primary/secondary switchover fails or the secondary site

cannot communicate with the production storage system correctly.

Usually, the secondary LUN of a remote replication pair is read-only. If the primary LUN is

faulty, the administrator can cancel secondary LUN write protection to set the secondary LUN

writable. Then the disaster recovery storage system can take over host services to ensure

business continuity.

The secondary LUN of a synchronous remote replication pair can be set to writable only when

the following two conditions are met:

The remote replication pair is in the split or interrupted state.

Data on the secondary LUN is consistent (when data on the secondary LUN is

inconsistent, the data is unavailable, and the secondary LUN cannot be set to writable).

V3 storage can record difference between the primary and secondary LUNs after host data is

written to the secondary LUN. After the production storage system at the primary site

recovers, users can perform incremental synchronization to quickly switch services back.

Primary/Secondary Switchover

A primary/secondary switchover is the process where the primary and secondary LUNs in a

remote replication pair exchange roles. HyperReplication allows users to perform

primary/secondary switchovers.

Figure 2-9 Primary/secondary switchover

Primary/Secondary switchovers are affected by the secondary LUN data state, which indicates

the availability of data on the secondary LUN. There are two secondary LUN data states:


18



Consistent: Data on the secondary LUN is a duplicate of the data on the primary LUN (at

the time the previous synchronization ended). In this state, data on the secondary LUN is

available but not necessarily the same as the current data on the primary LUN.

Inconsistent: Data on the secondary LUN is not a duplicate of the data on the primary

LUN (at the time the previous synchronization ended). In this state, data on the

secondary LUN is available.

As shown in the previous figure, the primary LUN at the primary site becomes the new

secondary LUN after the switchover, and the secondary LUN at the secondary site becomes

the new primary LUN. After users perform some simple operations on the host side (the major

operation is to map the new primary LUN to the secondary production hosts, which can be

performed in advance), the secondary production hosts at the secondary site take over services

and issue new read and write requests to the new primary LUN. A primary/secondary

switchover can be performed only when data on the secondary LUN is in the Consistent state.

Synchronization after a primary/secondary switchover is incremental synchronization.

The following must be noted before a primary/secondary switchover is performed for a

synchronous remote replication pair:

When the pair is in a normal state, a primary/secondary switchover can be performed.

In the split state, a primary/secondary switchover can be performed only when the

secondary LUN is set to writable.

Consistency Group

In medium- and large-sized database applications, data, logs, and modification information

are stored on different LUNs. If data on one of the LUNs is unavailable, data on the other

LUNs is also invalid. How to keep consistency between multiple remote replication pairs

must be considered if remote disaster recovery must be implemented for these LUNs

simultaneously. HyperReplication provides the consistency group function to maintain the

same synchronization pace among multiple remote replication pairs.

A consistency group is a set of multiple remote replication sessions, ensuring data consistency

in the scenario where a host writes data to multiple LUNs on a single storage system. After

data is written to a consistency group at the primary site, all data in the consistency group is

simultaneously copied to the secondary LUN using the synchronization function of the

consistency group, ensuring integrity and availability of the data used for backup and disaster

recovery.

HyperReplication allows users to add multiple remote replication pairs to a consistency group.

When users perform splitting, synchronization, or a primary/secondary switchover or set

secondary LUNs to writable for a consistency group, the operation applies to all members in

the consistency group. When a link fault occurs, all members of the consistency group enter

the abnormally interrupted state together. After the fault is rectified, data synchronization is

performed again to ensure availability of the data on the secondary storage system.

Primary LUNs in a consistency group can belong to different working controllers. The same

applies to secondary LUNs. This allows users to configure LUNs flexibly.

For more information about HyperReplication, please refer to the document:

HUAWEI OceanStor V3 converged storage systems HyperReplication technical white paper


19



2.1.4 OceanStor SRA

The Storage Replication Adapter (SRA) enables a storage system to closely cooperate with

the Site Recovery Manager (SRM) service and disaster recovery plans. The SRA uses

advanced disaster recovery technologies of storage systems to provide ongoing disaster

recovery management, rehearsal, and recovery modes for VMware virtual environments,

enhancing virtual environments' capabilities of resisting disasters.

The SRA provides the SRM with storage system information and interfaces to enable storage

system discovery, replication LUN discovery, failover tests, and disaster recovery. When the

SRM creates, tests, and executes a disaster recovery plan, the SRA immediately provides

corresponding resources for the SRM and cooperates with the SRM to complete ongoing tests,

site recovery automation, and migration process automation in a virtual environment.

The SRA is responsible to execute storage commands like “synchronize consistency group

(CG)”, “split CG”, “switchover CG”, “take snapshot”, and “LUN mapping” when executing

recovery tasks.

Figure 2-10 Role of SRA when performing recovery task

For more information about OceanStor SRA, please refer the following document:

eSDK OceanStor SRA technical white paper

2.2 VMware vSphere

2.2.1 VMware ESXi and vCenter

VMware vSphere leverages the power of virtualization to transform datacenters into

simplified cloud computing infrastructures and enables IT organizations to deliver flexible

and reliable IT services. VMware vSphere virtualizes and aggregates the underlying physical


20



hardware resources across multiple systems and provides pools of virtual resources to the

datacenter.

VMware components layers

As a cloud operating system, VMware vSphere manages large collections of infrastructure

(such as CPUs, storage, and networking) as a seamless and dynamic operating environment,

and also manages the complexity of a datacenter. The following component layers make up

VMware vSphere:

Infrastructure Services

Infrastructure Services are the set of services provided to abstract, aggregate, and allocate

hardware or infrastructure resources. Infrastructure Services can be categorized into:

VMware vCompute – the VMware capabilities that abstract away from underlying

disparate server resources. vCompute services aggregate these resources across many

discrete servers and assign them to applications.

VMware vStorage – the set of technologies that enables the most efficient use and

management of storage in virtual environments.

VMware vNetwork – the set of technologies that simplify and enhance networking in

virtual environments.

Application Services

Application Services are the set of services provided to ensure availability, security, and

scalability for applications. Examples include HA and Fault Tolerance.

VMware vCenter Server

VMware vCenter Server provides a single point of control of the datacenter. It provides

essential datacenter services such as access control, performance monitoring, and

configuration.

Clients

Users can access the VMware vSphere datacenter through clients such as the vSphere Client

or Web Access through a Web browser.


21



VMware components

The following figure shows the relationships between the component layers of VMware

vSphere.

Figure 2-11 VMware vSphere

VMware vSphere includes the following components:

VMware ESX or ESXi

VMware ESX or ESXi is a virtualization layer run on physical servers that abstracts processor,

memory, storage, and resources into multiple virtual machines.

VMware vCenter Server

VMware vCenter Server is the central point for configuring, provisioning, and managing

virtualized IT environments.


22



VMware vSphere Client

VMware vSphere Client is an interface that allows users to connect remotely to vCenter

Server or ESX/ESXi from any Windows PC.

VMware vSphere Web Access

VMware vSphere Web Access is a Web interface that allows virtual machine management and

access to remote consoles.

VMware Virtual Machine File System (VMFS)

VMFS is a high performance cluster file system for ESX/ESXi virtual machines.

VMware VMotion and Storage VMotion

VMware VMotion enables the live migration of running virtual machines from one physical

server to another with zero down time, continuous service availability, and complete

transaction integrity. Storage VMotion enables the migration of virtual machine files from one

datastore to another without service interruption. You can choose to place the virtual machine

and all its disks in a single location, or select separate locations for the virtual machine

configuration file and each virtual disk. The virtual machine remains on the same host during

Storage VMotion.

Migration with VMotion - Moving a powered-on virtual machine to a new host. Migration

with VMotion allows you to move a virtual machine to a new host without any interruption in

the availability of the virtual machine. Migration with VMotion cannot be used to move

virtual machines from one datacenter to another.

Migration with Storage VMotion - Moving the virtual disks or configuration file of a

powered-on virtual machine to a new datastore. Migration with Storage VMotion allows you

to move a virtual machine's storage without any interruption in the availability of the virtual

machine.

VMware High Availability (HA)

HA is a feature that provides high availability for applications running in virtual machines. If

a server fails, affected virtual machines are restarted on other production servers that have

spare capacity.

VMware Distributed Resource Scheduler (DRS)

DRS is a feature that allocates and balances computing capacity dynamically across

collections of hardware resources for virtual machines. This feature includes distributed

power management (DPM) capabilities that enable a datacenter to significantly reduce its

power consumption.

VMware Consolidated Backup (Consolidated Backup)

VMware Consolidated Backup is a feature that provides a centralized facility for agent-free

backup of virtual machines. It simplifies backup administration and reduces the impact of

backups on ESX/ESXi performance.


23



VMware Fault Tolerance (FT)

When Fault Tolerance is enabled for a virtual machine, a secondary copy of the original (or

primary) virtual machine is created. All actions completed on the primary virtual machine are

also applied to the secondary virtual machine. If the primary virtual machine becomes

unavailable, the secondary machine becomes active, providing continual availability.

vNetwork Distributed Switch (DVS)

Feature that includes a distributed virtual switch (DVS), which spans many ESX/ESXi hosts

enabling significant reduction of on-going network maintenance activities and increasing

network capacity. This allows virtual machines to maintain consistent network configuration

as they migrate across multiple hosts.

Pluggable Storage Array (PSA)

PSA is a storage partner plug-in framework that enables greater array certification flexibility

and improved array-optimized performance. PSA is a multipath I/O framework allowing

storage partners to enable their array asynchronously to ESX release schedules. VMware

partners can deliver performance-enhancing multipath load-balancing behaviors that are

optimized for each array.


24



VMware physical architecture

A typical VMware vSphere datacenter consists of basic physical building blocks such as x86

virtualization servers, storage networks and arrays, IP networks, a management server, and

desktop clients.

Figure 2-12 VMware vSphere datacenter physical topology

2.2.2 VMware Site Recovery Manager

VMware Site Recovery Manager (SRM) provides the workflow automation and process

management capabilities required for ensuring business continuity and disaster restart of

VMware Infrastructure. Site Recovery Manager uses modules called storage replication

adapters to leverage storage array replication. Using the storage replication adapter, the

product communicates with the utility managing the replication software through a

well-defined set of specifications. The protection of the VMware Infrastructure can extend

from individual replicated datastores to an entire virtualized site.

VMware’s virtualization of the data center offers advantages that can be extended to business

continuity and disaster recovery. These include:


25



The entire state of a virtual machine (memory, disk images, I/O, and device state) is

encapsulated. Encapsulation stores all information about a virtual machine as files.

Saving the state of a virtual machine to a file allows the transfer of an entire virtual

machine to another host.

Hardware independence eliminates the need for an exact replica of the hardware at the

recovery site. Because they are hardware independent, virtual machines can be restarted

on different hardware at the recovery site without requiring changes or reinstallation.

This flexibility eliminates the cost of purchasing and maintaining systems that cannot be

used productively.

Hardware independence allows an image of the system at the protected site to boot from

disk at the recovery site in minutes or hours instead of days.

VMware Site Recovery Manager leverages storage array-based replication between a

protection site and a recovery site. The workflow that is built into VMware Site Recovery

Manager automatically discovers the datastores that are replicated between the protected and

recovery sites. VMware Site Recovery Manager can be configured to support bi-directional

protection between two sites.

VMware Site Recovery Manager software is installed and configured on separate servers at

the protection site and at the recovery site. In addition, VMware Site Recovery Manager

requires the protected and recovery sites to be managed by their own VMware VirtualCenter

Server. VMware Site Recovery Manager is managed within VMware VirtualCenter, providing

a single point of management for virtual machines and the disaster restart process for those

virtual machines.

The data replication methods contains “VMware replication” and “Array-base replication”,

when “Array-base replication” is used, a plug-in called SRA is installed on vCenter Server to

execute storage commands need by SRM recovery tasks.

Figure 2-13 VMware SRM physical architecture


26



3 Multi-site hybrid cloud solution

3.1 Business Challenges

Data security and business continuity is always the key point of IT system. In nowadays, the

data becomes the core of business, which is much more important than fixed tangible assets

for enterprises, it is difficult simple measured in terms of money. Events like the Sichuan

earthquake and 911 showed that natural disasters (fire, flood, earthquake, etc) and

human-made disasters (misuse, viruses, etc) are unable to avoid, which bring heavy losses to

business. Enterprises must be able to efficiently protect and quickly recover data if a disaster,

mis-operation, or application fault occurs. To achieve these objectives, enterprises must not

only back up data locally, but also transfer the data to remote sites for storage. If no effective

data protection or remote data transfer methods are available, enterprises may suffer heavy

losses.

How to protect data, and ensure that the system is stable and reliable continuous, as well as

fast and reliable access, became the most important challenges in the construction of

information systems. Disaster recovery systems perfectly tackle the previous challenges. As

the core of a disaster recovery system, storage array–based remote replication is commonly

used.

Server virtualization brings lots of benefits to IT systems, and hybrid cloud is also a hot key

word in the last few years, which brings agile management to IT systems. How to build a

hybrid cloud with disaster recovery architecture becomes an issue for information system

builders.

3.2 Solution

3.2.1 Solution Architecture

This solution utilizes VMware SRM based on Huawei OceanStor V3 storages’ asynchronous

replication feature (HyperReplication/A) to implement a hybrid cloud across three sites, with

management services (active directory, domain services, shared file server, management

database and vCenter server) reside on public cloud, production VMs (ERP, CRM, and etc.)

reside on private site1, and testing VMs reside on private site2.

VMware SRM disaster recovery architecture is implemented between private site1 and site2.

Two recovery plans are created on VMware SRM side: Site1ToSite2 is used to protect VMs on

Site1, and Site2ToSite1 is used to protect VMs on Site2.


27



Data replication is based on HyperReplication/A (asynchronous replication) between

5800V3-Site1 and 5300V3-Site2. Two consistency groups are created: Site1ToSite2 is used to

replicate data from 5800V3-Site1 to 5300V3-Site2; Site2ToSite1 is used to replicate data from

5300V3-Site2 to 5800V3-Site1. The synchronous period is set to 10 seconds for the storage

consistency groups, thus the data lose time (RPO) is lower than 10 seconds.

Figure 3-1 Architecture of multi-site hybrid cloud solution

ESXi hosts are directly connected to storage via 16 Gbps FC (2 cables respectively connected to storage

controller A and B for each host).

Storages are directly connected to each other via 16 Gbps FC (2 cables respectively connect the two

storages’ controller A and B)

All management ports of ESXi hosts and storage controllers of Site1 and Site2 are connected Site3 via

WAN.

On Site1, datastore “os.site1” is used to store template VMs and OS disks of production VMs of Site1,

“vm1.site1 ~ vm4.site1” are used to store data disks of production VMs of Site1, “ph.site1” is used to

store placeholder VMs.

On Site2, datastore “os.site2” is used to store template VMs and OS disks of test/dev VMs of Site2,

“vm1.site2 ~ vm4.site2” are used to store data disks of test/dev VMs of Site2, “ph.site1” is used to store

placeholder VMs.

Infrastructure VMs run on Site3 (public cloud), Microsoft active directory and domain service run on

VM “AD-DNS”, samba service runs on VM “FileServer” (to store shared configuration files and log

files), vCenter sevices and SRM run on VM “vCenterSite1” and “vCenterSite2”, and SRM database

runs on VM “SRM-DB”.

OceanStor SRA is installed and configured on “vCenterSite1” and “vCenterSite2” to invoke storage

commands like “synchronize/split/switchover replication”, “create snapshot”, and “mapping to host”

when performing recovery tasks.

Production VMs runs on Site1, ERP database (Oracle 12C) runs on VM “ERP-DB-Site1”, ERP

application runs on “ERP-APP-Site1”, CRM database (SQL Server 2014) runs on VM

“CRM-DB-Site1”, and CRM application runs on “CRM-APP-Site1”.


28



Testing/Developing VMs runs on Site2, ERP database (Oracle 12C) runs on VM “ERP-DB-Test-Site2”,

ERP application runs on “ERP-APP-Test-Site2”, CRM database (SQL Server 2014) runs on VM

“CRM-DB-Test-Site2”, and CRM application runs on “CRM-APP-Test-Site2”.

3.2.2 Solution Configuration

Table 3-1 Hardware configuration

Site Item Specifications Quantity

Site1 ESXi hosts Huawei Tecal RH2288 V2 each with:

256GB memory

2 – Intel Xeon E5-2620 CPUs

1 – Intel 10Gbps Ethernet HBA card

1 – Qlogic 16Gbps FC HBA card

4

Private 10GE Switch Huawei S6700 1

Storage Huawei OceanStor 5800 V3 with:

2 – controllers each with 64GB cache

2 – 2U 25 slots disk enclosures

50 – 600GB 10K RPM SAS disks

6 – 16Gbps FC I/O modules (2 ports)

2 – 24Gbps SAS I/O modules (4 ports)

1

Site2 ESXi hosts Huawei Tecal RH2288 V2 each with:

256GB memory

2 – Intel Xeon E5-2620 CPUs

1 – Intel 10Gbps Ethernet HBA card

1 – Qlogic 16Gbps FC HBA card

2

Private 10GE Switch Huawei S6700 1

Storage Huawei OceanStor 5300 V3 with:

2 – controllers each with 16GB cache

1 – 2U 25 slots disk enclosures

12 – 2TB 7.2K RPM NL-SAS disks

25 – 600GB 10K RPM SAS disks

4 – 16Gbps FC I/O modules (2 ports)

1


29



Table 3-2 Software configuration

Site Item Software

Site1/Site2 ESXi hosts ESXi 5.5.0.update02-2068190

VM OS Red Hat Enterprise Linux 6.5

Microsoft Windows 2012 R2

Database Oracle Database 12.1.0.2

Microsoft SQL Server 2014

Application (Test tool

simulated) Huawei Order Entry test suite 1.1

Huawei TPC-E Like test suite 1.0

Storage Huawei UltraPath 8.01.023

Huawei OceanStor V3 HyperReplication License

Huawei OceanStor V3 HyperSnap License

Huawei OceanStor V3 SmartThin License

Site3 VM OS Red Hat Enterprise Linux 6.5

Microsoft Windows 2012 R2

Database Microsoft SQL Server 2014

Application VMware vCenter 5.5.0-2183112

VMware-srm-5.5.1-1647061

OceanStor SRA 2.0


30



Figure 3-2 Storage configuration of 5800V3-Site1

[1] The RAID level of storage pool is RAID6-10.

[2] All of the LUNs are thin provisioned, the initial capacity of Site1_PH is 2GB, and the initial capacity

of other LUNs is 500GB.

Figure 3-3 Storage configuration of 5300V3-Site2

[1] The RAID level of storage pool is RAID6-10.

[2] All of the LUNs are thin provisioned, the initial capacity of Site2_PH is 2GB, and the initial capacity

of other LUNs is 500GB.


31



3.3 Planning

3.3.1 Solution build-up

Figure 3-4 Solution build-up steps

Step 1: prepare physical environment

Step 2: configure active directory and domain services

Step 3: configure file share service

Step 4: configure database for SRM

Step 5: configure vCenter and ESXi

Step 6: configure storage and datastores

Step 7: build-up virtual machines

Step 8: configure secondary LUNs and replication

Step 9: Install VMware SRM and OceanStor SRA

Step 10: configure SRM (create SRM connection, configure resource mappings, configure array manager,

configure protection groups, configure recover plans, and test recover plans)


32



Table 3-3 IP configuration

Site Host Management IP / 16 Private IP / 25

Site1

esx5.site1 100.148.122.105 192.168.122.105

esx6.site1 100.148.122.106 192.168.122.106

esx7.site1 100.148.122.107 192.168.122.107

esx8.site1 100.148.122.108 192.168.122.108

5800V3-Site1 A 100.148.52.200

5800V3-Site1 B 100.148.52.201

CRM-DB-Site1 100.148.122.163 192.148.122.165

CRM-APP-Site1 100.148.122.164 192.148.122.166

ERP-DB-Site1 100.148.122.165 192.148.122.165

ERP-APP-Site1 100.148.122.166 192.148.122.166

Site2

esx7.site2 100.148.112.107 192.168.112.107

esx8.site2 100.148.112.108 192.168.112.108

5300V3-Site2 A 100.148.42.203

5300V3-Site2 B 100.148.42.204

CRM-DB-Test-Site2 100.148.112.163 192.148.112.165

CRM-APP-Test-Site2 100.148.112.164 192.148.112.166

ERP-DB-Test-Site2 100.148.112.165 192.148.112.165

ERP-APP-Test-Site2 100.148.112.166 192.148.112.166

Site3

AD-DNS 100.148.122.160

vCenterSite1 100.148.122.161

vCenterSite2 100.148.122.162

CRM-DB 100.148.122.159

FileServer 100.148.122.21


33



Table 3-4 VM configuration

VM vCPU Memory vDisk Datastore

CRM-DB-Site1 8 32GB

scsi0:0 100GB (Thin) os.site1


scsi0:2 100GB (Thin) vm1.site1




CRM-App-Site1 2 8GB scsi0:0 100GB (Thin) os.site1

ERP-DB-Site1 8 32GB







ERP-App-Site1 2 8GB scsi0:0 100GB (Thin) os.site1

Template-Windows2012R2-Site1 2 8GB scsi0:0 100GB (Thin) os.site1

Template-SQLServer2014-Site1 8 32GB scsi0:0 100GB (Thin) os.site1

Template-RedHat65-Site1 2 8GB scsi0:0 100GB (Thin) os.site1

Template-Oracle12C-Site1 8 32GB scsi0:0 100GB (Thin) os.site1

8 32GB







CRM-App-Test-Site2 2 8GB scsi0:0 100GB (Thin) os.site2

ERP-DB-Test-Site2 8 32GB







34





ERP-App-Test-Site2 2 8GB scsi0:0 100GB (Thin) os.site2

Template-Windows2012R2-Site2 2 8GB scsi0:0 100GB (Thin) os.site2

Template-SQLServer2014-Site2 8 32GB scsi0:0 100GB (Thin) os.site2

Template-RedHat65-Site2 2 8GB scsi0:0 100GB (Thin) os.site2

Template-Oracle12C-Site2 8 32GB scsi0:0 100GB (Thin) os.site2


35



3.3.2 Planned migration test

Figure 3-5 Planned migration test steps

Step 1: start ERP application with a RPO timer

Step 2: start CRM application with a RPO timer

Step 3: execute “planned migration” on recovery plan “Site1ToSite2”, migrate VMs from Site1 to Site2

Step 4: re-protect VMs after migration

Step 5: check RPO timer log

Step 6: re-start ERP & CRM application with RPO timers, check the performance on Site2

Step 7: execute “planned migration” on recovery plan “Site1ToSite2”, migrate VMs back to Site1



Step 10: re-start ERP & CRM application, check the performance on Site1

Step 11: collect test results and calculate RPO and RTO


36



3.3.3 Disaster recovery test

Figure 3-6 Disaster recovery test steps

Step 1: start ERP application with a RPO timer

Step 2: start CRM application with a RPO timer

Step 3: power off Site1 to simulate disaster

Step 4: execute “disaster recovery” on recovery plan “Site1ToSite2”, check the recover steps and status

Step 5: after recovered, check the RPO timer log

Step 6: power on Site1 to simulate disaster failback

Step 7: re-execute “disaster recovery” on recovery plan “Site1ToSite2” to fix errors in step4 (VM power

and storage commands failed because of disaster)

Step 8: re-protect VMs

Step 9: re-start ERP & CRM application with RPO timers, check the performance on Site2

Step 10: execute “planned migration” on recovery plan “Site1ToSite2”, migrate VMs back to Site1



Step 13: re-start ERP & CRM application, check the performance on Site1

Step 14: collect test results and calculate RPO and RTO


37



3.4 Infrastructure build-up

3.4.1 Prepare physical enviorment

Connect the physical components refer to the solution architecture. Connect the management

network of Site1, Site2 and Site3 (public cloud).

3.4.2 Configure active directory and domain service

Deploy a VM on Site3 (public cloud) using Windows 2012 R2 Template, name the VM and

computer name as “AD-DNS”. Add active directory role and configure the domain service,

create a domain “cdlab.huawei.com”.

3.4.3 Configure file share service

Deploy VM “FileServer” on Site3 using the Red Hat Enterprise 6.5 template, and configure

Samba service.

3.4.4 Configure database for SRM

Deploy a VM on Site3 (public cloud) using Windows 2012 R2 Template, name the VM and

computer name as “SRMDB-Site3”, join to the domain “cdlab.huawei.com”.

Install SQL Server 2014. Create two databases “SRMDB_Site1” and “SRMDB_Site2”.

Create SQL Server login “srmuser_site1” and “srmuser_site2”. Enable remote access to SQL

Server instance.

3.4.5 Configure vCenter and ESXi

Deploy VM “vCenterSite1” and “vCenterSite2” on Site3 using the Windows 2012 R2

template, join to the domain “cdlab.huawei.com”, add “.net framework 3.5” feature, install

vCenter Server 5.5.

Install ESXi 5.5 on all RH2288 V2 of Site1 and Site2, set the host name to “esx5.site1”,

“esx6.site1”, “esx7.site1”, “esx8.site1”, “esx7.site2” and “esx8.site2”, enable SSH for each

hosts. Add ESXi servers of Site1 to vCenterSite1, and ESXi hosts of Site2 to vCenterSite2.

Install OceanStor UltraPath on all ESXi servers. Install OceanStor UltraPath plugin on

“vCenterSite1” and “vCenterSite2”.


38



3.4.6 Configure storage and datastore

Step 1 Configure storage “5800V3-Site1”

Login to “5800V3-Site1” Device Manager via Chrome browser using the URL:

https://100.148.52.200:8088

Figure 3-7 Login to “5800V3-Site1”

Go to “Provisioning”, click “Create Disk Domain”, and name the disk domain as

“VMware.Site1”, .add 50 - 600 GB SAS disks.

Figure 3-8 Storage resource provisioning

https://100.148.52.200:8088/


39



Click “Create Storage Pool”, and name the storage pool as “VMware.Site1”, set disk domain

to “VMware.Site1”, set RAID policy to “RAID6-8D2P”, set Capacity to 8TB.

Figure 3-9 Create storage pool


40



Click “Create LUN”, and name the LUN to “VMware.Site1.OS”, enable “SmartThin”, set

“Capacity” to 4 TB and “Initially Allocated Capacity” to 500 GB, click “OK” to create.

Figure 3-10 Create LUN “VMware.Site1.OS”


41



Click “Create LUN”, and name the LUN to “VMware.Site1.PH”, enable “SmartThin”, set

“Capacity” to 5 GB and “Initially Allocated Capacity” to 1 GB, click “OK” to create.

Figure 3-11 Create LUN “VMware.Site1.PH”


42



Click “Create LUN”, and name the LUN prefix to “VMware.Site1.VM”, enable “SmartThin”,

set “Capacity” to 4 TB and “Initially Allocated Capacity” to 500 GB, set quantity to 4, click

“OK” to create.

Figure 3-12 Create LUN “VMware.Site1.VM_001~004”

Click “Create LUN Group”, name it as “VMware.Site1”, add the LUNs to the LUN Group.


43



Go to “Host > Host”, click “Create > Automatic Scan”, the 4 ESXi hosts are automatic added.

Figure 3-13 Automatic scan hosts

Go to “Host Group”, click “Create”, set the name to “VMware.Site1”, add the 4 hosts.

Figure 3-14 Create Host Group

Go to “Host > Mapping View”, click “Create”, name it as “VMware.Site1”, choose LUN

Group “VMware.Site1” and Host Group “VMware.Site1”, click “OK” to finish.


44



Step 2 Configure storage “5300V3-Site2”

Login to “5300V3-Site2” Device Manager via Chrome browser using the URL:

https://100.148.42.203:8088, refer step 16, create Disk Domain “VMware.Site2” using 25

SAS disks, create Storage Pool “VMware.Site2” with 8 TB capacity, create LUN

“VMware.Site2.OS” with 4TB capacity and 500GB initial size, create LUN

“VMware.Site2.PH” with 5 GB capacity and 1 GB initial size, create LUNs

“VMware.Site2.VM_001~004” with 4TB capacity and 500GB initial size, create LUN Group,

Host, Host Group, and Mapping View.

Step 3 Configure datastores on “vCenterSite1”

Login to “vCenterSite1” via vClient, go to “Hosts and Clusters > Site1 > Cluster1 >

100.148.122.105 (esx5.site1) > Configuration > Storage”, click “Rescan All”, after the task is

complete, click “Add Storage”, create datastore “os.site1”, “ph.site1”, “vm1.site1 ~ vm4.site1”

on LUNs “VMware.Site1.OS”, “VMware.Site1.PH”, “VMware.Site1.VM_001 ~ 004”.

Figure 3-15 Create datastores on “vCenterSite1”

Step 4 Configure datastores on “vCenterSite2”

Login to “vCenterSite2” via vClient, go to “Hosts and Clusters > Site2 > Cluster2 >

100.148.122.107 (esx7.site2) > Configuration > Storage”, click “Rescan All”, after the task is

complete, click “Add Storage”, create datastore “os.site2”, “ph.site2”, “vm1.site2 ~ vm4.site2”

on LUNs “VMware.Site2.OS”, “VMware.Site2.PH”, “VMware.Site2.VM_001 ~ 004”.

Figure 3-16 Create datastores on “vCenterSite2”

https://100.148.42.203:8088/


45



3.5 Virtual machine build-up

3.5.1 Template VMs build up

Step 1 Create the following template VMs on Site1.

Table 3-5 Template VMs on Site1

VM OS Application vDisk Datastore

Template-Windows201

2R2-Site1

Windows 2012 R2

Enterprise

100GB (Thin) os.site1

Template-SQLServer20

14-Site1

Windows 2012 R2

Enterprise

SQL Server

2014



Template-RedHat65-Sit

e1

Red Hat Enterprise

Linux 6.5


Template-Oracle12C-Si

te1

Red Hat Enterprise

Linux 6.5

Oracle 12C

Database



Step 2 Create the following template VMs on Site2

Table 3-6 Template VMs on Site2

VM OS Application vDisk Datastore

Template-Windows201

2R2-Site2

Windows 2012 R2

Enterprise


Template-SQLServer20

14-Site2

Windows 2012 R2

Enterprise

SQL Server

2014



Template-RedHat65-Sit

e2

Red Hat Enterprise

Linux 6.5


Template-Oracle12C-Si

te2

Red Hat Enterprise

Linux 6.5

Oracle 12C

Database




46



3.5.2 CRM application build up

Step 1 Create VM “CRM-DB-Site1” from template

Go to “vCenterSite1”, clone VM “Template-SQLServer2014-Site1” to “CRM-DB-Site1”,

configure the VM properties as following, customize the VM, and power on it. Configure IP

address and join to domain “cdlab.huawei.com”.

Table 3-7 Configuration of VM “CRM-DB-Site1”


CRM-DB-Site1 8 32GB scsi0:0 100GB (Thin) os.site1






Step 2 Generate test data

Mount “Disk 1” to “C:\SQLDB\System”. Create NTFS with 64KB allocation unit on “Disk 2

~ 5”. Mount them to “C:\SQLDB\Disk1~4”.

Start SQL Server services. Create database and generate TPC-E like data (10000 customers).

Step 3 Create VM “CRM-App-Site1” from template

Go to “vCenterSite1”, clone VM “Template-Windows2012R2-Site1” to “CRM-App-Site1”,


address and join to domain “cdlab.huawei.com”.

Table 3-8 Configuration of VM “CRM-App-Site1”


CRM-App-Site1 2 4GB scsi0:0 100GB (Thin) os.site1

Install TPC-E like benchmark tool. Install SQL Server ODBC and SQLCMD.

Mount shared directory of Site3.


47



Step 4 Configure workload and timer script

Create batch file “C:\run.bat” as follows:

@echo off

start cmd /c "C:\MSTPCE.1.12.0-1021\EGen\SUT_CE_Server\Release\SUT_CE_Server.exe -s

crm-db -d crmdb"

start cmd /c "C:\MSTPCE.1.12.0-1021\EGen\SUT_MEE_Server\Release\SUT_MEE_Server.exe

-s crm-db -d crmdb"

"C:\Program Files (x86)\Benchcraft\Benchcraft.exe"

taskkill /F /IM SUT_CE_Server.exe

taskkill /F /IM SUT_MEE_Server.exe

Create batch file “C:\timer.bat” as follows:

@echo off

del Z:\timer.log

sqlcmd -S crm-db -d crmdb -Q "DROP TABLE TIMER"

sqlcmd -S crm-db -d crmdb -Q "CREATE TABLE TIMER (ID int IDENTITY(1,1), TMSTMP DATETIME)"

for /L %%i in (1,1,14400) DO (

set /p="%%i, "<nul >> Z:\timer.log

sqlcmd -S crm-db -d crmdb -Q "print sysdatetime()" >> Z:\timer.log

sqlcmd -S crm-db -d crmdb -Q "set nocount on;insert into TIMER (TMSTMP)

VALUES(SYSDATETIME())"

sqlcmd -S crm-db -d crmdb -Q "WAITFOR DELAY '00:00:01'"

)

3.5.3 ERP application build up

Step 1 Create VM “CRM-DB-Site1” from template

Go to “vCenterSite1”, clone VM “Template-Oracle12C-Site1” to “ERP-DB-Site1”, configure

the VM properties as following, and power on it. Configure IP address and host name.

Table 3-9 Configuration of VM “ERP-DB-Site1”


ERP-DB-Site1 8 32GB scsi0:0 100GB (Thin) os.site1






Step 2 Generate test data

Create logical volume and file system. Create database and generate Order Entry data

(200 scale) using Huawei SwingBench Order Entry Test Suite 1.1.


48



Step 3 Create VM “ERP-App-Site1” from template

Go to “vCenterSite1”, clone VM “Template-RedHat65-Site1” to “CRM-App-Site1”,


address and host name. Configuration of VM “ERP-App-Site1”


ERP-App-Site1 2 4GB scsi0:0 100GB (Thin) os.site1

Configure SSH authentication to “ERP-DB”. Mount shared directory of “FileServer” resides

on Site3 (Public Cloud). Install and configure Oracle Instant Client.

Step 4 Configure workload and timer script

Upload “oe12c_suite_1.1.tar.gz” (Huawei SwingBench Order Entry Test Suite 1.1) to the

“/home/oracle” directory and unpack it Go to “oe12c” directory, and create script “timer.sh”

for RPO counting.

vi timer.sh

sqlplus -silent oe/oe@erp-db/erpdb <<EOF

DROP TABLE TIMER;

CREATE TABLE TIMER (ID NUMBER, TMSTMP TIMESTAMP);

COMMIT;

EOF

rm -rf /home/oracle/erpshare/timer.log

for ((i=0;i<=14400;i++))

do

(

sqlplus -silent oe/oe@erp-db/erpdb <<EOF

set pagesize 0

select $i || ', ' || to_char(systimestamp, 'YYYY-MM-DD HH24:MI:SS.FF') from dual;

insert into TIMER VALUES($i, systimestamp);

commit;

exit

EOF

) | tee -a /home/oracle/erpshare/timer.log

sleep 1

done


49



3.5.4 Applicaion build up for Site2

Refer 3.5.2 ~ 3.5.5, build up the following workload VMs on vCenterSite2 (typical test

environment for enterprise).

Table 3-10 Configuration of VMs on Site2


CRM-DB-Test-Site2 8 32GB







CRM-App-Test-Site2 2 4GB scsi0:0 100GB (Thin) os.site2

ERP-DB-Test-Site2 8 32GB







ERP-App-Test-Site2 2 4GB scsi0:0 100GB (Thin) os.site2


50



3.6 Configure storage replication

3.6.1 Add remote device on 5800V3-Site1

Step 1 Login in to Device Manager

Login in to 5800V3-Site1’s Device Manager via Chrome using the URL:

https://100.148.52.200:8088/.

Step 2 Add remote device

Go to “Data Protection > Remote Device”, Click “Add Remote Device”, Choose

“Manufacture of existing device”, Click “Next”.

Figure 3-17 Add remote device

https://100.148.52.200:8088/


51



Step 3 Configure remote link and credential

Choose one of the link, input remote connection user name (default: mm_user) and password

(default: mm_user@storage), Click “Next”, Click “Finish”.

Figure 3-18 Configure remote link and credential


52



Step 4 Add more remote links

Check on the option “Continue adding links”, click “Close”; check on the second link, Click

“OK”.

Figure 3-19 Add more remote links


53



Figure 3-20 Choose another remote link

3.6.2 Create remote replication on 5800V3-Site1

Step 1 Create replication target on 5300V3-Site2

Login in to 5300V3-Site2 via SSH, create replication target LUNs for 5800V3-Site1.

create lun name=VMware.Site1.OS pool_id=0 capacity=4TB lun_type=thin

initial_capacity=500GB

create lun name=VMware.Site1.VM_001 pool_id=0 capacity=4TB lun_type=thin









54



Step 2 Create remote replication

Login in to 5800V3-Site1’s Device Manager, go to go to “Data Protection > Remote

Replication”, Click “Create”; Choose “Asynchronous”, Click “Next”.

Figure 3-21 Create remote replication


55



Step 3 Add LUN pairs

Select Remote Device “5300V3-Site2”, Add LUN pairs for “VMware.Site1.OS” and

“VMware.Site1.VM_001~004”, Click “Next”.

Figure 3-22 Add LUN pairs


56



Step 4 Configure replication policies

Change Speed to “Highest”, Synchronization Method to “Timed wait when synchronization

begins”, set Interval to 10 seconds, click “Next”.

Figure 3-23 Replication policies


57



Step 5 Configure consistency group

Check on “Create a consistency group …” and click “Next” to finish.

Figure 3-24 Create consistency group

Step 6 Check the replication status

Figure 3-25 Replication status


58



Step 7 Add replication target to LUN group on 5300V3-Site2

Login in to 5300V3-Site2’s Device Manager, add replication target LUNs to LUN group

“VMware.Site2”.

Figure 3-26 Add target LUNs to LUN group

3.6.3 Create remote replication on 5300V3-Site2

Step 1 Create replication target on 5800V3-Site1

Login in to 5800V3-Site1 via SSH, create replication target LUNs for 5300V3-Site2.

create lun name=VMware.Site2.OS pool_id=0 capacity=4TB lun_type=thin










Step 2 Create remote replication

Login to 5300V3-Site2’s Device Manager ,and repeat the steps of 3.6.3 (Choose LUNs of

Site2) to create remote replication to 5800V3-Site1.


59



Step 3 Check the status of replication

Figure 3-27 Replication status

Step 4 Add replication target to LUN group on 5800V3-Site1

Login in to 5800V3-Site1’s Device Manager, add replication target LUNs to LUN group

“VMware.Site1”.

Figure 3-28 Add target LUNs to LUN group


60



3.7 Configure VMware SRM

3.7.1 Install SRM and configure resource

Install SRM and OceanStor SRA 2.0 on “vCenterSite1” and “vCenterSite2”.

Login vCenterSite1 using vClient, Go to “Home > Solutions and Applications > Site

Recovery”, right click vcentersite1, click “Configure connection”. Input remote vCenter

address and credentials.

Figure 3-29 Configure connection

Configure resource mappings, folder mappings, network mappings, and placeholder datastore

for “vCenterSite1” and “vCenterSite1”.

3.7.2 Configure array manager

Step 1 Rescan SRA

Go to “Array Managers > vCenterSite1/2 > SRAs”, click “Rescan SRA”.

Figure 3-30 Huawei OceanStor SRA information


61



Step 2 Add array manager for vCenterSite1

Go to “Array Managers > vCenterSite1”, click “Add Array Manager”. Configure display

name and credentials.

Figure 3-31 Configure address and credentials


62



Step 3 Add array manager for vCenterSite2

Go to “Array Managers > vCenterSite2”, click “Add Array Manager”. Configure display

name and credentials.

Figure 3-32 Configure address and credentials


63



Step 4 Enable array pair

Go to “5800V3-Site1 > Array Pairs”, Click “Enable”.

Figure 3-33 Enable array pair

Go to “5800V3-Site1 > Devices”, check the protected devices.

Figure 3-34 Protected devices


64



3.7.3 Configure protection groups

Step 1 Create protection group for vCenterSite1

Go to “Protection Groups”, and click “Create Protection Group”. Choose sites, array pair and

datastores.

Figure 3-35 Select datastore


65



Step 2 Create protection group for vCenterSite2

Go to “Protection Groups”, click “Create Protection Group”. Choose sites, array pair and

datastores.

Figure 3-36 Select datastore


66



3.7.4 Configure recover plans

Step 1 Create recover plan for vCenterSite1

Go to “Recovery Plans”, click “Create Recovery Plan”. Choose site and protection group. Set

test network to “Auto” and recovery plan name to “Site1ToSite2”.

Login to “vCenterSite2” via vClient, go to “Hosts and Clusters”, and check the existence of

placeholder VMs.

Figure 3-37 Check placeholder VMs


67



Step 2 Create recover plan for vCenterSite2

Go to “Recovery Plans”, click “Create Recovery Plan”. Choose site and protection group. Set

test network to “Auto” and recovery plan name to “Site2ToSite1”.

Login to “vCenterSite1” via vClient, go to “Hosts and Clusters”, and check the existence of

placeholder VMs.

Figure 3-38 Check placeholder VMs


68



Step 3 Configure SRA parameters

Login vCenterSite1 using vClient, Go to “Home > Solutions and Applications > Site

Recovery”, right click vCenterSite1, click “Advanced Settings”, go to “storage”, change the

default value 300 to 600 for the parameter storage.commandTimeout.

Figure 3-39 Configure SRA parameters

Repeat the configuration for vCenterSite2.


69



Step 4 Test recover plan for vCenterSite1

Login “vCenterSite1” using vClient, Go to “Home > Solutions and Applications > Site

Recovery > Recovery Plans”, choose “Site1ToSite2”, click “Test”.

Figure 3-40 Test recovery plan

Figure 3-41 Test Confirmation


70



Figure 3-42 Review test information

Go to “Site1ToSite2 > Recovery Steps”, check the recovery status.

Figure 3-43 Test steps


71



Figure 3-44 Test result

Login to 5300V3-Site2, go to “Provisioning > LUN > LUN Groups > VMware.Site2”, check

the snapshot mapped to the LUN group.

Figure 3-45 Storage snapshots

Login to “vCenterSite2”, go to “Hosts and Clusters > Virtual Machines > Search > Site1”,

check the status of tested virtual machines.

Figure 3-46 Status of tested VMs

Step 5 Clean up the recovery test for vCenterSite1

Return to vCenterSite1 Recovery Plan “Site1ToSite2”, Click “Cleanup” > “Next” > “Start”.


72



Figure 3-47 Cleanup tested recovery plan

Figure 3-48 Check the cleanup steps and results


73



Step 6 Test recover plan for vCenterSite2

Login “vCenterSite1” using vClient, Go to “Home > Solutions and Applications > Site

Recovery > Recovery Plans”, choose “Site2ToSite1”, click “Test”.

Figure 3-49 Test result

Login to 5800V3-Site1, go to “Provisioning > LUN > LUN Groups > VMware.Site1”, check

the snapshot mapped to the LUN group.

Figure 3-50 Storage snapshots


74



Login to “vCenterSite1”, go to “Hosts and Clusters > Virtual Machines > Search > Site2”,

check the status of tested virtual machines.

Figure 3-51 Status of tested VMs

Step 7 Clean up the recovery test for vCenterSite2

Return to vCenterSite1 Recovery Plan “Site2ToSite1”, Click “Cleanup” > “Next” > “Start”.

Figure 3-52 Cleanup tested recovery plan


75



Figure 3-53 Check the cleanup steps and results

3.8 Run planned migration from Site1 to Site2

3.8.1 Start application and RPO timer

Start ERP workload and RPO timer scripts on “ERP-APP-Site1”.

Start CRM workload and RPO timer scripts on “CRM-APP-Site1”.

3.8.2 Migrate VMs to Site2

Step 1 Invoke planned migration

Login to vCenterSite1 via vClient, go to “Site Recovery > Recovery Plans > Site1ToSite2”,

Click “Recovery”.

Figure 3-54 Begin recovery


76



Choose “Planned Migration”, click “next”.

Figure 3-55 Choose Planned Migration

Step 2 Check recovery steps

Go to “Recovery Steps”, check the status of each step when running recovery.

Figure 3-56 Recovery steps


77



Step 3 Check the status of recovered VMs

Login to vCenterSite2, go to “Hosts and Clusters > Site2 > Virtual Machines”, Search “Site1”

check the status of recovered virtual machines from Site1.

Figure 3-57 Status of recovered VMs

Step 4 Check the status of consistency groups

Go to “Site Recovery > Array Managers > 5300V3-Site2 > Devices”, check the consistency

groups’ status, it’s “split”.

Figure 3-58 Status of consistency group on SRA


78



3.8.3 Re-protect VMs after migration

Step 1 Invoke re-protect

Go to “vCenterSite1 > Site Recovery > Recovery Plan > Site1ToSite2”, Click “Reprotect”.

Figure 3-59 Invoke re-protect

Figure 3-60 Re-protect confirmation


79




Go to “Recovery Steps”, Check the status of each step when running re-protect.

Figure 3-61 Re-protect steps



groups’ status.

Figure 3-62 Consistency group status on SRA


80



3.8.4 Check RPO timer log

Step 1 Check ERP application timer

Login to virtual machine “ERP-APP-Site1” via SSH as user oracle, check the last timestamp

in ERP Database and the share folder from “FileServer” resides on Site3.

$ ssh oracle@ERP-APP-Site1

sqlplus oe/oe@erp-db/erpdb

SQL> select * from (select id, to_char(tmstmp,'YYYY-MM-DD HH24:MI:SS.FF') from timer

order by tmstmp desc) where rownum = 1;

ID TO_CHAR(TMSTMP,'YYYY-MM-DDHH2

---------- -----------------------------

976 2015-01-31 16:27:20.252199

SQL> exit

$ tail -n 1 erpshare/timer.log

976, 2015-01-31 16:27:20.250587

Step 2 Check CRM application timer

Login to virtual machine “CRM-APP-Site1” via MSTSC, check the last timestamp in CRM

Database and the share folder from “FileServer” resides on Site3.

sqlcmd -S crm-db -d crmdb -Q "select top 1 * from TIMER order by TMSTMP desc"

ID TMSTMP

----------- -----------------------

929 2015-01-31 16:27:20.237

type Z:\timer.log

……

929, 2015-01-31 16:27:20.1895644


81



3.8.5 Re-start application and RPO timer

Set the number of users to 50% of Site1and re-start ERP and CRM applications and RPO

timers.

3.8.6 Migrate VMs back to Site1

Repeat the steps of 3.8.2 to migrate VMs back to Site1. Check the recovery steps and VMs’

status after recovery.




82



3.8.7 Re-protect VMs after migration back

Repeat the steps of 3.8.3 to re-protect the VMs.











---------- -----------------------------

391 2015-01-31 17:09:06.967472

SQL> exit


391, 2015-01-31 17:09:06.966235





ID TMSTMP

----------- -----------------------

272 2015-01-31 17:09:06.197

type Z:\timer.log

272, 2015-01-31 17:09:06.1576491


83



3.8.9 Collect test result

Collect the transaction logs and I/O performance log of ERP & CRM application.

The following figure shows the transaction throughput performance (tps) of CRM application

during the planned migration process. The performance of CRM on Site1 in “No Protect”

state (before SRM configured), “Protected” state (SRM configured) and “Protect Broken”

state (Migrate to Site2 and back to Site1) are almost the same.

Figure 3-66 CRM transaction throughput

The number of users configuration is 20 when CRM running on Site1, and 10 on Site2.

The following figure shows the transaction throughput performance (tps) of ERP application

during the planned migration process. The performance of ERP on Site1 in “No Protect” state

(before SRM configured), “Protected” state (SRM configured) and “Protect Broken” state

(Migrate to Site2 and back to Site1) are almost the same.

Figure 3-67 ERP transaction throughput

The number of users configuration is 40 when ERP running on Site1, and 20 on Site2.

1108 1120

370 391

1144

0

200

400

600

800

1000

1200

1400

Site1 - No Protect

Site1 - Protected

Site2 - Protect Broken

Site2 - Protected


CRM tps

276.31 278.44

127.37 128.44

275.09

0

50

100

150

200

250

300

Site1 - No Protect

Site1 - Protected


Site2 - Protected


ERP tps


84



3.9 Run disaster recovery from Site1 to Site2

3.9.1 Start application and RPO timer

Start ERP workload and RPO timer scripts on “ERP-APP-Site1”.

Start CRM workload and RPO timer scripts on “CRM-APP-Site1”.

3.9.2 Simulate disaster

Step 1 Shutdown the power of Site1

Shutdown the power of Site1 to simulate disaster happened to the whole data center.

Step 2 Check the running status of Site1

Login to vCenterSite1 via vClient, go to “Hosts and Clusters”, and check the running status of

physical and virtual machines.

Figure 3-68 Running status of Site1


85



3.9.3 Recover VMs to Site2

Step 1 Invoke disaster recovery




Choose “Disaster Recovery”, click “next”.

Figure 3-70 Choose Disaster Recovery


86




Go to “Recovery Steps”, check the status of each step when running recovery.


Step 3 Check the status of recovered VMs

Login to vCenterSite2, go to “Hosts and Clusters > Site2 > Virtual Machines”, Search “Site1”

check the status of recovered virtual machines from Site1.



87




Check the Remote Replication consistency group’ status on the Device Manager of

5300V3-Site2, it’s interrupted.

Figure 3-73 Status of consistency group on Device Manager










---------- -----------------------------

538 2015-02-02 11:50:28.453449

SQL> exit


542, 2015-02-02 11:50:32.705360





ID TMSTMP

----------- -----------------------

435 2015-02-02 11:50:28.640

type Z:\timer.log

439, 2015-02-02 11:50:33.1979021


88



3.9.5 Simulate disaster failback

Step 1 Turn on the power of Site1

Turn on the power of Site1 to simulate disaster recovered.

Step 2 Check the running status of Site1

Login to vCenterSite1 via vClient, go to “Hosts and Clusters”, and check the running status of

physical and virtual machines.

Figure 3-74 Running status of Site1


89





groups’ status, it’s “split”.

Figure 3-75 Status of consistency group on SRA

3.9.6 Run recovery plan again

Step 1 Invoke recover again





90



3.9.7 Reprotect the VMs fail to Site2

Go to “vCenterSite1 > Site Recovery > Recovery Plan > Site1ToSite2”, Click “Reprotect”.

Check the recovery steps and consistency groups’ status after re-protect complete.

Figure 3-77 Invoke re-protect


Figure 3-79 Consistency groups’ status on SRA


91



3.9.8 Re-start application and RPO timer

Set the number of users to 50% of Site1and re-start ERP and CRM applications and RPO

timers.

3.9.9 Migrate VMs back to Site1

Repeat the steps of 3.8.6.

3.9.10 Re-protect VMs after migration back

Repeat the steps of 3.8.7.










---------- -----------------------------

458 2015-02-02 14:04:15.593342

SQL> exit


458, 2015-02-02 14:04:15.591405





ID TMSTMP

----------- -----------------------

55 2015-02-02 14:08:02.600

type Z:\timer.log

55, 2015-02-02 14:08:02.5571263

3.9.12 Collect test result

Collect the transaction logs and I/O performance log of ERP & CRM application.


92



3.10 Conclusion

3.10.1 Functionbility

As the above steps proven, this solution is able to protect VMs between Site1 and Site2. The

solution test result shows that customer could benefit from the architecture to avoid data

losing and business interruption. Thanks to the seconds-level synchronize frequency of

OceanStor V3 HyperReplication/A and the closely integrated plug-in OceanStor SRA 2.0

with VMware SRM, the solution is able to recover VMs when disaster happens with the RPO

lower than 10 seconds and RTO lower to 5 minutes.

Table 3-11 Recovery point (data loss time)

Application Disaster happen Recovered to Data loss time

ERP 2015-02-02 11:50:32.705360 2015-02-02 11:50:28.453449 4.252 sec

CRM 2015-02-02 11:50:33.1979021 2015-02-02 11:50:28.640 4.558 sec

The synchronous policy is set to “Wait 10 seconds when synchronization begins” for the storage’s

consistency group (the shortest time OceanStor V3 supported), so RTO is lower than 10 seconds.

Table 3-12 Recovery Time (business interrupt time)

Disaster happen Recovery begin Recovery

complete

Recover Time Interrupt time

2015-02-02

11:50:32

2015-02-02

11:52:15

2015-02-02

11:57:04

4.8 min 6.5 min

The recovery of SRM should be invoked manually, but it’s possible for administrators to run PowerShell

scripts to monitor the status of protect site and invoke recovery when disaster happens automatically.

The test in this solution is based on manually invoking the recovery plan, the recovery plan is completed

in 4.8 minutes, but the recovery begins 1.25 minutes after the disaster happens.


93



3.10.2 Performance

Thanks to the multi-time-segment caching technology of OceanStor V3 HyperReplciation/A

feature, there’s no performance impact brought by DR architecture on the application

perspective.

The following table shows the performance of ERP and CRM application (tps: transactions

per second) during the planned migration test. The performance before or after SRM

configured (without or with storage replication) is the same. The performance when SRM

configured to protect or no-protect state (replication ongoing or split) is the same.

Table 3-13 Transaction throughput during planned migration

Step ERP tps CRM tps

Before SRM configured (No replication) 276 1108

SRM configured to protect state (Async

replication with 10s sync period) 278 1120

SRM configured to no-protect state (storage

replication is split) 275 1144

3.10.3 Management

The solution utilizes VMware SRM integrated with OceanStor SRA 2.0 to provide

completely visualized disaster recovery architecture management. With OceanStor UltraPath

plug-in and VASA provider plug-in installed on vCenter, administrators are able to execute

tasks like “DR configuration”, “recovery plan test”, “planned migration”, “storage

management”, and “storage multipath management” in a single vClient.

Figure 3-80 DR and storage management framework


94



4 Appendix

4.1 Reference Documents

HUAWEI OceanStor V3 converged storage systems HyperReplication technical white paper

eSDK OceanStor SRA technical white paper

4.2 Terminology

Table 4-1 Terminology

Term Description

SRM Site Recovery Manager

SRA Storage Replication Adapter

HyperReplication/A Asynchronous replication of OceanStor V3 storage

ERP Enterprise Resource Planning

CRM Customer Relationship Management

RPO Recovery Point Objective

RTO Recovery Time Objective