Using HyperMirror of HUAWEI OceanStor V3
Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
The fast growth of enterprises has created an ever-increasing amount of data. This massive amount of data is
playing an important role in enterprise development because it not only records daily enterprise operation, but also
provides necessary information for decision making. How to ensure data reliability then becomes a great challenge.
In addition, with the wide use of heterogeneous virtualization, a new technology different from traditional RAID is
needed to ensure reliability of third-party LUN takeover. The HyperMirror feature provided by HUAWEI OceanStor
V3 converged storage systems enables data to be stored in an OceanStor V3 converged storage system or in a
heterogeneous storage environment in a 1:1 mirroring manner in different disk domains, thereby continuously
protecting mission-critical data.
This document describes two typical scenarios: using HyperMirror to continuously protect mission-critical Oracle
Database; using HyperMirror to perform online migration of mission-critical Oracle Database.
Li Yong
Storage Solutions, IT, Huawei Enterprise BG
2015-03-17 V1.0
Huawei Technologies Co., Ltd.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Contents
1 Overview ......................................................................................................................................... 4
1.1 Introduction .................................................................................................................................................................. 4
1.2 Purpose ......................................................................................................................................................................... 4
1.3 Intended Audience ........................................................................................................................................................ 5
1.4 Business Scenario ......................................................................................................................................................... 5
1.5 Customer Benefits ........................................................................................................................................................ 5
1.6 Workload Model ........................................................................................................................................................... 6
2 Products and Technologies ......................................................................................................... 7
2.1 OceanStor V3 Converged Storage Systems .................................................................................................................. 7
2.1.1 HyperMirror ............................................................................................................................................................... 8
2.1.2 SmartVirtualization .................................................................................................................................................. 11
2.2 Oracle Database and Cluster ....................................................................................................................................... 11
2.2.1 Oracle RAC and ASM ............................................................................................................................................. 12
2.2.2 Oracle System Architecture ..................................................................................................................................... 14
2.2.3 Oracle Application Types ......................................................................................................................................... 16
2.2.4 SwingBench ............................................................................................................................................................. 17
3 Using HyperMirror to Ensure Mission-Critical Databases' Reliability ........................... 19
3.1 Business Scenario ....................................................................................................................................................... 19
3.2 Huawei Solution ......................................................................................................................................................... 20
3.2.1 Solution Architecture ............................................................................................................................................... 20
3.3 Solution Configuration ............................................................................................................................................... 20
3.4 Verification Procedure ................................................................................................................................................ 22
3.4.1 Preparing the Environment ...................................................................................................................................... 22
3.4.2 Testing Basic Performance ...................................................................................................................................... 23
3.4.3 Testing Local Reliability Protection Implemented by HyperMirror ........................................................................ 23
3.4.4 Testing Heterogeneous Reliability Protection Implemented by HyperMirror ......................................................... 26
4 Verification Results .................................................................................................................... 29
4.1 Reliability Protection .................................................................................................................................................. 29
4.2 Data Synchronization Speed ....................................................................................................................................... 30
5 Appendix ...................................................................................................................................... 31
5.1 Reference Documents ................................................................................................................................................. 31
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 3
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
5.2 Glossary ...................................................................................................................................................................... 31
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 4
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
1 Overview
1.1 Introduction
The fast growth of enterprises has created an ever-increasing amount of data. This massive
amount of data is playing an important role in enterprise development because it not only
records daily enterprise operation, but also provides necessary information for decision
making. However, enterprises face various storage risks such as storage system failure, disk
failure, and irrecoverable damage to data. How to ensure data reliability is a great challenge.
In addition, with the wide use of heterogeneous virtualization, a new technology different
from traditional RAID is needed to ensure reliability of third-party LUN takeover.
HyperMirror, a volume mirroring feature provided by HUAWEI OceanStor V3 converged
storage systems, can store multiple physical copies in different disk domains to continuously
protect mission-critical data. Furthermore, HyperMirror can be used with SmartVirtualization
to store multiple physical copies in an OceanStor V3 converged storage system and a storage
system from another vendor and/or of another system architecture, thereby improving data
reliability.
This document describes how to implement continuous data protection and online data
migration based on HyperMirror in the Oracle Database 12c scenario. This document includes
the following contents:
Overview of HyperMirror
Working principle of HyperMirror
Using HyperMirror to continuously protect mission-critical data of Oracle Database
Using HyperMirror to implement online migration of Oracle Database
Best practices for using HyperMirror in Oracle Database scenarios
1.2 Purpose
This document aims to verify the feasibility and performance indicators of using HyperMirror
of OceanStor V3 converged storage systems in Oracle Database scenarios. It also provides
reference about IT system solutions for Huawei partners and customers, in order to help them
improve data reliability of IT systems.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 5
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
1.3 Intended Audience
This document is intended for Huawei employees as well as IT engineers from partners and
customers. It provides the usage and best practices of HyperMirror for IT partners, storage
architects, database architects, and IT system administrators who want to implement Oracle
database solutions based on HUAWEI OceanStor V3 converged storage systems.
It is assumed that the readers are familiar with the following products and technologies:
Oracle Database
Linux operating systems
Storage systems
1.4 Business Scenario
As a typical business scenario, databases play an important role in enterprise applications. To
continuously protect data and prevent applications and data from being affected by storage
space faults are the most urgent needs of customers. HyerMirror, a type of volume mirroring
technology, is developed to address such needs. HyperMirror enables two identical copies of
data to be stored in different storage spaces. If one storage space becomes unavailable, the
continuity of data and applications is not affected, improving business reliability. HyperMirror
is implemented in storage systems without the need to purchase and manage additional
hardware, thereby reducing the total cost of procurement.
Therefore, HyperMirror can easily address the following scenarios:
If mission-critical applications require robust business continuity, HyperMirror can be
used to enable the local storage system to continuously protect data, preventing a
physical fault in storage space from causing data loss.
If mission-critical applications require robust business continuity and reliability,
HyperMirror can be used with SmartVirtualization to continuously protect data in a
heterogeneous storage environment, preventing the failure of a storage system from
causing data loss.
If HyperMirror is used in a heterogeneous storage environment, SmartVirtualization must be used. For
details about the scenario, refer to the Using SmartVirtualization of OceanStor V3 Converged Storage
Systems in Oracle Database for Unified Management of Heterogeneous Storage Systems, Data
Migration, and Data Protection.
1.5 Customer Benefits
HyperMirror can be used to continuously protect customers' mission-critical databases based
on local mirroring, thereby improving data reliability. Furthermore, HyperMirror can be used
with SmartVirtualization to create mirror copies in different storage systems, without the need
to purchase and manage additional hardware, thereby reducing the total cost of procurement.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 6
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
1.6 Workload Model
This document uses the mainstream OLTP test model SwingBench Order Entry 2.0 to test the
use of SmartMigration for application SLA improvement, application migration, and LUN
type adjustment. This model defines a type of online order business and simulates a scenario
where a large number of users are querying products, placing orders, processing orders, and
viewing orders online. Those operations are the most common ones in transaction systems. In
this workload model, there are two main performance indicators: transactions per minute
(TPM) and average transaction response time. The TPM indicates the number of transactions
processed per minute. A higher TPM indicates higher productivity. The average transaction
response time directly affects the speed of user operations. A shorter response time indicates
better user experience.
Order Entry 2.0 defines 10 tables, storing information about products, customers, orders,
warehouses, and login. During the workload tests, 50% of operations are SELECT, 30%
INSERT, 20% UPDATE, and no DELETE operations. From the perspective of I/O layer, the
workload model is the most typical OLTP workload model, where small data blocks are
accessed at random and the ratio between reads and writes is 6:4.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 7
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
2 Products and Technologies
2.1 OceanStor V3 Converged Storage Systems
HUAWEI OceanStor V3 converged storage systems are next-generation unified storage
products designed for enterprise-class applications. Leveraging a storage operating system
oriented to cloud architecture, a powerful next-generation hardware platform, and a full range
of intelligent management software, OceanStor V3 converged storage systems deliver
industry-leading functionality, performance, efficiency, reliability, and ease of use. They
provide data storage for applications such as large-scale database OLTP/OLAP, file sharing,
and cloud computing, and can be used in industries ranging from government, finance,
telecommunications, energy, to media and entertainment (M&E). Meanwhile, OceanStor V3
converged storage systems can provide a wide range of efficient and flexible backup and
disaster recovery solutions to ensure business continuity and data security, delivering
excellent storage services.
For details about HUAWEI OceanStor V3 converged storage systems, click the following
link:
http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 8
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
2.1.1 HyperMirror
HyperMirror is a volume mirroring function provided by OceanStor V3 converged storage
systems. HyperMirror creates two available mirror copies of a LUN. If one mirror copy is
unavailable, the host can still access the LUN and services running on the host are not
affected. After the faulty mirror copy is recovered, it automatically synchronizes data from the
mirror LUN to ensure data consistency and availability.
HyperMirror provides the following functions:
Creates mirror copies for a thick LUN using local or external third-party LUNs.
Creates mirror copies for an eDevLUN using local or external third-party LUNs.
Non-disruptively converts a thick LUN that contains data into a mirror LUN.
Non-disruptively creates, splits, and deletes mirror copies.
Supports initial full synchronization upon the creation of a mirror copy and incremental
synchronization after a mirror copy is split.
HyperMirror does not support thin LUNs.
Continuous Data Protection Implemented by HyperMirror
HyperMirror creates a mirror LUN and multiple mirror copies for a LUN (called original
LUN) to prevent the failure of a single LUN from causing service interruption or data loss,
thereby improving service reliability.
HyperMirror requires that mirror copies do not reside in the same disk domain as the original
LUN. If the disk domain where the original LUN resides is faulty, the disk domains where
mirror copies reside are not affected. The normal mirror copy takes over all services from the
original LUN, ensuring host service continuity and data integrity.
Figure 2-1 Continuous protection of a local LUN
HyperMirror implementation is divided into three phases: creating a mirror LUN, synchronization, and splitting.
HyperMirrorMirror LUN
Local LUN
Mirror copy A Mirror copy B
Oce
an
Sto
r V
3 C
on
ve
rge
d S
tora
ge
Syste
m
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 9
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Creating a mirror LUN
A mirror LUN is created for the original LUN (which is a local or external LUN). The
mirror LUN inherits the storage space, basic properties, and services from the original
LUN. During the creation of the mirror LUN, host services are not interrupted, and local
mirror copy A is automatically generated. The original LUN is changed into the mirror
LUN, and storage space of the original LUN is handed over to mirror copy A. Then,
mirror copy B is created for the mirror LUN, and data is synchronized from mirror copy
A to mirror copy B. In doing so, the LUN with mirror copies A and B has the space
mirroring function.
I/O requests sent by a host to a mirror LUN are processed as follows:
− When a host sends a read request to a mirror LUN, the storage system reads data
from the mirror copies in round-robin mode through the mirror LUN. If one mirror
copy fails, host services are not affected.
− When a host sends a write request to a mirror LUN, the storage system writes data to
the mirror copies in dual-write mode.
Figure 2-2 Mirror LUN creation
Mirror
Mirror
copy
Mirror
copy
Synchronization
The following figure illustrates the synchronization principle. After a mirror copy
recovers from a fault or when data on a mirror copy is being made complete, incremental
data is synchronized from a complete mirror copy to that mirror copy, making data
between mirror copies identical.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 10
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Figure 2-3 Mirror LUN synchronization
Synchronization
Mirror LUN Mirror copy
Initial synchronization: Full copy is performed in the initial synchronization.
Mirror LUN Mirror copy
Incremental synchronization: Incremental copy is performed when a mirror copy recovers from a splited or interrupted state.
Dual Write After the Synchronization
Host
Write data d to replace data a.
Mirror LUN Mirror copy
When the host sends a write request, the same data is written to both the mirror LUN and mirror copy (dual write).
Data already stored
Data to be stored
Internal signal flow in the storage system
Host signal flow
Splitting
Splitting is implemented to isolate a mirror copy from its mirroring relationship with the
mirror LUN. After the mirror copy is isolated, the mirror LUN cannot perform mirroring
on the mirror copy. Subsequent data changes between the mirror copy and the mirror
LUN are recorded by the Data Change Log (DCL). When the mirroring relationship is
restored, incremental data synchronization is performed based on the DCL.
Figure 2-4 Mirror LUN splitting
Splitting a Pair
Mirror Copy After the Splitting
Mirror LUN
Mirror copy
Mirror copy
After a pair is split, data synchronization between the mirror LUN and mirror copy A is no longer implemented.
Host
Mirror copy
Mirror copy
Mirror LUN
After the split, data is no longer written to the mirror LUN and mirror copy A in dual-write mode. The data host of mirror copy A becomes unavailable, but mirror copy B can still work normally. Therefore, host services are not interrupted.
Subsequent data changes made between mirror copy A and the mirror LUN are recorded by the DCL for incremental data synchronization.
Data already stored
Internal signal flow in the storage system Host signal flow
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 11
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
2.1.2 SmartVirtualization
SmartVirtualization uses LUNs mapped from heterogeneous storage arrays to the local
storage array as logical disks (LDs) that can provide storage space for the local storage array
and creates eDevLUNs that can be mapped to the host on LDs. LDs provide data storage
space for data volumes, and the local storage array provides storage space for meta volumes
of eDevLUNs. SmartVirtualization ensures data integrity of external LUNs. eDevLUNs and
LUNs of heterogeneous storage arrays have different World Wide Names (WWNs).
The primary advantages of SmartVirtualization are as follows:
Wide compatibility
As storage arrays from different vendors have different understanding and compliance
for Small Computer System Interface (SCSI), incompatibility among storage arrays in
takeover becomes a big challenge. SmartVirtualization is enhanced for different storage
arrays in terms of interoperability and can identify and process incompatibility issues of
heterogeneous storage arrays. For example, it can identify and process LUN path faults
of heterogeneous storage arrays to ensure reliability and interoperability of
heterogeneous storage arrays. Heterogeneous storage arrays supported by
SmartVirtualization have been certified by Huawei Interoperability Lab.
Back-end multipathing
SmartVirtualization leverages back-end multipathing software to enable heterogeneous
storage arrays to be connected by redundant paths. In this way, if a physical path
between two heterogeneous storage arrays fails, services are not interrupted. As the
back-end multipathing software is enhanced for a heterogeneous storage array in terms
of interoperability, it can identify heterogeneous LUN paths, such as preferred paths,
non-preferred paths, or standby paths, and select the most appropriate path to deliver
I/Os. Meanwhile, it provides a range of path selection algorithms for back-end load
balancing.
Less storage space consumption
The eDevLUN is not a complete physical data mirror of the LUN in the third-party
storage system. Therefore, it occupies only a small amount of storage space.
Excellent function extension
Value-added features like remote replication and snapshot are available for the
eDevLUN, meeting higher data security and reliability requirements.
eDevLUNs and local LUNs have the same properties. Therefore, HyperMirror can be used to
provide mirror protection for heterogeneous LUNs. In addition, heterogeneous LUNs can be
used as mirror copies to improve service reliability.
For more information about the SmartMigration feature, refer to the following document:
HUAWEI OceanStor V3 Converged Storage Systems SmartVirtualization Technical White
Paper
2.2 Oracle Database and Cluster
Oracle Database is one of the most widely used relational databases. This section briefly
introduces Oracle Database 12c and focuses on Multitenant-related components and features,
including RAC, ASM, Multitenant, data files, database instance architecture, and application
types.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 12
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
2.2.1 Oracle RAC and ASM
As shown in Figure 2-5, an Oracle 12c RAC contains two types of nodes: Hub nodes and Leaf
nodes. Hub nodes have direct access to shared storage, whereas Leaf nodes access shared
storage through Hub nodes. When a database is deployed on an Oracle RAC, the nodes can be
grouped into multiple server pools. Each database is deployed in a server pool, and every
node in a server pool runs a database instance. Application servers access the virtual IP
addresses (VIPs) of nodes to store data. If a node fails, its VIP network is restored on another
node of Oracle RAC. Application servers reconnect to the Oracle database through a
reconnection mechanism. Setting connection character strings on application servers can
enable multiple modes of accessing Oracle RAC nodes, including load balancing and failover
modes. In these modes, a multi-node Oracle cluster is presented as a single database to
application servers.
The shared storage of the Oracle RAC Hub nodes includes Oracle Cluster Registry (OCR),
voting disks, and database. OCR records information about node statuses, voting disks
synchronize data between nodes, and the database is a set of files.
Figure 2-5 Oracle Flex Cluster 12c
Oracle ASM provides a simple storage management interface for database administrators to
manage servers and storage across different platforms. As a built-in file system and volume
manager, Oracle ASM is exclusive to Oracle database files. ASM simplifies file system
management, provides asynchronous I/O performance tuning, saves management time for
administrators, and offers a flexible, efficient database environment.
ASM can consolidate LUNs into a disk group and use Allocation Units (AUs) to allocate
storage space from the disk group. ASM supports three types of disk groups.
External: Data is not mirrored between LUNs, and the storage system provides data
protection.
Normal: A normal disk group consists of two failure groups between which data is
mirrored.
High: A high disk group consists of three failure groups among which data is mirrored.
When an OceanStor V3 storage system is used to create ASM disk groups, it is recommended
that controllers A and B evenly own the LUNs in the disk groups before external or normal
disk groups are created.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 13
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Oracle Flex ASM is a new Oracle ASM deployment model that increases database instance
availability and reduces Oracle ASM related resource consumption. Oracle Flex ASM
facilitates cluster based database consolidation, as it ensures that Oracle Database 12c
instances running on a particular server will continue to operate, should the Oracle Flex ASM
instance on that server fail.
Figure 2-6 ASM before Oracle Database 12c
Figure 2-7 Oracle 12c Flex ASM
Oracle RAC provides the following key characteristics, essential for HA data management:
Reliability — Oracle Database is known for its reliability. Oracle RAC takes this step
further by removing the database server as a single point of failure. If an instance fails,
the remaining instances in the cluster remain open and active. Oracle Clusterware
monitors all Oracle processes and immediate restarts any failed component.
Error detection — Oracle Clusterware automatically monitors all Oracle RAC databases
as well as other Oracle processes (Oracle ASM, instances, Listeners, etc.) and provides
fast detection of problems in the environment. It also automatically recovers from
failures often before users notice that a failure has occurred.
Recoverability — The Oracle Database includes many features that make it easy to
recover from various types of failures. If an instance fails in an Oracle RAC database, it
is recognized by another instance in the cluster and recovery will start automatically. Fast
Application Notification (FAN) and Fast Connection Failover (FCF) and especially the
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 14
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Oracle RAC 12c Application Continuity feature make it easy to mask any component
failure from the user.
Continuous Operations — Oracle RAC provides continuous service for both planned and
unplanned outages. If a server (or an instance) fails, the database remains open and
applications continue to be able to access data, allowing for business critical workloads
to finish, mostly without a delay in service delivery.
For more information about Oracle 12c RAC and ASM, refer to the following documents:
White Paper: Oracle Real Application Clusters (RAC)
A Technical Overview of New Features for Automatic Storage Management in Oracle
Database 12c
2.2.2 Oracle System Architecture
Figure 2-8 Oracle system architecture
As shown in the preceding figure, System Global Area (SGA) and Program Global Area
(PGA) of Oracle databases consume memory. SGA stores system information and page cache
information, and PGA stores session information. SGA contains the following parts:
Buffer Cache: buffers data blocks.
Redo Log Buffer: buffers log records as a recycle data group.
Share Pool: buffers data dictionaries and shared SQL information.
Oracle files are categorized as follows:
Control file: records the database structure, parameters, and locations of other data fails.
Data file: stores user data and temporary data.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 15
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Online log: record changes to data blocks and consists of several log groups. Files in the
log groups are mirrored to each other. After a log group is used up, data in written to the
next log group. After the first log group is used up, data is written to the first log group
again.
Archive log: In archive mode, databases copy fully written log groups to the archive area
for data restoration when anomalies occur.
Among Oracle processes, the Listener process monitors client connections. Clients are
connected in two modes:
In shared mode, the listener redirects client requests to the dispatcher process, which
places the request in the request queue of the large pool. Then the shared server obtains
and processes the request in the request queue and puts the processing results in the
response queue of the large pool. At last, the dispatcher returns the processing results to
the client.
In dedicated mode, a dedicated server process serves each client connection. After
receiving the request from a client, the Oracle server process looks for the data block in
the buffer cache. If the data block is found, data is read, computed, and changed in the
buffer cache. If the data block does not exist in the buffer cache, the Oracle server
process writes the data block from data files to the buffer cache, and then computes and
changes it.
Oracle uses the LRU algorithm to eliminate outdated data in the buffer cache so that the
released storage space can be used by new data blocks. Data that has been changed in the
buffer cache is called "dirty data", which is written by the DB writer (DBW) process to data
files.
To ensure data integrity and reliability, relational databases use "transaction" to indicate an
atomic operation. When processing a transaction, the Oracle server process records changed
data and the change time in the log buffer. When the transaction is delivered, the log writer
(LGWR) synchronizes the log data in the log buffer to online log files. The log buffer is a
memory area where data can be written in a circular manner. When the log buffer is one-third
full, the LGWR synchronizes the log records to the log file, regardless of whether the
transaction is committed. In addition, Oracle databases synchronize logs automatically every
three minutes.
By default, Oracle databases perform a checkpoint every 30 minutes. When the checkpoint is
performed, the Checkpoint (CKPT) process synchronizes the control file and triggers the
DBW to write dirty data to data files.
Online logs of Oracle are divided into several groups, each of which contains one or multiple
log files. When multiple log files exist, the files are mirrored to each other. Oracle databases
write logs to the log groups in sequence. When the last log group is filled, the databases write
to the first log group, restarting the cycle. Before a log group switch, Oracle checks whether
the dirty data recorded in the next log group is completely written to data files. If not, Oracle
waits until the DBW process writes all the dirty data to the data files before it starts the log
group switch.
When an Oracle database is in archive mode, the Archive (ARC) process copies filled logs to
the archive area. If data anomalies occur, the archived logs are used for precise data recovery.
Oracle Database 12c also supports a Multitenant that allows multiple PDBs to run in one
CDB. Figure 2-9 shows a CDB with four containers: the root, seed, and two PDBs (hrpdb and
salespdb). Each PDB has its own dedicated application. A different PDB administrator
manages each PDB. User SYS can manage the root and every PDB. At the physical level, this
CDB has a database instance and database files. The Multitenant feature brings better serviceability to the Oracle Database.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 16
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
The seed PDB is a system-supplied template that the CDB can use to create new PDBs. The seed PDB is
named PDB$SEED. You cannot add or modify objects in PDB$SEED.
Figure 2-9 Oracle Multitenant architecture
For more information about Oracle Multitenant Architecture, refer to the following document:
White Paper: Oracle Multitenant
2.2.3 Oracle Application Types
Data transactions are categorized as two types: OLTP and OLAP.
OLTP: A number of online users perform transactions.
OLAP: A small number of users perform long-term complex statistical queries.
OLTP applications have the following I/O characteristics:
From the perspective of database
The reading, writing, and changing of each transaction involve a small amount of data.
Database data must be up-to-date. Therefore, OLTP applications require a high database
availability.
Many users connect to and use the database concurrently.
The database must be highly responsive and able to complete a transaction within
seconds.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 17
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
From the perspective of storage
Every I/O is small-sized, ranging from 2 KB to 8 KB.
Disk data is randomly accessed.
At least 30% of data is generated by random writing operations.
Redo logs are written frequently.
2.2.4 SwingBench
In verification tests described in this document, SwingBench is used to stress test Oracle
Database. SwingBench is a free database benchmark tool provided by Oracle. The tool
provides four workload models, of which Order Entry is the most used one.
Order Entry is an online transactional workload model. This model defines a type of online
order business, simulating online product searching, placing of orders, managing of orders,
reviewing of orders, and other common system operations by a number of users. Order Entry
defines nine tables, storing information about products, customers, orders, warehouses, and
login. During the workload tests, 50% of operations are SELECT, 30% INSERT, 20%
UPDATE, and no DELETE operations. From the perspective of storage layer, Order Entry
represents the most typical OLTP model, where small data blocks are accessed at random and
the ratio between reads and writes is 6:4. This model uses two main performance indicators:
transactions per minute (TPM) and average transaction response time. The TPM indicates the
number of transactions processed per minute. A higher TPM indicates higher productivity.
The average transaction response time directly affects the speed of user operations. A shorter
response time indicates better user experience.
Loading Test Data
SwingBench provides two data loading modes for the Order Entry test: oewizard and
DataGenerator. In oewizard mode, all user data is stored in one tablespace, and table partitions
and indexes cannot be customized. This mode is suitable for testing less than 100 GB of data.
In DataGenerator mode, you can customize data loading scripts to distribute data optimally.
The scripts and configuration files used for loading data in the tests described in this
document can be found in the attachments.
Multi-client testing
Multiple SwingBench clients can jointly test an Oracle RAC. The coordinator coordinates
these clients.
Each SwingBench client can perform a test in the following two modes: invoking the
SwingBench program to perform tests in visual mode or invoking the CharBench program to
perform tests in CLI mode. The following table lists the key configuration items of the
swingconfig.xml file.
Table 2-1 Key configuration items of swingconfig.xml
Configuration Item Description
UserName/Password User name and password of a database
ConnectString Database connection string
NumberOfUsers Number of users
MinDelay/MaxDelay Interval between two transactions
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 18
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 19
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
3 Using HyperMirror to Ensure Mission-Critical Databases' Reliability
3.1 Business Scenario
As a typical business scenario, databases play an important role in enterprise applications. To
continuously protect data and prevent applications and data from being affected by storage
space faults are the most urgent needs of customers. HyerMirror, a type of volume mirroring
technology, is developed to address such needs. HyperMirror enables two identical copies of
data to be stored in different storage spaces. If one storage space becomes unavailable, the
continuity of data and applications is not affected, improving business reliability. HyperMirror
is implemented in storage systems without the need to purchase and manage additional
hardware, thereby reducing the total cost of procurement.
Therefore, HyperMirror can easily address the following scenarios:
If mission-critical applications require robust business continuity, HyperMirror can be
used to enable the local storage system to continuously protect data, preventing a
physical fault in storage space from causing data loss.
If mission-critical applications require robust business continuity and reliability,
HyperMirror can be used with SmartVirtualization to continuously protect data in a
heterogeneous storage environment, preventing the failure of a storage system from
causing data loss.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 20
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
3.2 Huawei Solution
3.2.1 Solution Architecture
Figure 3-1 Solution architecture
As shown in the preceding figure, Oracle Database runs on an OceanStor 5500 V3 converged
storage system. HyperMirror uses different storage spaces provided by the OceanStor 5500
V3 converged storage system to continuously protect Oracle Database and improve data
reliability.
SmartVirtualization is used to enable the OceanStor 5500 V3 converged storage system to
take over the S5500T. HyperMirror protects Oracle Database and improves data reliability
across the two storage systems.
3.3 Solution Configuration
Table 3-1 Hardware configuration
Hardware Component Quantity
Server: RH2288 V2 2
Intel® Xeon
® E5620 @ 2.40 GHz 1 x 8
Memory 8 GB x 6
QLogic 8 Gbit/s dual-port Fibre Channel
HBA 1 x 2
Oracle 12c RAC
2 x RH2288 V2
OceanStor 5500 V3
50 x 600 GB 10k rpm SAS disks
4 x
8G
bit/s
FC
Cluster Private Interconnection
S6700 10GE switch
S5500T
24 x 600 GB 10k rpm SAS disks
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 21
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Hardware Component Quantity
Storage system: OceanStor
5500 V300R001C10
1
600 GB 10k rpm SAS disk 50
Storage system: OceanStor
S5500T V200R002C20
1
600 GB 10k rpm SAS disk 24
Fibre Channel switch:
SNS2124
2
Table 3-2 Software configuration
Hardware Software Quantity
RH2288 V2 2
Operating system: Red Hat Enterprise Linux 6.5 1 x 2
Multipathing software: UltraPath for Linux
8.01.031
1 x 2
Database cluster: Oracle Grid 12.1.0.2 1 x 2
Database software: Oracle Database 12.1.0.2 1 x 2
Java operating environment: Oracle SUN JRE
7U64
1 x 2
Data loading tool: SwingBench 2.5 1 x 1
OceanStor 5500
V300R001C10
1
HyperMirror license 1 x 1
SmartVirtualization license 1 x 1
Table 3-3 Storage plan for the OceanStor 5500 V3 converged storage system
Function LUN Storage Pool Disk Domain
Data volume 7 x 300 GB StoragePool1 - RAID 6
(8D+2P)
DiskDomain1 (25 SAS
disks)
Data volume 6 x 500 GB StoragePool2 - RAID 6
(4D+2P)
DiskDomain2 (25 SAS
disks)
Online log volume 1 x 300 GB StoragePool1 - RAID 6
(8D+2P)
DiskDomain1 (25 SAS
disks)
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 22
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Function LUN Storage Pool Disk Domain
Archive log volume 2 x 300 GB StoragePool1- RAID 6
(8D+2P)
DiskDomain1 (25 SAS
disks)
Cluster voting volume 3 x 5 GB GridPool - RAID6
(4D+2P)
DiskDomain1 (25 SAS
disks)
Create two disk domains, each of which consists of 25 x 600 GB SAS disks. In DiskDomain1, create
GridPool and StoragePool1. GridPool is used to create cluster voting volumes for Oracle RAC.
StoragePool1 is used to create data volumes, online log volumes, and archive log volumes related to the
database. In DiskDomain2, create StoragePool2 that is used to create data volumes related to the
database. The remaining space is used to create mirror copies for data volumes in the storage pools.
Table 3-4 S5500T storage plan
Function LUN Storage Pool Disk Domain
Data Volume 7 x 300 GB StoragePool1 - RAID 6
(8D+2P)
DiskDomain1 (24 SAS
disks)
3.4 Verification Procedure
3.4.1 Preparing the Environment
Step 1 Prepare a physical environment.
Deploy a test environment based on the solution architecture shown in Figure 3-1.
Step 2 Install and configure the operating system on each host.
Install Red Hat Enterprise Linux 6.5 x86_64 on each host and configure the operating system
based on Oracle Database's requirements.
Step 3 Prepare storage devices.
Based on the plan, configure a SAN switch, create an alias, and set zones.
Based on the plan, configure storage, create host groups and hosts, and map LUNs.
Step 4 Install UltraPath.
Upload the UltraPath software package, which is used with the OceanStor 5500 V3 converged
storage system, to database nodes. Then, install UltraPath.
Step 5 Configure a UDEV policy.
On database nodes, configure a UDEV policy for LUNs.
Step 6 Install Oracle Database.
Deploy an Oracle cluster and install Oracle Database in the OceanStor 5500 V3 storage
system. Then, use SwingBench to generate 1 TB of test data.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 23
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
----End
3.4.2 Testing Basic Performance
In the Oracle environment deployed based on the OceanStor 5500 V3 storage system, use
SwingBench to perform a benchmark workload test (namely, the database I/O latency is about
10 ms). Perform the test for 30 minutes. Record the TPM, IOPS, and latency as reference
values of basic performance. Based on the reference values, assess the impact of HyperMirror
on Oracle Database.
3.4.3 Testing Local Reliability Protection Implemented by HyperMirror
Step 1 Creating a HyperMirror task.
1. Open a web browser and enter https://management IP address:8088.
The login page of DeviceManager of the OceanStor 5500 V3 storage system is
displayed.
2. Enter the user name and password.
3. Select the Data Protection tab page.
4. Click HyperMirror.
5. Create a HyperMirror task, start initial synchronization, and record the synchronization
time.
If a HyperMirror task is created for a LUN (called mirror LUN in the HyperMirror feature),
mirror copy A is generated in the pool where the mirror LUN resides.
During the creation of a HyperMirror task, mirror copy B must be generated for the mirror
LUN too. In the local mirroring scenario, mirror copy B and the mirror LUN do not reside in
the same disk domain, mirror copy B is a LUN automatically created by the storage system,
and data is automatically synchronized from mirror copy A to mirror copy B.
After a HyperMirror task is created, a mirror volume contains three LUNs: a mirror LUN,
mirror copy A, and mirror copy B. The two mirror copies reside in two disk domains. The
mirror LUN is mapped to the host as a data LUN that stores data generated by Oracle
Database.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 24
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Wait for the initial synchronization to complete.
The synchronization speed can be low, medium, high, or highest. After creating a HyperMirror task, you
can change the speed in the properties of a mirror copy.
One HyperMirror task can contain only two mirror copies.
The two databases are running properly. As shown in the following figure, database
HWPDB1 is protected by HyperMirror, and HWPDB2 is not.
The following figure shows the HyperMirror configuration.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 25
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Step 2 Simulate a fault.
After the synchronization is complete, remove multiple disks to simulate a disk failure.
Removing multiple disks will destroy the RAID property of the storage pool. Then, LUNs in
the storage pool will stop providing services.
Step 3 Check the database service.
All the LUNs in StoragePool2 are faulty. All the LUNs that serve HWPDB2 are faulty. Mirror
copy B of the data LUN that serves HWPDB1 is faulty, but mirror copy A is still available.
HWPDB1 protected by HyperMirror is not affected, ensuring business continuity. HWPDB2
that is not protected by HyperMirror cannot be opened.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 26
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Step 4 Reinsert disks.
After the removed disks are inserted, the disk failure is resolved, and the damaged RAID
property is also recovered. Then, LUNs in the storage pool become available and can provide
service again.
----End
3.4.4 Testing Heterogeneous Reliability Protection Implemented by HyperMirror
Step 1 Set up a relationship between the remote storage system and the local storage system.
In the remote storage system, create a host and identify the local storage system as a physical
host.
Step 2 In the local storage system, discover the remote storage system.
The local storage system automatically discovers the remote storage system.
Step 3 In the remote storage system, create LUNs that have the same size as the mirror LUN, and
map the LUNs to the local storage system. Then, create eDevLUNs that reside in a different
disk domain from the mirror LUN.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 27
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Step 4 Create a HyperMirror task.
Create a HyperMirror task with LUN Type set to eDevLUN, start initial synchronization, and
record the synchronization time.
Step 5 Simulate a fault.
Remove multiple disks that belong to the disk domain where the mirror LUN resides to
simulate a LUN failure.
Step 6 Check the database service.
Heterogeneous LUN mirroring protection is provided for the database. Therefore, the local
LUN failure does not cause all mirror copies to fail. The database still runs properly.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 28
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
----End
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 29
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
4 Verification Results
4.1 Reliability Protection
The verification described in section 3.4.3 "Testing Local Reliability Protection Implemented
by HyperMirror" and section 3.4.4 "Testing Heterogeneous Reliability Protection
Implemented by HyperMirror" proves that HyperMirror can protect mirror volumes of Oracle
Database. If a volume fails as a result of damage to storage media, the database can still work
properly.
The mirror copies in a HyperMirror task reside in different disk domains. Therefore, before
storage controllers become a performance bottleneck, the use of HyperMirror does not affect
performance. The splitting of a mirror copy from a HyperMirror task has only a negligible
impact on performance. The test result shown in the following figure proves the preceding
conclusion.
Figure 4-1 Performance impact of HyperMirror
The preceding figure shows the maximum IOPS of the storage system in the scenario where the I/O
latency of the OLTP database is less than 10 ms. In the preceding figure, three cases are compared:
without using HyperMirror, using a HyperMirror task in normal state, and using a HyperMirror task in
split state.
HyperMirror not
used
HyperMirror in
normal state
HyperMirror in
split state
Max. IOPS
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 30
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
If the mirror copies of a HyperMirror task are normal, the system performance does not change. If a
mirror copy of a HyperMirror task is split, the system performance decreases by 10%.
4.2 Data Synchronization Speed
HyperMirror provides four synchronization speeds. The following table lists the
synchronization time and average bandwidth that each speed option can provide for a
HyperMirror task created for a 300 GB LUN when there is no workload. In actual use, select
a synchronization speed based on the service impact and migration time that you can accept.
Table 4-1 Synchronization time and average bandwidth provided by each speed option
Speed Synchronization Time (Min) Average Bandwidth (MB/s)
Low 2366 2
Medium 340 15
High 108 48
Highest 36 142
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 31
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
5 Appendix
5.1 Reference Documents HUAWEI OceanStor V3 Converged Storage Systems HyperMirror Technical White
Paper
HUAWEI OceanStor V3 Converged Storage Systems SmartVirtualization Technical
White Paper
5.2 Glossary
Table 5-1 Glossary
Term Description
OLTP Online Transaction Processing
OLAP Online Analytical Processing
TPM Transactions Per Minute
UDEV Device manger of Linux kernel 2.6
RAC Oracle Real Application Clusters
ASM Oracle Automatic Storage Management
UltraPath Huawei storage multipathing management software
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 32
Using HyperMirror of HUAWEI OceanStor V3 Converged Storage Systems to Harden
Mission-Critical Oracle Database's Reliability
Huawei Technologies Co., Ltd.
Address: Huawei Industrial Base
Bantian, Longgang
Shenzhen 518129
People's Republic of China
Website: http://www.huawei.com
Email: [email protected]
Copyright © Huawei Technologies Co., Ltd. 2015. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of Huawei Technologies Co., Ltd.
Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and the
customer. All or part of the products, services and features described in this document may not be within the
purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,
and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.