33
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize OLAP Oracle Database's Performance and Availability This document is aimed at the scenario where HUAWEI OceanStor V3 converged storage systems are used to serve Online Analytical Processing (OLAP) Oracle Database 12c. This document focuses on how to efficiently deploy Oracle Database based on OceanStor V3 converged storage systems, and verifies OLAP applications in typical enterprise database scenarios. The best practices described in this document help you obtain higher application deployment efficiency and better application operation quality, thereby ensuring Oracle Database's performance and availability. Wang Yaohui Storage Solutions, IT, Huawei Enterprise BG 2015-03-17 V1.0 Huawei Technologies Co., Ltd.

Planning and Configuring HUAWEI OceanStor V3 …€¦ · Database based on OceanStor V3 converged storage systems, and verifies OLAP applications in typical enterprise ... Huawei

Embed Size (px)

Citation preview

Planning and Configuring HUAWEI

OceanStor V3 Converged Storage Systems

to Maximize OLAP Oracle Database's

Performance and Availability

This document is aimed at the scenario where HUAWEI OceanStor V3 converged storage systems are used to serve

Online Analytical Processing (OLAP) Oracle Database 12c. This document focuses on how to efficiently deploy Oracle

Database based on OceanStor V3 converged storage systems, and verifies OLAP applications in typical enterprise

database scenarios. The best practices described in this document help you obtain higher application deployment

efficiency and better application operation quality, thereby ensuring Oracle Database's performance and availability.

Wang Yaohui

Storage Solutions, IT, Huawei Enterprise BG

2015-03-17 V1.0

Huawei Technologies Co., Ltd.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Contents

1 Overview ......................................................................................................................................... 4

1.1 Introduction .................................................................................................................................................................. 4

1.2 Purpose ......................................................................................................................................................................... 4

1.3 Intended Audience ........................................................................................................................................................ 4

1.4 Business Scenario ......................................................................................................................................................... 4

1.5 Workload Model ........................................................................................................................................................... 5

1.6 Acronyms and Abbreviations ........................................................................................................................................ 5

2 Products and Technologies ......................................................................................................... 7

2.1 OceanStor V3 Converged Storage Systems .................................................................................................................. 7

2.1.1 Next-Generation Hardware ........................................................................................................................................ 8

2.1.2 Multi-Controller Architecture .................................................................................................................................... 8

2.1.3 Convergence Design .................................................................................................................................................. 8

2.1.4 Smart Software .......................................................................................................................................................... 8

2.1.5 Unified and Easy Management .................................................................................................................................. 9

2.1.6 RAID 2.0+ Block Virtualization ................................................................................................................................ 9

2.2 Oracle Database and Cluster ....................................................................................................................................... 10

2.2.1 Oracle RAC and ASM ............................................................................................................................................. 10

2.2.2 Oracle System Architecture ..................................................................................................................................... 13

2.2.3 Oracle Application Types ......................................................................................................................................... 15

3 Best Practices for Oracle Database Planning and Configuration ...................................... 17

3.1 SAN Networking ........................................................................................................................................................ 17

3.1.1 Setting Zones and vLANs ........................................................................................................................................ 18

3.2 Storage Configuration ................................................................................................................................................. 19

3.2.1 Planning ................................................................................................................................................................... 19

3.2.2 Disk Domains .......................................................................................................................................................... 20

3.2.3 Storage Pools ........................................................................................................................................................... 21

3.2.4 LUNs ....................................................................................................................................................................... 22

3.2.5 Mapping Views ........................................................................................................................................................ 23

3.3 Host Configuration ..................................................................................................................................................... 23

3.3.1 Queue Depth ............................................................................................................................................................ 23

3.3.2 I/O Alignment .......................................................................................................................................................... 24

3.3.3 Block Device Scheduling Algorithms ...................................................................................................................... 25

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 3

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3.3.4 Block Device I/O Settings ....................................................................................................................................... 25

3.4 Database Configuration .............................................................................................................................................. 26

3.4.1 Database Parameters ................................................................................................................................................ 26

3.4.2 Online Logs ............................................................................................................................................................. 27

3.4.3 Backup and Archiving ............................................................................................................................................. 27

4 Example of Oracle Database Planning and Configuration ................................................. 28

4.1 Solution Architecture .................................................................................................................................................. 28

4.2 Database Plan.............................................................................................................................................................. 29

4.3 Storage Configuration ................................................................................................................................................. 30

4.4 Host Configuration ..................................................................................................................................................... 30

4.5 Database Configuration .............................................................................................................................................. 31

4.6 Workload Verification Results .................................................................................................................................... 32

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 4

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

1 Overview

1.1 Introduction

This document is aimed at the scenario where HUAWEI OceanStor V3 converged storage

systems are used to serve Online Analytical Processing (OLAP) Oracle Database 12c. This

document focuses on how to efficiently deploy Oracle Database based on OceanStor V3

converged storage systems, and verifies OLAP applications in typical enterprise database

scenarios. The best practices described in this document help you obtain higher application

deployment efficiency and better application operation quality, thereby ensuring Oracle

Database's performance and availability.

1.2 Purpose

This document describes the plan and configuration for the scenario where OceanStor V3

converged storage systems are used to serve OLAP Oracle Database. It provides reference for

Huawei partners and customers, in order to help them simplify IT system planning and

deployment and reduce O&M risks.

1.3 Intended Audience

This document is intended for Huawei IT engineers and partners. It provides the best practices

of Oracle OLAP for IT partners, storage architects, database architects, and IT system

administrators who want to deploy Oracle OLAP applications based on HUAWEI OceanStor

V3 converged storage systems.

It is assumed that the readers are familiar with the following products and technologies:

Storage systems

Oracle Database

Linux operating systems

1.4 Business Scenario

Database services are typically classified into two types: OLTP and OLAP. OLTP is

commonly seen in traditional relational databases. It mainly involves basic and daily

transaction processing, such as stock exchanges and bank transactions. OLAP is commonly

seen in data warehouse systems. It supports complex analytical operations and provides

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 5

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

graphical and easy-to-understand query results to help decision makers obtain accurate

business operation information, so that they can make correct decisions.

With the wide use of database technology, enterprise information systems generate a large

amount of data. How to extract useful information from massive data to assist decision

making has become a great challenge facing decision makers of enterprises. How to

accelerate complex query and analysis in OLAP applications is the key to addressing the

challenge.

Oracle Database is one of the most commonly used databases and one of the most important

application scenarios for storage devices. Deploying OLAP Oracle Database on OceanStor V3

storage systems can ensure service reliability and improve database analysis and query

performance, helping enterprises make correct decisions.

This document describes the recommended plan and configuration, including networking,

storage plan and configuration, as well as host and database parameter settings, for the

scenario where OLAP Oracle Database is deployed based on an OceanStor V3 converged

storage system, in order to help customers mitigate performance and availability risks in IT

systems.

1.5 Workload Model

This document uses a TPCH-Like benchmark test model that consists of a suite of

business-oriented queries and concurrent data modifications. The queries have been chosen to

have broad industry-wide relevance. This benchmark illustrates the following aspects of

decision support systems:

Examining large volumes of data

Executing queries with a high degree of complexity

Giving answers to critical business questions

The test model defines eight tables, recording information about projects, orders, suppliers,

and customers. The workload test procedure includes loading, analysis, and updating

operations. In the loading phase, SQL*Loader is used to load a group of text files as external

tables to the database. In the analysis phase, 22 complex analytical SQL query statements are

executed in sequence. In the update phase, a group of existing data is deleted from the

database. From the perspective of I/O layer, the test model is one of the most typical OLAP

business models, where large blocks are accessed in sequence by multiple streams and the

ratio between reads and writes is 9:1.

1.6 Acronyms and Abbreviations

Table 1-1 Acronyms and abbreviations

Acronym and Abbreviation Full Name

AWR Automatic Workload Repository

OLAP Online Analytical Processing

OLTP Online Transaction Processing

PGA Progress Global Area

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 6

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Acronym and Abbreviation Full Name

SGA System Global Area

ETL Extract, Transform, Load

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 7

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

2 Products and Technologies

2.1 OceanStor V3 Converged Storage Systems

HUAWEI OceanStor V3 converged storage systems are next-generation unified storage

products designed for enterprise-class applications. Leveraging a storage operating system

oriented to cloud architecture, a powerful next-generation hardware platform, and a full range

of intelligent management software, OceanStor V3 converged storage systems deliver

industry-leading functionality, performance, efficiency, reliability, and ease of use. They

provide data storage for applications such as large-scale database OLTP/OLAP, file sharing,

and cloud computing, and can be used in industries ranging from government, finance,

telecommunications, energy, to media and entertainment (M&E). Meanwhile, OceanStor V3

converged storage systems can provide a wide range of efficient and flexible backup and

disaster recovery solutions to ensure business continuity and data security, delivering

excellent storage services.

For details about HUAWEI OceanStor V3 converged storage systems, click the following

link:

http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range

Figure 2-1 HUAWEI OceanStor V3 converged storage systems

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 8

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

2.1.1 Next-Generation Hardware

OceanStor V3 converged storage systems employ next-generation Intel multi-core processors,

PCIe 3.0 buses, 12 Gbit/s SAS 3.0 high-speed disk ports, and a variety of host ports such as

16 Gbit/s Fibre Channel, 10 Gbit/s FCoE, and 56 Gbit/s InfiniBand host ports. The storage

systems provide up to 28 GB/s of system bandwidth to meet the requirements of

bandwidth-intensive application scenarios. They also offer million-level IOPS performance,

outshining products from other vendors.

OceanStor V3 converged storage systems are equipped with exclusive SmartIO cards. A

SmartIO card supports 8 Gbit/s Fibre Channel, 16 Gbit/s Fibre Channel, 10 Gbit/s iSCSI, and

10 Gbit/s FCoE. Users can specify the protocols that a SmartIO card is required to support.

The deduplication/compression cards used by OceanStor V3 converged storage systems

support lossless data deduplication and compression, efficiently reducing data storage costs.

In addition, the storage systems can implement data encryption to secure data.

2.1.2 Multi-Controller Architecture

The multi-controller architecture used by OceanStor V3 converged storage systems supports

online horizontal expansion. An OceanStor V3 converged storage system can be

non-disruptively expanded to a maximum of eight controllers, 1 TB of cache, and 5 TB of

storage space, meeting customers' future capacity needs. The multi-controller architecture

allows load balancing among controllers and eliminates single points of failure, thereby

ensuring high availability and stable service running.

2.1.3 Convergence Design

Convergence of SAN and NAS: SAN and NAS services are converged to provide

elastic storage, simplify service deployment, improve storage resource utilization, and

reduce the total cost of ownership (TCO). Underlying storage resource pools directly

provide both block and file services, thereby shortening storage resource access paths to

ensure that the two services are equally efficient.

Convergence of heterogeneous storage systems: Based on the built-in heterogeneous

virtualization function, OceanStor V3 converged storage systems can efficiently manage

storage systems from other mainstream vendors and unify resource pools for central and

flexible resource allocation.

Convergence of entry-level, mid-range, and high-end storage systems: OceanStor V3

converged storage systems are the only storage systems in the industry that enable

entry-level, mid-range, and high-end storage systems to interwork seamlessly with each

other. Data can freely flow among storage products of different models without the

assistance of third-party systems.

Convergence of SSDs and HDDs: The advantages of traditional and solid-state storage

media are combined, bringing the performance of different types of storage media into

full play and striking an optimal balance between performance and cost.

Convergence of primary and backup storage: The built-in backup function enables

data to be efficiently backed up without additional backup software, simplifying backup

solution management.

2.1.4 Smart Software

Multi-tenancy and service level agreement (SLA): OceanStor V3 converged storage

systems intelligently allocate storage resources in cloud computing environments to meet

the needs of enterprises and organizations. The storage systems also leverage data

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 9

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

isolation and a range of data security policies such as data encryption and data

destruction to meet varying data security requirements. OceanStor V3 converged storage

systems provide four service levels and allocate storage resources based on service

priorities. Storage resources are first allocated to high-priority services to ensure system

performance and shorten response time.

Smart-series efficiency improvement suite: OceanStor V3 converged storage systems

use SmartTier (dynamic storage tiering), SmartMotion (intelligent data migration), and

innovative SmartVirtualization (heterogeneous virtualization) to achieve vertical,

horizontal, and cross-system 3D data flowing, improving storage resource utilization by

three times.

Hyper-series data protection software: Data protection software such as remote

replication, snapshot, and LUN copy software meets user needs for local, remote, and

multi-region data protection, maximizing business continuity and data availability.

2.1.5 Unified and Easy Management

Unified management: OceanStor V3 converged storage systems provide powerful

storage management software that supports global topology view, capacity analysis,

performance analysis, fault diagnosis, and end-to-end service visualization to manage a

wide range of devices.

Convenient management: OceanStor V3 converged storage systems can be initially

configured in 5 steps which take about 40 seconds, and expanded in two steps which take

about 15 seconds.

Mobile management: Users can use tablets and mobile phones to manage storage

systems in real time. System status is sent automatically, making constant attendance by

an engineer unnecessary.

2.1.6 RAID 2.0+ Block Virtualization

RAID 2.0+ block virtualization of the OceanStor V3 implements virtualization for underlying

disk management and upper-layer resource management. Inside the system, the storage space

of each disk is divided into fine-grained data blocks, which comprise RAID groups. In doing

so, data is evenly distributed to all disks in the storage pool. In addition, data block–based

resource management largely improves the resource management efficiency.

1. The V3 storage systems support SSDs, SAS disks, and NL-SAS disks. These disks

comprise disk domains. In a disk domain, disks of the same type comprise disk groups

(DGs).

2. In a DG, the storage space of disks is divided into chunks (CKs) of a fixed size. Then the

system consolidates CKs from random disks into CK groups (CKGs) based on RAID

algorithms.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 10

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3. CKGs are divided into logical storage space called extents, which have a fixed size too.

Extents are the minimum unit for comprising thick LUNs. On thin LUNs, extents are

further divided into smaller grains.

2.2 Oracle Database and Cluster

Oracle Database is one of the most widely used relational databases. This section briefly

introduces Oracle Database 12c and focuses on Multitenant-related components and features,

including RAC, ASM, Multitenant, data files, database instance architecture, and application

types.

2.2.1 Oracle RAC and ASM

As shown in Figure 2-2, an Oracle 12c RAC contains two types of nodes: Hub nodes and Leaf

nodes. Hub nodes have direct access to shared storage, whereas Leaf nodes access shared

storage through Hub nodes. When a database is deployed on an Oracle RAC, the nodes can be

grouped into multiple server pools. Each database is deployed in a server pool, and every

node in a server pool runs a database instance. Application servers access the virtual IP

addresses (VIPs) of nodes to store data. If a node fails, its VIP network is restored on another

node of Oracle RAC. Application servers reconnect to the Oracle database through a

reconnection mechanism. Setting connection character strings on application servers can

enable multiple modes of accessing Oracle RAC nodes, including load balancing and failover

modes. In these modes, a multi-node Oracle cluster is presented as a single database to

application servers.

The shared storage of the Oracle RAC Hub nodes includes Oracle Cluster Registry (OCR),

voting disks, and database. OCR records information about node statuses, voting disks

synchronize data between nodes, and the database is a set of files.

Figure 2-2 Oracle Flex Cluster 12c

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 11

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Oracle ASM provides a simple storage management interface for database administrators to

manage servers and storage across different platforms. As a built-in file system and volume

manager, Oracle ASM is exclusive to Oracle database files. ASM simplifies file system

management, provides asynchronous I/O performance tuning, saves management time for

administrators, and offers a flexible, efficient database environment.

ASM can consolidate LUNs into a disk group and use Allocation Units (AUs) to allocate

storage space from the disk group. ASM supports three types of disk groups.

External: Data is not mirrored between LUNs, and the storage system provides data

protection.

Normal: A normal disk group consists of two failure groups between which data is

mirrored.

High: A high disk group consists of three failure groups among which data is mirrored.

When an OceanStor V3 storage system is used to create ASM disk groups, it is recommended

that controllers A and B evenly own the LUNs in the disk groups before external or normal

disk groups are created.

Oracle Flex ASM is a new Oracle ASM deployment model that increases database instance

availability and reduces Oracle ASM related resource consumption. Oracle Flex ASM

facilitates cluster based database consolidation, as it ensures that Oracle Database 12c

instances running on a particular server will continue to operate, should the Oracle Flex ASM

instance on that server fail.

Figure 2-3 ASM before Oracle Database 12c

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 12

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Figure 2-4 Oracle 12c Flex ASM

Oracle RAC provides the following key characteristics, essential for HA data management:

Reliability — Oracle Database is known for its reliability. Oracle RAC takes this step

further by removing the database server as a single point of failure. If an instance fails,

the remaining instances in the cluster remain open and active. Oracle Clusterware

monitors all Oracle processes and immediate restarts any failed component.

Error detection — Oracle Clusterware automatically monitors all Oracle RAC databases

as well as other Oracle processes (Oracle ASM, instances, Listeners, etc.) and provides

fast detection of problems in the environment. It also automatically recovers from

failures often before users notice that a failure has occurred.

Recoverability — The Oracle Database includes many features that make it easy to

recover from various types of failures. If an instance fails in an Oracle RAC database, it

is recognized by another instance in the cluster and recovery will start automatically. Fast

Application Notification (FAN) and Fast Connection Failover (FCF) and especially the

Oracle RAC 12c Application Continuity feature make it easy to mask any component

failure from the user.

Continuous Operations — Oracle RAC provides continuous service for both planned and

unplanned outages. If a server (or an instance) fails, the database remains open and

applications continue to be able to access data, allowing for business critical workloads

to finish, mostly without a delay in service delivery.

For more information about Oracle 12c RAC and ASM, refer to the following documents:

White Paper: Oracle Real Application Clusters (RAC)

A Technical Overview of New Features for Automatic Storage Management in Oracle

Database 12c

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 13

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

2.2.2 Oracle System Architecture

Figure 2-5 Oracle system architecture

As shown in the preceding figure, System Global Area (SGA) and Program Global Area

(PGA) of Oracle databases consume memory. SGA stores system information and page cache

information, and PGA stores session information. SGA contains the following parts:

Buffer Cache: buffers data blocks.

Redo Log Buffer: buffers log records as a recycle data group.

Share Pool: buffers data dictionaries and shared SQL information.

Oracle files are categorized as follows:

Control file: records the database structure, parameters, and locations of other data fails.

Data file: stores user data and temporary data.

Online log: record changes to data blocks and consists of several log groups. Files in the

log groups are mirrored to each other. After a log group is used up, data in written to the

next log group. After the first log group is used up, data is written to the first log group

again.

Archive log: In archive mode, databases copy fully written log groups to the archive area

for data restoration when anomalies occur.

Among Oracle processes, the Listener process monitors client connections. Clients are

connected in two modes:

In shared mode, the listener redirects client requests to the dispatcher process, which

places the request in the request queue of the large pool. Then the shared server obtains

and processes the request in the request queue and puts the processing results in the

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 14

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

response queue of the large pool. At last, the dispatcher returns the processing results to

the client.

In dedicated mode, a dedicated server process serves each client connection. After

receiving the request from a client, the Oracle server process looks for the data block in

the buffer cache. If the data block is found, data is read, computed, and changed in the

buffer cache. If the data block does not exist in the buffer cache, the Oracle server

process writes the data block from data files to the buffer cache, and then computes and

changes it.

Oracle uses the LRU algorithm to eliminate outdated data in the buffer cache so that the

released storage space can be used by new data blocks. Data that has been changed in the

buffer cache is called "dirty data", which is written by the DB writer (DBW) process to data

files.

To ensure data integrity and reliability, relational databases use "transaction" to indicate an

atomic operation. When processing a transaction, the Oracle server process records changed

data and the change time in the log buffer. When the transaction is delivered, the log writer

(LGWR) synchronizes the log data in the log buffer to online log files. The log buffer is a

memory area where data can be written in a circular manner. When the log buffer is one-third

full, the LGWR synchronizes the log records to the log file, regardless of whether the

transaction is committed. In addition, Oracle databases synchronize logs automatically every

three minutes.

By default, Oracle databases perform a checkpoint every 30 minutes. When the checkpoint is

performed, the Checkpoint (CKPT) process synchronizes the control file and triggers the

DBW to write dirty data to data files.

Online logs of Oracle are divided into several groups, each of which contains one or multiple

log files. When multiple log files exist, the files are mirrored to each other. Oracle databases

write logs to the log groups in sequence. When the last log group is filled, the databases write

to the first log group, restarting the cycle. Before a log group switch, Oracle checks whether

the dirty data recorded in the next log group is completely written to data files. If not, Oracle

waits until the DBW process writes all the dirty data to the data files before it starts the log

group switch.

When an Oracle database is in archive mode, the Archive (ARC) process copies filled logs to

the archive area. If data anomalies occur, the archived logs are used for precise data recovery.

Oracle Database 12c also supports a Multitenant that allows multiple PDBs to run in one

CDB. Figure 2-6 shows a CDB with four containers: the root, seed, and two PDBs (hrpdb and

salespdb). Each PDB has its own dedicated application. A different PDB administrator

manages each PDB. User SYS can manage the root and every PDB. At the physical level, this

CDB has a database instance and database files. The Multitenant feature brings better

serviceability to the Oracle Database.

The seed PDB is a system-supplied template that the CDB can use to create new PDBs. The seed PDB is

named PDB$SEED. You cannot add or modify objects in PDB$SEED.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 15

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Figure 2-6 Oracle Multitenant architecture

For more information about Oracle Multitenant Architecture, refer to the following document:

White Paper: Oracle Multitenant

2.2.3 Oracle Application Types

Data transactions are categorized as two types: OLTP and OLAP.

OLTP: A number of online users perform transactions.

OLAP: A small number of users perform long-term complex statistical queries.

OLAP applications have the following I/O characteristics:

From the perspective of database administrators

No data or only a small amount of data is modified.

Complex query statements are invoked, and a large number of lines are scanned.

A query takes hours or even days, depending on the complexity of query statements.

Data aging can be measured by hour or day.

The query output is typically a statistical value, obtained using group by and order by.

From the perspective of storage sampling

Every I/O is large-sized, ranging from 64 KB to 1 MB.

Data is read in sequence.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 16

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

When read operations are being performed, write operations are performed in a

temporary tablespace if any.

Online logs are seldom written unless data is loaded in batches.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 17

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3 Best Practices for Oracle Database Planning and Configuration

This chapter describes the recommended plan and configuration, including networking,

storage plan and configuration, as well as host and database parameter settings, for the

scenario where OLAP Oracle Database 12c is deployed based on an OceanStor V3 converged

storage system.

3.1 SAN Networking

Oracle databases typically serve enterprises' mission-critical application systems. To ensure

business continuity, it is recommended that an Oracle RAC cluster be used. The network

must prevent single points of failure. As shown in the following figure, the SAN network

uses two switching planes that are physically isolated from each other. Each switching plane

consists of one switch or multiple interconnected switches. Each of the database nodes and

storage controllers is connected to both switching planes.

Figure 3-1 SAN network

Application layer

Management

networkPublic network RAC interconnection network

Oracle RACStorage

Area

Network

OceanStor 6800 V3

...

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 18

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3.1.1 Setting Zones and vLANs

Zones and vLANs can be set to serve the following purposes:

In the IT environment of an enterprise, a storage system typically serves multiple

application systems. Zones or vLANs can be set to prevent application systems from

affecting each other.

An excessive number of host paths will complicate system management and make

network adjustment more difficult during the O&M process. Zones or vLANs can be set

to improve IT infrastructure flexibility.

The overall policy for setting zones or vLANs is to ensure dual-switching networking.

That is, on each switching plane, each of the database nodes and storage controllers has at

least one logical channel. Based on this policy, consider networking from the following

aspects: performance, deployment complexity, and O&M flexibility.

In terms of performance, plan the number of host ports and paths based on the IOPS and

bandwidth requirements of hosts.

In terms of deployment and O&M, strike a balance between deployment complexity and

O&M flexibility. The following figure shows three zoning approaches. Zoning by application

has the lowest complexity and flexibility, zoning by host has the medium, and zoning by port

has the highest.

Figure 3-2 Zoning or vLAN setting granularity

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 19

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3.2 Storage Configuration

Figure 3-3 Resource allocation flowchart

3.2.1 Planning

Before using an OceanStor V3 converged storage system, it is recommended that you analyze

application performance and capacity requirements and plan the number of disks so that you

can minimize O&M risks.

In addition to a series of entry-level, mid-range, and high-end storage systems, Huawei

provides solid state storage that features low latency and high performance. In terms of

deploying Oracle Database, analysis of storage performance and capacity requirements is

necessary to selection of storage systems and disks. In storage performance assessment,

performance factors are typically first considered and then capacity factors.

In OLAP Oracle Database applications, I/O characteristics are multiple channels of sequential

reads. The major storage performance indicator is the amount of data processed per second,

namely, the bandwidth. Based on the following parameter values and formulas, you can

calculate the performance of RAID levels. Then, estimate the required number of disks.

Table 3-1 Assessment of OLAP application performance

RAID Level

OLAP Throughput (MB/s) For Example, 2 to 8 Gbit/s FC

RAID Group Comprised of Eight 10k rpm SAS Disks

RAID 10 MIN (MBPSPATH

, MBPSDISK

x N) MIN (1350, 50 x 8) = 400

RAID 5 MIN (MBPSPATH

, MBPSDISK

x N) MIN (1350, 50 x 8) = 400

RAID 6 MIN (MBPSPATH

, MBPSDISK

x N) MIN (1350, 50 x 8) = 400

MBPSDISK indicates the multi-stream sequential read throughput of a single disk. For OceanStor V3

converged storage systems, MBPSDISK of a 10k rpm HDD is 50, and that of a 7.2k rpm HDD is 30.

N indicates the number of disks that compose a RAID group.

MBPSPATH indicates the total bandwidth provided for the connections between all database hosts and

the storage system. In the formulas, the maximum bandwidth of a single 16 Gbit/s Fibre Channel link is

1350 MB/s.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 20

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Capacity estimation covers user tablespaces, online logs, archive logs, and backup sets. User

tablespaces and other database areas are typically stored in different LUNs for high

performance and reliability. In OLAP applications, online analysis is implemented and most

I/Os are read I/Os. In capacity estimation, the amount of data in user tablespaces and that in

temporary tablespaces are the major considerations. If archived data and backup data will be

stored, extra capacity must be considered too. Typically, the temporary tablespace size should

be set to the same as the user tablespace size, so that there is sufficient space for sorting

operations.

The use of RAID 2.0+ makes it difficult to estimate the number of disks based on application

performance and capacity requirements. Huawei UniSTAR eDesigner

(http://app.huawei.com/unistar/edesigner/solutionAction!showHome.action), which is a

solution design tool, includes two assessment tools oriented to storage systems. It is

recommended that customers use the two tools to plan the number of disks for Oracle

Database.

Product capability assessment: Based on a specific storage configuration, the

performance and capacity that can be provided by a storage system are assessed.

Storage configuration wizard: Based on specific application requirements, the storage

model and disk configuration are assessed.

Huawei provides verified data warehouse reference architectures based on OceanStor 5300

V3, 5500 V3, 5600 V3, and 5800 V3 converged storage systems. When designing a storage

solution, you can refer to HUAWEI OceanStor V3 Converged Storage Systems — OLAP

Oracle Database Reference Architectures.

3.2.2 Disk Domains

A disk domain is a combination of multiple disks. After disks are consolidated and a certain

amount of hot spare capacity is reserved, a disk domain provides storage resources for storage

pools in a unified manner.

One or more disk domains can be created in an OceanStor V3 converged storage system.

Multiple storage pools can be created in a disk domain.

A disk domain can consist of SSDs, SAS disks, and/or NL-SAS disks.

Disk domains are isolated from each other, including performance, storage resources,

and faults.

The percentage of hot space capacity in a disk domain is related to the hot spare policy and

the number and types of disks. The percentage is automatically maintained by the storage

system based on a reliability engineering method. If you need to plan storage capacity and

performance, it is recommended that you use Huawei UniSTAR eDesigner.

In terms of disk domain creation, the default hot spare policy is high. After a disk domain is

created, you can also change the policy to low or none based on reliability requirements of

applications. However, considering reliability, it is recommended that you do not change

the hot spare policy, especially not to change the policy to none.

OceanStor V3 converged storage systems support SSDs, SAS disks, and NL-SAS disks and

provide tiered storage. For OLAP Oracle Database, it is recommended that you use SSDs or

SAS disks, and do not use NL-SAS disks that provide low performance and reliability. OLAP

typically involves complex queries. A large number of data blocks are accessed, whereas the

access frequency is low. No obvious hotspot data exists. Therefore, it is recommended that

you do not use SmartTier for hotspot data identification and tiered storage.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 21

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

A disk domain in an OceanStor V3 converged storage system can consist of over 1000 disks.

However, considering performance and reliability, it is recommended that each storage tier

of a disk domain contain 100 disks at most.

3.2.3 Storage Pools

A storage pool, a container that stores storage space resources, is created in a disk domain. A

storage pool can dynamically allocate resources from a disk domain and define the RAID

level of each storage tier.

A storage tier is a collection of storage media providing the same level of performance in a

storage pool. Different storage tiers manage storage media with different performance levels

and provide different storage spaces for applications whose performance requirements vary.

There are three types of storage tiers: high-performance tier that consists of SSDs (including

SLC and eMLC), performance tier that consists of SAS disks (including 15k rpm and 10k

rpm), and capacity tier that consists of NL-SAS disks.

OceanStor V3 converged storage systems support seven RAID levels: RAID 6, RAID 10,

RAID 5, RAID 3, RAID 50, RAID 1, and RAID 0. The most commonly used RAID levels are

RAID 6, RAID 10, and RAID 5. From the perspective of stripe width, RAID 6 and RAID 5

are classified into RAID 6-4 (2D2P), RAID 6-6 (4D2P), RAID 6-10 (8D2P), RAID 5-3

(2D1P), RAID 5-5 (4D1P), and RAID 5-9 (8D1P). It is recommended that you set a RAID

level for OLAP Oracle Database based on the following rules:

RAID 6 is preferred, especially for mission-critical database applications that require

high reliability.

If capacity takes precedence over reliability, RAID 5 is recommended.

If a capacity tier is configured for OLAP Oracle Database, it is strongly recommended

that RAID 6 be set for the capacity tier.

OceanStor V3 converged storage systems employ innovative RAID 2.0+. When creating a

storage pool, you can set the extent size. An extent consists of one or more chunk groups

(namely, RAID stripes). Extent is the basic unit used by a storage pool for allocating space to

a LUN. The default extent size is 4 MB, which is applicable to most scenarios. Extent is also

the basic unit of block virtualization. Functions such as hotspot data identification and

migration and data motion are implemented based on extents too. It is recommended that

you set the extent size for OLAP Oracle Database based on the following rules:

4 MB is preferred.

If you want to improve the speed of data motion, it is recommended that you set the

extent size to a larger value.

During the creation of a storage pool, you can set an alarm threshold for the capacity

allocation ratio. The default threshold is 80%. Capacity alarming is particularly important in

scenarios where value-added features such as thin LUN, snapshot, remote replication, and

clone are used. You can set a proper alarm threshold based on the speed of application data

growth to prevent insufficient capacity of the storage pool from causing application

interruption.

When creating a storage pool in developer mode, you are allowed to set a stripe depth. For

Oracle OLAP applications, it is recommended that you set a large stripe depth value (for

example, 512 KB) because the application type is large data block reading, where data is not

frequently written and the amount of written data is small.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 22

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3.2.4 LUNs

In terms of LUNs created for OLAP Oracle Database, it is recommended that you set their

major parameters based on the following rules:

Owner controller (owner_controller): If the database workload is heavy, it is

recommended that you evenly allocate LUNs to controllers to ensure load balancing

among these controllers.

Capacity allocation policy (allocation_policy): Four capacity allocation policies are

provided: Automatic allocation, Allocate from the high performance tier first,

Allocate from the performance tier first, and Allocate from the capacity tier first.

This parameter is used with tier storage. Typically, it is recommended that you set the

policy to Automatic allocation for Oracle storage. If you want to accelerate access to an

identified hotspot area, you can set the policy to Allocate from the high performance

tier first for that area. If you do not want archived data or backup data to consume the

space of the performance tier, you can set the policy to Allocate from the capacity tier

first.

Read cache policy (read_cache_policy): OLAP applications are characterized by

sequential workload. If an OLAP application shares storage with other applications, the

reclamation policy is recommended.

Write cache policy (write_cache_policy): OLAP applications are characterized by

sequential workload. If an OLAP application shares storage with other applications, the

reclamation policy is recommended.

Prefetch policy (prefetch_policy): Four prefetch policies are provided: non-prefetch,

constant prefetch, variable prefetch, and intelligent prefetch. For details about the four

prefetch policies, see the OceanStor V3 converged storage system user manual. OLAP

Oracle Database is characterized by multiple channels of sequential large I/Os. Therefore,

it is recommended that you use the non-prefetch policy if the number of concurrent

accesses is large, and use the intelligent prefetch policy if the number of concurrent

accesses is small.

I/O priority (io_priority): Three levels are provided. The default level is Middle. It is

recommended that you set the I/O priority of the log area to High and that of the archive

area to Low.

LUN type (lun_type): Two types are provided: Thick and Thin. If performance takes

precedence over capacity (for example, a mission-critical production system), it is

recommended that you set the LUN type to Thick. If capacity takes precedence over

performance (for example, a testing/development system), it is recommended that you

set the LUN type to Thin. Thin is not recommended for scenarios where a high write

bandwidth is required, such as a backup target or temporary tablespace area.

LUN capacity and quantity

Different from a traditional RAID group that contains 10+ disks at most, a storage pool that

employs the RAID 2.0+ mechanism contains LUNs across all disks in the disk domain. The

number of disks in a disk domain can be more than 100. To bring disk performance into full

play, it is recommended that you configure LUN capacity and quantity based on the following

rules:

Total number of LUNs in a disk domain: Not less than the number of disks x 4 ÷ 32 is

recommended. (4 is the proper number of concurrent access requests per disk, and 32 is

the default maximum queue depth of a LUN).

LUN capacity: If the preceding condition is met, prefer large-sized LUNs to simplify

management overheads. In terms of the maximum capacity of a LUN, you must consider

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 23

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

the restrictions posed by the operating system and Oracle Database. For example, Oracle

11g ASM requires that the capacity of a LUN do not exceed 2 TB.

3.2.5 Mapping Views

A mapping view defines a logical mapping among LUNs, array ports, and host ports. It is

recommended that you create a mapping view based on the following rules:

A LUN group is an object designed to facilitate LUN resource management. Typically,

LUNs that serve the same application should be added to one LUN group, for example,

LUNs that belong to each storage area of Oracle RAC.

A host group is a collection of hosts that need to share storage resources. Each host

contains multiple initiators (host ports). It is recommended that you create a host for each

server and add all initiators of a server to the host created for that server.

Port groups help users allocate storage ports in a more fine-grained manner. Port groups

are not necessary. However, it is recommended that you allocate a port group for an

OLAP application to improve O&M flexibility and reduce performance impacts between

applications. To prevent single points of failure, each port group must contain at least

one port on each controller.

3.3 Host Configuration

3.3.1 Queue Depth

I/O queue depth is an important parameter that determines the performance of Oracle

Database. Two parameters on the operating system layer affect the I/O queue depth: block

device's queue depth and HBA's queue depth. It is recommended that you set a block device's

queue depth and HBA's queue depth based on the following rules:

In a Linux operating system, the block device's maximum queue depth is 128. It is

recommended that you do not change the value. An HBA's queue parameters vary

depending on the HBA type and driver. For details, see the specifications provided by the

HBA vendor. For example, the QLogic 8 Gbit/s dual-port Fibre Channel HBA allows the

maximum queue depth of each LUN to be 32. It is recommended that you do not

change the maximum queue depth of an HBA. To increase the overall I/O queue depth,

it is recommended that you increase the number of LUNs.

For an AIX operating system, it is recommended that you install UltraPath and do not

use the multipathing software delivered with the operating system or provided by a third

party. After UltraPath is installed, the block device's maximum queue depth is changed to

32. It is recommended that you do not change the value. If UltraPath is not installed, the

block device's default maximum queue depth is 5. It is strongly recommended that you

change the value to 32 or larger. In an AIX operating system, the HBA's default

maximum queue depth is 200. You can change the value based on application

requirements.

For a Windows operating system, the maximum I/O queue depth of a LUN is also

determined by the specifications provided by the HBA vendor. It is recommended that

you do not change the value.

For other operating systems, adjust the queue depth as instructed in the user manual of

that operating system.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 24

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

In AIX, you can run lsattr -El fcs0 to query the parameter settings of HBA fcs0. See the following

figure. num_cmd_elems indicates the maximum I/O queue depth of fcs0, and the default value is 200.

You can run chdev -Pl fcs0 -a num_cmd_elems=512 to change the maximum I/O queue depth of fcs0

to 512.

In AIX, you can run lsattr -El hdisk1 to query the maximum I/O limit (max_transfer) and maximum

queue depth (queue_depth) of hdisk1. You can run chdev -Pl hdisk1 -a queue_depth=32 to change

the maximum queue depth of hdisk1 to 32.

3.3.2 I/O Alignment

If MBR partitions are created in Linux or Windows whose version is earlier than Windows

Server 2003, the first 63 sectors of a disk are reserved for the master boot record and partition

table. The first partition starts from the 64th sector by default. As a result, misalignment

occurs between data blocks (database or file system) delivered by hosts and data blocks stored

in the disk array, causing poor I/O processing efficiency.

When creating MBR partitions in Linux, it is recommended that you enter the expert mode of

the fdisk command and set the start location of the first partition to the start location of the

second extent on a LUN. (The default extent size is 4 MB.) The following is a quick

command used to create an MBR partition in /dev/sdb. The partition uses all space of

/dev/sdb. The start sector is set to 8192, namely, 4 MB.

printf "n\np\n1\n\n\nx\nb\n1\n 8192\nw\n" | fdisk /dev/sdb

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 25

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

The step-by-step command used to set partition alignment is described as follows:

fdisk /dev/sdb

n (Create a partition.)

p (Set the partition type to maser.)

1 (Set this partition as the first partition.)

(Press Enter. Set the start location of the partition to the default start sector, namely, 64.)

(Press Enter. Set the end location of the partition to the default last sector.)

x (Enter the expert mode.)

b (Set the start location of the partition.)

1 (Set the first partition.)

8192 (Set the start location to 8192, namely, 4 MB.)

w (Write data to disks and quit.)

Another way to resolve partition misalignment in Linux is to use GPT partitions. The

following is a quick command used to create a GPT partition in /dev/sdb. The partition uses

all space of /dev/sdb. The start sector is set to 8192, namely, 4 MB.

parted -s -- /dev/sdb "mklabel gpt" "unit s" "mkpart primary 8192 -1" "print"

To create MBR partitions in Windows whose version is earlier than Windows Server 2003, it

is recommended that you run diskpart to set partition alignment.

diskpart> select disk 1

diskpart> create partition primary align=4096

3.3.3 Block Device Scheduling Algorithms

Linux 2.6 kernel supports four types of block device scheduling algorithms: noop,

anticipatory, deadline, and cfq. In OLAP application scenarios, if traditional disks are used,

the deadline algorithm is recommended; if SSDs are used, the noop algorithm is

recommended. Change the configuration file /boot/grub/menu.lst and add

elevator=algorithm name in the kernel line. This method applies to all block devices.

kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet

elevator=deadline

3.3.4 Block Device I/O Settings

In Linux, max_sectors_kb, a block device parameter, restricts the maximum I/O data block

size. Set max_sectors_kb to the database I/O size. For example, the typical data block size

used by an OLAP database is 32 KB. The db_file_multiblock_read_count parameter

specifies the number of concurrent I/O blocks. If db_file_multiblock_read_count is set to 32,

the concurrent I/O size of the database is 1024 KB. In this case, it is recommended that you

change the value of max_sectors_kb to 1024.

In the configuration file /etc/rc.local, you can add the following command that enables the

value of max_sectors_kb to be changed to 1024 upon each startup.

echo 1024 > /sys/block/sdb/queue/max_sectors_kb

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 26

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

3.4 Database Configuration

3.4.1 Database Parameters

Database Block Size

The block size used by Oracle Database ranges from 2 KB to 32 KB, with the default value of

8 KB. Users can change the block size as required. For a typical OLTP application, the

database block size is usually set to the default value 8 KB. However, for an OLAP

application, it is recommended that the database block size be set to a larger value, so that the

application performance is significantly improved.

The db_block_size parameter indicates the database block size. You need to set it when creating a

database. After that, you cannot change the value.

Memory Allocation

In Oracle database applications, the memory should be fully used without compromising the

normal system running. It is recommended that 80% of the physical host memory be allocated

to Oracle Database. In OLAP applications, a large number of sorting operations are performed.

When dedicated servers are used to access the database, sorting operations occur in the PGA.

It is recommended that at least 50% of the memory allocated to Oracle Database be allocated

to the PGA for improving system performance.

Assume that MEM indicates the physical memory of a host. Set database instance parameters

memory_target and memory_max_target to 0. Set sga_max_size and sga_target to 50% x MEM. Set

pga_aggregate_target to 50% x MEM.

Parallel Operations

OLAP Oracle Database involves a large number of complex query and analytical statements,

with the possibility of massive full-table scanning operations. If the host is equipped with

multi-core CPUs, parallel queries can greatly improve the query performance but consume a

lot of CPU resources of the host. It should be noted that, the maximum number of parallel

queries is related to the concurrency capabilities of tables.

Assume that the number of CPUs in a host is CPU_COUNT and the number of cores on each CPU is

PARALLEL_THREADS_PER_CPU. Then, you can set parallel_max_servers to

PARALLEL_THREADS_PER_CPU x CPU_COUNT x 4 x 5, indicating that a maximum of

parallel_max_servers parallel operations can be concurrently performed. Set parallel_min_servers to

a value that ranges from 0 to parallel_max_servers, indicating the number of processes concurrently

running after the database instance starts.

I/O Policy

If a database is deployed in file system mode, the I/O policy can be set to synchronous I/O,

asynchronous I/O, direct I/O, and/or merged I/O. To achieve a better I/O efficiency, it is

recommended that you enable both asynchronous I/O and direct I/O.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 27

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Database parameter filesystemio_option can be set to ASYNCH, DIRECTIO, SETALL, or NONE.

ASYNCH indicates that the asynchronous I/O policy is enabled for the file system. DIRECTIO

indicates that the direct I/O policy is enabled for the file system. SETALL indicates that the

asynchronous I/O and direct I/O polices are enabled for the file system. NONE indicates that the

asynchronous I/O and direct I/O polices are disabled for the file system. It is recommended that you set

filesystemio_option to SETALL.

3.4.2 Online Logs

If OLAP Oracle Database is in use, data is frequently written to the analytical database during

the Extract, Transform, Load (ETL) process. It is recommended that you set the size of the

online log file to a large value. However, it is recommended that the size do not exceed 128

MB. Otherwise, if a failure occurs, the recovery will be time-consuming. Oracle Database

writes logs in a cyclical manner. To prevent the log waiting issue, it is recommended that you

create at least four log groups for each instance, enable each log group to contain two log files,

and place one of the two log files in the archive area.

3.4.3 Backup and Archiving

It is strongly recommended that you enable the archive mode for the databases running on the

production system and configure a backup policy to ensure that data is backed up at least once

a day.

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 28

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

4 Example of Oracle Database Planning and Configuration

4.1 Solution Architecture

Figure 4-1 Tiny-size data warehouse reference architecture

Oracle 12c RAC

2 x RH2288 V2 [1]

5300 V3 [2]

50 x 600 GB 10k rpm SAS disks

4 x

16 G

bit/s

FC

Tiny-Size Data Warehouse Solution2 TB[3] data, 3 GB/s[4] analytical bandwidth

Cluster Private Interconnection

S6700 10GE switch

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 29

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

[1] The detailed configuration of the RH2288 V2 is as follows: 2 x E5-2660 CPUs, 256 GB memory, 1 x

QLogic 16 Gbit/s dual-port Fibre Channel HBA, and 1 x Intel 10 Gbit/s Ethernet HBA.

[2] The detailed configuration of the OceanStor 5300 V3 converged storage system is as follows: 32 GB

cache, 1 x controller enclosure (25 x 600 GB 10k rpm SAS disks, 2 x 16 Gbit/s dual-port Fibre Channel

I/O modules), 1 x disk enclosure (25 x 600 GB 10k rpm SAS disks), 1 x disk domain containing all the

50 disks (25 TB capacity), 1 x storage pool (RAID 6, 12 TB capacity, and 512 KB stripe size), 16 x 500

GB LUNs (eight LUNs used as the data area, two LUNs used as the log area, and six LUNs used as the

ETL area).

[3] The tested database contains 2.5 TB of table and index data as well as 1 TB of temporary tablespace.

[4] The tested stable analytical bandwidth is 3 GB/s and the tested peak analytical bandwidth is 4.2 GB/s,

based on the TPCH-Like universal order and warehouse analytical model.

4.2 Database Plan

The solution simulates a 2 TB tiny-size Oracle OLAP data warehouse and uses the universal

analytical model (TPCH-Like) as the workload model. The following table lists the plan of

capacity, workload, and disk quantity. (The storage configuration wizard of UniSTAR

eDesigner is used to plan the disk quantity, with the RAID level of the storage pool set to

RAID 6.)

Table 4-1 Plan of database capacity, workload, and disk quantity

Size Amount of Data Bandwidth Disk Quantity

Tiny-size 2 TB > 3 GB/s 50

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 30

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

4.3 Storage Configuration

Figure 4-2 Storage configuration

[1] The disk domain configuration is as follows: The hot spare policy is High (default policy).

[2] The storage pool configuration is as follows: The RAID level is RAID 6-10. The capacity is 12 TB.

The stripe depth is 512 KB. Other parameters keep the default values.

[3] The LUN configuration of the data area is as follows: Both the read and write policies are

reclamation. The prefetch policy is non-prefetch. LUNs are evenly allocated to controllers A and B.

The LUN configuration of the log area is as follows: Both the read and write policies are reclamation.

The prefetch policy is intelligent prefetch. LUNs are evenly allocated to controllers A and B.

The LUN configuration of the ETL area is as follows: Both the read and write policies are reclamation.

The prefetch policy is intelligent prefetch. LUNs are evenly allocated to controllers A and B.

[4] The external redundancy policy is configured for all ASM disk groups. Other parameters keep the

default values.

[5] An ASM volume is created in the +ETL disk group and uses all space of the +ETL disk group.

ACFS is created in the ASM volume and mounted to the /opt/oracle/ETL directory as a data loading

area.

[6] For details about the database configuration, see Table 4-3.

4.4 Host Configuration In the configuration example, partitions are not created. Therefore, I/O alignment

configuration is not involved.

The block device's queue depth and HBA's queue depth are not changed.

The block device's I/Os scheduling policy is set to deadline.

The Linux huge page is set to 200 GB (about 80% of memory capacity).

+DATA +LOG +ETL+GRID

...50 x 600 GB 10k rpm SAS disks

15 TB allocated 9 TB free 1 TB hot spare

4288 GB free

1 x 100 GB LUN

...

8 x 500 GB LUNs 2 x 500 GB LUNs 6 x 500 GB LUNs

...

/opt/oracle/ETL

2.5 TB table & index data

1 TB temporary tablespace2 TB text data

LoadAnalytical Queries

Disk

Disk Domain [1]

LUN [3]

ASM Disk Group[4]

ACFS [5]

Storage Pool [2]

Oracle Database[6]

8100 GB allocated

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 31

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

The following table provides the specific host parameter settings.

Table 4-2 Host parameter settings

Configuration File Parameter Value

/etc/sysctl.conf vm.nr_hugepages 102400 (2 MB)

kernel.shmmax 214748364800 (bytes)

kernel.shmall 52428800 (4 KB)

/etc/security/limits.conf oracle soft nproc 16384

oracle hard nproc 65536

oracle soft nofile 16384

oracle hard nofile 65536

oracle soft memlock 209715200 (1 KB)

oracle hard memlock 209715200 (1 KB)

/sys/block/sd*/queue/scheduler deadline

/sys/block/sd*/queue/max_sector

s_kb

1024

4.5 Database Configuration A total of 10 online log groups exist. Each instance is assigned five groups that are stored

in +LOG.

In the solution, backup components are not deployed. In actual implementation, it is

strongly recommended that you consider backup components and configure a backup

policy.

The archive log area can retain logs of the latest 15 days at least.

The following table provides database parameter settings.

Table 4-3 Database parameter settings

Parameter Value

db_create_file_dest +DATA

db_block_size 32768

db_file_multiblock_read_count 32

sga_target 50 GB

pga_aggregate_target 80 GB

lock_sga TRUE

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 32

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Parameter Value

use_large_pages ONLY

db_files 512

process 1024

parallel_max_servers 256

parallel_force_local true

4.6 Workload Verification Results

Table 4-4 Workload verification results

Size Amount of Data Stable Bandwidth

Peak Bandwidth

Disk Quantity

Tiny size 2.5 TB of table and index data

1 TB of temporary data

3 GB/s 4.2 GB/s 50

Figure 4-3 Execution process of multi-stream analytical query

The y-axis indicates the bandwidth (expressed in MB/s). The x-axis indicates the execution time

(expressed in seconds). rmbps indicates the read bandwidth. wmbps indicates the write bandwidth

(temporary tablespace sorting).

The test covers four analytical query streams. Each host involves two streams. Each stream contains 22

complex queries that represent typical data warehouse applications. The query statement sequence and

parameters of each stream are generated at random. Different query statements have different needs for

computing and I/O resources. In some phases that involve a large number of I/O operations, the stable

bandwidth is at least 3 GB/s, and the peak bandwidth is 4.2 GB/s.

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0 545 1091 1755 2302 3039 3613 4159 4705 5358 5903 6496

rmbps wmbps

Stable: 3 GB/s

Peak: 4.2 GB/s

Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 33

Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize

OLAP Oracle Database's Performance and Availability

Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base

Bantian, Longgang

Shenzhen 518129

People's Republic of China

Website: http://www.huawei.com

Email: [email protected]

Copyright © Huawei Technologies Co., Ltd. 2015. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means without prior written

consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.

All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice

The purchased products, services and features are stipulated by the contract made between Huawei and the

customer. All or part of the products, services and features described in this document may not be within the

purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,

and recommendations in this document are provided "AS IS" without warranties, guarantees or

representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the

preparation of this document to ensure accuracy of the contents, but all statements, information, and

recommendations in this document do not constitute a warranty of any kind, express or implied.