Upload
dangphuc
View
280
Download
1
Embed Size (px)
Citation preview
Planning and Configuring HUAWEI
OceanStor V3 Converged Storage Systems
to Maximize OLAP Oracle Database's
Performance and Availability
This document is aimed at the scenario where HUAWEI OceanStor V3 converged storage systems are used to serve
Online Analytical Processing (OLAP) Oracle Database 12c. This document focuses on how to efficiently deploy Oracle
Database based on OceanStor V3 converged storage systems, and verifies OLAP applications in typical enterprise
database scenarios. The best practices described in this document help you obtain higher application deployment
efficiency and better application operation quality, thereby ensuring Oracle Database's performance and availability.
Wang Yaohui
Storage Solutions, IT, Huawei Enterprise BG
2015-03-17 V1.0
Huawei Technologies Co., Ltd.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Contents
1 Overview ......................................................................................................................................... 4
1.1 Introduction .................................................................................................................................................................. 4
1.2 Purpose ......................................................................................................................................................................... 4
1.3 Intended Audience ........................................................................................................................................................ 4
1.4 Business Scenario ......................................................................................................................................................... 4
1.5 Workload Model ........................................................................................................................................................... 5
1.6 Acronyms and Abbreviations ........................................................................................................................................ 5
2 Products and Technologies ......................................................................................................... 7
2.1 OceanStor V3 Converged Storage Systems .................................................................................................................. 7
2.1.1 Next-Generation Hardware ........................................................................................................................................ 8
2.1.2 Multi-Controller Architecture .................................................................................................................................... 8
2.1.3 Convergence Design .................................................................................................................................................. 8
2.1.4 Smart Software .......................................................................................................................................................... 8
2.1.5 Unified and Easy Management .................................................................................................................................. 9
2.1.6 RAID 2.0+ Block Virtualization ................................................................................................................................ 9
2.2 Oracle Database and Cluster ....................................................................................................................................... 10
2.2.1 Oracle RAC and ASM ............................................................................................................................................. 10
2.2.2 Oracle System Architecture ..................................................................................................................................... 13
2.2.3 Oracle Application Types ......................................................................................................................................... 15
3 Best Practices for Oracle Database Planning and Configuration ...................................... 17
3.1 SAN Networking ........................................................................................................................................................ 17
3.1.1 Setting Zones and vLANs ........................................................................................................................................ 18
3.2 Storage Configuration ................................................................................................................................................. 19
3.2.1 Planning ................................................................................................................................................................... 19
3.2.2 Disk Domains .......................................................................................................................................................... 20
3.2.3 Storage Pools ........................................................................................................................................................... 21
3.2.4 LUNs ....................................................................................................................................................................... 22
3.2.5 Mapping Views ........................................................................................................................................................ 23
3.3 Host Configuration ..................................................................................................................................................... 23
3.3.1 Queue Depth ............................................................................................................................................................ 23
3.3.2 I/O Alignment .......................................................................................................................................................... 24
3.3.3 Block Device Scheduling Algorithms ...................................................................................................................... 25
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 3
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3.3.4 Block Device I/O Settings ....................................................................................................................................... 25
3.4 Database Configuration .............................................................................................................................................. 26
3.4.1 Database Parameters ................................................................................................................................................ 26
3.4.2 Online Logs ............................................................................................................................................................. 27
3.4.3 Backup and Archiving ............................................................................................................................................. 27
4 Example of Oracle Database Planning and Configuration ................................................. 28
4.1 Solution Architecture .................................................................................................................................................. 28
4.2 Database Plan.............................................................................................................................................................. 29
4.3 Storage Configuration ................................................................................................................................................. 30
4.4 Host Configuration ..................................................................................................................................................... 30
4.5 Database Configuration .............................................................................................................................................. 31
4.6 Workload Verification Results .................................................................................................................................... 32
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 4
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
1 Overview
1.1 Introduction
This document is aimed at the scenario where HUAWEI OceanStor V3 converged storage
systems are used to serve Online Analytical Processing (OLAP) Oracle Database 12c. This
document focuses on how to efficiently deploy Oracle Database based on OceanStor V3
converged storage systems, and verifies OLAP applications in typical enterprise database
scenarios. The best practices described in this document help you obtain higher application
deployment efficiency and better application operation quality, thereby ensuring Oracle
Database's performance and availability.
1.2 Purpose
This document describes the plan and configuration for the scenario where OceanStor V3
converged storage systems are used to serve OLAP Oracle Database. It provides reference for
Huawei partners and customers, in order to help them simplify IT system planning and
deployment and reduce O&M risks.
1.3 Intended Audience
This document is intended for Huawei IT engineers and partners. It provides the best practices
of Oracle OLAP for IT partners, storage architects, database architects, and IT system
administrators who want to deploy Oracle OLAP applications based on HUAWEI OceanStor
V3 converged storage systems.
It is assumed that the readers are familiar with the following products and technologies:
Storage systems
Oracle Database
Linux operating systems
1.4 Business Scenario
Database services are typically classified into two types: OLTP and OLAP. OLTP is
commonly seen in traditional relational databases. It mainly involves basic and daily
transaction processing, such as stock exchanges and bank transactions. OLAP is commonly
seen in data warehouse systems. It supports complex analytical operations and provides
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 5
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
graphical and easy-to-understand query results to help decision makers obtain accurate
business operation information, so that they can make correct decisions.
With the wide use of database technology, enterprise information systems generate a large
amount of data. How to extract useful information from massive data to assist decision
making has become a great challenge facing decision makers of enterprises. How to
accelerate complex query and analysis in OLAP applications is the key to addressing the
challenge.
Oracle Database is one of the most commonly used databases and one of the most important
application scenarios for storage devices. Deploying OLAP Oracle Database on OceanStor V3
storage systems can ensure service reliability and improve database analysis and query
performance, helping enterprises make correct decisions.
This document describes the recommended plan and configuration, including networking,
storage plan and configuration, as well as host and database parameter settings, for the
scenario where OLAP Oracle Database is deployed based on an OceanStor V3 converged
storage system, in order to help customers mitigate performance and availability risks in IT
systems.
1.5 Workload Model
This document uses a TPCH-Like benchmark test model that consists of a suite of
business-oriented queries and concurrent data modifications. The queries have been chosen to
have broad industry-wide relevance. This benchmark illustrates the following aspects of
decision support systems:
Examining large volumes of data
Executing queries with a high degree of complexity
Giving answers to critical business questions
The test model defines eight tables, recording information about projects, orders, suppliers,
and customers. The workload test procedure includes loading, analysis, and updating
operations. In the loading phase, SQL*Loader is used to load a group of text files as external
tables to the database. In the analysis phase, 22 complex analytical SQL query statements are
executed in sequence. In the update phase, a group of existing data is deleted from the
database. From the perspective of I/O layer, the test model is one of the most typical OLAP
business models, where large blocks are accessed in sequence by multiple streams and the
ratio between reads and writes is 9:1.
1.6 Acronyms and Abbreviations
Table 1-1 Acronyms and abbreviations
Acronym and Abbreviation Full Name
AWR Automatic Workload Repository
OLAP Online Analytical Processing
OLTP Online Transaction Processing
PGA Progress Global Area
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 6
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Acronym and Abbreviation Full Name
SGA System Global Area
ETL Extract, Transform, Load
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 7
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
2 Products and Technologies
2.1 OceanStor V3 Converged Storage Systems
HUAWEI OceanStor V3 converged storage systems are next-generation unified storage
products designed for enterprise-class applications. Leveraging a storage operating system
oriented to cloud architecture, a powerful next-generation hardware platform, and a full range
of intelligent management software, OceanStor V3 converged storage systems deliver
industry-leading functionality, performance, efficiency, reliability, and ease of use. They
provide data storage for applications such as large-scale database OLTP/OLAP, file sharing,
and cloud computing, and can be used in industries ranging from government, finance,
telecommunications, energy, to media and entertainment (M&E). Meanwhile, OceanStor V3
converged storage systems can provide a wide range of efficient and flexible backup and
disaster recovery solutions to ensure business continuity and data security, delivering
excellent storage services.
For details about HUAWEI OceanStor V3 converged storage systems, click the following
link:
http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range
Figure 2-1 HUAWEI OceanStor V3 converged storage systems
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 8
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
2.1.1 Next-Generation Hardware
OceanStor V3 converged storage systems employ next-generation Intel multi-core processors,
PCIe 3.0 buses, 12 Gbit/s SAS 3.0 high-speed disk ports, and a variety of host ports such as
16 Gbit/s Fibre Channel, 10 Gbit/s FCoE, and 56 Gbit/s InfiniBand host ports. The storage
systems provide up to 28 GB/s of system bandwidth to meet the requirements of
bandwidth-intensive application scenarios. They also offer million-level IOPS performance,
outshining products from other vendors.
OceanStor V3 converged storage systems are equipped with exclusive SmartIO cards. A
SmartIO card supports 8 Gbit/s Fibre Channel, 16 Gbit/s Fibre Channel, 10 Gbit/s iSCSI, and
10 Gbit/s FCoE. Users can specify the protocols that a SmartIO card is required to support.
The deduplication/compression cards used by OceanStor V3 converged storage systems
support lossless data deduplication and compression, efficiently reducing data storage costs.
In addition, the storage systems can implement data encryption to secure data.
2.1.2 Multi-Controller Architecture
The multi-controller architecture used by OceanStor V3 converged storage systems supports
online horizontal expansion. An OceanStor V3 converged storage system can be
non-disruptively expanded to a maximum of eight controllers, 1 TB of cache, and 5 TB of
storage space, meeting customers' future capacity needs. The multi-controller architecture
allows load balancing among controllers and eliminates single points of failure, thereby
ensuring high availability and stable service running.
2.1.3 Convergence Design
Convergence of SAN and NAS: SAN and NAS services are converged to provide
elastic storage, simplify service deployment, improve storage resource utilization, and
reduce the total cost of ownership (TCO). Underlying storage resource pools directly
provide both block and file services, thereby shortening storage resource access paths to
ensure that the two services are equally efficient.
Convergence of heterogeneous storage systems: Based on the built-in heterogeneous
virtualization function, OceanStor V3 converged storage systems can efficiently manage
storage systems from other mainstream vendors and unify resource pools for central and
flexible resource allocation.
Convergence of entry-level, mid-range, and high-end storage systems: OceanStor V3
converged storage systems are the only storage systems in the industry that enable
entry-level, mid-range, and high-end storage systems to interwork seamlessly with each
other. Data can freely flow among storage products of different models without the
assistance of third-party systems.
Convergence of SSDs and HDDs: The advantages of traditional and solid-state storage
media are combined, bringing the performance of different types of storage media into
full play and striking an optimal balance between performance and cost.
Convergence of primary and backup storage: The built-in backup function enables
data to be efficiently backed up without additional backup software, simplifying backup
solution management.
2.1.4 Smart Software
Multi-tenancy and service level agreement (SLA): OceanStor V3 converged storage
systems intelligently allocate storage resources in cloud computing environments to meet
the needs of enterprises and organizations. The storage systems also leverage data
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 9
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
isolation and a range of data security policies such as data encryption and data
destruction to meet varying data security requirements. OceanStor V3 converged storage
systems provide four service levels and allocate storage resources based on service
priorities. Storage resources are first allocated to high-priority services to ensure system
performance and shorten response time.
Smart-series efficiency improvement suite: OceanStor V3 converged storage systems
use SmartTier (dynamic storage tiering), SmartMotion (intelligent data migration), and
innovative SmartVirtualization (heterogeneous virtualization) to achieve vertical,
horizontal, and cross-system 3D data flowing, improving storage resource utilization by
three times.
Hyper-series data protection software: Data protection software such as remote
replication, snapshot, and LUN copy software meets user needs for local, remote, and
multi-region data protection, maximizing business continuity and data availability.
2.1.5 Unified and Easy Management
Unified management: OceanStor V3 converged storage systems provide powerful
storage management software that supports global topology view, capacity analysis,
performance analysis, fault diagnosis, and end-to-end service visualization to manage a
wide range of devices.
Convenient management: OceanStor V3 converged storage systems can be initially
configured in 5 steps which take about 40 seconds, and expanded in two steps which take
about 15 seconds.
Mobile management: Users can use tablets and mobile phones to manage storage
systems in real time. System status is sent automatically, making constant attendance by
an engineer unnecessary.
2.1.6 RAID 2.0+ Block Virtualization
RAID 2.0+ block virtualization of the OceanStor V3 implements virtualization for underlying
disk management and upper-layer resource management. Inside the system, the storage space
of each disk is divided into fine-grained data blocks, which comprise RAID groups. In doing
so, data is evenly distributed to all disks in the storage pool. In addition, data block–based
resource management largely improves the resource management efficiency.
1. The V3 storage systems support SSDs, SAS disks, and NL-SAS disks. These disks
comprise disk domains. In a disk domain, disks of the same type comprise disk groups
(DGs).
2. In a DG, the storage space of disks is divided into chunks (CKs) of a fixed size. Then the
system consolidates CKs from random disks into CK groups (CKGs) based on RAID
algorithms.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 10
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3. CKGs are divided into logical storage space called extents, which have a fixed size too.
Extents are the minimum unit for comprising thick LUNs. On thin LUNs, extents are
further divided into smaller grains.
2.2 Oracle Database and Cluster
Oracle Database is one of the most widely used relational databases. This section briefly
introduces Oracle Database 12c and focuses on Multitenant-related components and features,
including RAC, ASM, Multitenant, data files, database instance architecture, and application
types.
2.2.1 Oracle RAC and ASM
As shown in Figure 2-2, an Oracle 12c RAC contains two types of nodes: Hub nodes and Leaf
nodes. Hub nodes have direct access to shared storage, whereas Leaf nodes access shared
storage through Hub nodes. When a database is deployed on an Oracle RAC, the nodes can be
grouped into multiple server pools. Each database is deployed in a server pool, and every
node in a server pool runs a database instance. Application servers access the virtual IP
addresses (VIPs) of nodes to store data. If a node fails, its VIP network is restored on another
node of Oracle RAC. Application servers reconnect to the Oracle database through a
reconnection mechanism. Setting connection character strings on application servers can
enable multiple modes of accessing Oracle RAC nodes, including load balancing and failover
modes. In these modes, a multi-node Oracle cluster is presented as a single database to
application servers.
The shared storage of the Oracle RAC Hub nodes includes Oracle Cluster Registry (OCR),
voting disks, and database. OCR records information about node statuses, voting disks
synchronize data between nodes, and the database is a set of files.
Figure 2-2 Oracle Flex Cluster 12c
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 11
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Oracle ASM provides a simple storage management interface for database administrators to
manage servers and storage across different platforms. As a built-in file system and volume
manager, Oracle ASM is exclusive to Oracle database files. ASM simplifies file system
management, provides asynchronous I/O performance tuning, saves management time for
administrators, and offers a flexible, efficient database environment.
ASM can consolidate LUNs into a disk group and use Allocation Units (AUs) to allocate
storage space from the disk group. ASM supports three types of disk groups.
External: Data is not mirrored between LUNs, and the storage system provides data
protection.
Normal: A normal disk group consists of two failure groups between which data is
mirrored.
High: A high disk group consists of three failure groups among which data is mirrored.
When an OceanStor V3 storage system is used to create ASM disk groups, it is recommended
that controllers A and B evenly own the LUNs in the disk groups before external or normal
disk groups are created.
Oracle Flex ASM is a new Oracle ASM deployment model that increases database instance
availability and reduces Oracle ASM related resource consumption. Oracle Flex ASM
facilitates cluster based database consolidation, as it ensures that Oracle Database 12c
instances running on a particular server will continue to operate, should the Oracle Flex ASM
instance on that server fail.
Figure 2-3 ASM before Oracle Database 12c
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 12
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Figure 2-4 Oracle 12c Flex ASM
Oracle RAC provides the following key characteristics, essential for HA data management:
Reliability — Oracle Database is known for its reliability. Oracle RAC takes this step
further by removing the database server as a single point of failure. If an instance fails,
the remaining instances in the cluster remain open and active. Oracle Clusterware
monitors all Oracle processes and immediate restarts any failed component.
Error detection — Oracle Clusterware automatically monitors all Oracle RAC databases
as well as other Oracle processes (Oracle ASM, instances, Listeners, etc.) and provides
fast detection of problems in the environment. It also automatically recovers from
failures often before users notice that a failure has occurred.
Recoverability — The Oracle Database includes many features that make it easy to
recover from various types of failures. If an instance fails in an Oracle RAC database, it
is recognized by another instance in the cluster and recovery will start automatically. Fast
Application Notification (FAN) and Fast Connection Failover (FCF) and especially the
Oracle RAC 12c Application Continuity feature make it easy to mask any component
failure from the user.
Continuous Operations — Oracle RAC provides continuous service for both planned and
unplanned outages. If a server (or an instance) fails, the database remains open and
applications continue to be able to access data, allowing for business critical workloads
to finish, mostly without a delay in service delivery.
For more information about Oracle 12c RAC and ASM, refer to the following documents:
White Paper: Oracle Real Application Clusters (RAC)
A Technical Overview of New Features for Automatic Storage Management in Oracle
Database 12c
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 13
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
2.2.2 Oracle System Architecture
Figure 2-5 Oracle system architecture
As shown in the preceding figure, System Global Area (SGA) and Program Global Area
(PGA) of Oracle databases consume memory. SGA stores system information and page cache
information, and PGA stores session information. SGA contains the following parts:
Buffer Cache: buffers data blocks.
Redo Log Buffer: buffers log records as a recycle data group.
Share Pool: buffers data dictionaries and shared SQL information.
Oracle files are categorized as follows:
Control file: records the database structure, parameters, and locations of other data fails.
Data file: stores user data and temporary data.
Online log: record changes to data blocks and consists of several log groups. Files in the
log groups are mirrored to each other. After a log group is used up, data in written to the
next log group. After the first log group is used up, data is written to the first log group
again.
Archive log: In archive mode, databases copy fully written log groups to the archive area
for data restoration when anomalies occur.
Among Oracle processes, the Listener process monitors client connections. Clients are
connected in two modes:
In shared mode, the listener redirects client requests to the dispatcher process, which
places the request in the request queue of the large pool. Then the shared server obtains
and processes the request in the request queue and puts the processing results in the
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 14
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
response queue of the large pool. At last, the dispatcher returns the processing results to
the client.
In dedicated mode, a dedicated server process serves each client connection. After
receiving the request from a client, the Oracle server process looks for the data block in
the buffer cache. If the data block is found, data is read, computed, and changed in the
buffer cache. If the data block does not exist in the buffer cache, the Oracle server
process writes the data block from data files to the buffer cache, and then computes and
changes it.
Oracle uses the LRU algorithm to eliminate outdated data in the buffer cache so that the
released storage space can be used by new data blocks. Data that has been changed in the
buffer cache is called "dirty data", which is written by the DB writer (DBW) process to data
files.
To ensure data integrity and reliability, relational databases use "transaction" to indicate an
atomic operation. When processing a transaction, the Oracle server process records changed
data and the change time in the log buffer. When the transaction is delivered, the log writer
(LGWR) synchronizes the log data in the log buffer to online log files. The log buffer is a
memory area where data can be written in a circular manner. When the log buffer is one-third
full, the LGWR synchronizes the log records to the log file, regardless of whether the
transaction is committed. In addition, Oracle databases synchronize logs automatically every
three minutes.
By default, Oracle databases perform a checkpoint every 30 minutes. When the checkpoint is
performed, the Checkpoint (CKPT) process synchronizes the control file and triggers the
DBW to write dirty data to data files.
Online logs of Oracle are divided into several groups, each of which contains one or multiple
log files. When multiple log files exist, the files are mirrored to each other. Oracle databases
write logs to the log groups in sequence. When the last log group is filled, the databases write
to the first log group, restarting the cycle. Before a log group switch, Oracle checks whether
the dirty data recorded in the next log group is completely written to data files. If not, Oracle
waits until the DBW process writes all the dirty data to the data files before it starts the log
group switch.
When an Oracle database is in archive mode, the Archive (ARC) process copies filled logs to
the archive area. If data anomalies occur, the archived logs are used for precise data recovery.
Oracle Database 12c also supports a Multitenant that allows multiple PDBs to run in one
CDB. Figure 2-6 shows a CDB with four containers: the root, seed, and two PDBs (hrpdb and
salespdb). Each PDB has its own dedicated application. A different PDB administrator
manages each PDB. User SYS can manage the root and every PDB. At the physical level, this
CDB has a database instance and database files. The Multitenant feature brings better
serviceability to the Oracle Database.
The seed PDB is a system-supplied template that the CDB can use to create new PDBs. The seed PDB is
named PDB$SEED. You cannot add or modify objects in PDB$SEED.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 15
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Figure 2-6 Oracle Multitenant architecture
For more information about Oracle Multitenant Architecture, refer to the following document:
White Paper: Oracle Multitenant
2.2.3 Oracle Application Types
Data transactions are categorized as two types: OLTP and OLAP.
OLTP: A number of online users perform transactions.
OLAP: A small number of users perform long-term complex statistical queries.
OLAP applications have the following I/O characteristics:
From the perspective of database administrators
No data or only a small amount of data is modified.
Complex query statements are invoked, and a large number of lines are scanned.
A query takes hours or even days, depending on the complexity of query statements.
Data aging can be measured by hour or day.
The query output is typically a statistical value, obtained using group by and order by.
From the perspective of storage sampling
Every I/O is large-sized, ranging from 64 KB to 1 MB.
Data is read in sequence.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 16
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
When read operations are being performed, write operations are performed in a
temporary tablespace if any.
Online logs are seldom written unless data is loaded in batches.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 17
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3 Best Practices for Oracle Database Planning and Configuration
This chapter describes the recommended plan and configuration, including networking,
storage plan and configuration, as well as host and database parameter settings, for the
scenario where OLAP Oracle Database 12c is deployed based on an OceanStor V3 converged
storage system.
3.1 SAN Networking
Oracle databases typically serve enterprises' mission-critical application systems. To ensure
business continuity, it is recommended that an Oracle RAC cluster be used. The network
must prevent single points of failure. As shown in the following figure, the SAN network
uses two switching planes that are physically isolated from each other. Each switching plane
consists of one switch or multiple interconnected switches. Each of the database nodes and
storage controllers is connected to both switching planes.
Figure 3-1 SAN network
Application layer
Management
networkPublic network RAC interconnection network
Oracle RACStorage
Area
Network
OceanStor 6800 V3
...
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 18
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3.1.1 Setting Zones and vLANs
Zones and vLANs can be set to serve the following purposes:
In the IT environment of an enterprise, a storage system typically serves multiple
application systems. Zones or vLANs can be set to prevent application systems from
affecting each other.
An excessive number of host paths will complicate system management and make
network adjustment more difficult during the O&M process. Zones or vLANs can be set
to improve IT infrastructure flexibility.
The overall policy for setting zones or vLANs is to ensure dual-switching networking.
That is, on each switching plane, each of the database nodes and storage controllers has at
least one logical channel. Based on this policy, consider networking from the following
aspects: performance, deployment complexity, and O&M flexibility.
In terms of performance, plan the number of host ports and paths based on the IOPS and
bandwidth requirements of hosts.
In terms of deployment and O&M, strike a balance between deployment complexity and
O&M flexibility. The following figure shows three zoning approaches. Zoning by application
has the lowest complexity and flexibility, zoning by host has the medium, and zoning by port
has the highest.
Figure 3-2 Zoning or vLAN setting granularity
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 19
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3.2 Storage Configuration
Figure 3-3 Resource allocation flowchart
3.2.1 Planning
Before using an OceanStor V3 converged storage system, it is recommended that you analyze
application performance and capacity requirements and plan the number of disks so that you
can minimize O&M risks.
In addition to a series of entry-level, mid-range, and high-end storage systems, Huawei
provides solid state storage that features low latency and high performance. In terms of
deploying Oracle Database, analysis of storage performance and capacity requirements is
necessary to selection of storage systems and disks. In storage performance assessment,
performance factors are typically first considered and then capacity factors.
In OLAP Oracle Database applications, I/O characteristics are multiple channels of sequential
reads. The major storage performance indicator is the amount of data processed per second,
namely, the bandwidth. Based on the following parameter values and formulas, you can
calculate the performance of RAID levels. Then, estimate the required number of disks.
Table 3-1 Assessment of OLAP application performance
RAID Level
OLAP Throughput (MB/s) For Example, 2 to 8 Gbit/s FC
RAID Group Comprised of Eight 10k rpm SAS Disks
RAID 10 MIN (MBPSPATH
, MBPSDISK
x N) MIN (1350, 50 x 8) = 400
RAID 5 MIN (MBPSPATH
, MBPSDISK
x N) MIN (1350, 50 x 8) = 400
RAID 6 MIN (MBPSPATH
, MBPSDISK
x N) MIN (1350, 50 x 8) = 400
MBPSDISK indicates the multi-stream sequential read throughput of a single disk. For OceanStor V3
converged storage systems, MBPSDISK of a 10k rpm HDD is 50, and that of a 7.2k rpm HDD is 30.
N indicates the number of disks that compose a RAID group.
MBPSPATH indicates the total bandwidth provided for the connections between all database hosts and
the storage system. In the formulas, the maximum bandwidth of a single 16 Gbit/s Fibre Channel link is
1350 MB/s.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 20
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Capacity estimation covers user tablespaces, online logs, archive logs, and backup sets. User
tablespaces and other database areas are typically stored in different LUNs for high
performance and reliability. In OLAP applications, online analysis is implemented and most
I/Os are read I/Os. In capacity estimation, the amount of data in user tablespaces and that in
temporary tablespaces are the major considerations. If archived data and backup data will be
stored, extra capacity must be considered too. Typically, the temporary tablespace size should
be set to the same as the user tablespace size, so that there is sufficient space for sorting
operations.
The use of RAID 2.0+ makes it difficult to estimate the number of disks based on application
performance and capacity requirements. Huawei UniSTAR eDesigner
(http://app.huawei.com/unistar/edesigner/solutionAction!showHome.action), which is a
solution design tool, includes two assessment tools oriented to storage systems. It is
recommended that customers use the two tools to plan the number of disks for Oracle
Database.
Product capability assessment: Based on a specific storage configuration, the
performance and capacity that can be provided by a storage system are assessed.
Storage configuration wizard: Based on specific application requirements, the storage
model and disk configuration are assessed.
Huawei provides verified data warehouse reference architectures based on OceanStor 5300
V3, 5500 V3, 5600 V3, and 5800 V3 converged storage systems. When designing a storage
solution, you can refer to HUAWEI OceanStor V3 Converged Storage Systems — OLAP
Oracle Database Reference Architectures.
3.2.2 Disk Domains
A disk domain is a combination of multiple disks. After disks are consolidated and a certain
amount of hot spare capacity is reserved, a disk domain provides storage resources for storage
pools in a unified manner.
One or more disk domains can be created in an OceanStor V3 converged storage system.
Multiple storage pools can be created in a disk domain.
A disk domain can consist of SSDs, SAS disks, and/or NL-SAS disks.
Disk domains are isolated from each other, including performance, storage resources,
and faults.
The percentage of hot space capacity in a disk domain is related to the hot spare policy and
the number and types of disks. The percentage is automatically maintained by the storage
system based on a reliability engineering method. If you need to plan storage capacity and
performance, it is recommended that you use Huawei UniSTAR eDesigner.
In terms of disk domain creation, the default hot spare policy is high. After a disk domain is
created, you can also change the policy to low or none based on reliability requirements of
applications. However, considering reliability, it is recommended that you do not change
the hot spare policy, especially not to change the policy to none.
OceanStor V3 converged storage systems support SSDs, SAS disks, and NL-SAS disks and
provide tiered storage. For OLAP Oracle Database, it is recommended that you use SSDs or
SAS disks, and do not use NL-SAS disks that provide low performance and reliability. OLAP
typically involves complex queries. A large number of data blocks are accessed, whereas the
access frequency is low. No obvious hotspot data exists. Therefore, it is recommended that
you do not use SmartTier for hotspot data identification and tiered storage.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 21
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
A disk domain in an OceanStor V3 converged storage system can consist of over 1000 disks.
However, considering performance and reliability, it is recommended that each storage tier
of a disk domain contain 100 disks at most.
3.2.3 Storage Pools
A storage pool, a container that stores storage space resources, is created in a disk domain. A
storage pool can dynamically allocate resources from a disk domain and define the RAID
level of each storage tier.
A storage tier is a collection of storage media providing the same level of performance in a
storage pool. Different storage tiers manage storage media with different performance levels
and provide different storage spaces for applications whose performance requirements vary.
There are three types of storage tiers: high-performance tier that consists of SSDs (including
SLC and eMLC), performance tier that consists of SAS disks (including 15k rpm and 10k
rpm), and capacity tier that consists of NL-SAS disks.
OceanStor V3 converged storage systems support seven RAID levels: RAID 6, RAID 10,
RAID 5, RAID 3, RAID 50, RAID 1, and RAID 0. The most commonly used RAID levels are
RAID 6, RAID 10, and RAID 5. From the perspective of stripe width, RAID 6 and RAID 5
are classified into RAID 6-4 (2D2P), RAID 6-6 (4D2P), RAID 6-10 (8D2P), RAID 5-3
(2D1P), RAID 5-5 (4D1P), and RAID 5-9 (8D1P). It is recommended that you set a RAID
level for OLAP Oracle Database based on the following rules:
RAID 6 is preferred, especially for mission-critical database applications that require
high reliability.
If capacity takes precedence over reliability, RAID 5 is recommended.
If a capacity tier is configured for OLAP Oracle Database, it is strongly recommended
that RAID 6 be set for the capacity tier.
OceanStor V3 converged storage systems employ innovative RAID 2.0+. When creating a
storage pool, you can set the extent size. An extent consists of one or more chunk groups
(namely, RAID stripes). Extent is the basic unit used by a storage pool for allocating space to
a LUN. The default extent size is 4 MB, which is applicable to most scenarios. Extent is also
the basic unit of block virtualization. Functions such as hotspot data identification and
migration and data motion are implemented based on extents too. It is recommended that
you set the extent size for OLAP Oracle Database based on the following rules:
4 MB is preferred.
If you want to improve the speed of data motion, it is recommended that you set the
extent size to a larger value.
During the creation of a storage pool, you can set an alarm threshold for the capacity
allocation ratio. The default threshold is 80%. Capacity alarming is particularly important in
scenarios where value-added features such as thin LUN, snapshot, remote replication, and
clone are used. You can set a proper alarm threshold based on the speed of application data
growth to prevent insufficient capacity of the storage pool from causing application
interruption.
When creating a storage pool in developer mode, you are allowed to set a stripe depth. For
Oracle OLAP applications, it is recommended that you set a large stripe depth value (for
example, 512 KB) because the application type is large data block reading, where data is not
frequently written and the amount of written data is small.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 22
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3.2.4 LUNs
In terms of LUNs created for OLAP Oracle Database, it is recommended that you set their
major parameters based on the following rules:
Owner controller (owner_controller): If the database workload is heavy, it is
recommended that you evenly allocate LUNs to controllers to ensure load balancing
among these controllers.
Capacity allocation policy (allocation_policy): Four capacity allocation policies are
provided: Automatic allocation, Allocate from the high performance tier first,
Allocate from the performance tier first, and Allocate from the capacity tier first.
This parameter is used with tier storage. Typically, it is recommended that you set the
policy to Automatic allocation for Oracle storage. If you want to accelerate access to an
identified hotspot area, you can set the policy to Allocate from the high performance
tier first for that area. If you do not want archived data or backup data to consume the
space of the performance tier, you can set the policy to Allocate from the capacity tier
first.
Read cache policy (read_cache_policy): OLAP applications are characterized by
sequential workload. If an OLAP application shares storage with other applications, the
reclamation policy is recommended.
Write cache policy (write_cache_policy): OLAP applications are characterized by
sequential workload. If an OLAP application shares storage with other applications, the
reclamation policy is recommended.
Prefetch policy (prefetch_policy): Four prefetch policies are provided: non-prefetch,
constant prefetch, variable prefetch, and intelligent prefetch. For details about the four
prefetch policies, see the OceanStor V3 converged storage system user manual. OLAP
Oracle Database is characterized by multiple channels of sequential large I/Os. Therefore,
it is recommended that you use the non-prefetch policy if the number of concurrent
accesses is large, and use the intelligent prefetch policy if the number of concurrent
accesses is small.
I/O priority (io_priority): Three levels are provided. The default level is Middle. It is
recommended that you set the I/O priority of the log area to High and that of the archive
area to Low.
LUN type (lun_type): Two types are provided: Thick and Thin. If performance takes
precedence over capacity (for example, a mission-critical production system), it is
recommended that you set the LUN type to Thick. If capacity takes precedence over
performance (for example, a testing/development system), it is recommended that you
set the LUN type to Thin. Thin is not recommended for scenarios where a high write
bandwidth is required, such as a backup target or temporary tablespace area.
LUN capacity and quantity
Different from a traditional RAID group that contains 10+ disks at most, a storage pool that
employs the RAID 2.0+ mechanism contains LUNs across all disks in the disk domain. The
number of disks in a disk domain can be more than 100. To bring disk performance into full
play, it is recommended that you configure LUN capacity and quantity based on the following
rules:
Total number of LUNs in a disk domain: Not less than the number of disks x 4 ÷ 32 is
recommended. (4 is the proper number of concurrent access requests per disk, and 32 is
the default maximum queue depth of a LUN).
LUN capacity: If the preceding condition is met, prefer large-sized LUNs to simplify
management overheads. In terms of the maximum capacity of a LUN, you must consider
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 23
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
the restrictions posed by the operating system and Oracle Database. For example, Oracle
11g ASM requires that the capacity of a LUN do not exceed 2 TB.
3.2.5 Mapping Views
A mapping view defines a logical mapping among LUNs, array ports, and host ports. It is
recommended that you create a mapping view based on the following rules:
A LUN group is an object designed to facilitate LUN resource management. Typically,
LUNs that serve the same application should be added to one LUN group, for example,
LUNs that belong to each storage area of Oracle RAC.
A host group is a collection of hosts that need to share storage resources. Each host
contains multiple initiators (host ports). It is recommended that you create a host for each
server and add all initiators of a server to the host created for that server.
Port groups help users allocate storage ports in a more fine-grained manner. Port groups
are not necessary. However, it is recommended that you allocate a port group for an
OLAP application to improve O&M flexibility and reduce performance impacts between
applications. To prevent single points of failure, each port group must contain at least
one port on each controller.
3.3 Host Configuration
3.3.1 Queue Depth
I/O queue depth is an important parameter that determines the performance of Oracle
Database. Two parameters on the operating system layer affect the I/O queue depth: block
device's queue depth and HBA's queue depth. It is recommended that you set a block device's
queue depth and HBA's queue depth based on the following rules:
In a Linux operating system, the block device's maximum queue depth is 128. It is
recommended that you do not change the value. An HBA's queue parameters vary
depending on the HBA type and driver. For details, see the specifications provided by the
HBA vendor. For example, the QLogic 8 Gbit/s dual-port Fibre Channel HBA allows the
maximum queue depth of each LUN to be 32. It is recommended that you do not
change the maximum queue depth of an HBA. To increase the overall I/O queue depth,
it is recommended that you increase the number of LUNs.
For an AIX operating system, it is recommended that you install UltraPath and do not
use the multipathing software delivered with the operating system or provided by a third
party. After UltraPath is installed, the block device's maximum queue depth is changed to
32. It is recommended that you do not change the value. If UltraPath is not installed, the
block device's default maximum queue depth is 5. It is strongly recommended that you
change the value to 32 or larger. In an AIX operating system, the HBA's default
maximum queue depth is 200. You can change the value based on application
requirements.
For a Windows operating system, the maximum I/O queue depth of a LUN is also
determined by the specifications provided by the HBA vendor. It is recommended that
you do not change the value.
For other operating systems, adjust the queue depth as instructed in the user manual of
that operating system.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 24
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
In AIX, you can run lsattr -El fcs0 to query the parameter settings of HBA fcs0. See the following
figure. num_cmd_elems indicates the maximum I/O queue depth of fcs0, and the default value is 200.
You can run chdev -Pl fcs0 -a num_cmd_elems=512 to change the maximum I/O queue depth of fcs0
to 512.
In AIX, you can run lsattr -El hdisk1 to query the maximum I/O limit (max_transfer) and maximum
queue depth (queue_depth) of hdisk1. You can run chdev -Pl hdisk1 -a queue_depth=32 to change
the maximum queue depth of hdisk1 to 32.
3.3.2 I/O Alignment
If MBR partitions are created in Linux or Windows whose version is earlier than Windows
Server 2003, the first 63 sectors of a disk are reserved for the master boot record and partition
table. The first partition starts from the 64th sector by default. As a result, misalignment
occurs between data blocks (database or file system) delivered by hosts and data blocks stored
in the disk array, causing poor I/O processing efficiency.
When creating MBR partitions in Linux, it is recommended that you enter the expert mode of
the fdisk command and set the start location of the first partition to the start location of the
second extent on a LUN. (The default extent size is 4 MB.) The following is a quick
command used to create an MBR partition in /dev/sdb. The partition uses all space of
/dev/sdb. The start sector is set to 8192, namely, 4 MB.
printf "n\np\n1\n\n\nx\nb\n1\n 8192\nw\n" | fdisk /dev/sdb
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 25
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
The step-by-step command used to set partition alignment is described as follows:
fdisk /dev/sdb
n (Create a partition.)
p (Set the partition type to maser.)
1 (Set this partition as the first partition.)
(Press Enter. Set the start location of the partition to the default start sector, namely, 64.)
(Press Enter. Set the end location of the partition to the default last sector.)
x (Enter the expert mode.)
b (Set the start location of the partition.)
1 (Set the first partition.)
8192 (Set the start location to 8192, namely, 4 MB.)
w (Write data to disks and quit.)
Another way to resolve partition misalignment in Linux is to use GPT partitions. The
following is a quick command used to create a GPT partition in /dev/sdb. The partition uses
all space of /dev/sdb. The start sector is set to 8192, namely, 4 MB.
parted -s -- /dev/sdb "mklabel gpt" "unit s" "mkpart primary 8192 -1" "print"
To create MBR partitions in Windows whose version is earlier than Windows Server 2003, it
is recommended that you run diskpart to set partition alignment.
diskpart> select disk 1
diskpart> create partition primary align=4096
3.3.3 Block Device Scheduling Algorithms
Linux 2.6 kernel supports four types of block device scheduling algorithms: noop,
anticipatory, deadline, and cfq. In OLAP application scenarios, if traditional disks are used,
the deadline algorithm is recommended; if SSDs are used, the noop algorithm is
recommended. Change the configuration file /boot/grub/menu.lst and add
elevator=algorithm name in the kernel line. This method applies to all block devices.
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
elevator=deadline
3.3.4 Block Device I/O Settings
In Linux, max_sectors_kb, a block device parameter, restricts the maximum I/O data block
size. Set max_sectors_kb to the database I/O size. For example, the typical data block size
used by an OLAP database is 32 KB. The db_file_multiblock_read_count parameter
specifies the number of concurrent I/O blocks. If db_file_multiblock_read_count is set to 32,
the concurrent I/O size of the database is 1024 KB. In this case, it is recommended that you
change the value of max_sectors_kb to 1024.
In the configuration file /etc/rc.local, you can add the following command that enables the
value of max_sectors_kb to be changed to 1024 upon each startup.
echo 1024 > /sys/block/sdb/queue/max_sectors_kb
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 26
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
3.4 Database Configuration
3.4.1 Database Parameters
Database Block Size
The block size used by Oracle Database ranges from 2 KB to 32 KB, with the default value of
8 KB. Users can change the block size as required. For a typical OLTP application, the
database block size is usually set to the default value 8 KB. However, for an OLAP
application, it is recommended that the database block size be set to a larger value, so that the
application performance is significantly improved.
The db_block_size parameter indicates the database block size. You need to set it when creating a
database. After that, you cannot change the value.
Memory Allocation
In Oracle database applications, the memory should be fully used without compromising the
normal system running. It is recommended that 80% of the physical host memory be allocated
to Oracle Database. In OLAP applications, a large number of sorting operations are performed.
When dedicated servers are used to access the database, sorting operations occur in the PGA.
It is recommended that at least 50% of the memory allocated to Oracle Database be allocated
to the PGA for improving system performance.
Assume that MEM indicates the physical memory of a host. Set database instance parameters
memory_target and memory_max_target to 0. Set sga_max_size and sga_target to 50% x MEM. Set
pga_aggregate_target to 50% x MEM.
Parallel Operations
OLAP Oracle Database involves a large number of complex query and analytical statements,
with the possibility of massive full-table scanning operations. If the host is equipped with
multi-core CPUs, parallel queries can greatly improve the query performance but consume a
lot of CPU resources of the host. It should be noted that, the maximum number of parallel
queries is related to the concurrency capabilities of tables.
Assume that the number of CPUs in a host is CPU_COUNT and the number of cores on each CPU is
PARALLEL_THREADS_PER_CPU. Then, you can set parallel_max_servers to
PARALLEL_THREADS_PER_CPU x CPU_COUNT x 4 x 5, indicating that a maximum of
parallel_max_servers parallel operations can be concurrently performed. Set parallel_min_servers to
a value that ranges from 0 to parallel_max_servers, indicating the number of processes concurrently
running after the database instance starts.
I/O Policy
If a database is deployed in file system mode, the I/O policy can be set to synchronous I/O,
asynchronous I/O, direct I/O, and/or merged I/O. To achieve a better I/O efficiency, it is
recommended that you enable both asynchronous I/O and direct I/O.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 27
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Database parameter filesystemio_option can be set to ASYNCH, DIRECTIO, SETALL, or NONE.
ASYNCH indicates that the asynchronous I/O policy is enabled for the file system. DIRECTIO
indicates that the direct I/O policy is enabled for the file system. SETALL indicates that the
asynchronous I/O and direct I/O polices are enabled for the file system. NONE indicates that the
asynchronous I/O and direct I/O polices are disabled for the file system. It is recommended that you set
filesystemio_option to SETALL.
3.4.2 Online Logs
If OLAP Oracle Database is in use, data is frequently written to the analytical database during
the Extract, Transform, Load (ETL) process. It is recommended that you set the size of the
online log file to a large value. However, it is recommended that the size do not exceed 128
MB. Otherwise, if a failure occurs, the recovery will be time-consuming. Oracle Database
writes logs in a cyclical manner. To prevent the log waiting issue, it is recommended that you
create at least four log groups for each instance, enable each log group to contain two log files,
and place one of the two log files in the archive area.
3.4.3 Backup and Archiving
It is strongly recommended that you enable the archive mode for the databases running on the
production system and configure a backup policy to ensure that data is backed up at least once
a day.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 28
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
4 Example of Oracle Database Planning and Configuration
4.1 Solution Architecture
Figure 4-1 Tiny-size data warehouse reference architecture
Oracle 12c RAC
2 x RH2288 V2 [1]
5300 V3 [2]
50 x 600 GB 10k rpm SAS disks
4 x
16 G
bit/s
FC
Tiny-Size Data Warehouse Solution2 TB[3] data, 3 GB/s[4] analytical bandwidth
Cluster Private Interconnection
S6700 10GE switch
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 29
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
[1] The detailed configuration of the RH2288 V2 is as follows: 2 x E5-2660 CPUs, 256 GB memory, 1 x
QLogic 16 Gbit/s dual-port Fibre Channel HBA, and 1 x Intel 10 Gbit/s Ethernet HBA.
[2] The detailed configuration of the OceanStor 5300 V3 converged storage system is as follows: 32 GB
cache, 1 x controller enclosure (25 x 600 GB 10k rpm SAS disks, 2 x 16 Gbit/s dual-port Fibre Channel
I/O modules), 1 x disk enclosure (25 x 600 GB 10k rpm SAS disks), 1 x disk domain containing all the
50 disks (25 TB capacity), 1 x storage pool (RAID 6, 12 TB capacity, and 512 KB stripe size), 16 x 500
GB LUNs (eight LUNs used as the data area, two LUNs used as the log area, and six LUNs used as the
ETL area).
[3] The tested database contains 2.5 TB of table and index data as well as 1 TB of temporary tablespace.
[4] The tested stable analytical bandwidth is 3 GB/s and the tested peak analytical bandwidth is 4.2 GB/s,
based on the TPCH-Like universal order and warehouse analytical model.
4.2 Database Plan
The solution simulates a 2 TB tiny-size Oracle OLAP data warehouse and uses the universal
analytical model (TPCH-Like) as the workload model. The following table lists the plan of
capacity, workload, and disk quantity. (The storage configuration wizard of UniSTAR
eDesigner is used to plan the disk quantity, with the RAID level of the storage pool set to
RAID 6.)
Table 4-1 Plan of database capacity, workload, and disk quantity
Size Amount of Data Bandwidth Disk Quantity
Tiny-size 2 TB > 3 GB/s 50
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 30
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
4.3 Storage Configuration
Figure 4-2 Storage configuration
[1] The disk domain configuration is as follows: The hot spare policy is High (default policy).
[2] The storage pool configuration is as follows: The RAID level is RAID 6-10. The capacity is 12 TB.
The stripe depth is 512 KB. Other parameters keep the default values.
[3] The LUN configuration of the data area is as follows: Both the read and write policies are
reclamation. The prefetch policy is non-prefetch. LUNs are evenly allocated to controllers A and B.
The LUN configuration of the log area is as follows: Both the read and write policies are reclamation.
The prefetch policy is intelligent prefetch. LUNs are evenly allocated to controllers A and B.
The LUN configuration of the ETL area is as follows: Both the read and write policies are reclamation.
The prefetch policy is intelligent prefetch. LUNs are evenly allocated to controllers A and B.
[4] The external redundancy policy is configured for all ASM disk groups. Other parameters keep the
default values.
[5] An ASM volume is created in the +ETL disk group and uses all space of the +ETL disk group.
ACFS is created in the ASM volume and mounted to the /opt/oracle/ETL directory as a data loading
area.
[6] For details about the database configuration, see Table 4-3.
4.4 Host Configuration In the configuration example, partitions are not created. Therefore, I/O alignment
configuration is not involved.
The block device's queue depth and HBA's queue depth are not changed.
The block device's I/Os scheduling policy is set to deadline.
The Linux huge page is set to 200 GB (about 80% of memory capacity).
+DATA +LOG +ETL+GRID
...50 x 600 GB 10k rpm SAS disks
15 TB allocated 9 TB free 1 TB hot spare
4288 GB free
1 x 100 GB LUN
...
8 x 500 GB LUNs 2 x 500 GB LUNs 6 x 500 GB LUNs
...
/opt/oracle/ETL
2.5 TB table & index data
1 TB temporary tablespace2 TB text data
LoadAnalytical Queries
Disk
Disk Domain [1]
LUN [3]
ASM Disk Group[4]
ACFS [5]
Storage Pool [2]
Oracle Database[6]
8100 GB allocated
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 31
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
The following table provides the specific host parameter settings.
Table 4-2 Host parameter settings
Configuration File Parameter Value
/etc/sysctl.conf vm.nr_hugepages 102400 (2 MB)
kernel.shmmax 214748364800 (bytes)
kernel.shmall 52428800 (4 KB)
/etc/security/limits.conf oracle soft nproc 16384
oracle hard nproc 65536
oracle soft nofile 16384
oracle hard nofile 65536
oracle soft memlock 209715200 (1 KB)
oracle hard memlock 209715200 (1 KB)
/sys/block/sd*/queue/scheduler deadline
/sys/block/sd*/queue/max_sector
s_kb
1024
4.5 Database Configuration A total of 10 online log groups exist. Each instance is assigned five groups that are stored
in +LOG.
In the solution, backup components are not deployed. In actual implementation, it is
strongly recommended that you consider backup components and configure a backup
policy.
The archive log area can retain logs of the latest 15 days at least.
The following table provides database parameter settings.
Table 4-3 Database parameter settings
Parameter Value
db_create_file_dest +DATA
db_block_size 32768
db_file_multiblock_read_count 32
sga_target 50 GB
pga_aggregate_target 80 GB
lock_sga TRUE
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 32
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Parameter Value
use_large_pages ONLY
db_files 512
process 1024
parallel_max_servers 256
parallel_force_local true
4.6 Workload Verification Results
Table 4-4 Workload verification results
Size Amount of Data Stable Bandwidth
Peak Bandwidth
Disk Quantity
Tiny size 2.5 TB of table and index data
1 TB of temporary data
3 GB/s 4.2 GB/s 50
Figure 4-3 Execution process of multi-stream analytical query
The y-axis indicates the bandwidth (expressed in MB/s). The x-axis indicates the execution time
(expressed in seconds). rmbps indicates the read bandwidth. wmbps indicates the write bandwidth
(temporary tablespace sorting).
The test covers four analytical query streams. Each host involves two streams. Each stream contains 22
complex queries that represent typical data warehouse applications. The query statement sequence and
parameters of each stream are generated at random. Different query statements have different needs for
computing and I/O resources. In some phases that involve a large number of I/O operations, the stable
bandwidth is at least 3 GB/s, and the peak bandwidth is 4.2 GB/s.
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 545 1091 1755 2302 3039 3613 4159 4705 5358 5903 6496
rmbps wmbps
Stable: 3 GB/s
Peak: 4.2 GB/s
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 33
Planning and Configuring HUAWEI OceanStor V3 Converged Storage Systems to Maximize
OLAP Oracle Database's Performance and Availability
Huawei Technologies Co., Ltd.
Address: Huawei Industrial Base
Bantian, Longgang
Shenzhen 518129
People's Republic of China
Website: http://www.huawei.com
Email: [email protected]
Copyright © Huawei Technologies Co., Ltd. 2015. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of Huawei Technologies Co., Ltd.
Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and the
customer. All or part of the products, services and features described in this document may not be within the
purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,
and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.