1

Accelerate Oracle Performance

by Using SmartCache of T Series

Unified Storage

This document describes why and how to use SmartCache in Oracle OLTP database to accelerate the

transaction performance.

SmartCache is an advanced feature of OceanStor T series unified storage, with which hot blocks on

traditional hard disks could be detected and migrated to SSD cache pool, and later hit in the pool. Through

this technology, the physical read response time and the OLTP throughput could be significantly improved.

Using SmartCache in OLTP database, the transaction response time could be commonly reduced to 50%

or lower, and the transaction throughput could be increased to 200% or higher.

Jarvis WANG

wangyaohui@huawei.com

IT Storage Solution & Verification, Enterprise BG

2012-9-28 Version 1.1

2

SmartCache is an advanced feature of OceanStor T series storage, with which hot blocks

on traditional hard disks could be found and migrated to SSD cache pool, and later be hit

in the pool.

The following diagram shows the principle of SmartCache. Hot blocks on traditional hard

disks are identified by SmartCache Engine, and then be migrated to SSD cache pool. The

hot blocks in SSD cache pool have big chances to be re-read again, in other words, hit in

cache pool. The read response time could be significantly improved if there‟s enough hit

ratio. If the data of the cached hot blocks is updated, the new data will be written to

hard disks and updated in the cache pool simultaneously. When the cache pool is full

and there‟re new hot blocks detected, old blocks previous loaded into the cache pool will

be evicted out using LRU-LIKE algorithm.

T Series Storage

SmartCache Engine

Cold I/O Hot I/O

Hot Block

Oracle Database

HDD RAID Group SSD Cache Pool

What is SmartCache

3

As we know, in OLTP (online transactional processing) systems, Oracle is random I/O

intensive, and “hot” areas exist. Besides, the increasing of capacity and performance is

not in-phase for enterprise hard disk. The capacity of enterprise hard disk is too big and

too slow for OLTP database nowadays. Using SmartCache in Oracle OLTP database, the

hot blocks could be cached in SSDs and the physical random read response time could

be improved. User could benefit from the low latency, high random IOPS and low power

consumption features of SSDs.

The following table shows the top 5 wait events in one oracle OLTP database running on

traditional magnet disks (10K RPM SAS disk). 96.87% of DB time is spent on waits of “db

file sequential read”, and each wait costs 15 milliseconds. The high latency of the slow

magnet disks is definitely the bottleneck of the OLTP system.

The following chart shows read load distribution of a LUN which stores a big table and its

indexes. The LBA is spitted to 100 units and read I/O is counted for each unit. We could

clearly find the hot areas of the LUN.

Why SmartCache

OLTP

bottleneck

OLTP Hot

Areas

4

The following table shows the capacity and performance change of enterprise hard disk

in the last 5 years, the capacity is increased year by year, but the performance stays the

same level. When deploying Oracle OLTP database on hard disk, users will buy much

more spindles than the capacity needs to suport the high random I/O requirements,

while the free capacity can‟t be used to store other things because that will make the

performance of database bad. Tiering storage or pure SSD solution are now strongly

recommended in Oracle OLTP database, and T series storage with SmartCache is an

advanced technology fit into the requirements.

Year Capacity RPM OLTP Throughput OLAP Throughput

2008 73 GB 15 K 200 IO/s 30 MB/s

2009 146 GB 15 K 200 IO/s 30 MB/s

2010 300 GB 15 K 200 IO/s 30 MB/s

2011 600 GB 15 K 200 IO/s 30 MB/s

2012 900 GB 15 K 200 IO/s 30MB/s

HUAWEI OceanStor T Series Unified Storage (T Series) is new-generation storage

product for mid-range and high-end storage applications. It boasts integration of

block-level and file-level data storage, supports for a variety of storage protocols, and

GUI-based central storage management.

0

2

4

6

8

10

12

14

16

18

1 11 21 31 41 51 61 71 81 91

% R

ead

Lo

ad

T Series

Storage

Capacity and

Performance

of Hard Disk

5

The most important performance related features of T series are DST (dynamic storage

tiering) and SmartCache. When using file storage or NAS, DST could be used to

dynamically detect hot files and put them to tier1 storage, while cold files which are not

frequently visited are left in tier2 storage. When using block storage or SAN,

SmartCache could be used to detect hot blocks and put them into SSD cache pools.

Oracle database is widely deployed on SAN storage, SmartCache could significantly

reduce the physical read response time for oracle database if you choose T series

storage.

The following chart shows the change of transaction response time (New-Order Trans.)

and „db file sequential read‟ wait time before and after setting SmartCache for oracle

OLTP database. The transaction response time is reduced from 247ms to 70ms,

reduced to 28%. The physical read wait time is reduced from 15ms to 5ms,

reduced to 33%. The end users are significantly benefited from SmartCache and

SSDs.

Performance

Acceleration

6

The following chart shows the change of maximum active users and transaction

response time before and after setting SmartCache for oracle OLTP database. The

maximum active users (Act. Users) are increased from 2000 to 8000, increased to

400%. The transaction response time (Avg. DB RT) is reduced from 247 to 117,

reduced to 47%. The IT system could host much more users in the future when

setting SmartCache for oracle database.

The tests are performed using TPCC-LIKE workload, in which the read ratio is about 60%. Workloads with higher read ratio

will benefit more from SmartCache and SSDs.

15

5

247.02

70.05

0

2

4

6

8

10

12

14

16

0.00

50.00

100.00

150.00

200.00

250.00

300.00

Base Line - Without Smart Cache With Smart Cache

ave

rage

re

ad t

ime

(ms)

Tra

nsa

ctio

n R

esp

on

se T

ime

(ms)

db file sequential read New-Order Trans.

2000

8000

247.02

117.37

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0.00

50.00

100.00

150.00

200.00

250.00

300.00

Base Line - Without Smart Cache With Smart Cache

Act

ive

End

Use

rs

Tra

nsa

ctio

n R

esp

on

se T

ime

(ms)

Act. Users Avg. DB RT (ms)

7

It‟s very simple to use SmartCache, you don‟t have to install software on oracle host

systems or apply patches for oracle database, and you don‟t have to migrate any

database object, just plug SSDs in the empty slot of T series disk enclosure and enable

SmartCache, the transaction response time will be reduced significantly and the

transaction throughput will be doubled, trebled or more improved.

The following steps introduce how to enable SmartCache for LUNs in ISM (Integrated

Storage Management Software) of T series storage.

1. Import and active SmartCache license on ISM

Connect T series storage using ISM, in “Settings” page, click “License Management”.

In “License Management” dialog, go to “Inactivated License” tab, click “Import” button, select the

license file and click “open” button.

How to Use SmartCache

8

Click “Active” button to active the license.

2. Set SmartCache for each controller

In ISM “SmartCache” page, select controller “A”, click “Disk Management” button, and then add

SSDs to the SmartCache pool “A”, then click “OK”.

9

In ISM “SmartCache” page, select controller B, click “Disk Management” button, and then add 3

SSDs to the SmartCache pool B, then click “OK”.

3. Enable SmartCache for LUNs

In ISM “LUNs” page, select the LUNs which is used to store schema objects of OLTP database, click

“Advanced” drop down list, select “Enable SmartCache”.

10

4. Check size of cached data and hit ratio

You can check the SmartCache counters 4 hours after SmartCache is enabled in detailed LUN

properties.

In ISM “LUNs” page, select a LUN on which SmartCache is enabled, you will see the detailed

properties of the LUN. The counter “SmartCache Capacity” stands for the amount of cached data

in SmartCache, the counter “Percentage of SmartCache” stands for the percentage of the LUN‟s

cached data in SmartCache, and the counter “Hit Ratio of SmartCache” stands for the LUN‟s block

hit ratio in SmartCache.

11

5. Done

12

This chapter introduces best practices when using SmartCache in T series storage.

T series storage supports two kinds of SSDs: SLC (Single-Layer Chip) and eMLC

(Enterprise Multi-Layer Chip). SLC has much better random write performance and more

number of block erase count, but more expensive than eMLC. SLC and eMLC almost

have the same random read performance. SmartCache is read cache, the update data of

cached blocks is both written to LUNs and SSD cache pool, so the data will NOT be lost

when the SSDs fail. So choosing eMLC is a better idea when using SmartCache.

The maximum size of SmartCache and the maximum size of LUNs with SmartCache

enabled are limited by storage controller‟s memory size. The following table lists the

configuration limit of SmartCache.

Storage Max. LUN size (TB) Max. SmartCache pool size (GB)

S5500T 32 1200

S5600T 32 2400

S5800T 64 3600

S6800T 64 4800

The size of SmartCache could be changed dynamically. First you could set SmartCache

pool size to 10% of user data, if the transaction response time is reduced to a good

level, try to reduce the SmartCache size. If the transaction response time is also bad,

double the cache size until the response time good or to the size limit.

If you have more than one disk enclosures for T series storage, average distributing the

SSDs onto the level 0 expanded enclosures is recommended. Through this way, you

could gain maximum I/O throughput improvement.

Oracle is widely deployed on Linux Operating system. There‟re 4 kinds of I/O scheduler

on block devices in Linux kernel 2.6: “noop”, “anticipatory”, “deadline”, and “cfq”. The

default I/O scheduler is “cfq”, which is not suitable for LUNs with SmartCache enabled.

SmartCache Best Practices

SLC or eMLC

Size of

SmartCache

Linux I/O

Scheduler

How to Place

SSDs

13

For Oracle database, “deadline” is recommended for LUNs without SmartCache enabled

and “noop” is recommended for LUNs with SmartCache enabled.

Using the following command, you could change the I/O scheduler of “/dev/sdb” to “noop” and “/dev/sdc” to “deadline”:

echo noop > /sys/block/sdb/queue/scheduler

echo deadline > /sys/block/sdc/queue/scheduler

14

Copyright © Huawei Technologies Co., Ltd. 2012. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd.

Trademark Notice

HUAWEI, and are trademarks or registered trademarks of Huawei Technologies Co., Ltd.

Other trademarks, product, service and company names mentioned are the property of their respective owners.

General Disclaimer

The information in this document may contain predictive statements including, without limitation,

statements regarding the future financial and operating results, future product portfolio, new

technology, etc. There are a number of factors that could cause actual results and developments

to differ materially from those expressed or implied in the predictive statements. Therefore, such

information is provided for reference purpose only and constitutes neither an offer nor an

acceptance. Huawei may change the information at any time without notice.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Industrial Base

Bantian Longgang

Shenzhen 518129, P.R. China

Tel: +86-755-28780808

www.huawei.com

PROVIDED BY HUAWEI STORAGE PERFORMANCE LAB