1

Hüsnü ŞensoyHüsnü Şensoy

Turkcell Telecommunication Services VLDB ExpertOracle ACE DirectorMember of Global DWH Leaders & Oracle CABOracle DBA of 2009

2

Agenda• Backup & Recovery Strategies for Oracle Databases

– Motivation behind those strategies– Revisiting “Incrementally Updated Backup”– Revisiting “FRA”– How to bring your database back without restore ?– Sick backup will not work– Centralized scheduling & monitoring

• 11g Release 2 Backup & Recovery New Features with real Telco data warehouse data – Brand new compression algorithms

• Summary

3

Turkcell Overview

• Leading GSM operator of Turkey established in February 1994.

• Third GSM operator in Europe in terms of subscriber (+36 million).

• First and only Turkish company ever to be listed on New York Stock Exchange.

• Member of Board of Directors of GSMA since 2003.

• 25th company of INFOTECH 100 list.

4

BACKUP & RECOVERY STRATEGIES BACKUP & RECOVERY STRATEGIES FOR ORACLE DATABASESFOR ORACLE DATABASES

5

Design Considerations• Define your backup &

recovery policies upfront– A well documented

strategy that can be used to bring everything back

– KISS:KISS: Even a junior DBA should be able to bring your database back.

– Standardize, standardize, standardize…

– Be prepared to justify the cost in terms of business impact of downtime

6

Design Considerations• Proactively validate

database and backup integrity– Physical errors– Logical

inconsistencies– Transmission

errors• Do you perform

regular full recoveries to separate host and storage?

7

Design Considerations

• Centralized backup reporting:– Is there a single point of

access for all my databases’ backup logs ?

– What is the average backup duration for database X ?

– How do brand new tape drives affect backup performance ?

8

What’s in there ?•7 RAC databases•More than 20 services

What’s in there ?•7 RAC databases•More than 20 services

20

Gb

it/s

12

0 I

nte

l Co

res

64

0 G

B M

em

ory

25

TB

DATADATA

ARCHIVEARCHIVEFRAFRA

What type of Architecture ?

VASRESVASRES

VASNIFVASNIF

VASCMTVASCMT

APPDBAPPDB

BSSARCHBSSARCH

BSSOSSBSSOSS

VASSEVASSE

9

How Do We Backup ? Incrementally Updated Backup

Strategy Initial image copy backup to

FRA Fast incremental backups

thereafter Image copy is rolled forward

with incremental backup on regular basis to create full on-disk backup

Full database backup times only depend on the amount of blocks changed since last incremental backup.

The longest backup time is only ~30 minutes, with ZLIB backup compression and logical block checking turned on.

run{

backup as compressed backupset

check logical incremental level 1

for recover of copy with

tag DAILY_COPY database

filesperset 1;

recover copy of database with

tag DAILY_COPY;

}

This is the shortest, cleanest, and most elegant backup script that I have seen in all my years at Turkcell.

This is the shortest, cleanest, and most elegant backup script that I have seen in all my years at Turkcell.

10

Setting Up FSetting Up Flash Recovery Area(Oracle Database 11g Release 1)

• Self managed & organized logical storage area.

• Setup as part of Universal Installer wizard. – Redo log copy, control

file copy, archived logs, and Flashback logs are automatically stored there.

– RMAN automatically utilizes FRA for all disk backups.

• Or, just enable it by setting two init.ora parameters :

– db_recovery_file_dest_size– db_recovery_file_dest

11

FFlash RRecovery AArea• ASM is the best

infrastructure to be used as FRA destination:– Raw device performance.– No other solution (except

Sun ZFS file system with its online FS check capability) will practically let you implement large storage pools as ASM does.

– Ease of management.• ASM allows you to

provision the same diskgroup to multiple FRA destinations.

ASM Diskgroup (+FRA)ASM Diskgroup (+FRA)

DB1 FRADB1 FRA DB2 FRADB2 FRA DB3 DB3 FRAFRA DB4 FRADB4 FRA

12

Restore-Free Recovery

13

What Are the Commands?

From hoursto

minutes

14

• Backups on disk or tape might be damaged due to– Physical problems on media (fabric problems,

dust, cosmic rays, etc)– Media library errors (error in checksum

computation)

• How you can increase the probability that your backups are healthy ?

Backup Validation

15

Possible Solutions

16

RMAN Backup ValidationRMAN> backup check logical validate

datafilecopy all

filesperset 1;

• This will report– For any inconsistent data, index, or other type of blocks.– Number of total and empty blocks examined.– Highest change number of each datafile copy.

17

Centralized Scheduling & Monitoring

• Develop standard backup job scheduling and monitoring routines.

• This enables you to:– See all backup schedules at once– Check details of previously completed backups

(duration, logs, etc.)– Easily modify backup scripts and bulk deploy

them.

18

Grid Control Backup Jobs

Manage backup of all databases of the cluster by using just one screen

19

20

21

11G RELEASE 2 RMAN 11G RELEASE 2 RMAN COMPRESSIONCOMPRESSION

22

11gR2 RMAN Compression11gR2 RMAN Compression

23

Test SetupTest Setup

Data

•Marketing data from Turkcell data warehouse•2.2 billion records (140G)•No segment compression•PCTFREE 1•16K block size tablespace

Number of Channels •8 RMAN Channels

Compression Types

•NONE•BASIC•LOW•MEDIUM•HIGH

Collected Metrics

•Compression Ratio•Duration•I/O Throughput•CPU Utilization

24

Backup Compression Summary• In Oracle Database 11g Release 2, RMAN extends

its compression capabilities to fit any CPU power and I/O throughput combination.

• MEDIUM compression level can backup faster than BASIC with the same compression ratio and 3X faster with 50% less CPU utilization.

• Even if you don’t have need to reduce backup sizes, LOW/MEDIUM compression level might be faster than uncompressed backup depending on your I/O throughput, by significantly reducing the amount of data/sec written by RMAN.

25

Best Practices Summary

• A well defined, documented, standard, manageable, and fast backup & recovery strategy is a MUST if you manage tens (even hundreds) of databases.

• Whatever solution you pick, the indicator of a good backup & recovery strategy is simple:– It shouldn’t depend on the size of database.

• FRA over ASM and RMAN satisfies these requirements with zero cost.