View
115
Download
0
Category
Tags:
Preview:
DESCRIPTION
Citation preview
Database Virtualization and
Instant CloningKyle Hailey
http://dboptimizer.com
Database Cloning Challenge
Business want data now.
Business don’t understand DBAs.
Databases getting bigger & harder to copy.
Developers want more copies.
Reporting wants more copies.
Everyone has storage constraints.
If you can’t satisfy the business demands your process is broken.
Two Parts
I. Cloning TechnologyII. Accelerate your business
Part I : Cloning Technology
3. Virtual2. Thin Provision1. Physical
=
database
1. Physical Cloning
Problem
Developers
QA and UAT
Reports
First copy
Production
• CERN - European Organization for Nuclear Research
• 145 TB database• 75 TB growth each year• Dozens of developers want copies.
workaroundsDevelopers
QA and UAT
ReportsShared
Sub set copy
Production
Many copies
Physical Clones
Database SubsetsShared Databases
Subsets
ProductionThe Production ‘Wall’
Classic problem is that queries that run fast on subsets hit the wall in production.
Developers are unable to test against all data
Shared Full
Shared access = Poor Productivity
Developers and tester get frustrated
Databases become old and unrepresentative of production.
Requires complex scheduling and management
Never enough environments
Average customer makes 12 copies of production- Charles Garry, Database Product Manager Oracle
Physical Copies – Time & Space consuming
Time consumingTime to make copies, days to weeks
RMAN backup, archive logs, copy data over, recover
Meetings , days to weeksSystem, Storage ,Database ,Network Admins , manager coordination
Space consuming 12 devs x 10TB production = 120TB20 report DBs x 40 TB = 800TB
Slow Clones mean bottlenecks
Setup Develop
Setup
Setup Develop
QA
$40M
$75M
$850M
$27,000M
Storage
IT
Develop
Business
ERP Project Failures 2011
• NYC CityTime : delays $63 M => $760 M • Montclair Uni: delays sues PeopleSoft• Idaho : delays ERP cost millions
Standish : IT Project Failure Rate
1994 1996 1998 2000 2002 2004 2009
31% 40% 28% 23% 15% 18% 24%
★http://www.galorath.com/wp/software-project-failure-costs-billions-better-estimation-planning-can-help.php*http://www.pcworld.com/article/246647/10_biggest_erp_software_failures_of_2011.html
Clone 1 Clone 3
99% of blocks are Identical
Clone 2
2. Thin Provision Cloning
Clone 1 Clone 2 Clone 3
Thin Provision
2. Thin Provision Cloning
Requires Cornerstone Technology
I. clonedbII. Copy on Write Snapshots
a) EMC BCV b) EMC SRDF or Recover Pointc) Vmware
III. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS
2. Thin Provision Cloning
RMAN backup
dNFSsparse file
I. clonedb
RMAN backup
dNFSsparse file
I. clonedb
I. CloneDB• dNFS 11.2.0.2+
– cd $ORACLE_HOME/rdbms/lib– make -f ins_rdbms.mk dnfs_on
• Clonedb.pl initSOURCE.ora output.sql– MASTER_COPY_DIR="/rman_backup”– CLONE_FILE_CREATE_DEST="/nfs_mount”– CLONEDB_NAME="clone"
• sqlplus / as sysdba @output.sql– startup nomount PFILE=initclone.ora – Create control file backup location– dbms_dnfs.clonedb_renamefile ('/backup/file.dbf' , '/clone/file.dbf');– alter database open resetlogs;
Tim Hallwww.oracle-base.com/articles/11g/clonedb-11gr2.php
I. CloneDB
RMAN
physical
Target A NFS Server
Clone 1
Clone 2
Read only Clone 3
Target B
Clone 1
Clone 2
Clone 3
I. clonedbII. Copy on Write + snapshots
a) EMC BCVb) EMC SRDF or Recover Pointc) Vmware
III. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS
2. Thin Provision Cloning
D
ActiveFile
SystemSnapshot
CBA
File System constant - EMC
II. Copy on Write a) EMC snapshots
D
ActiveFile
SystemSnapshot
DCBA
Changes written in place - EMCChanges to new area - VMware
II. Copy on Write a) EMC snapshots
• Goal: backup– Create BCV , take snaps of BCV– Zone and mask LUN to target host– Full copy of disk, snapshot, recover
• Snapshots– 16 snapshots– Zero or one snapshots of snapshots
• “Golden Copy”– EMC uses a save area, area for changes to the snapshot– Initial snapshot has to stay
II. Copy on Write a) EMC BCV
Non-Prod FilerEMC Filer
ProductionDatabase
Database Luns
Target A
Target B
Target C
Clone 1
Clone 2
Clone 3
Clone 4
SnapshotsMax 16
a) EMC BCVIII. Allocate on Write
BCV Day 1
snapshot
Snapshotor break
Day 2
Timefinder needed for snapshots on across multiple LUNs
• SRDF – Symetrix Remote Data Facility– Stream Changes to a remote machine
• Recover Point– Capture changes on wire– Split changes send to remote site
II. Copy on Write b) EMC SRDF
Non-Prod FilerEMC Filer
ProductionDatabase
Database Luns
Target A
Target B
Target C
Clone 1
Clone 2
Clone 3
Clone 4
File system level
SRDF orRecover Point
Snapshot day 1
Snapshot day 1
Snapshot day 2
Snapshot day 2
II. Copy on Write b) EMC SRDF
D
ActiveFile
SystemSnapshot
DCBA
Changes written to a new locationChanges to new area - VMware
II. Copy on Write, c) VMware
II Copy on Write c) VMwareData Director : Linked Clones
• Not support for Oracle databases• Golden Copy : rebuild after 32 snapshots• x86 host databases only• Performance issues
– “Having several linked clones can affect the performance of the source database and the performance of the linked clones.”http://bit.ly/QOXbyE (on http://pubs.vmware.com )
– “If you are focused on performance, you should prefer a full clone over a linked clone.”http://www.vmware.com/support/ws5/doc/ws_clone_typeofclone.html
– Performance worse with more snapshots– Performance worse with more concurrent users
I. clonedbII. Copy on Write
a) EMC BCVb) EMC SRDF or Recover Pointc) Vmware
III. Allocate on Writea) ZFSb) Netapp (EMC VNX)c) DxFS
2. Thin Provision Cloning
Data Blocks
root
a. ZFSb.Netapp (EMC VNX)c. Dxfs
III. Allocate on Write
a) ZFS
• 1 disk = 1 filesystem• ~1990: N disks = 1 FS• 2001: ZFS starts• 2005: ZFS ships• 2008: ZFS storage appliance ships• 2010: Delphix heads ZFS open source
III. Allocate on Write
FS vs. ZFS• FS per Volume
• FS limited bandwidth
• Storage stranded
• Many FS in a pool
• Grow automatically
• All bandwidth
Storage PoolVolume
FS
Volume
FS
Volume
FS ZFS ZFS ZFS
3a) ZFSIII. Allocate on Write
a) ZFS
Snapshot rootLive root
Delphix Proprietary and Confidential
ZilIntent Log
III. Allocate on Write
ZFS Appliance + RMAN
1. ZFS Appliance– Project db_master– Project db_clone – 4 file systems: datafile, redo, archive, alerts
2. Source Database– NFS Mount ZFS Appliance – Backup with RMAN
3. ZFS Appliance• Select db_master• Snapshots• Then each filesystem on db_master clone it onto db_clone
4. Target Host– NFS Mount db_clone– recover clone
cloning-solution-353626.pdf
a) ZFS III. Allocate on Write
Oracle ZFS Appliance + RMAN
1. physical
ZFS Storage Appliance
RMANcopy
RMAN Copyto NFS mount
Target A
Clone 1
Clone 1
Snapshot
NFS
a) ZFS III. Allocate on Write
cloning-solution-353626.pdf
Oracle ZFS Appliance + Dataguard
Dataguard
ZFS Storage Appliance
datafiles
NFS
Target A
Clone 1
Clone 1
Snapshot
NFS
a) ZFS III. Allocate on Write
Datafiles via NFS
maa-db-clone-szfssa-172997.pdf
NetApp FilerNetApp Filer
ProductionDatabase
Database Luns
snapshot
Target A
Target B
Target C
Clone 1
Clone 2
Clone 3
Clone 4
File system level
snapshot
clonesSnap mirror
Snapshot Manager for Oracle
Flexclone
b) NetappIII. Allocate on Write
Requires a database
NetApp FilerNetApp Filer
ProductionDatabase
Database Luns
snapshot
Target A
Target B
Clone 1
Clone 2
snapshot
snapshot
snapshot
b) NetappIII. Allocate on Write
NetApp FilerNetApp Filer
Physical Database
Database Luns
snapshot
Target A
Target B
Clone 1
Clone 2
snapshot
snapshot
snapshot
b) NetappIII. Allocate on Write
• Compression • typically 3x
• Block sharing• DxFS optimized for databases
• Write optimizations• Space allocation and destroy• Shared blocks in memory
c) DxFSIII. Allocate on Write
I. clonedbII. Copy on Write Snapshots
a) EMC BCV b) EMC SRDF or Recover Pointc) Vmware
III. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS
2. Thin Provision Cloning
• Clonedb • EMC BCV
• Netapp• EMC SRDF• EMC VNX
2. Review: Thin Provision Cloning
2. Thin Provision Cloning
2. Thin Provision Cloning
3. Database Virtualization
• Oracle SMU• Delphix
Virtualization Layer
Virtualization
SMU
a) Oracle 12c SMUOracle Snap Management Utility for ZFS Appliance
• Requires ZFS Appliance• Supports Linux , Solaris 10+, Windows 2008+• GUI
– snapshot source databases – provision virtual databases
3. Database Virtualization
Virtualization Layer
x86 hardware
Allocate StorageAny type
SMU
Choose your virtualization Layer:• Delphix and Oracle SMU automated out of box• Netapp and EMC SRDF/Recover Point require massive scripting• Delphix and possibly SMU share blocks in memory
ZFS Storage Appliance
One time backup of source database
Database
Production
Instance
File system
RMAN APIs
Delphix Compress Data
Database
Production
Instance
File system
Data is compressed typically 1/3 size
Incremental forever change collection
Database
Production
Instance
File system
Changes are collected automatically foreverData older than retention widow freed
Typical Architecture
Database
File system
Production
Instance
Database
File system
Development
Instance
Database
File system
QA
Instance
Database
UAT
Instance
File system
Clones share duplicate blocks
Development
Instance
Database
Production
Instance
File system
vDatabase
QA
Instance
UAT
Instance
vDatabase vDatabaseNFS
Source Database Clone Copies of Source Database
Fiber Channel
Part II: Virtualization Case Studies
1. Development2. QA3. Recovery
1. Development Acceleration
Database Virtualizationlayer
3 clones of same source database 3 virtual clones of same source database
Virtualization layer orchestrates I/O access between database and storage
Source Timeflow
Source Timeflow
Source Database
Target HostVirtual
Database
NFS
Fiber
Fiber
RMAN over TCP
Source
Fresh
VDB
Source
VDB
VDB
VDB
Frequent
Fresh data
Full clonesFrequent (many)
fast
Self Service
Devv2.6 v2.6v2.6
QA UAT
v2.6Production
v2.6 v2.6v2.6v2.7
v2.6 v2.6v2.6v2.8
Source Control for the database data
v2.6v2.6 v2.6v2.6
v2.6v2.7 v2.6v2.7
v2.6v2.8 v2.6v2.8
DevProd
2.6
Dev
QA
Prod
2.6
Dev
QA
UAT
Prod
2.6
Dev
QA
UAT
Prod
Dev
QA
UAT
2.6
2.7
Dev
QA
UAT
Prod
Dev
QA
UAT
2.6
2.7
Dev
QA
UAT2.8
Dev
QA
UAT
Prod
Dev
QA
UAT
2.6
2.7
Dev
QA
UAT2.8
Data Control = Source Control for the Database
Collaborative Provisioning
VDB2
Source2
Source3
Source1
VDB3
VDB1
Collaborative sources
“I looked like a hero”Tony Young, CIO Informatica
1: Review of Development
• Provision , Self Service– Fast– Full– Fresh– Frequent (Many)
• Source control for the database• Federated cloning
2. Rapid QA
dSource
Pass to QA
Developer VDB
QA VDB
Source
Punt to Dev
Developer VDB
QA VDB
Developer VDB
dSource
Forensic Analysis
VDB
dSource
Upgrades, Patches, RAT
VDB
A/B Testing
• Production vs Virtual– invisible index on Prod– Creating index on virtual
• Flashback vs Virtual• Confidence testing
– change reporting SQL– compare new code over last 30 days
2: review QA
• Fork copies off of Dev to QA• QA fork copies back to Dev• Instant replay
– performance A/B– Upgrade, patching– Forensic analysis of Prod
3. Backup and Recovery
MONSUN WEDTUE THU SATFRI MONSUN TUE
MONSUN WEDTUE THU SATFRI MON
SUN TUE
Dat
a Pr
oces
sing
Re
quire
d
Current Volume of Work
Source
Rollback / Time machine
VDB
Source
Rollback / Time machine
VDB
VDB Rolled back
Source
Recovery
VDB
V2P
3: review Backup
• Backup 50 days in size of 1 copy• logical recovery on prod, dev• Restore virtual to production
Review Part II: Data Virtualization1. Development: Full, Fresh, Fast, Data SCCS = SDCS2. QA : forking, replay, forensics3. Recovery: 50:1, time machine, V2P
Review Part I: Technology1. Full Clones2. Thin Provision : dbclone, EMC, Netapp3. Database Virtualization : SMU, Delphix
over 10 times
"perhaps the single largest storage consolidation opportunity history“
Oracle 12c
80MB buffer cache ?
5000
Tnxs
/ m
inLa
tenc
y
300 ms
1 5 10 20 30 60 100 200
with
1 5 10 20 30 60 100 200Users
200GBCache
5000
Tnxs
/ m
inLa
tenc
y
300 ms
1 5 10 20 30 60 100 200Users
1 5 10 20 30 60 100 200
200GBCache
8000
Tnxs
/ m
inLa
tenc
y
600 ms
1 5 10 20 30 60 100 200Users
1 5 10 20 30 60 100 200
Database Virtualization
Matrix of featuresCloneDB ZFS
ApplianceDelphix Data
DirectorNetApp EMC
Time Flow No Yes Yes No Yes No
HardwareAgnostic
Yes No Yes Yes No No
Snapshots No Unlimited Unlimited 31 255 16 (96 read only)
Snapshots of snapshots
No Unlimited Unlimited 30 255 1
Automated Snapshots
No No Yes No Yes No
Automated Provisioning
No No Yes No No No
Any O/S Yes Yes Yes Nox86 only
Yes Yes
Max size None None None ? 16-100TB
?
Conclusion : Enterprise Solutions• VMware Data Director • EMC Timefinder
– offer limited ability to benefit from cloning• Clonedb ***
– fast easy way to create many clones of the same copy– limited to 11.2.0.2 and systems with sparse file system capability– suffers the golden image problem and performance
• NetApp Flexclone, Snap Manager for Oracle– offers a rolling solution – limited database awareness– file system clones– limited snapshots– Vendor lock-in
• Oracle ZFS Appliance– Vendor Lock-in
• Delphix– Agility : Automation, unlimited snapshots, clones, multi-database
Appendix• CloneDB
– http://www.oracle-base.com/articles/11g/clonedb-11gr2.php• ZFS
– http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf• ZFS Appliance
– ZFS setup for Oracle http://www.oracle.com/technetwork/server-storage/solaris/config-solaris-zfs-wp-167894.pdf– using RMAN http://www.oracle.com/technetwork/articles/systems-hardware-architecture/cloning-solution-353626.pdf– using dataguard http://www.oracle.com/technetwork/database/features/availability/maa-db-clone-szfssa-172997.pdf
• Data Director– http://pubs.vmware.com/datadirector-25/topic/com.vmware.ICbase/PDF/vfabric-data-director-25-administration-guide.pdf
p107 linked clone not support for Oracle• EMC
– https://community.emc.com/servlet/JiveServlet/previewBody/11789-102-1-45992/h8728-snapsure-oracle-dnfs-wp.pdf– http://www.centroid.com/knowledgebase/blog/cloning-oracle-databases-with-emc-snapview-and-recoverpoint– http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:39960485246162 BCV and SRDF
• NetApp– http://www.redbooks.ibm.com/redpapers/pdfs/redp4133.pdf– OEM for Netapp http://www.youtube.com/watch?v=J7fnfLS5Dxg&feature=youtu.be
• Oracle SMU– http://www.oracle.com/us/products/servers-storage/storage/nas/smu-bus-wp-final-1857065.pdf– http://www.oracle.com/us/products/servers-storage/storage/nas/snap/smu-datasheet-1903178.pdf– http://www.oracle.com/us/products/servers-storage/storage/nas/snap/overview/index.html
• Oracle OEM Snapshot Manager for Netapp– http://www.youtube.com/watch?v=J7fnfLS5Dxg&feature=youtu.be– Demo starts at 3:10 goes to 19:43
Other
• Can’t run multiple clones on same host with ASM https://forums.oracle.com/forums/thread.jspa?messageID=1810251
• EMC VNX http://virtualgeek.typepad.com/virtual_geek/2012/05/vnx-inyo-is-going-to-blow-some-minds.html
NetApp Limits
Controller Size Limit
32 bit controllers 16TB
FAS3140/FAS3040/FAS3050 40TB
FAS3160/FAS3070 50TB
FAS6040/FAS3170 70TB
FAS6080 100TB
All sources have to be in the same aggregate to be snapshot together.
Limit of 255 snapshotssnaps are limited to the same aggregate (storage pool)Aggregates have size limits depending on controller
b) NetappIII. Allocate on Write
b) NetappIII. Allocate on Write
b) NetappIII. Allocate on Write
Netapp vs
EMC
Production
Initial Incremental Incremental
Redo
Clones
Redo
a) ZFS III. Allocate on Write
Goal - How Can we do it?
Data Director
CloneDB NetApp
EMC
Delphix
SMU
ZFS
Flawed
Limited UseManual
low touch: virtualization
Automated (easy)
Manual (Hard)
Full
Func
tiona
lity
Doe
sn’t
wor
k
Application Life Cycle Management
Number of required clones change during Project and Extended Project Life CycleProjects with phased Go-Lives can have Implementation, Handover ‘Cut-Over’ and Operations ‘RUN’ all running concurrently.
Development & maintenance is hard
UserAcceptance
TestTestDevelopment
Production
Training & Education
ProductionSupport
New release / upgrades lifecycle…Bespoke, enhancements, configuration changes etc…
Production Support lifecycle…Internal bug fixes, 3rd Party Patches, Data fixes etc..
UserAcceptance
TestTestDevelopment
Training & Education
ProductionSupport
Multiple Clones used by Production Support Analysts, DBA team and for internal training
DBA/
Note: All environments periodically refreshed from production
Database’s not just used by developers
Data Migration TeamData Mapping & CleansingDevelopment of ETL (Extraction, Transformation & Load) Routines…
UserAcceptance
TestUser
AcceptanceTestSystem
ValidationTesting
Data MigrationData
MigrationData Migration
Technical TeamsSystem IntegrationEnterprise Scheduling Automated TestingPerformance Testing…
Application Functional Specialists
Development of System Set-upBusiness Process ConfigurationSandbox…
Business Process OwnersApplication FunctionalProcess validationCRP (Conference Room Pilot) …
UserAcceptance
TestUser
AcceptanceTest
UserAcceptance
Test
AutomatedTestingAutomated
TestingAutomatedTesting
Source Timeflow
Source Timeflow
Source Database
Target HostVirtual
Database
NFS
Fiber
Fiber
RMAN over TCP
Recommended