DevOps, Databases and The Phoenix Project UGF4042 from OOW14

Virtual Data Platform: or

Revolutionizing Database Cloning

How can the DBA make the biggest impact on the company

1

http://kylehailey.comkyle@delphix.com

The Goal : Theory of Constraints

Improvement not made at the constraint is an illusion

factory floor optimization

Factory floor

resinMolding

Trimmer

Leak detection

Labeling

Capping/Filling

Pallet - izing

Shipping

Factory floor

resinMolding

Trimmer

Leak detection

Labeling

Capping/Filling

Pallet - izing

Shipping

constraint

Not a relay race

resinMolding

Trimmer

Leak detection

Labeling

Capping/Filling

Pallet - izing

Shipping

Tune before constraint

constraint

Tuning here

Stock piling

resinMolding

Trimmer

Leak detection

Labeling

Capping/Filling

Pallet - izing

Shipping

Tune after constraint

constraint

Tuning here

Starvation

Factory floor : straight forward

constraint

Goal: find constraint optimize it

Theory of Constraints work for IT ?

• Goals Clarify • Metrics Define • Constraints Identify • Priorities Set • Iterations Fast

• CI• Cloud • Agile • Kanban• Kata

“IT is the factory floor of this century”

The Phoenix Project

What is the constraint

in IT ?

What are the top 5 constraints in IT?

1. Dev environments setup2. QA setup3. Code Architecture4. Development5. Product management

- Gene Kim

“One of the most powerful things that organizations can do is to enable development and testing to get environment they need when they need it“

Data is the constraint

60% Projects Over Schedule

85% delayed waiting for data

Data is the Constraint

CIO Magazine Survey:

only getting worseGartner: Data Doomsday, by 2017 1/3rd IT in crisis

• Data Constraint• Solution• Use Cases

In this presentation :

• Data ConstraintI. strains ITII. price is hugeIII. companies unaware

• Solution• Use Cases

– Storage & Systems– Personnel – Time

moving data is hard

Typical Architecture

Production

Instance

File system

Database

Typical Architecture

Production

Instance

Backup

File system

Database

File system

Database

Typical Architecture

Production

Instance

Reporting Backup

File system

Database

Instance

File system

Database

File system

Database

Typical Architecture

Production

Instance

File system

Database

Instance

File system

Database

File system

Database

File system

Database

Instance Instance

Instance

File system

Database

File system

Database

Dev, QA, UAT Reporting Backup

Triple Tax

Typical Architecture

Production

Instance

File system

Database

Instance

File system

Database

File system

Database

File system

Database

Instance Instance

Instance

File system

Database

File system

Database

Data floods infrastructure

92% of the cost of business,

in financial services business , is “data”

www.wsta.org/resources/industry-articles

Most companies have 2-9% IT spending , ½ on “data”

http://uclue.com/?xq=1133

• Data ConstraintI. strains ITII. price is hugeIII. companies unaware

• Solution• Use Cases

In this presentation :

Four Areas data tax hits

– IT Capital resources $– IT Operations personnel $– Application Development $$$– Business $$$$$$$

price is Huge

$Hardware–Servers–Storage–Network–Data center floor space, power, cooling

$ Never enough environments

• People– DBAs– SYS Admin– Storage Admin– Backup Admin – Network Admin

• Hours : 1000s just for DBAs • $100s Millions for data center modernizations

$ IT Operations

• Inefficient QA: Higher costs of QA• QA Delays : Greater re-work of code• Sharing DB Environments : Bottlenecks• Using DB Subsets: More bugs in Prod• Slow Environment Builds: Delays

$ Application Development

Ability to capture revenue

• Business Applications – Delays cause lost revenue

• Business Intelligence – Old data = less intelligence

$ Business

• Data ConstraintI. strains ITII. price is hugeIII. companies unaware

• Solution• Use Cases

In this presentation :

companies unaware

companies unaware

Developer or AnalystBoss, Storage Admin, DBA

Metrics–Time –Old Data –Storage –Analysts –Audits

companies unaware

• Data ConstraintI. strains ITII. price is hugeIII. companies unaware

• Solution• Use Cases

In this presentation :

Clone 1 Clone 3Clone 2

99% of blocks are identical

Solution

Clone 1 Clone 2 Clone 3

Thin Clone

• EMC – 16 snapshots on Symmetrix– Write performance impact– No snapshots of snapshots

• Netapp– 255 snapshots

• ZFS– Compression– Unlimited snapshots– Snapshots of Snapshots

• DxFS– “”– Storage agnostic– Shared cache in memory

Technology Core : file system snapshots

Also check out new SSD storage such as:Pure Storage, EMC XtremIO

Fuel not equal car

Challenges 1. Technical2. Bureaucracy

Bureaucracy

Developer Asks for DB Get Access

Manager approves

DBA Request system

Setup DB

System Admin

Requeststorage

Setup machine

Storage Admin

Allocate storage (take snapshot)

Why are hand offs so expensive?

1hour1 day

9 days

Bureaucracy

Technical Challenge

Database Luns

Production FilerTarget A

Target B

Target C

snapshotclones

InstanceInstance

InstanceInstance

InstanceInstance

InstanceInstance

Instance

Source

Database LUNs

snapshotclonesProduction Filer

Development Filer

Technical Challenge

Instance

Target A

Target B

Target C

InstanceInstance

InstanceInstance

InstanceInstance

Instance

Technical Challenge

Copy Time FlowPurge

Production

File System

Instance

DevelopmentStorage

21 3

Clone (snapshot)CompressShare Cache

ProvisionMount, recover, renameSelf Service, Roles & Security

Instance

– EMC + SRDF – Netapp + SMO– Oracle EM 12c + data guard + Netapp /ZFS – Delphix

2 12 1

3 1 2

How to get a Data Virtualization?Source

sync

Deployautomation

Storagesnapshots

21 3

Goal : virtualize, govern, deliver

04/09/2023 44

• Security• Masking• Chain of custody

• Self Service• Roles• Restrictions

• Developer• Data Versioning • Refresh, Rollback

• Audit:• Live Archive Snap Shots

Thin CloningData Virtualization

Data Supply Chain

Intel hardware

Unstructured Data

Install Delphix on x86 hardware

Allocate Any Storage to Delphix

Allocate StorageAny type

Pure Storage + DelphixBetter Performance for 1/10 the cost

One time backup of source database

Database

Production

File systemFile system

InstanceInstanceInstance

DxFS (Delphix) Compress Data

Database

Production

Data is compressed typically 1/3 size

File system

InstanceInstanceInstance

Incremental forever change collection

Database

Production

File system

Changes

• Collected incrementally forever• Old data purged

File system Time Window

Production

InstanceInstanceInstance

Snapshot 1 – full backup once only at link time

Jonathan Lewis © 2013 Virtual DB

50 / 30

a b c d e f g h i

We start with a full backup - analogous to a level 0 rman backup. Includes the archived redo log files needed for recovery. Run in archivelog mode.

Snapshot 2 (from SCN)

Jonathan Lewis © 2013

b' c'

a b c d e f g h i

The "backup from SCN" is analogous to a level 1 incremental backup (which includes the relevant archived redo logs). Sensible to enable BCT.

Delphix executes standard rman scripts

Apply Snapshot 2

Jonathan Lewis © 2013

a b c d e f g h ib' c'

The Delphix appliance unpacks the rman backup and "overwrites" the initial backup with the changed blocks - but DxFS makes new copies of the blocks

Drop Snapshot 1

Jonathan Lewis © 2013

b' c'a d e f g h i

The call to rman leaves us with a new level 0 backup, waiting for recovery. But we can pick the snapshot root block. We have EVERY level 0 backup

Creating a vDB

Jonathan Lewis © 2013

b' c'a d e f g h i

The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward)

My vDB(filesystem)

Your vDB(filesystem)

b' c'a d e f g h i

Creating a vDB

Jonathan Lewis © 2013

b' c'a d e f g h i

The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward)

My vDB(filesystem)

Your vDB(filesystem)

i’

b' c'a d e f g h ib' c'a d e f g h i

Database Virtualization

Three Physical CopiesThree Virtual Copies

Data Virtualization Appliance

Before Virtual Data

Production Dev, QA, UAT

Instance

Reporting Backup

File system

Database

Instance

File system

Database

File system

Database

File system

Database

Instance Instance

Instance

File system

Database

File system

Database

“triple data tax”

With Virtual DataProduction

Instance

Database

Dev & QA

Instance

Database

Reporting

Instance

Database

Backup

Instance Instance Instance

Database

InstanceInstance

Database

InstanceInstance

File system

Database

Data Virtualization Appliance

• Problem in the Industry• Solution• Use Cases

In this presentation :

1. Development and QA 2. Production Support3. Business

Use Cases

1. Development and QA2. Production Support3. Business

Use Cases

Development : bottlenecks

Frustration Waiting

Old Unrepresentative Data

Development : subsets

False NegativesFalse PositivesBugs in Production

Development : bugs

http://martinfowler.com/bliki/NoDBA.html

Development : slow env build times

Development: Virtual Data

• Unlimited • Full size • Self Service

Development

Virtual Data: Easy

Instance

Instance

Instance

Instance

Source

DVA

Development Virtual Data: Parallelize

gif by Steve Karam

Development Virtual Data: Full size

Development Virtual Data: Self Service

QA : Virtual Data• Fast • Parallel• Rollback• A/B testing

QA : Long Build times

96% of QA time was building environment$.04/$1.00 actual testing vs. setup

QA Build QAQA Build QA

BugX

1 2 3 4 5 6 70

10203040506070

Delay in Fixing the bug

Cost ToCorrect

Software Engineering Economics – Barry Boehm (1981)

Dev

QA

Instance

Prod

DVATime Flow

• Low Resource• Find bugs Fast

QA Virtual Data : Fast

QA with Virtual Data: Rewind

Instance

Instance

Development

Prod

QA with Virtual Data: A/B

Instance

Instance

Instance

Index 1

Index 2

Data Version Control

04/09/2023 77

Dev

QA

2.1

Dev

QA

2.2

2.1 2.2

Instance

Prod

DVA

1. Development and QA2. Production Support3. Business

Use Cases

• Backups• Recovery• Forensics• Migration• Consolidation

Recovery

Backups

Recovery Instance

Instance

Recover VDB

Drop

Source

DVA

Forensics

Instance

Instance

Development DVA

Source

Development (the new production)

Instance Instance

Development DVA

Source

Instance

Development

Migration

Consolidation

1. Development and QA2. Production Support3. Business Intelligence

Use Cases

Business Intelligence

• ETL• Temporal• Confidence Testing• Federated Databases• Audits

Business Intelligence: ETL and Refresh Windows

1pm 10pm 8am noon

Business Intelligence: batch taking too long

1pm 10pm 8am noon20112012201320142015

20112012201320142015

1pm 10pm 8am noon

10pm 8am noon 9pm

6am 8am 10pm

Business Intelligence: ETL and DW Refreshes

Instance

Prod

Instance

DW & BI

• Collect only Changes• Refresh in minutes

Instance Instance

Prod BI and DW

ETL24x7

DVA Instance

Virtual Data: Fast Refreshes

Temporal Data

Confidence testing

Modernization: Federated

Instance

Instance

Instance

Instance

Source1

Source2

DVA

Modernization: Federated

“I looked like a hero”Tony Young, CIO Informatica

Modernization: Federated

Audit

04/09/2023 98

Instance

Prod

DVA

Live Archive

1. Development & QA2. Production Support3. Business

Use Case Summary

How expensive is the Data Constraint?

DVA at Fortune 500 :

Dev throughput increase by 2x

• Faster Financial Close• Faster BI refreshes• Faster surgical recovery• More Project tracks• Faster Projects

How expensive is the Data Constraint?

• Projects “12 months to 6 months.”– New York Life

• Insurance product “about 50 days ... to about 23 days”– Presbyterian Health

• “Can't imagine working without it”– State of California

Virtual Data Quotes

• Problem: Data is the constraint • Solution: Virtualize Data• Results:

• Half the time for projects• Higher quality• Increase revenue

Summary

105

Oaktable World & hands on labs

We are here

Oaktable World

MosconeSouth

Thank you!

• Kyle Hailey| Oracle ACE and Technical Evangelist, Delphix – Kyle@delphix.com– kylehailey.com– slideshare.net/khailey

Oracle 12c

80MB buffer cache ?

200GBCache

5000

Tnxs

/ m

inLa

tenc

y

300 ms

1 5 10 20 30 60 100 200

with

1 5 10 20 30 60 100 200Users

8000

Tnxs

/ m

inLa

tenc

y

600 ms

1 5 10 20 30 60 100 200Users

1 5 10 20 30 60 100 200

$1,000,000 1TB cache on SAN

$6,000200GB shared cache on Delphix

Five 200GB database copies are cached with :

04/09/2023 113

04/09/2023 114

Business Intelligence

a) 24x7 Batches

b) Temporal queries

c) Confidence testing

Thin Cloning

Snap Manager

SnapManagerRepository

Protection Manager

Snap Drive

Snap Manager

Snap Mirror

Flex Clone

RMANRepository

Production

Development

DBA

Storage Admin

1 tr-3761.pdf

Netapp

NetApp Filer - DevelopmentNetApp Filer - Production

Database Luns

Snap mirror

Snapshot Manager for Oracle

Flexclone

Repository Database

SnapDrive

Protection Manage

Production

Development

1 NetappTarget A

Target B

Target C

InstanceInstance

InstanceInstance

InstanceInstance

Instance

Where we want to be

Database

File system

Production

Instance

Database

Development

Instance

Database

QA

Instance

Database

UAT

Instance

Snapshots

Instance Instance Instance Instance

EM 12c: Snap Clone

Production Development

Flexclone Flexclone

Netapp Snap Manager for Oracle

II. Data constraint price is Huge : 4. Business

II. Data constraint price is Huge : 4. Business

Storage

IT Ops

Dev

Revenue

0 5000 10000 15000 20000 25000 30000Billion $

III. Data Constraint companies unaware

#1 Biggest Enemy :

IT departments believe– best processes – greatest technology– Just the way it is

Are you Innovative?

III. Data Constraint companies unaware

Why do I need an iPhone ?

Don’t we already do that ?

SQL scriptsAlter database begin backupBack up datafilesRedoArchiveAlter database end backup

RMAN

Merge Dev1 to ForkMerge to dev2

Dev2

Dev1Merge to dev1

Merge Dev2 to Fork

Trunk

Merge Dev1 to Fork

Merge Dev2 to Fork

DBVC

Fork

Fork

Fork

Fork

DBmaestro

1. Federated2. Migration3. Auditing

Modernization

If you can’t satisfy the business demands then your process is broken.

What is the constraint in IT