Unlocking the potential of
dev/test/prod automation
October 2016
Before We Begin
You will be on mute for the duration of the event
Please type a message in the Questions box in the Control
Panel if you can’t hear us (please check your speakers
and webinar audio settings first)
If you have questions during the session, please submit
them on the Q&A bar on your webinar dashboard and we
will address them at the end
A recording of the full webinar will available online
Presenters
Gary Hallam
CTO, Director of Presales, EMEA at Delphix
Gary has a development & consulting background with a lot of global project delivery
experience. He ran a remote tech pre-sales team at Oracle before joining Delphix.
Yaniv spent the last years raising awareness about the challenges around database
development and deployment, and how to support database Continuous Delivery.
© 2016 Delphix Corporation 4 4 © 2016 Delphix Corporation
Over 30 of the Global100 run on Delphix
Financial Healthcare Telecom Manufacturing Retail Technology
5 © 2016 Delphix Corporation
Delphix value
Accelerate application release cycles
Meet data privacy requirements
Make large scale data accessible
Reduce infrastructure requirements
All at the same time
Simplify and automate database deployment processes in an
agile environment and reduce critical application downtime.
10X Faster time to market
15% Increase in team efficiency & productivity
80% Reduction in application downtime due to human errors
100% Control, 100% Compliance - control and audit who, does what, when and why!
© 2016 Delphix Corporation 9 9 © 2016 Delphix Corporation
Continuous Delivery “Stack”
VIRTUALIZATION
AUTOMATION • Self service environment creation
• Continuous integration, release delivery
• Virtualized runtime environment
• Instantiate base system image
• Install and configure software
• Configuration as code
• Connect services across systems
• Dynamic scaling
• Find and correct problems
• Analyze and tune performance
CONFIGURATION
MANAGEMENT
ORCHESTRATION
MONITORING
© 2016 Delphix Corporation 10 10 © 2016 Delphix Corporation
Data Is The Missing Link
VIRTUALIZATION
AUTOMATION • Self service environment creation
• Continuous integration, release delivery
• Virtualized runtime environment
• Instantiate base system image
• Install and configure software
• Configuration as code
• Connect services across systems
• Dynamic scaling
• Find and correct problems
• Analyze and tune performance
CONFIGURATION
MANAGEMENT
ORCHESTRATION
MONITORING
DATA MANAGEMENT
▸ How do you supply
production data?
▸ How do you share
the data?
▸ How do you stand
up data in new
environments?
Your data has never done this before
12 © 2016 Delphix Corporation
Data Virtualization: How it works
STORAGE STORAGE
NON PRODUCTION PRODUCTION
STORAGE
APP
DBMS
STORAGE
APP
DBMS
APP
DBMS
APP
DBMS
NON PRODUCTION PRODUCTION
STORAGE STORAGE STORAGE
APP
DBMS
APP
DBMS
APP
DBMS
APP
DBMS
Collect & compress
production data
Capture All Data
Create, rewind, refresh,
point-in-time virtual copies
Self Service
Share data blocks instead
of duplicating data
Virtual Data
Continuous change-data
recording
Time Machine
13 © 2016 Delphix Corporation
Delphix Architecture Schematic
Other Databases
Flat files (Applications)
EBS Data Sources
and VDBs
DELPHIX ENGINE
Production
Sources
Target Host
VDB
Target Host
VDB
TCP
Synchronization
(Incremental
Backups)
Any
Storage
Data
Self Service
14 © 2016 Delphix Corporation
DEV TEAMS: Power Data Features
to Accelerate SDLC
Governance Model Across Dev Teams, IT Ops
AUTOMATE
Provision apps,
DBs with right data
in minutes
MASK
Replace private,
sensitive data with
masked values
RETAIN
Store 30 days in space
of one for restore,
archive
REPLICATE
Replicate within, across
sites, or to the cloud
TRANSFORM
Convert Unix data into to
Linux for modernization
OPS TEAMS: Full Security, Control,
& Resource Allocation
ROLLBACK
Reset to run A/B tests, recover
from errors and data loss
SYNCHRONIZE
Dial multiple applications to
exact same time
BOOKMARK
Mark release changes,
business events
BRANCH
Pass environments to/from
Dev and QA
REFRESH
Refresh to the latest data
for high fidelity testing,
analytics
15 © 2016 Delphix Corporation
User Interfaces and Automation API
Dev Ops
16 © 2016 Delphix Corporation
Accelerate Releases – “Shift Left” Quality
200 TB
Storage saved
20 min
Environment turnaround
60%
Lower Defect Rates
Case Study: Fortune 500 Financial Institution
• Massive growth in online platform usage, demand for features
• Struggling with quality issues impacting customer experience
• Explosion in data growth, rising IT costs
0
100
200
300
400
500
Dev Testing UAT Production
Bugs Discovered Before After
400
300
200
100
0
© 2016 Delphix Corporation 17 17 © 2016 Delphix Corporation
Accelerate with Synchronised Application Time Travel
Switch
Branch/Version
ORCL
SQL
/application
TEMPLATE
Synchronize data
delivery across
sources
Enable short
Feedback Loops
CONTAINER Self-Service
© 2016 Delphix Corporation 18 18 © 2016 Delphix Corporation
Developers Can Test and Undo Changes Quickly
Developer resets in minutes. A new project requires a
destructive schema change.
The developer creates a
bookmark
Developer tests schema change.
Change is not correct and
development is unusable.
Test Developers
can’t work
Developer Developer Developer Developer
Developer makes changes to
their script and successfully re-
executes.
Test Developers
continue to work
1:30:56 P.M.
© 2016 Delphix Corporation 19 19 © 2016 Delphix Corporation
Increase Quality and Speed Delivery with Self-Service
Tester resets in minutes,
continues testing
Tester logs defect and creates a
bookmark of the same name and
shares with development
Tester encounters error
1:30:56 P.M.
Developer
Errors are found earlier using
real business data
Developer opens defect ticket
and creates a branch off of the
bookmark in minutes
Developer
After fixing the defect, the
developer activates the
branch he was previously
working on and continues
development
© 2016 Delphix Corporation 20 20 © 2016 Delphix Corporation
What About Data Security of Virtual Data?
PRODUCTION DATA
20%
80%
NON-PRODUCTION DATA
NON-PRODUCTION DATA:
» REPRESRENTS LARGE SURFACE AREA
OF RISK
On average, 8-12 copies for every production
source
» ENTAILS DISPARATE DATA SOURCES
secured with a variety of tools and processes
» IS CONSTANTLY GROWING
and changing over time
Data Masking Can Dramatically Reduce The Surface of Risk
21 © 2016 Delphix Corporation
Evolution of Data Masking
CAPABILITY
HOME
GROWN
SCRIPTS
VIRTUAL
DATA
MASKING
EXTRACT
TRANSFORM
LOAD
STORED
PROCEDURE
▸Mask Once and Share
▸Simplicity: Easy to implement,
easy to use
▸Database Agnostic
▸Self-service & automation
▸Enterprise Architecture
22 © 2016 Delphix Corporation
Delphix Data Masking
• Verify all sensitive data is
masked
• Alert admins if vulnerabilities are
identified
• Deliver assessment to auditors
• Mask data without any programming
• Maintain usability with fictitious, but
realistic data
• Apply masking with consistency,
repeatability
• Identify sensitive data across
sources
• Assign masking algorithms to
match data
• Report risk profile across the
enterprise
Profile Secure Audit
Name
SSN#
Phone#
CC#
Name
SSN#
Phone#
CC#
23 © 2016 Delphix Corporation
Data-centric security and reduced risk
Secure, virtual data sets:
Reduce privileged user risk by automatically masking data prior to access
Reduce surface area of risk by eliminating admin touch points
Secure data record from vandalism with independent, granular time machine
18 hours
Mask Data
6 hours
Standalone
Masking
10 min
Mask Data
4 hours
Deliver, Provision Masked Data
Provision masked data in minutes
Realistic, irreversible, repeatable
Maintains referential integrity
24 © 2016 Delphix Corporation
Resolve the Data Bottleneck for Continuous Integration
Implement DevOps capability – rapid spin-up / tear down with full datasets
Self Service Automation Integration
25 © 2016 Delphix Corporation
A
5%
subset
A
20%
subset
Production Data Manipulation Test Data Factory Developer Self-Service
Mask
Subset
Add
Synthetic
Dev QA Test
A
A
A
A
A
A
A
10% subset,
masked
A
Full
copy
A
Prod +
synth
A
Prod +
synth
A
Masked
copy
Collect, bookmark,
& store
Bookmark, reset,
branch, etc.
Collect, compress,
& synchronize
Provision, mask,
& replicate
A
Full
copy
A
Prod +
synth
A
10%
subset
Data Delivery Mechanism
1/10th
1/10th
1/10th
Storage-Efficient Time Machine Integrated Masking Self-Service
A Test Data Management Factory
Int Stage Prod
Dev Dev
Dev Sandbox
‘Break Glass’
Out of
Process
Change Team
A Dev
Team
B Dev QA
Used to take days, now its so easy!!!
But now, people are starting to make changes… And we need to mange them and push them forward!
Int Stage Prod
Dev Dev
Dev Sandbox
‘Break Glass’
Out of
Process
Change
…
Team
A Dev
Team
B Dev QA
We have undocumented changes…
And conflicting changes
We need to merge everything… and deal with conflicts…
We MUST Identify that drift… as scripts that worked, can no longer be trusted
or else…
And be able to easily merge everything where it belongs…
▪ We need tools and processes to support that, but…
− Challenging manual source control process
− Static deployments code overrides and configuration drift
− Dynamic deployments tools unaware of version control
− No release automation red-flags – don’t know when to stop the line…
File Version Control Process
DB Development Process
Check-Out Script
Modify Script
Get updated Script from
DB
Check-In Script
Compile Script in DB
Debug Script in DB
?
?
?
?
A
A’
72% Admit database may not be in sync with the source repository
X Int QA Stage Prod
Dev
Dev
Dev
Model
‘Break Glass’
Out of Process
Change
X
X
X
X
X
X
…
Configuration drift…
60% Of those manually building scripts have to fix or tweak them regularly as part of a deployment process
Safe to automate?
70% of those using compare & sync tools have to review and fix the results as they can't always trust them to automatically deploy correctly
‘ ’
Revision history
Actions
Standard IDE
Change management
Int QA Stage Prod
Dev Dev
Dev Model
‘Break Glass’
Out of Process
Change
Validate
Configuration Drift prevention / conflict identification and
Validated execution
OR
Validate
Validate
Dynamic in nature Static in nature
Dev - Baseline aware analysis Ops – Validated execution
- Understand the nature of the changes
- Raise red flags on conflicts
- Support out-of-process changes
- Utilize baseline aware analysis
–
…
Track all changes Alert security monitors Create an audit log for compliance reports
Define a process Automate the process Prevent/Alert out of process changes
Int Stage Prod
Dev Dev Dev
Sandbo
x
‘Break Glass’
Out of
Process
Change
…
Team
A Dev
Team
B
Dev
QA
We have undocumented changes…
And conflicting changes
We need to merge everything… and deal with conflicts…
We MUST Identify that drift… as scripts that worked, can no longer be trusted
or else…
And be able to easily merge everything where it belongs…
Check out/in changes
Merge changes
Identify changes
vs baseline
Identify drift
Merge back changes /
Get latest version
Validate deployment
Unique: Database Enforced Source Control
Unique: Enhanced Security and Regulatory Compliance
Unique: Database Merge and Build Automation
Database Release Automation
Delphix & DBmaestro
LLGR2
SQL
/app_config
JUL 1,
13:31:51
JUL 1,
09:00:00
JUL 1,
13:28:29
JUL 1,
11:30:12
Synchronize data
delivery across
sources
Enable short
Feedback Loops
JUL 1,
15:11:27
Baseline Label 1.1 Label 1.2 VALIDATE
UNLOCK INNOVATION
Questions…