Introduction to Oracle GoldenGate 12c and Oracle Data Integrator 12c
Bryan Ko
Senior Sales Consultant
Data Integration Solutions
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2
Data Integration Trends Data Integration Tools Today Need to Do More
Big Data is the new normal
Real-time data jumps to the forefront
Application data now lives in the Cloud and
in a multitude of disparate environments
Conventional ETL tools are struggling to
meet performance demands
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3
New Requirements for Data Integration
Integrated and
High Productivity
Tooling
High Performance,
High Availability Cloud Ready
Any Data Latency,
Real-time
Any Data,
Any Source
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Oracle Data Integration 12c Delivering Real-Time Integration for Big Data and Cloud
Future Ready
Extreme Performance
Fast Time to Value
Oracle Data Integrator
Oracle GoldenGate
Oracle Enterprise
Data Quality
Oracle Data
Service Integrator Big Data
Database
Apps
Cloud
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
New Features in Data Integrator 12c
Improved developer productivity through
declarative flow based mapping
Extreme performance through parallelism
and low session overheads
Enterprise integration through big data and
business analytics capabilities
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
Enhanced Experience for Improved Productivity Flow Based Modeling
Declarative, flow based mapping to easily
model complex transformation logic combined
with technology specific out of the box
knowledge modules
Component re-usability to capture shared
logic throughout mappings (sources, targets
and transformations)
Step–by–step debugger with complete user
control / access to uncommitted data, and
defining break points
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
Performance Optimizations
In-session parallelism
– Parallel task execution
– Narrower execution windows
Low overhead session during execution
– When executing repeated scenarios, restarting
sessions
– Efficient micro-batch loading for real time data
– Reduced logging, better cache utilization reduce
network traffic for entire IT system
Faster Loads for Improved Run-time Caching
Total Execution Time
TRADITIONAL
ODI 12.1.2
SrcSet1
SrcSet2
SrcSet3
Integration
Integration
Total Execution Time
SrcSet1
SrcSet2
SrcSet3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
Integration Between GoldenGate and ODI 12c
Support real-time and bulk from the
same interface
Deploy GoldenGate processes via ODI Bulk Load Scenario Real Time Scenario
Enable GoldenGate journalizing on
interfaces without restructuring.
Manage GoldenGate Capture/Delivery
instances in ODI Topology
Set common GoldenGate parameters
graphically
Capture Delivery
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
Enterprise Manager Cloud Control Integration
Discovery of ODI components
– Agents: Standalone and JEE
– Repositories
– Source and Target databases
Monitoring
– Availability Monitoring
ODI homepage to quickly assess ODI infrastructure status
Graphical representation of ODI infrastructure
– Performance Monitoring
ODI components performance
Scenarios, Load Plans, Sessions execution statistics
Historical metrics
Overview
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
Enterprise Manager Cloud Control Integration
Performance Monitoring
– SLA Management
Alerts creation
– SQL Drill Down
Navigate from ODI session to Database session statistics (leveraging DB EMGC
Diagnostics and Tuning Packs)
Other Features
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Loading and Transforming Big Data
Access
Transform
Loads
Oracle Data
Integrator
Supports Hadoop standards
Reverse engineer Hadoop
metadata
Check, validate and ensure
data integrity with Hadoop
Load data into HDFS/Hive
Generate HiveQL and execute
in Hadoop
Leverage existing Hadoop
transformations
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
Goal Provide a migration path to OWB users
Integrate OWB and ODI runtime for
phased migration
How OWB repositories can now be managed
in an ODI topology
OWB jobs can be executed and
monitored in ODI
OWB to ODI Migration Utility
Oracle Warehouse Builder Integration Overview
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Key New Features in GoldenGate 12c Advancing Oracle’s Leadership in Data Replication
Intelligent and Integrated Delivery for
higher performance and IT productivity
Optimization for Database 12c
Expanded heterogeneity and solutions for
Oracle Applications
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
Current Approach to High Volume Replication
Customer
defines
split
SQL
SQL
SQL
SQL
Delivery
Trail
Files
Delivery
Delivery
Delivery
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
Lightweight
Streaming
API
Intelligent & Integrated Delivery for Oracle
Unique, next generation, high-speed data apply for Oracle Database
Leverages the Oracle Database parallel apply servers for auto dependency-
aware parallel delivery
Simplifies deployment while doubling performance
Trail
Files Delivery Capture
Oracle Database
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Coordinated Delivery for Non-Oracle
Customer
Defines
Split
SQL
SQL
SQL
Delivery Single replication
process for customer
Orchestrates multiple apply threads
Simplifies configuration and increases performance effortlessly
Trail
Files Capture
Thread
Thread
Thread
GoldenGate coordinates threads to
maintain ordering
Non- Oracle Databases
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
Oracle 12c Container Database
Ora
cle
Go
lde
nG
ate
Optimization for Oracle Database 12c
Oracle & Non-Oracle Database(s)
Ora
cle
Go
lde
nG
ate
Supports traditional non-consolidated
databases and the new multitenant
architecture, Database12c specific
features and data types
Enables enterprises to move to private
cloud without downtime and meet
availability SLAs
Reduces overhead by capturing changed
data at the container level
Enhanced security for credentials and
encryption keys
Integrated with Data Guard and FSFO for
automated failover and high availability
Multitenant
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
Expanded Heterogeneity
Oracle
IBM DB2 LUW
IBM DB2 z Series
IBM DB2 i Series /AS400
Microsoft SQL Server
Sybase ASE
Teradata
C-Tree
Enscribe
SQL/MP
SQL/MX
MySQL
JMS Message Queues
New in Oracle GoldenGate 12c
• MySQL 5.6
• MySQL Cluster 7.1 and 7.2
• Sybase 15.7 -------------------------------------------------------
• DB2 LUW 10.1
• Teradata 14.10
• SQL Server 2012
Platform Specific Upgrades
• SQL Server Compressed Back-Up
• MySQL Clusters (NDB)
Netezza
TimesTen
PostgreSQL
GreenPlum
HP Neoview
Flat File
ETL products
Coherence
Capture & Delivery Delivery Only
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Integration with Oracle Coherence
Oracle TopLink
Tight Integration with Oracle Coherence
enables real-time updates to Coherence
cache
Refreshes invalidated object in the
Coherence cache when the database is
directly modified
Coherence users can access real-time
data without any changes required to the
source system
Oracle Coherence Grid Edition 12.1.2
Oracle & Non-Oracle Database(s)
Capture C
oh
ere
nce
Ad
ap
ter
Trail
Files
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20
Source
E-Business
Database
Target
E-Business
Database
LAN / WAN / Internet Over TCP/IP
Oracle to Oracle Initial Data Load
Real-Time Change Data Feeds via GoldenGate
New Certified Oracle Application Solutions
Low-downtime Oracle E-Business Suite database migration
ATG Web Commerce active-active data center synchronization
Trail
Files Delivery Capture Trail
Files
Database Switchover
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
Emerging Capabilities in Oracle GoldenGate Management, Monitoring, Adapters
Advanced Monitoring
Veridata: Heterogeneous database
comparison and repair
Integration with Hadoop
Cloud friendly replication
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Oracle preferred monitoring
tool, covers entire Oracle stack
– database, storage, FMW etc
Extends command and control
functions; start and stopping of
processes
Enhanced Functionality for Enterprise Manager Plug-in
Management Pack for Oracle GoldenGate What’s Coming
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Oracle GoldenGate Veridata What’s Coming
Compare Compare
Repair
Oracle Weblogic Server
integration
Compare Group XML Import
Streamlined Navigation
Repair Out of Sync
Data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24
GoldenGate for Integration with Hadoop
Stream transactional data into
Hadoop without source impact
and in real time
Implement custom adapter or
handler for the big data system
using Oracle GoldenGate's Java
Adapter
Full control of data delivery to
any Java exposed Big Data
technology
Oracle &
Non-Oracle
Database(s)
Custom Adapter
Capture Trail
Files Pump
Pump
Parameter
File
Adapter
Properties
File
Enabling Coexistence of Relational Data with Big Data Systems
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25
Enterprise-Class Data Integration with Oracle Multiple Use-Cases for Heterogeneous Systems
100% Faster Speed into Enterprise Data
Warehouse on Oracle Exadata
Real-Time Data from Heterogeneous Sources to ODS
Supporting Application Services
Data Warehousing for Operations from
Restocking to Marketing
Real-Time Data Warehousing and Hot Standby for Disaster
Recovery
Operational Reporting for E-Business Suite
and Data Services for SOA
Data integration from Heterogeneous Sources
for SOA and BAM
Real-Time Data Feed from SQL
Server Sources to Teradata DW for
Finance & Marketing analysis
Real-Time Data Warehousing, Siebel CRM Upgrades, Zero Downtime Database
Migrations
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26
Real-time data integration from 14
gaming and promotional management
systems on SQL Server in 8 properties
Delivers to Teradata DW for fast
analysis of financial data and customer
promotions for improved customer
experience
Reduced operational costs due to ease
of deployment and management
Real-Time Visibility for Customer Service
Ameristar Casinos Real-Time Heterogeneous Data Replication for Data Warehousing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27
Real-time data for revenue, central
reservations, and rate plan reports running
on Oracle Exadata
100% improvement in data loading speed,
288X reduction in latency, down to 5 min.
Access to up-to-date data globally
Uses timely data for increasing efficiency
and improving customer service via
visibility into room occupancy
Timely Insight , Globally
StarWood Hotels & Resorts Near Real-Time Reporting on Oracle Exadata
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28
Oracle GoldenGate is the best way to
migrate data to Exadata and Exalytics, for
zero downtime
Oracle Data Integrator transformations run
natively and use Exadata optimized SQL
Oracle Enterprise Data Quality leverages
both Exalogic and Exadata extreme de-
duplication performance
Simplifies operations for big data and
analytics, while maximizing performance
Data Integration on Engineered Systems
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29
FASTER TIME TO VALUE
FUTURE READY
EXTREME PERFORMANCE
Use B&C Final
image
Summary
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30
Join the Data Integration Community
Twitter twitter.com/ORCLGoldenGate
Facebook facebook.com/OracleDataIntegration
Oracle GoldenGate YouTube channel youtube.com/oraclegoldengate
Oracle.com/goto/dataintegration
LinkedIn “Oracle Data Integration”
Oracle’s Data Integration blog blogs.oracle.com/dataintegration
Oracle Technolog
y Network
Oracle’s Public Help Forum http://forums.oracle.com/forums/forum.jspa?forumID=860&start=0
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 32