Oracle Business Intelligence
Three Sides of the Coin with Oracle Data Integration – ODI, OGG, EDQMaria Gyurova, UKOUG, 8 Dec 20015,
• BI Specialist at Peak Indicators
• Oracle Consultant with ETL and DWH experience
• OCP DBA 9i, 10g, 11g
• Speaking at BGOUG, SIOUG, UKOUG
• Linkedin https://www.linkedin.com/in/mariagyurova
• Twitter @mgyurova
• Personal Blog https://oraclespot.wordpress.com/
• Salsa dancer having a tall husband and a lovely daughter
About me
© Peak Indicators Limited 2Three Sides of the Coin with Oracle Data Integration
• We are the largest Oracle BI specialist partner in Europe
• Our sole focus is on delivering value to our customers
• We are the only Specialized UK Oracle Partner that
currently offers and delivers hosted “private clouds” and
Managed Services for Oracle BI
About Peak IndicatorsIntroduction
© Peak Indicators Limited 3
• We serve some of the world's largest companies, including the UK's top Finance, Retail,
Consumer Goods, Public Sector and Telecoms organisations
Three Sides of the Coin with Oracle Data Integration
The purpose of this session is to show the integration of Oracle Data Integrator(ODI) 12c with Oracle
Golden Gate(OGG) 12c and Oracle Enterprise Data Quality(EDQ) 12c:
• ODI 12c and Real-Time Data Integration
• OGG 12c
• OGG Configuration
• Change Data Capture(CDC)
• EDQ 12c
• EDQ Configuration
• Use EDQ within ODI
• Live Demo
AgendaOverview
© Peak Indicators Limited 4Three Sides of the Coin with Oracle Data Integration
• Use it only if required
• 10 Rules for Real-Time Data Integration at
http://www.tdwi.org
• RT DI with OGG
• Support for CDC
• ODI for Real-Time Data Warehousing
• Benefits
• Move only changed data
• Continuous feeds from operational systems
• Eliminate performance impact on source systems
• Enable near real-time decision making with
real-time data flows
ODI 12cReal-Time Data Integration(RT DI)
© Peak Indicators Limited 5Three Sides of the Coin with Oracle Data Integration
Oracle Data Integration Solution
Oracle Data Integrator
ELT/ETL
Data Transformation
Oracle GoldenGate
Real-time CDC
Data Replication
Enterprise Data Quality
Data Cleansing
Data Profiling
Real-Time Data Integration
Data Source(s)
Operational BI /
Real-Time Reporting
Staging Area for
Operational BI
Real-Time
Data Warehouse
Periodical
ODI Load
GoldenGate
• Real-Time Data Integration
• Heterogeneous Database Replication Platform
• Can be used to support requirements below
OGG 12cIntroduction
© Peak Indicators Limited 6Three Sides of the Coin with Oracle Data Integration
Logical OGG Architecture
UNIDIRECTIONAL
Reporting Instance
BI-DIRECTIONAL
Instant Failover Active-Active
PEER-TO-PEER
High Availability
BROADCAST
Data Distribution
CONSOLIDATIONData Warehouse/Mart
CASCADINGScalability
Extract
Extract
Data source for
Initial Load
Data source for
Change synchronization
Replicat
Replicat
Initial Load
Change Synchronization
Manager Manager
Network
Data
Pump
Collector
Target DB
Trail
• Install OGG on the source and target to capture the source data and deliver to the target
• Oracle database configuration
• alter database add supplemental log data;
• alter system switch logfile;
• alter database force logging;
• enable archivelog mode of the database
• ODI integrates with OGG using GoldenGate JAgent – in Topology Navigator defined as a Data
Server
OGG Configuration
© Peak Indicators Limited 7Three Sides of the Coin with Oracle Data Integration
• Allows ODI to track changes in the source data
• Reduces the source data flow to only changed data and avoid processing unchanged data
• Oracle GoldenGate KMs enable ODI to leverage the power of OGG for real-time CDC
• Journalizing modes:
• Simple Journalizing – Track changes in individual datastores
• Consistent Set Journalizing – Track changes to a group of datastores taking into account the
referential integrity between the datastores
CDC
© Peak Indicators Limited 8Three Sides of the Coin with Oracle Data Integration
• Demo will show how to use CDC in Consistent Set Journalizing
CDC
© Peak Indicators Limited 9Three Sides of the Coin with Oracle Data Integration
• Capture and Delivery processes
• Remote trail files path of the Capture should be the Target trail file path
CDC
© Peak Indicators Limited 10Three Sides of the Coin with Oracle Data Integration
CDC
© Peak Indicators Limited 11Three Sides of the Coin with Oracle Data Integration
Set up Journalizing
1 2 3
• Journals: Where changes are recorded
• Capture processes: Journalizing captures the changes in the source datastores
• Subscribers: Processes that use the tracked changes. They subscribe to a model's CDC to have
the changes tracked for them. Changes are captured only if there is at least one subscriber to the
changes.
• Journalizing views: Provide access to the captured changes; used by integration processes to
retrieve the changed data
CDC
© Peak Indicators Limited 12Three Sides of the Coin with Oracle Data Integration
Journalizing Components
• Extend Window: The Consistency Window is a range of available table changes. The extend window
operation computes this window to take into account new changes captured since the latest Extend
• Lock Subscribers: Subscribers consume the changes separately. This operation performs a subscriber(s)
specific "snapshot" of the changes
• Mappings: Create mappings to process the changed data
• Unlock Subscribers: Commits the use of the changes
• Purge Journal: After all subscribers have consumed the changes all the entries are deleted
CDC
© Peak Indicators Limited 13Three Sides of the Coin with Oracle Data Integration
Use Changed Data
CDC
© Peak Indicators Limited 14Three Sides of the Coin with Oracle Data Integration
Use Changed Data
1
3 CDC Mappings2
45
or automate in a package
• Data quality management environment
• Facilitates MDM, Data Governance, Data Integration, BI
EDQ 12c
© Peak Indicators Limited 15Three Sides of the Coin with Oracle Data Integration
understand,
improve,
protect data
govern data
govern data
improve,
protect data
govern data
Introduction
• Discards records containing ‘*’ in the product name
EDQ 12c
© Peak Indicators Limited 16Three Sides of the Coin with Oracle Data Integration
EDQ Processes and Jobs
EDQ 12c
© Peak Indicators Limited 17Three Sides of the Coin with Oracle Data Integration
EDQ Processes and Jobs
• EDQ augments ODI’s Data Quality capabilities
• Design Data Quality processes in EDQ and invoke them from a package in ODI
EDQ 12c
© Peak Indicators Limited 18Three Sides of the Coin with Oracle Data Integration
Use EDQ within ODI
Gold
Partners
Specialized
Oracle Business Intelligence
Foundation
Gold
Partners
Specialized
Oracle Data Integration 11g
www.peakindicators.com
+44 (0) 124 638 900019
ODI, OGG and EDQ IntegrationUse CDC, EDQ jobs from ODI
• How to use
• ODI with OGG
• CDC
• ODI with EDQ
• OGG enables ODI to leverage the power of:
• Real-Time Data Integration
• Real-Time Data Warehouse
• Real-Time Reporting
• EDQ leverages ODI’s Data Quality capabilities
• incorporate Data Quality within ETL process
Summary
© Peak Indicators Limited 20Three Sides of the Coin with Oracle Data Integration
OGG ODI EDQ
Source
Q & A
© Peak Indicators Limited 21Three Sides of the Coin with Oracle Data Integration
Helping your Business Intelligence Journey
Gold
Partners
Specialized
Oracle Business Intelligence
Foundation
Gold
Partners
Specialized
Oracle Data Integration 11g
www.peakindicators.com
+44 (0) 124 638 9000