Three Sides of the Coin with Oracle Data Integration 12c - ODI, OGG, EDQ

Oracle Business Intelligence

Three Sides of the Coin with Oracle Data Integration – ODI, OGG, EDQMaria Gyurova, UKOUG, 8 Dec 20015,

• BI Specialist at Peak Indicators

• Oracle Consultant with ETL and DWH experience

• OCP DBA 9i, 10g, 11g

• Speaking at BGOUG, SIOUG, UKOUG

• Linkedin https://www.linkedin.com/in/mariagyurova

• Twitter @mgyurova

• Personal Blog https://oraclespot.wordpress.com/

• Salsa dancer having a tall husband and a lovely daughter

About me

© Peak Indicators Limited 2Three Sides of the Coin with Oracle Data Integration

• We are the largest Oracle BI specialist partner in Europe

• Our sole focus is on delivering value to our customers

• We are the only Specialized UK Oracle Partner that

currently offers and delivers hosted “private clouds” and

Managed Services for Oracle BI

About Peak IndicatorsIntroduction

© Peak Indicators Limited 3

• We serve some of the world's largest companies, including the UK's top Finance, Retail,

Consumer Goods, Public Sector and Telecoms organisations

Three Sides of the Coin with Oracle Data Integration

The purpose of this session is to show the integration of Oracle Data Integrator(ODI) 12c with Oracle

Golden Gate(OGG) 12c and Oracle Enterprise Data Quality(EDQ) 12c:

• ODI 12c and Real-Time Data Integration

• OGG 12c

• OGG Configuration

• Change Data Capture(CDC)

• EDQ 12c

• EDQ Configuration

• Use EDQ within ODI

• Live Demo

AgendaOverview

© Peak Indicators Limited 4Three Sides of the Coin with Oracle Data Integration

• Use it only if required

• 10 Rules for Real-Time Data Integration at

http://www.tdwi.org

• RT DI with OGG

• Support for CDC

• ODI for Real-Time Data Warehousing

• Benefits

• Move only changed data

• Continuous feeds from operational systems

• Eliminate performance impact on source systems

• Enable near real-time decision making with

real-time data flows

ODI 12cReal-Time Data Integration(RT DI)

© Peak Indicators Limited 5Three Sides of the Coin with Oracle Data Integration

Oracle Data Integration Solution

Oracle Data Integrator

ELT/ETL

Data Transformation

Oracle GoldenGate

Real-time CDC

Data Replication

Enterprise Data Quality

Data Cleansing

Data Profiling

Real-Time Data Integration

Data Source(s)

Operational BI /

Real-Time Reporting

Staging Area for

Operational BI

Real-Time

Data Warehouse

Periodical

ODI Load

GoldenGate

• Real-Time Data Integration

• Heterogeneous Database Replication Platform

• Can be used to support requirements below

OGG 12cIntroduction

© Peak Indicators Limited 6Three Sides of the Coin with Oracle Data Integration

Logical OGG Architecture

UNIDIRECTIONAL

Reporting Instance

BI-DIRECTIONAL

Instant Failover Active-Active

PEER-TO-PEER

High Availability

BROADCAST

Data Distribution

CONSOLIDATIONData Warehouse/Mart

CASCADINGScalability

Extract

Extract

Data source for

Initial Load

Data source for

Change synchronization

Replicat

Replicat

Initial Load

Change Synchronization

Manager Manager

Network

Data

Pump

Collector

Target DB

Trail

• Install OGG on the source and target to capture the source data and deliver to the target

• Oracle database configuration

• alter database add supplemental log data;

• alter system switch logfile;

• alter database force logging;

• enable archivelog mode of the database

• ODI integrates with OGG using GoldenGate JAgent – in Topology Navigator defined as a Data

Server

OGG Configuration

© Peak Indicators Limited 7Three Sides of the Coin with Oracle Data Integration

• Allows ODI to track changes in the source data

• Reduces the source data flow to only changed data and avoid processing unchanged data

• Oracle GoldenGate KMs enable ODI to leverage the power of OGG for real-time CDC

• Journalizing modes:

• Simple Journalizing – Track changes in individual datastores

• Consistent Set Journalizing – Track changes to a group of datastores taking into account the

referential integrity between the datastores

CDC

© Peak Indicators Limited 8Three Sides of the Coin with Oracle Data Integration

• Demo will show how to use CDC in Consistent Set Journalizing

CDC

© Peak Indicators Limited 9Three Sides of the Coin with Oracle Data Integration

• Capture and Delivery processes

• Remote trail files path of the Capture should be the Target trail file path

CDC

© Peak Indicators Limited 10Three Sides of the Coin with Oracle Data Integration

CDC

© Peak Indicators Limited 11Three Sides of the Coin with Oracle Data Integration

Set up Journalizing

1 2 3

• Journals: Where changes are recorded

• Capture processes: Journalizing captures the changes in the source datastores

• Subscribers: Processes that use the tracked changes. They subscribe to a model's CDC to have

the changes tracked for them. Changes are captured only if there is at least one subscriber to the

changes.

• Journalizing views: Provide access to the captured changes; used by integration processes to

retrieve the changed data

CDC

© Peak Indicators Limited 12Three Sides of the Coin with Oracle Data Integration

Journalizing Components

• Extend Window: The Consistency Window is a range of available table changes. The extend window

operation computes this window to take into account new changes captured since the latest Extend

• Lock Subscribers: Subscribers consume the changes separately. This operation performs a subscriber(s)

specific "snapshot" of the changes

• Mappings: Create mappings to process the changed data

• Unlock Subscribers: Commits the use of the changes

• Purge Journal: After all subscribers have consumed the changes all the entries are deleted

CDC

© Peak Indicators Limited 13Three Sides of the Coin with Oracle Data Integration

Use Changed Data

CDC

© Peak Indicators Limited 14Three Sides of the Coin with Oracle Data Integration

Use Changed Data

1

3 CDC Mappings2

45

or automate in a package

• Data quality management environment

• Facilitates MDM, Data Governance, Data Integration, BI

EDQ 12c

© Peak Indicators Limited 15Three Sides of the Coin with Oracle Data Integration

understand,

improve,

protect data

govern data

govern data

improve,

protect data

govern data

Introduction

• Discards records containing ‘*’ in the product name

EDQ 12c

© Peak Indicators Limited 16Three Sides of the Coin with Oracle Data Integration

EDQ Processes and Jobs

EDQ 12c

© Peak Indicators Limited 17Three Sides of the Coin with Oracle Data Integration

EDQ Processes and Jobs

• EDQ augments ODI’s Data Quality capabilities

• Design Data Quality processes in EDQ and invoke them from a package in ODI

EDQ 12c

© Peak Indicators Limited 18Three Sides of the Coin with Oracle Data Integration

Use EDQ within ODI

Gold

Partners

Specialized

Oracle Business Intelligence

Foundation

Gold

Partners

Specialized

Oracle Data Integration 11g

www.peakindicators.com

enquiries@peakindicators.com

+44 (0) 124 638 900019

ODI, OGG and EDQ IntegrationUse CDC, EDQ jobs from ODI

• How to use

• ODI with OGG

• CDC

• ODI with EDQ

• OGG enables ODI to leverage the power of:

• Real-Time Data Integration

• Real-Time Data Warehouse

• Real-Time Reporting

• EDQ leverages ODI’s Data Quality capabilities

• incorporate Data Quality within ETL process

Summary

© Peak Indicators Limited 20Three Sides of the Coin with Oracle Data Integration

OGG ODI EDQ

Source

Q & A

© Peak Indicators Limited 21Three Sides of the Coin with Oracle Data Integration

Helping your Business Intelligence Journey

Gold

Partners

Specialized

Oracle Business Intelligence

Foundation

Gold

Partners

Specialized

Oracle Data Integration 11g

www.peakindicators.com

enquiries@peakindicators.com

+44 (0) 124 638 9000