Golden gate11g overview - Edgars Rungis

1 Copyright © 2011, Oracle and/or its affiliates. All rights

reserved.

Oracle GoldenGate 11g Overview

Edgars Ruņģis

Technology Consultant

Oracle in Latvia


reserved.

Do You ...

Growing data volumes and shrinking batch

windows for ETL?

Need fresh data 24X7 to optimize operations?

Mission-critical OLTP with performance

impact?

Business Intelligence/ DWH

Want to get onto more efficient and effective

standard infrastructure?

Can’t afford downtime?

Why not fix data assets during an

infrastructure move?

Migration Data-based Application Integration Integrating today via manual code?

Looking ahead to SOA but concerned about

the data?

Need to handle large files and batch

processes?

Support global operations with real-time

data?

High Availability/Disaster Recovery

Strict SLAs for high availability and

performance?

Need single technology for heterogeneous

systems?

Leverage standby databases for transaction

processing?


reserved.

Agenda

• Introduction to Oracle GoldenGate

• Technology Overview

• Use Cases and Benefits

• Summary

• Q&A


reserved.

What is Oracle GoldenGate ?

Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of database transactions across heterogeneous environments in real time

Database

Distributed Systems

Replica/ Standby

Log-based, changed data

Data Warehouse


reserved.

Performance

Flexible & Extendable

Reliable

• Sub-second latency for data movement

• Low-impact capture, routing, transformation, and

delivery of transactional data

• Open, modular architecture

• Supports heterogeneous sources and targets

• Agile and adaptable to modern data integration architectures

• Maintains transactional integrity

• Resilient against interruptions and failures

• Fast Recovery after process interruptions.

Oracle GoldenGate Differentiators


reserved.

Agenda




• Summary

• Q&A


reserved.

How Oracle GoldenGate Works

Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s)

Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs.


reserved.


Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s)


Trail: stages and queues data for routing.


reserved.


Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s)



Pump: distributes data for routing to target(s).


reserved.


Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s)




Route: data is compressed, encrypted for routing to target(s).


reserved.


Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s)





Delivery: applies data with transaction integrity, transforming the data as required.


reserved.





Delivery: applies data with transaction integrity, transforming the data as required.

Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s) Bi-directional



reserved.

Unidirectional Query Offloading

Zero-Downtime Migration

Bi-Directional Hot Standby or

Active-Active for HA

Peer-to-Peer Load Balancing

Multi-Master

Broadcast Data Distribution

Integration/Consolidation Data Warehouse

BPM

BAM

CEP

Data Distribution via Messaging

Oracle GoldenGate Differentiator: Flexibility


reserved.

Differentiator: Performance and Scalability GoldenGate Capture

• Log-based change data capture

– High volume

– Low overhead

• Decoupled architecture

– Multiple capture processes may be used to scale, but generally not required

– Possible to split “hot” tables into a separate capture process

• Filtering and compression

– Unnecessary change data is discarded immediately

– Updates and deletes are “compressed” by default


reserved.

Differentiator: Performance and Scalability GoldenGate Delivery


– Multiple delivery processes may be used to scale

– Possible to split “hot” tables into a separate delivery process

– Possible for multiple delivery processes to split the work for a single table

• Transaction grouping

– Small transactions are grouped by default to reduce commit overhead


reserved.

Differentiator: Transaction Integrity

• Transaction boundaries

– Recorded in GoldenGate trail file

– Transactions ordered in commit sequence

– Boundaries adjusted automatically due to record filtering or trail

splits

• Transaction integrity

– Checkpointing and recovery are based on transaction

boundaries

– Original commit sequence is maintained by the delivery process


reserved.

Differentiator: Reliability


– Individual processes can be restarted automatically

– Tolerance to network outages (configurable)

• Recovery

– Recovery ensures that no operations are skipped or duplicated after

failure of any kind

– Recovery of the Capture process is more involved than recovery the

Delivery process

– Although GoldenGate processes are completely decoupled, the

Delivery process must be aware of a recovery performed by the

Capture process or pump


reserved.

Databases O/S and Platforms

Oracle GoldenGate Capture:

Oracle

DB2 for v 9.7

DB2 for v 10 on z/OS

Microsoft SQL Server for 2008 R1, R2

Sybase ASE, 15.5

Teradata

Enscribe

SQL/MP

SQL/MX

MySQL

JMS message queues

Oracle GoldenGate Delivery:

All listed above, plus:

TimesTen, IBM System I, Netezza & Greenplum

ETL product

Linux

Sun Solaris

Windows 2000, 2003, XP, 2008

HP NonStop

HP-UX

IBM AIX

IBM z Series

zLinux

18

Oracle GoldenGate 11g Supported Platforms


reserved.

• A high-speed, low impact data

comparison solution

• Identifies and reports data discrepancies

between heterogeneous databases

without interrupting their availability

• Benefits:

• Reduce financial/legal risk exposure

• Speed and simplify IT work in comparing

data sources

• No disruption to business systems

• Improved failover to backup systems

• Confident decision-making and reporting

Oracle GoldenGate Veridata – Data Comparison

without Downtime


reserved.

Oracle Management Pack for OGG Complete Visibility (1 / 2): Director

1. Oracle GoldenGate Director:

Has basic configuration,

management, monitoring, and

alerting. Based on legacy pull

infrastructure.


reserved.

Oracle Management Pack for OGG Complete Visibility (2 / 2): Monitor

2. Oracle GoldenGate Monitor:

Has advanced monitoring,

alerting (SNMP support), lag

graphs, and historical

repository. Based on new

agent infrastructure, which

will be the foundation for all

future integration projects.

Supports OGG 11.1.1.1.1+


reserved.

Agenda




• Summary

• Q&A


reserved.

Oracle GoldenGate 11g use cases Low-Impact Real-Time Data Integration & Transactional Replication

New DB/HW/OS/APP

Fully Active Distributed DB

Reporting Database

Data Warehouse

Global Data Centers

ODS Data Integrator

Zero Downtime Upgrade & Migration

Query Offloading, Disaster Recovery

Data Synchronization across the Enterprise

Real-time BI, Operational Reporting, MDM

Event Driven Architecture, SOA

Highly Available / Disaster Recovery

Log-based, changed data

Database

Message Bus

Legacy

Message Bus


reserved.

Eliminate All Downtime for Continuous Availability

Available – no issues

Planned outage

• Upgrade

• Migration

• Maintenance

Unplanned outage

• System failure

• Data failure


reserved.

• Zero Downtime Upgrades, Migrations and Maintenance

– Migration from non-Oracle databases to Oracle 11gR2 & Exadata

– Upgrade Oracle Database 8i, 9i, 10g to 11gR2

– Upgrade/Migrate Database Server/OS

– Perform Database Maintenance

– Application Upgrade (Siebel CRM and JD Edwards)

• Disaster Recovery for Non-Oracle Databases and Cross- Oracle Database versions

• Continuous Availability via Active-Active Databases

Oracle GoldenGate for Continuous Availability


reserved.

• Standard:

– Database Upgrade Assistant

• Alternatives

– Export/Import

– DataPump

– Transportable Tablespaces

– Streams

– Logical Standby

– Oracle GoldenGate (zero downtime)

What are Oracle Database Upgrade Options ?


reserved.

Eliminate Downtime During Upgrades to Oracle 11g R2

• Zero database

downtime for

upgrades from 8i, 9i,

10g to 11g

• Leverage new

features of Oracle

Database 11g without

impacting business

operations

• Minimize risks with

failback option

Failback Data Flow

Oracle Database 8i/9i/10g

Oracle Database

11g Release 2

Application

Switchover

Compare & Verify

With Oracle GoldenGate

Veridata


reserved.

A Certified App Upgrade Solution for

JD Edwards and Siebel CRM

• Upgrade/migrate/main

tain database,

hardware, OS and/or

application

• Minimize risk with

failback option

• Improve success with

phased user migration

Real-time changed data

Failback Data Flow

Non-Oracle Database /Server

Oracle Database

Sun Server

Siebel / JDE

Lower Version Siebel / JDE

Higher Version


reserved.

Eliminate unplanned downtime with

Active Data Guard & GoldenGate

• Active Data Guard for

disaster recovery and

data protection for:

• Oracle-to-Oracle

• Oracle GoldenGate for

disaster recovery and

data protection for:

• Non-Oracle platforms

• Active-Active

• Cross-OS and Oracle

database version

requirements

Post-Switchover Data Flow

Application

Production Live Standby

Can be used for

reporting &

testing

Switchover

Compare & Verify

With Oracle GoldenGate

Veridata


reserved.

Improve Production System Performance and

Lower Costs

• Off-load queries from

production systems in

Active-Active or

Heterogeneous

configurations

• E.g. different OS,

database version,

database

• E.g. legacy system

query off-load

Real-time changed data

Production OLTP

Oracle Database

Transaction Processing Read-Only Activity


reserved.

Increase ROI on Existing Servers & Synchronize Data

Across Globe

• Utilize secondary

systems for

transactions

• Enable continuous

availability during

unplanned and

planned outages

• Synchronize data

across data centers

around the globe

Source & Target Region A

Source & Target

Region B

Application Application


reserved.

Drivers for Moving to Real-Time BI and Data

Warehouse

• Day+ old data

• Batch data extracts within specified

“off business hours”

• Process interruptions impact data

recoverability

Legacy, Packaged Apps, OLTP Databases

Enterprise Data Warehouse

Traditional Data Warehouse

ETL

Real-Time Data Warehouse

Enterprise Data Warehouse

Transformation

ELT

Legacy, Packaged Apps, OLTP

Real-Time

Continuous Feeds

• Timely, relevant data – better decisions

• No batch windows on OLTP

• Complete recoverability after

interruptions


reserved.

• Solution

• Using OGG for log-based capture of database

transactions from source to minimize overhead

• Load to target with sub-second latency

• Transformation performed on the database

using E-LT in mini-batches

• Fast ETL/E-LT handled by database processing

• Benefits

• No resource / performance impact to OLTP

• Live data available for better decision making

• Get double-duty from database investment by

using it for transformations

• Maximizes availability of source systems and

DW due to smaller batch windows.

Next Generation Data Warehousing: Enabling Real Time BI High Performance E-LT, Real-Time Data Warehousing

Oracle Data

Integrator

EMP DEPT

DIM

FACT

DIM

DIM DIM

ODS Schema DW Schema

Oracle

GoldenGate


reserved.

Real Time vs. Right Time

Capture in real time, consume in right time

IT should not be a constraint for business: Invest in future-ready solutions

Consider technical benefits: cost savings, reliability and performance improvements


reserved.

BPM

BAM

SOA Message Bus

Event Driven Architecture, SOA


reserved.

Agenda




• Summary

• Q&A


reserved.

•Captures once, delivers to many targets for different uses

• Non-invasive, log-based capture

• Moves only committed data, reduces bandwidth needs

Reduced Overhead and TCO

• Subsecond latency even with high data volumes

•Preserves transaction integrity

•Ensures data recoverability

High Performance with Reliability

•Provides decoupled, modular architecture

• Supports heterogeneous sources and targets, and different latency needs

•Coexists and integrates with ELT/ETL and messaging solutions

Flexibility and Ease of Use

Advantages of Oracle GoldenGate 11g


reserved.

GoldenGate is Oracle’s Strategic Replication Solution

• Best-in-class leader in real-time data integration solutions

• Complements existing Oracle products

• Continuous Availability for heterogeneous systems

• Real-time data access for Reporting & BI/EPM

• Zero-downtime migrations / upgrades to Oracle

Database, Hardware, and Applications

• Over 1200 customers including Fortune 500 companies,

representing Financial Services, Communications,

Healthcare, Public Sector, Retail, Utilities industries

• 4 of top 5 largest commercial banks

• 3 of top 3 busiest ATM networks

• 5 of top 5 financial data services companies

• 5 of top 5 telecommunications providers

• 3 of top 5 largest food & drug stores

http://brandsoftheworld.com/brands/0002/6850/Liberty_Mutual.eps

http://images.google.com/imgres?imgurl=http://hrff.jalmc.org/_Images/News/adp.jpg&imgrefurl=http://www.ortop.org/sd.htm&h=671&w=1500&sz=74&hl=en&start=1&um=1&tbnid=lSxuKOYqXblBlM:&tbnh=67&tbnw=150&prev=/images?q=adp&imgsz=xxlarge&svnum=10&um=1&hl=en&rlz=1T4GZHY_enUS230US230&sa=N

http://www.google.com/imgres?imgurl=http://www.androidguys.com/wp-content/uploads/2011/01/sprint-logo.jpg&imgrefurl=http://www.androidguys.com/2011/01/18/sprint-add-10-fee-smart-phones-effective-january-30th/sprint-logo/&h=325&w=600&sz=94&tbnid=4KxUWz0_-gpywM:&tbnh=73&tbnw=135&prev=/search?q=Sprint+logo&tbm=isch&tbo=u&zoom=1&q=Sprint+logo&usg=__fOAuD9MkRB0yPYgzD3mJWRvnMS8=&sa=X&ei=avRqTuS1JqvciALEkoCTDg&ved=0CBQQ9QEwAA

http://www.siriusxm.com/wo/i/logodownload/img/logo.eps

http://www.google.com/imgres?imgurl=http://logok.org/wp-content/uploads/2010/07/dell_logo.jpg&imgrefurl=http://logok.org/dell/&h=480&w=640&sz=33&tbnid=vDCkrslBpqF2fM:&tbnh=103&tbnw=137&prev=/search?q=Dell+logo&tbm=isch&tbo=u&zoom=1&q=Dell+logo&usg=__nk3s3cLpJsfrEW05IseCXCl1_bA=&sa=X&ei=5_RqTt6HCePZiALb0PC-Dg&ved=0CBgQ9QEwAg

http://www.google.com/imgres?imgurl=http://www.credit-card-logos.com/images/visa_credit-card-logos/visa_logo_6.gif&imgrefurl=http://www.credit-card-logos.com/&h=113&w=180&sz=2&tbnid=6RgFXMad8jPZ6M:&tbnh=63&tbnw=101&prev=/search?q=Visa+logo&tbm=isch&tbo=u&zoom=1&q=Visa+logo&usg=__gUcOb1jLcLWi-BDQpHJ5scq4BUo=&sa=X&ei=DfVqTq_PDonXiAL6k9DBDg&ved=0CBUQ9QEwAQ

http://www.google.com/imgres?imgurl=http://upload.wikimedia.org/wikipedia/en/7/76/Mayo-clinic-logo.png&imgrefurl=http://en.wikipedia.org/wiki/File:Mayo-clinic-logo.png&h=300&w=300&sz=19&tbnid=A5QcsnkrPLNbvM:&tbnh=116&tbnw=116&prev=/search?q=mayo+clinic+logo&tbm=isch&tbo=u&zoom=1&q=mayo+clinic+logo&usg=__SrXRqCQTmrVAupL7I6lKwG4Nex0=&sa=X&ei=vfVqToP1BY_RiAKctqWhDg&ved=0CBoQ9QEwAg

http://www.google.com/imgres?imgurl=http://www.kokkada.com/wp-content/images/2009/04/bse-logo.jpg&imgrefurl=http://www.kokkada.com/sensex-bombay-stock-exchange-bse/&h=288&w=432&sz=31&tbnid=NlkoKyI0y0z2CM:&tbnh=84&tbnw=126&prev=/search?q=bombay+stock+exchange+logo&tbm=isch&tbo=u&zoom=1&q=bombay+stock+exchange+logo&usg=__xDyibMVps8n3-4Qr6pGr0LkVtxc=&sa=X&ei=B_ZqToD9N-nXiAKLjZnGDg&ved=0CBoQ9QEwAw

http://www.google.com/imgres?imgurl=http://news.walgreens.com/images/20007/wag_logo2.gif&imgrefurl=http://news.walgreens.com/photo+archive/logos/&h=58&w=229&sz=2&tbnid=WY8d8gw2DxRGXM:&tbnh=27&tbnw=108&prev=/search?q=walgreens+logo&tbm=isch&tbo=u&zoom=1&q=walgreens+logo&usg=__YxNNh5bwFk3VPCwMW13a0WwyXyM=&sa=X&ei=XfZqTobII9LWiALdjcWmDg&ved=0CCoQ9QEwBA

http://www.google.com/imgres?imgurl=http://www.meddutch.com/sites/default/files/pictures/faa.gif&imgrefurl=http://www.meddutch.com/?q=content/faa-medicals&h=998&w=967&sz=312&tbnid=uukb5SmuS8jCaM:&tbnh=149&tbnw=144&prev=/search?q=FAA+logo&tbm=isch&tbo=u&zoom=1&q=FAA+logo&usg=__q7RPSPNK3OV4Y5Y-dOMkBySBIgw=&sa=X&ei=t_ZqTvvSIKfYiAL52-DTDg&ved=0CBUQ9QEwAQ

http://www.google.com/imgres?imgurl=http://www.yourlogoresources.com/wp-content/uploads/2011/08/walmart-logo.png&imgrefurl=http://www.yourlogoresources.com/walmart-logo/&h=787&w=2846&sz=70&tbnid=38GNj6G-fd-PLM:&tbnh=41&tbnw=150&prev=/search?q=Walmart+logo&tbm=isch&tbo=u&zoom=1&q=Walmart+logo&usg=__XIZZd_QfnsMe8X81zdMNviqsJGo=&sa=X&ei=pfdqTtv_CMniiALb-Ki-Dg&ved=0CBcQ9QEwAg


reserved.

Q&A


reserved.


reserved.