Wechsel von Oracle Streams Practice parameters (These are not based on Streams configuration) ... Unidirectional – one direction Bi-Directional

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Wechsel von Oracle Streams

nach Oracle GoldenGate 12c

Joachim Jaensch Principal Sales Consultant


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3


Agenda

2

3

4

Oracle Streams versus Oracle GoldenGate

4

1

Streams to GoldenGate Migration Utility

Migration Example (One direction)

Summary, Resources & Contacts

Generated Config Files

Modified OBEY Files

Transition (Starting, Stopping, Error Situations)

5 Q&A

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 5

Oracle Replication Solutions History

Oracle Advanced Replication

Oracle Streams (OS)

Oracle GoldenGate (OGG) Strategic Oracle Replication Solution

Deprecated with Oracle 12c

Deprecated with Oracle 12c

Oracle Change Data Capture (CDC) Desupported with Oracle 12c


Oracle Streams Processes

Oracle Source Database

Redologs

Databases (DBs) connected by Database Link(s)

Multiple Links possible

Flow Control between Capture and Apply

Capture, Propagation, Apply inside into the DBs

Multiple Capture, Propagation, Apply possible

Replication via Logical Change Records (LCRs)

Uni- and Bi-Directional

Oracle Target Database

Capture

Queue

------ LCRs

Queue

----- LCRs

Apply

Propagation

LAN / WAN / Internet

Over TCP/IP

Flow Control Flow Control

Sender Receiver


Capture Trail

Files

Source Database Target Database

LAN / WAN / Internet

Over TCP/IP

Trail

Files Pump Apply

Databases are not connected

Processes de-coupled from each other

Capture, Pump, Apply outside of the DBs*

Multiple Capture, Pump, Apply possible

Replication via Trail Files / Trail Records

Uni- and Bi-Directional

Heterogeneous Solution

Oracle GoldenGate Processes

Redologs Optional but recommended

* Except Integrated Capture and Integrated Apply


Mapping OS processes to OGG processes

Oracle Streams

Oracle GoldenGate



Oracle Database Dictionary

Oracle Streams

Oracle GoldenGate

Apply Propagation Capture




Capture „prm“ File Pump „prm“ File Apply „prm“ File

Oracle Streams

Oracle GoldenGate




Capture Pump Apply Trail

Files Trail

Files


Capture „prm“ File Pump „prm“ File Apply „prm“ File

Oracle Streams

Oracle GoldenGate



Object Type Streams GoldenGate Replicate DB Objects

Rules in DB identify which objects to replicate for each process

TABLE or MAP statements in parameter file identify which objects to replicate for each process

Skip specific objects Rules in negative rule set identify objects to skip

TABLEEXCLUDE, MAPEXCLUDE

DDL Rules for DDL objects DDL statement (extract , replicat)

Transformations Declarative transformations on rules MAP statement identifies transformations

Handlers DML handlers (statement or procedural) error handlers

(Inbound server invokes DML and error handlers)

Conflict detection Always on COMPARECOLS (ON UPDATE ALL, ON DELETE ALL) ALLCOLS configured for TRANDATA and SCHEMATRANDATA

Conflict resolution Update conflict resolution RESOLVECONFLICT - UPDATEROWEXISTS

Mapping of Methods from Streams to GoldenGate


Streams Oracle GoldenGate

Capture

A process that scans the Oracle database redo log to capture

DML and DDL changes made to database objects.

Primary Capture (Primary Extract)

The Oracle GoldenGate program that reads data from the

change logs of a data source (e.g. redo log of the Oracle

database) or the from source tables and write it to trail files.

Propagation

Propagation is mainly used if information is pushed from the

capture queue of a local database to the apply queue at the

destination database.

Routing (Optional but recommended!)

Data is sent via TCP/IP to the target systems.

Data Pump (optional): A Secondary Extract process that

reads from an extract file or trail. The trail is populated by a

Primary Extract process that reads from the data source.

Apply

A process that dequeues messages from a specific queue and

either applies each message directly, discards it, passes it as a

parameter to an apply handler, or re-enqueues it.

Delivery (Replicat, Apply)

The Oracle GoldenGate process that applies data to target

tables or moves it to another application or destination.

Queuing

The abstract storage unit used by a messaging system to store

messages. Normally Streams uses buffered queues which

means that messages are only kept in memory.

Trail Files

A series of files on disk where Oracle GoldenGate stores data

temporarily in preparation for further processing. Oracle

GoldenGate records checkpoints in the trail for online change

synchronization.

Description of Elements


Agenda

2

3

4

Oracle Streams versus Oracle GoldenGate 1





Modified OBEY Files


5 Q&A



“Documentation for Streams to GoldenGate Migration Tool”

1. Oracle Database Package: streams2ogg

2. Documentation (PDF, 1.53 MB)

My Oracle Support Note 1912338.1:

Downloads:

This tool will help:

1. Eliminate manual conversion errors

2. Minimize the GoldenGate learning curve for Streams customers


Streams to GoldenGate Migration Utility Load & Run

Load package as Streams Administrator and answer 2 questions:

Run procedure from Streams Administrator schema using either:

All generated files will be in the staging directory.

exec streams2ogg.main or exec streams2ogg.customize (if performing customizations)

1. @streams2ogg 2. Name of Oracle Streams administrator schema (e.g. STRADMIN) 3. Name of staging directory for storing the generated files (D:/temp/<db_name>)


Streams to GoldenGate - Best Practices: Conversion

“Best Practices: Oracle Streams to Oracle GoldenGate conversion”

Documentation (PDF, 196.19 kB)

My Oracle Support Note 1383303.1:

Download:

This note will help:

1. Transition from Oracle Streams to Oracle GoldenGate

2. Step by step explanation: Stop Oracle Streams Start Oracle GoldenGate


Files generated:

“streams2ogg“ – Generated Files

File Name

Description

ecapname.prm Extract file name based on capture name

pcapname.prm Pump file name based on capture name/propagation name

rappname.prm Replicat file name based on apply name

mgr.prm Manager file name – contains PORT number, other OGG mgr Best Practice parameters (These are not based on Streams configuration)

handlers.sql SQL script to configure Streams handlers after replicat process is registered (does not include handler code)

streams2ogg.trc Debug/Trace file – always generated

streams2ogg.rpt Report file – always generated


File Name

Description

ggconfig.oby GGSCI obey file to create extract, pump, replicat processes

ggconfig2.oby GGSCI obey file to create additional extract processes based on first extract SCN

ggconfig_trandata.oby GGSCI obey file to configure database supplemental logging

create_subdirectories.sh Shell script to generate directory structure under the trail file storage directory

reg_extract.sh Helper shell script for ggconfig2.oby OGG 12.1.2.0 Register extract .... SCN option

watch.sh, watch.sql watch_status.sql

Helper shell and SQL script for ggconfig2.oby for OGG 12.1.2.1 Register extract ... SHARE option watch_status.sql can be used to get status of capture initialization

ogg_name_map.csv Comma-separated name/value pairs for customizing streams2ogg file generation

Files generated:

Streams2OGG – Generated Files (cont.)


Migration from OS to OGG – Step by Step

Action Description Comment

Prepare and Run the Migration Utility:

exec streams2ogg.<proc_name> (Source & target database!)

Analyze your Oracle Streams Environment and decide what you must migrate to OGG. Without parameter modifications: MAIN (one step) With modifications of „csv“ file: CUSTOMIZE (two step)

MOS Note: 1912338.1

Check (and modify) the configuration

files (Source & target database!)

If you have checked the configuration files and all looks well than you are ready to transition from Streams to GoldenGate

Stopping OS Starting OGG

Unidirectional – one direction Bi-Directional – two directions

Check for open transactions and Apply errors Stop OS Apply at target database Find last applied SCN from OS replication at the target database Start OGG Replicate using AFTERCSN parm Stop OS Capture and OS Propagation at source database

MOS Note: 1383303.1


Agenda

2

3

4






Modified OBEY Files


5 Q&A


Assumptions & Requirements (Databases, OGG)

Running Streams Environment At least one replicated transaction

Oracle GoldenGate Install the GoldenGate version 12c (12.1.2.1) software on all Source & Target Systems

“streams2ogg” generates Integrated Capture & Integrated Apply parameter files The Migration Utility generates configuration and parameter files for GoldenGate 12c. For older versions you must modify the generated files after „streams2ogg“ was run.

Oracle Database 11.2.0.4 or higher Integrated Apply not available before

Oracle Database Initialization Parameters ENABLE_GOLDENGATE_REPLICATION = TRUE (11.2.0.4 or 12.1.0.2 +)

STREAMS_POOL_SIZE = (similar to Streams)


Migration Sample – Current Streams Environment

C_CN1_NOCDBP P_PN1_NOCDBP

Apply

A_AN1_NOCDBS

A_AN1_NOCDBP P_PN1_NOCDBS C_CN1_NOCDBS

NOCDBP NOCDBS Primary

Database Secondary Database




Integrated Capture

Trail Files

Pump Integrated Apply

Trail Files



TSTSTR.STR_HEARTBEAT_Table DML & DDL Replication Default TAGs (NULL, ‘00‘) Update Conflict Handler both sides None DML, Apply & Error Handlers

Migration Sample – Only one Direction migrated

A_AN1_NOCDBP P_PN1_NOCDBS C_CN1_NOCDBS



Integrated Apply

Trail Files

Pump

Trail Files

Integrated Capture

Trail Files

Trail Files

Pump

Migration Sample – Both directions migrated



TSTSTR.STR_HEARTBEAT_Table DML & DDL Replication Default TAGs (NULL, ‘00‘) Update Conflict Handler both sides None DML, Apply & Error Handlers

Integrated Capture

Integrated Apply


Understand „streams2ogg“ and Migration Behaviour

1. “streams2ogg” works for one database

Run the program for all source and target databases. In this sample for primary and secondary database

2. Migration from Streams to GoldenGate must be done for all replication directions

Migrate step by step one replication direction Begin with replication from primary to secondary database and continue with replication direction from secondary to primary database.

Streams2OGG NOCDBP

Migration Direction 1

Migration Direction 2

Streams2OGG NOCDBS


Migration – Direction 1: NOCDBP NOCDBS

Streams2OGG Load & Run the Utility on both Databases:

Streams2OGG NOCDBP

Streams2OGG NOCDBS




Streams2OGG NOCDBP

Streams2OGG NOCDBS

Generated Config Files Check & Modify the generated Config Files:

Parameter & OBEY Files NOCDBP

Parameter & OBEY Files NOCDBS

ggconfig.oby MGR.prm E1NOCDBP.prm P1NOCDBP.prm

ggconfig.oby MGR.prm R_NOCDBS.prm




Streams2OGG NOCDBP

Streams2OGG NOCDBS

NOCDBP NOCDBS

Migrate Capture, Propagation, Apply Migration Direction: NOCDBP to NOCDBS:

Streams









Streams2OGG NOCDBP

Streams2OGG NOCDBS

NOCDBP NOCDBS

Migrate Capture, Propagation, Apply Migration Direction: NOCDBP to NOCDBS: GoldenGate







Install the Database Package „streams2ogg“ SQL> connect /@nocdbp as sysdba

Connect durchgeführt.

NLS Date Format:

----------------

DD.MM.RR

GLOBAL_NAME

--------------------

NOCDBP.DE.ORACLE.COM

SQL> @streams2ogg

...

...

...

3233 END streams2ogg;

3234 /

Geben Sie einen Wert für gg_username ein: stradmin

Geben Sie einen Wert für staging_dir ein: d:\temp\nocdbp

Package Body wurde erstellt.

SQL> show errors

Keine Fehler.

Install this package in both databases:

Primary Database: NOCDBP

Username: stradmin Directory Path: d:\temp\nocdbp

Secondary Database: NOCDBS

Username: stradmin Directory Path: d:\temp\nocdbs

NOCDBP


Run „streams2ogg“ on NOCDBP and NOCDBS

SQL> connect /@nocdbp as sysdba


GLOBAL_NAME

--------------------


SQL> exec streams2ogg.main

PL/SQL-Prozedur erfolgreich abgeschlossen.

SQL> connect /@nocdbs as sysdba


GLOBAL_NAME

--------------------

NOCDBS.DE.ORACLE.COM

SQL> exec streams2ogg.main


SQL>

Execute only „streams2ogg.main“

One Step Execution

- Create mapping file „ogg_name_map.csv“

- Create GoldenGate configuration files


Run „streams2ogg“ on NOCDBP and NOCDBS

SQL> connect /@nocdbp as sysdba


GLOBAL_NAME

--------------------


SQL> exec streams2ogg.customize


SQL> connect /@nocdbs as sysdba


GLOBAL_NAME

--------------------


SQL> exec streams2ogg.customize


SQL>

1. Execute „streams2ogg.customize“

Two Step Execution

- Create mapping file „ogg_name_map.csv“

- Modify mapping file (optional)

2. Execute „streams2ogg.main“

- Create GoldenGate configuration files


Generated Config Files (streams2ogg)


“streams2ogg.main“ – Generated Files: Primary DB

2 Files streams2ogg.rpt - Report File

streams2ogg.trc - Trace File

1 CSV-File ogg_name_map.csv - Mapping Process Names

2 Files create_subdirectories.sh – Trail Files Folder

reg_extract.sh - Capture Beginn

5 Parameter-Files mgr.prm - Manager

e1nocdbp.prm - Capture (Extract) E1NOCDBP

p1nocdbp.prm - Pump P1NOCDBP

r_nocdbp.prm - Apply R_NOCDBP

nocdbp_stradmin.prm - INCLUDE File Login (Edit!)

3 OBEY-Files ggconfig.oby - OBEY-File

ggconfig2.oby - OBEY-File

ggconfig_trandata.oby - OBEY-File

1 Handlers File handlers.sql - empty, no Handlers


“streams2ogg.main“ – Generated Files: Secondary DB




2 Files create_subdirectories.sh – Trail Files Folder

reg_extract.sh - Capture Beginn

5 Parameter-Files mgr.prm - Manager

e1nocdbs.prm - Capture (Extract) E1NOCDBS

p1nocdbs.prm - Pump P1NOCDBS

r_nocdbs.prm - Apply R_NOCDBS

nocdbs_stradmin.prm - INCLUDE File Login (Edit!)

3 OBEY-Files ggconfig.oby - OBEY-File

ggconfig2.oby - OBEY-File

ggconfig_trandata.oby - OBEY-File

1 Handlers File handlers.sql - empty, no Handlers


“streams2ogg.customize“ – Gener. Mapping File only







Primary Database

Secondary Database

You can delete Streams processes before you run the „main“ procedure of „streams2ogg“ package


“streams2ogg“ – Generated Mapping Files (...csv)

-- Generated by streams2ogg (Version 3.0) 05.02.15 15:13:03,985000

C_CN1_NOCDBP,"e1nocdbp"

A_AN1_NOCDBP,"r_nocdbp"

P_PN1_NOCDBP,"p1nocdbp"

STRADMIN.C_QN1_NOCDBP,"aa"

./dirdat,"./dirdat"

MANAGER PORT,"7809"

DYNAMICPORTLIST,"15000-15040"

Conflict Detection,"Y"

OGG VERSION,"12.1.2.1"

DBLOGIN,"BOTH"


C_CN1_NOCDBS,"e1nocdbs"

A_AN1_NOCDBS,"r_nocdbs"

P_PN1_NOCDBS,"p1nocdbs"

STRADMIN.C_QN1_NOCDBS,"aa"

./dirdat,"./dirdat"

MANAGER PORT,"7809"




DBLOGIN,"BOTH"

NOCDBP

NOCDBS

If both OGG manager are running on the same machine you must use different TCP/IP Ports.

Files are identical for streams2ogg.main & streams2ogg.customize


OGG – Parameter Files for Capture & Pump


extract P1NOCDBP

-- propagation P_PN1_NOCDBP

INCLUDE ./dirprm/nocdbp_oggadmin.prm

rmthost <host name>, mgrport 7809, PARAMS -f -B1048576, TCPBUFSIZE 1048576, TCPFLUSHBYTES 1048576

rmttrail ./dirdat/NOCDBS/STRADMIN/A_QN1_NOCDBS/zz

discardfile ./dirrpt/p1nocdbp.dsc, PURGE, MEGABYTES 500

PASSTHRU

--If there are multiple TABLE statements, the first match is considered valid. Place the wildcarded statements at the end.

TABLE "TSTSTR"."STR_HEARTBEAT_TABLE";


extract E1NOCDBP

-- capture C_CN1_NOCDBP


exttrail ./dirdat/NOCDBP/aa

REPORTCOUNT EVERY 5 MINUTES, RATE

WARNLONGTRANS 1H, CHECKINTERVAL 30m

LOGALLSUPCOLS

UPDATERECORDFORMAT COMPACT

DDL INCLUDE OBJNAME "TSTSTR"."STR_HEARTBEAT_TABLE"

DDLOPTIONS REPORT

TRANLOGOPTIONS EXCLUDETAG +

--If there are multiple TABLE statements, the first match is considered valid. Place the wildcarded statements at the

end.



NOCDBP

E1NOCDBP.prm

P1NOCDBP.prm

For GoldenGate: oggadmin user created!

NOCDBP_OGGADMIN.prm looks like NOCDBS_OGGADMIN.prm but have a different TNSNAMES alias

? Why two times ?

? Why DB Login?


OGG – Parameter File for Apply & INCLUDE File


replicat R_NOCDBS

-- apply A_AN1_NOCDBS

GETENV (NLS_LANG)

INCLUDE ./dirprm/nocdbs_oggadmin.prm

ASSUMETARGETDEFS

discardfile ./dirrpt/r_nocdbs.dsc, PURGE, MEGABYTES 500


DBOPTIONS DEFERREFCONST

DBOPTIONS SETTAG 00

DBOPTIONS SUPPRESSTRIGGERS

DBOPTIONS _NO_DISABLE_ON_RETRY_ERROR

--If there are multiple MAP statements, the first match is considered valid. Place the wildcarded statements at the end.


DDLOPTIONS REPORT

DDLOPTIONS NOTAG

DDLOPTIONS UPDATEMETADATA

MAP "TSTSTR"."STR_HEARTBEAT_TABLE", TARGET "TSTSTR"."STR_HEARTBEAT_TABLE",

COMPARECOLS (

ON UPDATE ALL,

ON DELETE ALL),

RESOLVECONFLICT (

UPDATEROWEXISTS, (DEFAULT, OVERWRITE));

NOCDBS

--Generated by streams2ogg (Version 3.0) 05.02.15 15:13:04,554000

-- Uncomment one of the following two

userid oggadmin@NOCDBS, password OGGADMIN

-- useridalias stradmin

R_NOCDBS.prm

NOCDBS_OGGADMIN.prm



Modified OBEY Files (Migration Direction 1)



ALLOWNESTED


dblogin userid oggadmin@NOCDBP, password OGGADMIN

-- dblogin useridalias stradmin

-- Capture (Extract) NOCDBP

register extract e1nocdbp with database

add extract e1nocdbp, integrated tranlog, begin now

add exttrail ./dirdat/NOCDBP/aa, extract e1nocdbp, megabytes 500

-- Pump NOCDBP

add extract p1nocdbp, exttrailsource ./dirdat/NOCDBP/aa

add rmttrail ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz, extract p1nocdbp, megabytes 500

-- Define this Apply later (Reverse Direction)


-- dblogin userid oggadmin@NOCDBP, password OGGADMIN


--

-- Apply NOCDBP

--

-- register replicat r_nocdbp database

-- add replicat r_nocdbp integrated, exttrail ./dirdat/NOCDBP/OGGADMIN/A_QN1_NOCDBP/zz, nodbcheckpoint

-- shell echo Streams2ogg processing is complete. Please review the report file: d:\temp_nocdbp/streams2ogg.rpt NOCDBP

GoldenGate Command File

OGG – OBEY File “ggconfig.oby“ primary Side



OGG – OBEY File “ggconfig.oby“ secondary Side


ALLOWNESTED

-- Define Capture and Pump later (Reverse Direction)


-- dblogin userid stradmin@NOCDBS, password <password>


-- Capture (Extract) NOCDBS

-- register extract e1nocdbs with database

-- add extract e1nocdbs, integrated tranlog, begin now

-- add exttrail ./dirdat/NOCDBS/aa, extract e1nocdbs, megabytes 500

-- Pump NOCDBS

-- add extract p1nocdbs, exttrailsource ./dirdat/NOCDBS/aa

-- add rmttrail ./dirdat/NOCDBP/STRADMIN/A_QN1_NOCDBP/zz, extract p1nocdbs, megabytes 500

-- Apply NOCDBS


dblogin userid oggadmin@NOCDBS, password OGGADMIN


register replicat r_nocdbs database

add replicat r_nocdbs integrated, exttrail ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz, nodbcheckpoint

-- shell echo Streams2ogg processing is complete. Please review the report file: d:\temp_nocdbs/streams2ogg.rpt NOCDBS

GoldenGate Command File



Transition

Stopping Oracle Streams &

Starting Oracle GoldenGate


1. Changes that started and committed before the extract was created are managed by Streams.

2. Changes that started before the extract was created and commited after the extract had been created but before the capture was stopped are replicated by the Streams capture process as well.

3. Because the Oracle GoldenGate replicat process will be configured to start from the time that Streams Capture will stop, the messages that were processed between the

started extract and the stopped capture will be managed by Streams and not by Oracle GoldenGate.

1

2

3

Transition (1)

Start OGG

Extract

Stop Streams Capture


4. Changes that started after the extract was created and before the capture was stopped, but commited after the Streams Capture was stopped are managed by Oracle GoldenGate.

5. Changes that are processing after the started extract and after the stopped capture are managed by GoldenGate.

6. Changes that had started before the Oracle GoldenGate Extract was created and commited after the Streams Capture was stopped are ignored by any process.

4

5

Start OGG

Extract

Stop Streams Capture

6

Transition: (2)


Transition Sequence

1. Install Oracle GoldenGate on source and target

2. Create and start the Oracle GoldenGate Extract & Pump process

3. Create GoldenGate Replicat process

4. Check source DB for open transactions

5. Change COMMIT_SERIALIZATION to FULL

6. Stop and check Streams Apply process and find last applied SCN

7. Start GoldenGate Replicat process using AFTERCSN parameter

8. Stop Streams Capture process and Propagation on source

9. Remove Streams Configuration (See Note 1383303.1, Pages 10+11 before!)


connect stradmin/STRADMIN@nocdbp

set linesize 200

set pagesize 50

col username format a10

col machine format a15

col object_name format a20

col owner format a15

col program format a20

SELECT vs.username,

vs.program,

vs.machine,

vt.xidusn,

vt.xidslot,

vt.xidsqn,

vt.start_time, TO_CHAR(SYSDATE,'YYYY-MM-DD HH24:MI:SS')

FROM gv$transaction vt, gv$session vs

WHERE vt.addr = vs.taddr

ORDER BY start_time;


GLOBAL_NAME

--------------------


Es wurden keine Zeilen ausgewählt

Transition: Check for Long Running Transactions


Transition: OGG Capture from NOCDBP – Report (1) ***********************************************************************

Oracle GoldenGate Capture for Oracle

Version 12.1.2.0.0 17185003 OGGCORE_12.1.2.0.0_PLATFORMS_130924.1316

Windows x64 (optimized), Oracle 12c on Sep 24 2013 19:40:53

Copyright (C) 1995, 2013, Oracle and/or its affiliates. All rights reserved.

Starting at 2015-02-11 15:43:14

***********************************************************************

Operating System Version:

Microsoft Windows 7 , on x64

Version 6.1 (Build 7601: Service Pack 1)

Process id: 4920

Description:

***********************************************************************

** Running with the following parameters **

***********************************************************************

2015-02-11 15:43:14 INFO OGG-03059 Operating system character set identified as windows-1252.

2015-02-11 15:43:14 INFO OGG-02695 ANSI SQL parameter syntax is used for parameter parsing.

extract e1nocdbp


userid oggadmin@NOCDBP, password ********

2015-02-11 15:43:14 INFO OGG-03522 Setting session time zone to source database time zone 'GMT'.




LOGALLSUPCOLS


DDL INCLUDE OBJNAME "TSTSTR"."STR_HEARTBEAT_TABLE“

DDLOPTIONS REPORT

TRANLOGOPTIONS EXCLUDETAG 00


2015-02-11 15:43:14 INFO OGG-01635 BOUNDED RECOVERY: reset to initial or altered checkpoint.

Excluded operations: 0

...

ggsci> start extract E1NOCDBP


...

2015-02-11 15:43:14 INFO OGG-02089 Source redo compatibility version is: 12.1.0.2.

2015-02-11 15:43:16 INFO OGG-02068 Integrated capture successfully attached to logmining server OGG$CAP_E1NOCDBP using OGGCapture API.

2015-02-11 15:43:16 INFO OGG-02086 Integrated Dictionary will be used.

2015-02-11 15:43:16 INFO OGG-01052 No recovery is required for target file ./dirdat/NOCDBP/aa000000, at RBA 0 (file not opened).

2015-02-11 15:43:16 INFO OGG-01478 Output file ./dirdat/NOCDBP/aa is using format RELEASE 12.1.

***********************************************************************

** Run Time Messages **

***********************************************************************


TABLE resolved (entry TSTSTR.STR_HEARTBEAT_TABLE): TABLE "TSTSTR"."STR_HEARTBEAT_TABLE";

Using the following key columns for source table TSTSTR.STR_HEARTBEAT_TABLE: DB_NAME.

2015-02-11 15:43:45 INFO OGG-01021 Command received from GGSCI: STATS.

2015-02-11 15:46:15 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.

14 records processed as of 2015-02-11 15:49:05 (rate 0,delta 0)







2015-02-11 16:23:23 INFO OGG-01021 Command received from GGSCI: STOP.

***********************************************************************

* ** Run Time Statistics ** *

***********************************************************************

Report at 2015-02-11 16:23:23 (activity since 2015-02-11 15:43:20)

Output to ./dirdat/NOCDBP/aa:

From Table TSTSTR.STR_HEARTBEAT_TABLE:

# inserts: 0

# updates: 48

# deletes: 0

# discards: 0

...

Transition: OGG Capture from NOCDBP – Report (2)


Transition: OGG Pump to NOCDBS – Report (1) ***********************************************************************





Starting at 2015-02-11 15:46:51

***********************************************************************




Process id: 5716

Description:

***********************************************************************


***********************************************************************



extract p1nocdbp




rmthost localhost, mgrport 7817, PARAMS -f -B1048576, TCPBUFSIZE 1048576, TCPFLUSHBYTES 1048576

rmttrail ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz

discardfile ./dirrpt/p1nocdbp.dsc, PURGE, MEGABYTES 500

PASSTHRU


2015-02-11 15:46:51 INFO OGG-01851 filecaching started: thread ID: 116121680.

2015-02-11 15:46:51 INFO OGG-01815 Virtual Memory Facilities for: COM

anon alloc: MapViewOfFile anon free: UnmapViewOfFile

file alloc: MapViewOfFile file free: UnmapViewOfFile

target directories:

D:\ogg1212_src\dirtmp.

...

ggsci> start extract P1NOCDBP


Transition: OGG Pump to NOCDBS – Report (2) Database Version:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

PL/SQL Release 12.1.0.2.0 - Production

CORE 12.1.0.2.0 Production

TNS for 64-bit Windows: Version 12.1.0.2.0 - Production

NLSRTL Version 12.1.0.2.0 – Production

Database Language and Character Set:

NLS_LANGUAGE = "AMERICAN"

NLS_TERRITORY = "AMERICA"

NLS_CHARACTERSET = "WE8MSWIN1252"

2015-02-11 15:46:56 INFO OGG-01226 Socket buffer size set to 1048576 (flush size 1048576).

2015-02-11 15:46:56 INFO OGG-01052 No recovery is required for target file ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000000, at RBA 0

(file not opened).

2015-02-11 15:46:56 INFO OGG-01478 Output file ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz is using format RELEASE 12.1.

***********************************************************************


***********************************************************************

Opened trail file ./dirdat/NOCDBP/aa000000 at 2015-02-11 15:46:56

TABLE resolved (entry TSTSTR.STR_HEARTBEAT_TABLE): TABLE "TSTSTR"."STR_HEARTBEAT_TABLE";

PASSTHRU mapping resolved for source table TSTSTR.STR_HEARTBEAT_TABLE




***********************************************************************


***********************************************************************


Output to ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz:

From Table TSTSTR.STR_HEARTBEAT_TABLE:

# inserts: 0

# updates: 48

# deletes: 0

# discards: 0

...


Transition: Stop & Check Apply process for NOCDBS SQL> connect stradmin/STRADMIN@nocdbs


GLOBAL_NAME

--------------------


SQL>

SQL> Begin

2

3 DBMS_APPLY_ADM.STOP_APPLY ('A_AN1_NOCDBS');

4

5 End;

6 /


SQL>

SQL> COLUMN APPLY_NAME HEADING 'Apply Process Name' FORMAT A20

SQL>

SQL> SELECT APPLY_NAME,

2 STATUS,

3 MAX_APPLIED_MESSAGE_NUMBER

4 FROM DBA_APPLY

5 where apply_name = 'A_AN1_NOCDBS';

Apply Process Name STATUS MAX_APPLIED_MESSAGE_NUMBER

-------------------- -------- --------------------------

A_AN1_NOCDBS DISABLED 2931340

SQL>

SQL> col source_database format a20

SQL> col recipient_name format a20

SQL>

SQL> select count(*) "Number of Apply Errors" from dba_apply_error;

Number of Apply Errors

----------------------

0

SQL>


Transition: OGG Apply to NOCDBS – Report (1) ***********************************************************************

Oracle GoldenGate Delivery for Oracle




Starting at 2015-02-11 16:49:24

***********************************************************************




Process id: 8656

Description:

***********************************************************************


***********************************************************************



replicat r_nocdbs

GETENV (NLS_LANG) NLS_LANG = .AL32UTF8


userid oggadmin@NOCDBS, password ********

ASSUMETARGETDEFS




DBOPTIONS SETTAG 00


DDL INCLUDE OBJNAME "TSTSTR"."STR_HEARTBEAT_TABLE“

DDLOPTIONS REPORT

DDLOPTIONS NOTAG



COMPARECOLS (ON UPDATE ALL, ON DELETE ALL), RESOLVECONFLICT (UPDATEROWEXISTS, (DEFAULT, OVERWRITE));

...

ggsci> start replicat R_NOCDBS aftercsn 2931340


...




target directories:

D:\ogg1212_tar\dirtmp.

CACHEMGR virtual memory values (may have been adjusted)

CACHESIZE: 2G

CACHEPAGEOUTSIZE (default): 8M

PROCESS VM AVAIL FROM OS (min): 4G

CACHESIZEMAX (strict force to disk): 3.41G

Database Version:









NLS_CHARACTERSET = "WE8MSWIN1252"

2015-02-11 16:49:24 INFO OGG-02527 Integrated Replicat does not populate a trace table.

2015-02-11 16:49:24 INFO OGG-02545 Parameter GROUPTRANSOPS is ignored by Integrated Replicat when parallelism is greater than 1.

2015-02-11 16:49:35 INFO OGG-02530 Integrated replicat successfully attached to inbound server OGG$R_NOCDBS.

***********************************************************************


***********************************************************************

Opened trail file ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000000 at 2015-02-11 16:49:35


2015-02-11 16:49:35 INFO OGG-03506 The source database character set, as determined from the trail file, is we8mswin1252.

2015-02-11 16:49:38 INFO OGG-01373 User requested start after CSN 2931161.

2015-02-11 16:49:38 INFO OGG-01374 Transaction delivery commencing at position Seqno 0, RBA 3627, Transaction ID 5.17.2488, CSN 2931340,

9 transaction(s) skipped.

...

Transition: OGG Apply to NOCDBS – Report (2)


Transition: OGG Apply to NOCDBS – Report (3) ...

Using the following key columns for target table TSTSTR.STR_HEARTBEAT_TABLE: DB_NAME.


Switching to next trail file ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000001 at 2015-02-11 16:56:25 due to EOF, with current RBA 12943


Processed extract process graceful restart record at seq 1, rba 845.

Switching to next trail file ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000002 at 2015-02-11 16:56:25 due to EOF, with current RBA 908





***********************************************************************


***********************************************************************

Last record for the last committed transaction is the following:

_______________________________________________________________________

Trail name : ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000002

Hdr-Ind : E (x45) Partition : . (x04)

UndoFlag : . (x00) BeforeAfter: A (x41)

RecLength : 110 (x006e) IO Time : 2015-02-11 17:01:02.001000

IOType : 134 (x86) OrigNode : 255 (xff)

TransInd : . (x03) FormatType : R (x52)

SyskeyLen : 0 (x00) Incomplete : . (x00)

AuditRBA : 42 AuditPos : 6023184

Continued : N (x00) RecCount : 1 (x01)

2015-02-11 17:01:02.001000 UnifiedUpdate Len 110 RBA 10378

Name: TSTSTR.STR_HEARTBEAT_TABLE

_______________________________________________________________________

Reading ./dirdat/NOCDBS/OGGADMIN/A_QN1_NOCDBS/zz000002, current RBA 10617, 77 records


From Table TSTSTR.STR_HEARTBEAT_TABLE to TSTSTR.STR_HEARTBEAT_TABLE:

# inserts: 0

# updates: 77

# deletes: 0

# discards: 0


Transition: Stop Streams Capture & Propagation

SQL> connect stradmin/STRADMIN@nocdbp


GLOBAL_NAME

--------------------


SQL>

SQL> Begin

2

3 DBMS_CAPTURE_ADM.STOP_CAPTURE(

4 capture_name=>'C_CN1_NOCDBP');

5

6 End;

7 /


Transaktion mit COMMIT abgeschlossen.

CAPTURE_NAME STATUS

---------------- --------

C_CN1_NOCDBP DISABLED

OGG$CAP_E1NOCDBP ENABLED

SQL> connect stradmin/STRADMIN@nocdbp


GLOBAL_NAME

--------------------


SQL>

SQL> Begin

2

3 DBMS_PROPAGATION_ADM.STOP_PROPAGATION(

4 propagation_name=>'P_PN1_NOCDBP');

5

6 End;

7 /


Transaktion mit COMMIT abgeschlossen.

PROPAGATION_NAM STATUS

--------------- --------

P_PN1_NOCDBP DISABLED

OGG$CAP_E1NOCDBP – GoldenGate Integrated Capture


OS processes in NOCDBP after Migration of Direction 1

Migrated Oracle Streams processes are in status „DISABLED“: C_CN1_NOCDBP and P_PN1_NOCDBP

Oracle Streams Apply process OGG$E1NOCDBP is an internal process of OGG Integrated Capture E1NOCDBP

Oracle Streams Capture process OGG$CAP_E1NOCDBP is the internal LogMiner process of OGG Integrated Capture E1NOCDBP

Oracle Streams Apply process A_AN1_NOCDBP is still in status „ENABLED“ because Direction 2 ist not yet migrated


OS processes in NOCDBS after Migration of Direction 1

Oracle Streams process C_CN1_NOCDBS and P_PN1_NOCDBS are still in status „ENABLED“ because Direction 2 ist not yet migrated

Migrated Oracle Streams Apply processes is in status „DISABLED“: C_AN1_NOCDBS

Oracle Streams Appy process OGG$R_NOCDBS is the Inbound Server of OGG Integrated Apply R_NOCDBS


Transition Error Situations


***********************************************************************





Starting at 2015-02-11 15:38:03

***********************************************************************

...

...

...




LOGALLSUPCOLS


DDL INCLUDE OBJNAME "TSTSTR"."STR_HEARTBEAT_TABLE„

DDLOPTIONS REPORT



...

...

...

2015-02-11 15:38:09 ERROR OGG-02082 Unable to use + as input for the EXCLUDETAG parameter. A valid hex value is expected.

2015-02-11 15:38:09 ERROR OGG-01668 PROCESS ABENDING.

First supported in OGG 12.1.2.1

TRANLOGOPTIONS EXCLUDETAG 00

OGG Capture: E1NOCDBP

Reason is: OGG VERSION,"12.1.2.0"


***********************************************************************





Starting at 2015-02-11 15:50:26

***********************************************************************

...

...

...

replicat r_nocdbs

GETENV (NLS_LANG)

NLS_LANG is not set


userid oggadmin@NOCDBS, password ********

ASSUMETARGETDEFS




DBOPTIONS SETTAG 00


DBOPTIONS _NO_DISABLE_ON_RETRY_ERROR

...

...

...

2015-02-11 15:50:27 ERROR OGG-00219 Invalid value for DBOPTIONS: Invalid database option (_NO_DISABLE_ON_RETRY_ERROR).

2015-02-11 15:50:27 ERROR OGG-01668 PROCESS ABENDING.

66

First supported in OGG version 12.1.2.1

outcommented

OGG Apply: R_NOCDBR

Reason is: OGG VERSION,"12.1.2.0"


Remove Streams Components – All Directions

Don’t use DBMS_STREAMS_ADM.REMOVE_STREAMS_CONFIGURATION

Remove step by step Processes, Rulesets, Queues and Queue-Tables

After Migration of Direction 1: (see next Slide)

1. Stop and Drop Capture & Propagation Processes in primary Database

2. Remove Capture Queues (normal & exception) in primary Database

3. Stop and Drop Apply Process in secondary Database

4. Remove Apply Queues (normal & exception) in secondary Database

After Migration of Direction 2:

1. Stop and Drop Capture & Propagation Processes in secondary Database

2. Remove Capture Queues (normal & exception) in secondary Database

3. Stop and Drop Apply Process in primary Database

4. Remove Apply Queues (normal & exception) in primary Database


Remove Streams Components – Direction 1

C_CN1_NOCDBP A_AN1_NOCDBS P_PN1_NOCDBP

Capture Process

DBMS_ _ADM.STOP_ ...

Normal Queue

(DROP_UNUSED_RULE_SETS=TRUE)

DBMS_ _ADM.DROP_ ...

Queue Table

Queue Table

Exception Queue

Apply Process

Normal Queue

Queue Table

Queue Table

Exception Queue

DBMS_STREAMS_ADM.REMOVE_QUEUE

(DROP_UNUSED_QUEUE_TABLE=TRUE)

CAPTURE PROPAGATION

APPLY

Propagation


Agenda

2

3

4






Modified OBEY Files


5 Q&A


Summary

Simple, no Apply Handlers

Only replication direction from NOCDBP to NOCDBS was migrated

The same steps are used for migration direction 2 (NOCDBS to NOCDBP)

Migration sample

Migration utility

Easy to use

Parameter files must be checked and modify

Recommentation: Migrate per replication direction

More complex Streams environments at customer sides (e.g. Hub & Spoke)

Additional Issues: (e.g. Multiple Extracts per source database sharing same logminer dictionary Parallel Apply: must set to PARALLELISM = 1 (COMMIT_SERIALIZATION = FULL)


Resources

http://www.oracledwh.de/downloads/14_Oracle_Data_Integration/Data_Integration_Solutions_start.html

OGG

http://www.oracledwh.de/downloads/14_Oracle_Data_Integration/Data_Integration_Solutions_start.html


Resources

streams2ogg


Resources

PPTs in

PDF format


Contacts

Oracle Consulting

Oracle Advanvced Customer Support (ACS) Thomas Hauser thomas.hauser@oracle .com 0211/74839552

Alexander Böttcher alexander.boettcher@oracle .com 040/89091529

Oracle Presales Thomas Niewel thomas.niewel@oracle .com 089/14301899

Joachim Jaensch joachim.jaensch@oracle .com 089/14301326


Agenda

2

3

4






Modified OBEY Files


5 Q&A



“streams2ogg“ - After GoldenGate was configured


C_CN1_NOCDBP,"e1nocdbp“

A_AN1_NOCDBP,"r_nocdbp"

P_PN1_NOCDBP,"p1nocdbp"

OGGADMIN.OGG$Q_E1NOCDBP,"aa"

STRADMIN.C_QN1_NOCDBP,"bb"

./dirdat,"./dirdat"

MANAGER PORT,"7809"




DBLOGIN,"BOTH"


C_CN1_NOCDBS,"e1nocdbs"

A_AN1_NOCDBS,"r_nocdbs"

P_PN1_NOCDBS,"p1nocdbs"

STRADMIN.C_QN1_NOCDBS,"aa"

./dirdat,"./dirdat"

MANAGER PORT,"7809"




DBLOGIN,"BOTH"

NOCDBP

NOCDBS

If Capture and Apply are running on the same machine you must use different Manager Ports

OGGADMIN.OGG$Q_E1NOCDBP is an internal Oracle Streams process for GoldenGate Integrated Capture


Transition for Direction 2 Stop OS Apply and start OGG Apply


Transition: Stop & Check Apply process for NOCDBP


GLOBAL_NAME

--------------------



Apply Process Name STATUS MAX_APPLIED_MESSAGE_NUMBER

-------------------- -------- --------------------------

A_AN1_NOCDBP DISABLED 4037608

Number of Apply Errors

----------------------

0

Es wurden keine Zeilen ausgewählt


Transition: OGG Apply to NOCDBP – Report (1) ***********************************************************************





Starting at 2015-05-19 10:56:51

***********************************************************************




Process id: 6292

Description:

***********************************************************************


***********************************************************************



replicat r_nocdbp

GETENV (NLS_LANG) NLS_LANG is not set



ASSUMETARGETDEFS

discardfile ./dirrpt/r_nocdbp.dsc, PURGE, MEGABYTES 500



DBOPTIONS SETTAG 00



DDLOPTIONS REPORT

DDLOPTIONS NOTAG



COMPARECOLS (ON UPDATE ALL, ON DELETE ALL), RESOLVECONFLICT (UPDATEROWEXISTS, (DEFAULT, OVERWRITE));

...

ggsci> start replicat R_NOCDBP aftercsn 4037608


Transition: OGG Apply to NOCDBP – Report (2) ...




target directories:

D:\ogg1212_src\dirtmp.

CACHEMGR virtual memory values (may have been adjusted)

CACHESIZE: 2G

CACHEPAGEOUTSIZE (default): 8M

PROCESS VM AVAIL FROM OS (min): 4G

CACHESIZEMAX (strict force to disk): 3.41G

Database Version:









NLS_CHARACTERSET = "WE8MSWIN1252“

2015-05-19 10:56:53 INFO OGG-02527 Integrated Replicat does not populate a trace table.

2015-05-19 10:56:53 INFO OGG-02545 Parameter GROUPTRANSOPS is ignored by Integrated Replicat when parallelism is greater than 1.

2015-05-19 10:57:05 INFO OGG-02530 Integrated replicat successfully attached to inbound server OGG$R_NOCDBP.

***********************************************************************


***********************************************************************

Opened trail file ./dirdat/NOCDBP/OGGADMIN/A_QN1_NOCDBP/zz000000 at 2015-05-19 10:57:05


2015-05-19 10:57:05 INFO OGG-03506 The source database character set, as determined from the trail file, is we8mswin1252.

2015-05-19 10:57:08 INFO OGG-01373 User requested start after CSN 4037608.

2015-05-19 10:57:08 INFO OGG-01374 Transaction delivery commencing at position Seqno 0, RBA 8637, Transaction ID 2.24.2661, CSN 4037771,

30 transaction(s) skipped

...


Transition: OGG Apply to NOCDBP – Report (3) ...

MAP resolved (entry TSTSTR.STR_HEARTBEAT_TABLE):

MAP "TSTSTR"."STR_HEARTBEAT_TABLE", TARGET "TSTSTR"."STR_HEARTBEAT_TABLE", COMPARECOLS ( ON UPDATE ALL, ON DELETE ALL),

RESOLVECONFLICT ( UPDATEROWEXISTS, (DEFAULT, OVERWRITE));

Using following columns in default map by name: DB_NAME, CURRENT_TIME

Using the following key columns for target table TSTSTR.STR_HEARTBEAT_TABLE: DB_NAME.



2015-05-19 10:58:17 INFO OGG-01021 Command received from GGSCI: REPORT.

***********************************************************************


***********************************************************************

Last record for the last committed transaction is the following:

___________________________________________________________________

Trail name : ./dirdat/NOCDBP/OGGADMIN/A_QN1_NOCDBP/zz000000

Hdr-Ind : E (x45) Partition : . (x04)

UndoFlag : . (x00) BeforeAfter: A (x41)

RecLength : 110 (x006e) IO Time : 2015-05-19 10:58:02.000000

IOType : 134 (x86) OrigNode : 255 (xff)

TransInd : . (x03) FormatType : R (x52)

SyskeyLen : 0 (x00) Incomplete : . (x00)

AuditRBA : 52 AuditPos : 26147356

Continued : N (x00) RecCount : 1 (x01)

2015-05-19 10:58:02.000000 UnifiedUpdate Len 110 RBA 10071

Name: TSTSTR.STR_HEARTBEAT_TABLE

___________________________________________________________________

Reading ./dirdat/NOCDBP/OGGADMIN/A_QN1_NOCDBP/zz000000, current RBA 10310, 7 records


From Table TSTSTR.STR_HEARTBEAT_TABLE to TSTSTR.STR_HEARTBEAT_TABLE:

# inserts: 0

# updates: 7

# deletes: 0

# discards: 0

...


Pseudo Streams processes after migration finished

Oracle Streams process C_CN1_NOCDBS and P_PN1_NOCDBS are still in status „ENABLED“ because Direction 2 ist not yet migrated

Primary database NOCDBP

Secondary database NOCDBS


Integrated Cloud Applications & Platform Services

Documents

Wechsel von Oracle Streams Practice parameters (These are not based on Streams configuration) ... Unidirectional – one direction Bi-Directional