Upload
stewart-bryson
View
598
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Prior to the introduction of Data Integrator (ODI), Oracle had another data integration tool: Warehouse Builder (OWB). Usually positioned as an ETL tool, OWB excelled in environments with a strong footprint in the Oracle Database. Oracle's statement of direction has been clear: to deliver a unified data integration platform, combining the best from both tools into a true world class product. With ODI 12c, that day has arrived. In this presentation, I’ll demonstrate the features available for migrating from OWB to ODI 12c. I’ll also describe a phased approach for doing a “right-time” conversion to ODI 12c, which involves migrating bite-sized chunks of OWB processes over to ODI when that migration adds legitimate value for the customer.
Citation preview
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
The Time is Now!
2
Stewart Bryson
medium.com/@stewartbryson @stewartbryson linkedin.com/in/stewartbryson
Migrating from Oracle Warehouse Builder to Oracle Data Integrator 12c
© 2014 RED PILL Analytics
Who is Red Pill Analytics?
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Red Pill Analytics: Leadership
4
Stewart Bryson !
Oracle ACE Director !
Previously From: Rittman Mead
Informix Software !
Data Warehousing & Business Intelligence
since 1996
Kevin McGinley !
Oracle ACE Director !
Previously From: Accenture
BI Consulting Group !
Data Warehousing & Business Intelligence
since 1997
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
About Red Pill Analytics
Other Services
!
• Expert Retainer • Support • Training
5
!
!
•DevOps for BI and DI •Hosted or On-premise •Used in Capacity Analytics
Capacity Analytics
!
•Development-as-a-service •Agile methodology • Faster/cheaper than traditional consulting • Subscription billing
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Technology Network White Paper
6
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Technology Network White Paper
7
http://bit.ly/owb-odi
Google Search: otn odi bryson
© 2014 RED PILL Analytics
Oh Captain My Captain
© 2014 RED PILL Analytics
Goodbye Old Friend
© 2014 RED PILL Analytics
Migrations are Hard
© 2014 RED PILL Analytics
Add Value
© 2014 RED PILL Analytics
Have a Mission Statement
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Mission Statement
13
Any task undertaken to migrate content from OWB to ODI will add immediate value to our BI stakeholders.
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ODI 12c: Overview
14
Flow-Based Declarative Design with Component-Based
Development
New Mappings replace interfaces and allow multi-step
design patterns
Reusable Mappings and Datasets for Declarative Design
similar to 11g
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ODI 12c: Overview
15
Components Provide Multi-Step Process
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ODI 12c: Overview
15
Components Provide Multi-Step Process
Data Lineage built into the Mapping Editor
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ODI 12c: Overview
16
Warehouse Builder-like Components including
Pivot/Unpivot, Join, Expression, Set, etc.
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ODI 12c: Overview
17
Component-Style KM’s encapsulate modular, reusable
pieces of logic specific to particular Components
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
19
Enterprise Installation includes Fusion Middleware
and supports Standalone Colocated Agents
Handles configuration of the Master and Work repositories and other FMW tables (Work
Tables)
JEE Agents can utilize Weblogic Coherence for high
availability
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
20
Warning: FMW Needs to be installed
for JEE Agents
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
20
Warning: FMW Needs to be installed
for JEE Agents
Ignore This!The Enterprise
Installation installs WLS and FMW
behind the scenes
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
21
Features Specifically Selected
WLS and FMW are installed as an internal
feature
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
22
We get all this cool stuff!
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Installing ODI 12c: Enterprise Installation
23
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Repository Creation Utility
24
Recent Oracle Database versions, including 11gR2 and 12c, as well as DB2 and SQL
Server
Master Repository contains Topology information, Work
Repository contains development information
Creates Fusion Middleware tables, including Fusion Work
Tables
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Agents
26
ODI Agents execute load processes, including compiled
Load Plans and Scenarios
Somewhat comparable to the OWB Control Center (much, much better)
Deployed as JEE Agents or Standalone Agents
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Domain Configuration
27
Use built-in FMW and WLS functionality to “harden” our
ODI Agents
Our JEE-deployed ODI Agent is comparable to an ETL Server
Utilizing Coherence gives us performance increases in
certain circumstances, fault tolerance, etc.
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Domain Configuration
28
Domain Configuration
Templates make it easy to mix and match deployed functionality to
our domain
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Domain Configuration
29
Service Table Schema (STB) stores relevant
information from our install to
feed the templates
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Domain Configuration
30
In the end, we have a WLS
Domain complete with Admin Server and
Managed Server
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Domain Configuration
31
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration
32
Execute OWB processes as standard ODI processes, including Mappings and
Process Flows
Full execution from OWB Control Center captured in standard ODI audit tables
Orchestrate our loads wherever we want: ODI Load Plans or OWB Process Flows
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Topology
33
Data Server configured with
OWB Workspace Owner
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Topology
34
JDBC Configuration to
the OWB Workspace
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Topology
35
Configure the Workspace Owner
as the Physical Schema
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Topology
36
Setting a Logical Schema to
abstract the Physical Schema and Data Server
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Topology
37
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
38
Use the ODI Package Editor and the new
OdiStartOwbJob tool
We can execute Mappings or
Process Flows
OdiStartOwbJob "-WORKSPACE=OWBREP" "-LOCATION=OWF_MGR" "-OBJECT_NAME=SBATCH/MAIN_LOAD" "-OBJECT_TYPE=PROCESSFLOW"
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
39
ODI Scenario for Process Flow
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
39
ODI Scenario for Process Flow
Main Process Flow execution
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
39
ODI Scenario for Process Flow
Main Process Flow execution
Sub-Process Flow execution
Sub-Process Flow execution
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
39
ODI Scenario for Process Flow
Mapping Execution
Main Process Flow execution
Sub-Process Flow execution
Sub-Process Flow execution Mapping Execution
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
40
Full Auditing Results
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
OWB Runtime Integration Execution
41
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility
42
Supports 11.2.0.4 of Warehouse Builder with Patch
17830453
Apply Patch 17053768 to Data Integrator if not already
included in release
Command-line utility to perform the OWB to ODI
Migration
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Run Modes
FAST_CHECK Performs a read-only check of the OWB repository and reports back the items than can and cannot be migrated.
DRY_RUN Performs a migration to ODI using the ODI 12c SDK, but does not perform a commit at the end of the process.
RUN (Default) Executes the migration and commits migrated objects to the target ODI 12c repository.
43
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
44
ODI Master Repository Connection Information
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
44
ODI Master Repository Connection Information
ODI Work Repository Information
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
44
ODI Master Repository Connection Information
ODI Work Repository Information
OWB Workspace Information
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
44
ODI Master Repository Connection Information
ODI Work Repository Information
Migration Options
OWB Workspace Information
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
45
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
45
Configuration FileODI and OWB passwords
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
46
Migration Results
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migrated Mappings
Migrated Models
Migration Utility: Configuration
47
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Migration Utility: Configuration
48
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Warehouse Builder (Our Old Friend)
50
Wildly Successful •Flow-based design •High-performing code (EL-T) •Ships with Database 10g and 11g •PL/SQL Code Generation •Freemium model
Deprecated •Certified (not shipped) with Database 12c • In maintenance mode Limited Integration Capabilities
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Data Integrator 12c
52
Best of both worlds! •Now with flow-based design •High-performing code (EL-T) •Knowledge Module architecture •Enterprise Manager integration •Fusion Middleware integration
Is there value in changing? • “Our organization has a major investment in OWB” • “Our mappings and process flows just work”
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
If Only There Was a Way to…
53
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
If Only There Was a Way to…
54
Maintain investment in OWB •Continue to execute legacy code •Only port our code when it presents value to do so
Blaze new trails with ODI 12c •Fantastic new features are dying to be used •The Knowledge Module (KM) architecture pays huge dividends
Deploy a coherent strategy •Shouldn’t feel like two tools
© 2014 RED PILL Analytics
It’s not an easy choice
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 1: Run OWB Process Flows in ODI
56
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 1: Run OWB Process Flows in ODI
57
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 2: Orchestrate our OWB Mappings in ODI
58
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 2: Orchestrate our OWB Mappings in ODI
59
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 2: Orchestrate our OWB Mappings in ODI
60
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 3: Re-implement in ODI When Necessary
61
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 3: Re-implement in ODI When Necessary
62
www.RedPillAnalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Phase 3: Re-implement in ODI When Necessary
63