Enterprise Data Quality - OracleDirect Sales Division : Oracle€¦ · PPT file · Web view · 2015-05-05Calling EDQ from an Oracle application such as Siebel, MDM, EBS, etc. Apps:

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1


The following is intended to outline our general product direction. It is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and

timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Enterprise Data Quality

Martin Boyd – Senior Director Product Strategy,


MDM Partner Summit – May 2014


Agenda

EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap


Enterprise Data Quality Growth

• Revenues up 10x in 11 quarters (rolling 12 months basis)

• Completely replaced ODQ (Trillium and Informatica OEMs)

• 14 new EDQ/EDQP deals in NAS last quarter alone


“Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point, you either have to stop and clear the windshield or risk everything.”

Ken Orr, The Cutter Consortium


• How do you know?

• What is the business impact?

• What should you do about it?

Data Quality – Is Your Data “Fit for Purpose”?


Data Quality Maturity Progression

CHAOS!

DQ Tools

DQ Process/ System

DQ + MDM– Single Domain

DQ + MDM– Multiple Domains

No tools, no process – the Wild West!

Project based – benefits are short lived

DQ ‘Masters’ data in legacy applications with repeatable process

Hubs persist and sync data – leveraged impact

Greater coverage, greater impact

Gover

nanc

e


Gartner's BI/EPM Maturity Model


Agenda



Enterprise Data QualityIntegrated Solution for All Data Quality Problems

• Process metrics• Quality metrics

• Case Management• Remediation

• Party (individuals, households) match

• Entity match

• Semantic (category) match

• Statistical match

• Match review• Merge/survivorship

• Global parse• Category parse• Extract

• Transform• Address verification

& geocoding

• Substitute • Enrich• Classify

• Statistics• Patterns• Phrases

• Duplicates• Completeness• Max/min values

Profile

Standardize

Match

Govern

Quickly understand data content

Drive conformance to standards

Identify & merge duplicates

Monitor effectiveness & resolve problems

Com

mon

Acc

ess/

UI



Broadest DQ offering• Best of breed capabilities for both Party Data and Product

Data• Profiling, standardization, matching, case management,

governance

Most usable DQ offering• Completely integrated offering – designed to work together• Designed for business and technical users• Transparent operation and results – no black boxes

Pervasive operation for enterprise quality governance • Within legacy systems and MDM Hubs• As part of migration/system load• On data entry/capture• As part of data movement/transfer

Profile

Standardize

Match

Govern

Quickly understand data content

Drive conformance to standards

Identify & merge duplicates

Monitor effectiveness & resolve problems

Com

mon

Acc

ess/

UI

Enterprise DQ Platform


Typical Customer/Party Data IssuesVariation or

Error Example Variation or Error Example

Sequence errors • Mark Douglas or Douglas Mark Transcription mistakes

• Hannah, Hamah

Involuntary corrections • Browne – Brown Missing or extra

tokens• George W Smith, George Smith, Smith

Concatenated names • Mary Anne, Maryanne Foreign sourced

data• Khader AL Ghamdi, Khadir A.

AlGamdeyNicknames and aliases • Chris – Christine, Christopher, Tina Unpredictable

use of initials• John Alan Smith, J A Smith

Noise • Full stops, dashes, slashes, titles, apostrophes

Transposed characters

• Johnson, Jhonson

Abbreviations • Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan Milo

Truncations • Credit Suisse First Bost Inaccurate dates• 12/10/1915, 21/10/1951, 10121951,

00001951

Prefix/suffix errors • MacDonald/McDonald/Donald Transliteration

differences• Gang, Kang, Kwang

Spelling & typing errors

• P0rter, Beht Phonetic errors • Graeme – Graham


Typical Product/Item Data Issues

10hp motor 115V Yoke mount

mtr, ac(115) 10 horsepower 115volts

MOT-10,115V, 48YZ,YOKE

This 10hp yoke mounted motor is rated for 115V with a 5 year warranty

10 Caballos, Motor, 115 Voltios

TEAO HP = 10.0 1725RPM 115V 48YZ YOKE MTR

Motor, TEAO, 1725 RPM, 48YZ, 15 Voltios, Montaje de Yugo, hp = 10

Item MotorClassification 26101600Power 10 horsepowerVoltage 115Mounting Yoke


Urban Myth: Customer and Product Data can use the Same Approach

Party Data (customer, supplier, employee, citizen etc.)

Product Data (part, SKU, item, asset etc.)

Over 30,000 different product categoriesVariations by country/locale

Requires: • Pre-built processes• Pattern-based matching

Name & Address Data• Relatively fixed syntax• Mis-spellings and name equivalents• Verify against postal files

Requires:• Contextual understanding• Ability to learn new domains quickly

Product Data• No fixed syntax – few standards• Infinite variability – format, content, syntax• Different rules for each product category


Main EDQ Console, Focused on the User

ProjectBrowser

Main Canvas

Results Browser

Tool Palette


Immediate drill-down to examine real data

Field-level completeness, uniqueness, validity etc.



Drill-down to see actual data values and determine

required rules, standards etc.



Drill-down to see actual data values and determine

required rules, standards etc.


Build-out Full DQ Process

Process can be ‘simple’ or ‘complex’

Process thumbnail assists in navigation


Key Feature: Pre-built processors • Comprehensive DQ Functionality with a Single User Interface and Repository


Key Feature: Language, Region and Country-Specific Capabilities

Country and region-specific processors Fully unicode compliant


Agenda



Pricing Objectives Objective: Maximize revenue opportunity from both Tech and Apps with minimal jump-

ball issues– Tech: Sell EDQ into data integration scenarios– Apps: Sell EDQ into application enablement scenarios

Apps Changes (as of March 13th, 2014)– Merge EDQ Parsing & Standardization + EDQ Match & Merge– Merge EDQ Product Data Parsing & Standardization + EDQ Product Data Match & Merge and make EDQ

platform a pre-req– Provide Weblogic & Oracle DB SE as RULs for EDQ

Tech changes (as of March 13th, 2014)– New SKU for Audit and Dashboard– New SKU for Real-time Processing (call EDQ as web service)– Remove requirement for ODI license– Cannot be called from Oracle applications (MDM, EBS, etc.)


EDQ Pricing Changes on Apps PL

Customer/Party data

EDQ Product Data Match and Merge

EDQ Match and Merge

EDQ Product Data Parse and Standardize

EDQ Parse and Standardize

Product/Item data

• Includes Connectors to Customer Hub (UCM) and Product Hub

• Structure is 1:1 replacement for Informatica OEM

$150k

$125k

$150k

$125k

Oracle Confidential 25

EDQ Product Data Extension$150K

• EDQ Product Data Extension • Combines EDQ Product Data Match and Merge and

EDQ product Data Parse and Standardize• Has pre-req of EDQ Parse and Standardize

EDQ Profile & Audit$150K

EDQ Profile & Audit$150K

ED

Q A

ddre

ss V

erifi

catio

n $6

3.3K

ED

Q A

ddre

ss V

erifi

catio

n $6

3.3K

EDQ Standardize and Match $275K

New/Changed SKU

Apps PricelistGo-forward

Apps PricelistPrevious


Tech & Apps Comparison

EDQ Standardization and Match$275k

EDQ Profile & Audit$150k EDQ Profiling for Data Integration

$100K

EDQ Batch Processing for Data Integration

$100K

Apps Pricelist

• Includes Application Connectors• CRM/UCM/Product Hub/Endeca etc.

• Includes option for EDQ Product Data Extension for deep product data capabilities

• Batch & Real-time SKUs are restricted:• Cannot be called by Oracle App (sold off Apps PL)• Cannot return real-time call to Oracle App user• Do not include App Connectors

• EDQ Real-time Processing SKU requires PM approval (CA)

EDQ Real-time Processing for Data

Integration (CA)

$100K

EDQ Audit & Dashboard for Data Integration $50K

Changed SKU

New SKU

EDQ Product Data Extension - $150k

ED

Q A

ddre

ss V

erifi

catio

n $6

3.3K

ED

Q A

ddre

ss V

erifi

catio

n fo

r D

ata

Inte

grat

ion

$63

.3K

Tech Pricelist


Is this deal a good fit?

Tech: Sell EDQ into data integration scenarios

Tech should sell if:– EDQ is standalone, or being called

from a DI system (ODI, OGG, SOA, competitor ETL, etc.)

Hand over to Apps if:– Calling EDQ from an Oracle application

such as Siebel, MDM, EBS, etc.

Apps: Sell EDQ into application enablement scenarios

Apps should sell if:– EDQ is standalone or enabling an

Oracle application (something sold on Application pricelist – EBS, CRM, MDM, etc.)

Hand over to Tech if:– It’s all about ODI and data integration

with no apps impact

Note: A Tech license can be migrated to Apps using a net-net-min 10% method – so this will NOT be free and will involve admin hassles for your customer, but should not be a big issue if there is license expansion involved


Pre-Requisites & RULs – Apps

• Arrows show inter-module pre-reqs• Everything builds from Profiling

Other details:• EDQ includes a restricted use

license of Weblogic Basic and Oracle Database EE

• EDQ and EDQ Product Data Extension can be installed on the same server, but this is not certified and careful though should be given to potential resource conflicts. Should be evaluated case by case.

• Servers can be segmented using OVM (EDQ works under VMWare, but Oracle does not respect it for licensing purposes)

• Oracle charges for all processors (live, QA, Development, etc.)

• EDQ runs on EXA*

EDQ Standardization and Match$275k

EDQ Profile & Audit$150k

EDQ Product Data Extension - $150k

ED

Q A

ddre

ss V

erifi

catio

n $6

3.3K

Requires

Requires

Requires


Agenda



• How do you know?

• What is the business impact?

• What should you do about it?

Data Quality – Is Your Data “Fit for Purpose”?


Common Data Quality Use Cases

System Consolidation/Migration• Enforce new system standards

on legacy data

Compliance• Drive consistent data and processes to

meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)

Enterprise DQ Services • Common DQ services to clean-up and

govern application data across multiple applications

Business Intelligence Enablement• Enforce BI standards on disparate

data

MDM Enablement • Verify, standardize, match and

and merge data from disparate sources

Application Enablement• Clean-up and govern application

data for a single application (CRM, HR, PLM, Retail search, etc.)




on legacy data






data





Poor data quality is the most common source of project delay

and failure – but it is easily avoided with systematic data

preparation


Why Focus on System Migration/Consolidation?

• More than 80% of data migration projects run over time and/or over budget. Cost overruns average 30%. Time overruns average 41%.Bloor Group

• 80% of organizations… will underestimate the costs related to the data acquisition tasks by an average of 50 percent”Gartner

Because poor data biggest single risk to the project

33


Case Management for Governance

Usage• Cases/alerts are assigned a work queues and a priority• Data specialists sign in and review/resolve issues• Management reports allow monitoring of work queues

and productivity• Helpful for

o One-time cleanse/migrationo Ongoing governance program

Features• Hierarchical Case/alert functionality• Configurable Workflows• Automatic prioritization of cases/alerts• Timers• Email Notification Support• Comprehensive audit trail• Immediate ad-hoc reporting

Review and resolve exceptions from the DQ process


Data Prep for System Migration/Implementation

EDQ Process

Apps and hubs

Governance and Case Management to ‘Perfect’ Data

• DQ Insight (Dashboard)• Reporting• Trend Analysis• Case Management• Workflow• Remediation

Legacy Data ‘Fit for Purpose’ Data




on legacy data






data





Even after initial data preparation, ongoing data governance is

required




on legacy data






data




data for a single application (CRM, HR, PLM, Retail search, etc.)Business Intelligence is perhaps the

most obvious example of garbage-in, garbage-out


Impact of Data Quality on BI

“Only 30% of BI/DW implementations fully succeed. The top two reasons for failure? Budget constraints and data quality.”

Gartner

“More than 50 percent of data warehouse projects will have limited acceptance, or will be outright failures, as a result of a

lack of attention to data quality issues” Gartner

“It does not really matter how good your management sponsorship or your business-driven motivation is. If you do not have the data, or the data does not have sufficient quality, any

BI implementation will fail.” Kimball


EDQ for DW/BI1. ODI or GoldenGate move data into DW2. ODI maps data into Staging tables3. EDQ ‘cleanses’ data4. ODI moves from staging into production tables

Data in DW is ‘Fit for Purpose’BI/AnalyticsSource of truth

ProductionTables

Data Sources

BI/analytics

Single source of truth for reference

StagingtablesODI

EDQParsing, Cleansing,

Standardization, Matching

Replicasourcetables

ODI

Golden Gate

ODI




on legacy data






data





Sometimes best practices are mandated by law and have large

penalties for non-compliance (FATCA, OFAC, etc.)




on legacy data






data





Even where hub implementation is a core objective, DQ often comes first

• To prepare the way• Lower implementation risk • Deliver tactical ROI


Enterprise Grade MDMModern. Purpose-Built. Multi-Domain. SaaS.

Cus

tom

er

Sup

plie

r

Ass

et

Pro

duct

Loca

tion

Em

ploy

ee

Fina

ncia

l

Oracle MDM

EPM/BI

CRM

Legacy…

CRM

Other Sources

ERP

http://www.pixmule.com/




on legacy data






data





Enforcing common DQ standards across the enterprise is a simple,

tactical fix to systemic problems and a good foundation for enterprise

governance


EDQ Web ServicesEnforce common DQ standards across the enterprise

Common Services

Applications App 1 App 2 App 3

Library of enterprise standard DQ services

Any EDQ process may be called as a real-time web service

Call any process from any application to

1. Enforce common standards

2. Minimize architectural changes


Zebra’s - Oracle Eco System


Governance Vision in ActionLibrary of enterprise standard DQ services

Data Steward “Cockpit”

• DQ and Process Visibility• Rules Management• Case Management

Case Management


Agenda



Key Investment Drivers Cloud

– Cloud DQ services– DQ capabilities built into Fusion Apps– Cloud DQ governance enablement

Governance II– Improved DQ issue lifecycle support

Integration of DQ and DQ Services– Oracle Middleware – Oracle Applications– External Applications

N.B. Several mandatory requirements for cloud adoption, including full HA/Clustering, Elastic provisioning support etc.


Q&A

Martin Boyd – Senior Director Product Strategy, Enterprise Data [email protected]

mailto:[email protected]


Documents

Enterprise Data Quality - OracleDirect Sales Division : Oracle€¦ · PPT file · Web view · 2015-05-05Calling EDQ from an Oracle application such as Siebel, MDM, EBS, etc. Apps: