A Service-Oriented Approach to Unifying Data Nicholas Gall Senior Vice President & Principal Analyst [email protected]

A Service-Oriented Approach to Unifying Data

A Service-Oriented Approach to Unifying Data

Nicholas GallSenior Vice President &Principal Analyst

[email protected]

© 2003 META Group, Inc., Stamford, CT-USA, +1 (203) 973-6700, metagroup.com

A Most Frequently Asked Question

Given that we want to evolve towards XML, Web services, and SOA, What are the best practices for unifying shared

business data (aka reference and master data)?– Customer– Product/Material– Supplier/Vendor/Partner

Current answer: Relational Data+EAI+EII+ETL Emerging answer: XML, Web Services, SOA What’s the key difference?


Modularity: The Key to Efficient Innovation

An on demand enterprise is one that is able to efficiently innovate its business processesand the IT systems that enable them The key to efficient innovation is modularity

Modularity enables Independent Comprehension Independent Development Independent Change

SOA minimizes coordination and maximizes consistency by modularizing change Find what changes and encapsulate it

Applying SOA to Master Data will make it easier to coordinate changes by modularizing the data


Mastering Reference Data Confusion

Phrase Google Hits

"reference data" 1,050,000

"reference data management" 1,700

"master data" 86,800

"master data management" 13,100

"master reference data" 209

"reference master data" 132

“customer reference data” 196

"customer master data" 951

“customer data integration” 6,720

"product reference data" 120

"product master data" 682

"master patient" 7,450

"patient master" 2,050

"master patient index" 6,280

"patient master index" 554

"reference data" "master data" 774

(or is it “Referencing Master Data Confusion”?)


Key Issue:How to Manage Ownership of Master Data?

Who owns the rights to query the data?

Who owns the rights to define the data?

Who owns the rights to create/delete the data?

Who owns the rights to change the data?

Master Data Management (MDM) must evolvefrom ad hoc to architected

How Widely Shared is Ownership?•App-wide•Process-wide•BU-wide•Enterprise-wide•Partner-wide•Industry-wide•World-wide


Service-Oriented Architecture:An Internetwork Approach to Modularity

An SOA is an internetwork architecture that should beInteroperable

– Should not be tied to a particular platform architecture (J2EE or .Net)

Networked– Should do integration “on the fly”

using a network of intermediariesGeneric

– Should support a wide range of applications

Federated– Should overlay a wide range of

software to minimize rip and replaceSimple

– Should define standards using only “on the wire” identifiers, formats & protocols (IFaPs), NOT APIs

Extensible– Should be easily and dynamically

adaptable to new uses and implementations

Federated

Simple

Generic

Enablers

Providers

Complementors

Users

E x

t e

n s

i b

l e

The Hourglass Model


Example:Applying SOA to Modularize Shipping

The Hourglass Model for

Intermodal Containerized Shipping

……

Oil

CSX Southern Pacific

Train

Freightliner

Truck

SS Minnow

Ship

Ms. Ohio

Barge

Wal-Mart Warehouse

Warehouse

Intermodal Container

Grain Consumer Electronics

FormatContainerEnvelopePacket

Message

For the transportation industry, horizontal integration, which emerged only in the 1950s, was made possible by the development of and agreement upon a standard freight container. When it was

publicly demonstrated in 1956 that standard containers could move successfully on a land-sea intermodal journey, a commercial revolution was started. It was the container’s unique role as common denominator among modes that was revolutionary.


Example:Applying SOA to Modularize MDM

An SOA for Customer MDM should be Interoperable

– Should not specify how apps represent customer data internally Networked

– Should do data routing, cleansing, and transformation “on the fly” using a network of intermediaries

Generic– Should define a customer metadata model that can be used

across all applications Federated

– Should independently bind different aspects of the customer metadata model to application-specific customer data models

– Should overlay existing middleware and applications to minimize rip and replace

Simple:– Should define customer metadata standards in terms of “on the

wire” IFaPs, NOT APIs Extensible

– Should enable extension of the customer metadata model to new application-specific customer aspects “on the fly”


Next Steps to a Service-Oriented Approach to Unifying Data

Use the SOA “hourglass” criteria as a guide to better modularity Design for change by focusing on services that are

general purpose, federated, simple, and extensible Focus on network concepts of interoperability

IFaPs “over the wire” not APIs “at the endpoints” Use extensibility to incrementally get there from

here Minimize rip and replace caused by business or IT

innovations Modularize data to minimize the complexity of

changing it

TRANSFORMATION STEPS

© 2004 Ascential Software Corporation. All rights reserved. Ascential is a trademark of Ascential Software Corporation or its affiliates and may be registered in the United States or other jurisdictions. Reproduction and redistribution is prohibited.

Creating a Single Enterprise-Wide View of Your Customers

A Joint BEA & Ascential Webinar

April 18, 2023 © 2004 Ascential Software Corporation. All rights reserved. Ascential is a trademark of Ascential Software Corporation or its affiliates and may be registered in the United States or other jurisdictions. Reproduction and redistribution is prohibited.

So Where is the Master Customer Data?

Most organizations are doing some customer data consolidation Data Warehouses Enterprise Applications (System of Record)

So why doesn’t everyone have a complete view of their customers? These approaches fail to meet the generic, federated,

simple, and extensible requirements

What’s needed is an approach that leverages what you have, but focuses on these requirements


Seeking a Single View of Customer

CRM

Call Center

ERP

DataWarehouse

?

Duplicated, mismatched,

and contradictory

customer data

Application

Portal

Workflow


BEA & Ascential Joint SOMDA Solution

CRM

Call Center

ERP

DataWarehouse

Common Metadata

ModelSource

System

ProfilingApplication

Portal

Workflow

?

Step 1: Understand Source Systems



CRM

Call Center

ERP

Common Metadata

Model

Extract& Load Staging

DataWarehouse

Application

Portal

Workflow

?

Step 2: Extract & Load Data to a Staging Database


SurviveEnrich MatchCertifyStandardizeLoadTransform


CRM

Call Center

ERP

Staging

DataWarehouse

MatchingService

CustomerX-ref

Application

Portal

Workflow

?

Step 3: Match & Link Records to Create a Customer Cross-Reference




CRM

Call Center

ERPApplication

Portal

Workflow

Staging

DataWarehouse

MatchingService

CustomerX-ref

Composite Data

Services

Query, Transform, Cache & Secure

Step 4: Create Application centric Data Services in Liquid Data




CRM

Call Center

ERPApplication

Portal

Workflow

Staging

DataWarehouse

MatchingService

CustomerX-ref

Composite Data

Services


Ascential

BEA

BEA for application development, Ascential for x-ref keys and data analytics




CRM

Call Center

ERPApplication

Portal

Workflow

Staging

DataWarehouse

MatchingService

CustomerX-ref

Composite Data

Services


Ascential

BEA

BEA and Ascential are working on XMI based interchange of Meta Data

Meta Data Sync via XMI


The BEA Platform for Application Development


Parallel Execution Engine

DISCOVERDISCOVER

Discover data

content and

structure

PREPAREPREPARE

Standardize, match, and correct data

TRANSFORM and DELIVERTRANSFORM and DELIVER

Transform, enrich, and deliver data

ProfileStage™ProfileStage™ QualityStage™QualityStage™ DataStage™DataStage™

Meta Data Management

Real-Time Integration Services

Enterprise Connectivity

and Event Management

Service-Oriented Architecture

The Ascential Solution for MDM Data Integration

Ascential Enterprise Integration Suite™

Complementary To BPM, EAI, and EII Technologies

Open, Service-Oriented Architecture

Integrated Data Profiling & Data Quality

Complex Data Transformation and Routing

Reusable Components & Rules

Unlimited Performance with Linear Scalability

Enterprise Meta Data Management

Anytime, Anywhere Connectivity

Industry Standard Compliant (XML, EDI, JMS, JCA)

Industry-Ready Integration Solutions


Core Components of this Approach

Data Integration Service (ETL) Matching Service Data Integration (Composite Services)


CustomerX-ref

ETL Service

CRM

Call Center

ERP

DataWarehouse

MatchingService

1) AT&T – One Hundred 1st Street2) AT&T Corp – 1700 El Camino Real3) AT&T Wireless – 250 Guadalupe Ave

1) ATEndT– Unknown

1) AT&T Corp – 100 North 1st

1) AT&T2) AT&T Wireless

Staging Standardize Match Survive

Correct source

Review Table

ATEndT– Unknown

Company Table

Location Table

AT&T

AT&T Wireless

100 North 1st. Street

250 Guadalupe Ave

CRM 1 ERP 1

Company x-ref

CRM 3

CRM 1 ERP 1

Location x-ref

CRM 3

1700 El Camino Real Ave

CRM 2

CRM 2

DW 1

DW 2


Matching Service

CustomerX-ref

MatchingService

Standardize MatchReturnMatchKeys

AT&T Inc. 100 North First CRM 1 ERP 1CRM 2 DW 1

Company Table

Location Table

AT&T

AT&T Wireless

100 North 1st. Street

250 Guadalupe Ave

CRM 1 ERP 1

Company x-ref

CRM 3

CRM 1 ERP 1

Location x-ref

CRM 3

1700 El Camino Real Ave

CRM 2

CRM 2

DW 1

DW 2

AT&T 100 North 1st. Street


Liquid Data for Composite Data Services

Liquid Data Functionality: Services spanning multiple sources Complex transforms for service definition Caching and security policies Modeling to organize services Meta data management to ease service

maintenance Query optimization for performance Validation rules for data consistency WSDL generation: 4 clicks between query

definition and WSDL generation

Impact Breakthrough productivity for creating and

managing data services

CRM SQL

Liquid Data

Client Applications

Customer

PaymentsOrders

ERP


Check security & cache Cache results

Create optimized Execution plan

Merge & TransformData

1

2 4

5

Request Result

Get data from underlying sources

Sub-query Sub-query Function Call

3

Liquid Data Engine’s Internal WorkingCreates & executes an optimized path for each service request


Summary

A comprehensive master data approach is required

Understand and cross-reference source systems

Identify inbound records and relate to cross-reference

Liquid Data can then be used to create Composite Data Services

BEA and Ascential together can provide the necessary pieces


Next Steps

Download the white paper: A Reference Architecture for Master Customer Data Management

For more information, go to: http://www.ascential.com/mdmsoa/ http://www.bea.com/mdmsoa/

Contact your local BEA and Ascential representatives to get started

© 2004 Ascential Software Corporation. All rights reserved. Ascential is a trademark of Ascential Software Corporation or its affiliates and may be registered in the United States or other jurisdictions. Reproduction and redistribution is prohibited.

Questions?

Upcoming WebinarsUpcoming Webinars

August 31 - Centralizing Master Customer Data In Your Service Oriented Architecture, Nick Gall, Meta Group

September 2 - SOA Essentials: Modeling Legacy Applications as Web Services

September 8- Implementing Business Process Monitoring and Management Solutions

September 23 -Why a Message Broker is critical to your Enterprise Service Bus

http://www.bea.com/

http://www.bea.com/

Documents

A Service-Oriented Approach to Unifying Data Nicholas Gall Senior Vice President & Principal Analyst [email protected]