Art of Integration Service Oriented Architectures (SOA) and the Enterprise

Services Bus (ESB)

Origination: Chris Huff, TWC

Current Philosophy – The Theme Continues

Common Theme? What’s missing: q  Ubiquitous Standards

q  Open standards q  Open community

UDDI

WS-*

RPC

RMI

COM/DCOM

Goal à Flexibility and Reuse SOAP

What is still missing: q  Consistency q  Discipline REST

Service Oriented Architectures

�  EAI, ESB, SOA, ESA, EIEIO…

�  BPM, Workflow

�  ESB is very similar to that of "Enterprise Application Integration" (EAI) tools, apart from three significant terms: �  Web services, ubiquitous and lightweight. 1

�  “The emergence of the ESB Concept is closely linked with the lasting trends that have been slowly transforming the EAI market for the last few years: standardization of infrastructures with �  J2EE �  Microsoft .NET �  Web services �  the redistribution of roles of integration software component

vendors.” 1

Current Integration Philosophy

1 - http://searchwebservices.techtarget.com/tip/1,289483,sid26_gci913058,00.html

PROCESS

APPLICATION

INTEGRATION

DATA

Objective: Provide efficiently all non-merchandise Materials & Services tosupport the operation

Item Object

Vendor Object

Store Object

Home Mart2.1 Procure Materials & Services – Level Sub Process Detail

2.1.5Manage Master

Data

2.1.6Purchase

Materials &Services

2.1.4Manage & MonitorSupplier Contracts

2.1.1Develop/Maintain

ProcurementPolicies

Purchase Order Object

Inventory FileVendor MasterFile

Item MasterFile

START END

2.1.7Receive Materials

& Services

A/PPaymentRequestProcess

Purchase OrderFile

PurchasingModule

Optical ArchivingSystem

Phone Calls

Paper Forms

Manual System

Services

Intranet Application

PoliciesProceduresProcess Flows

Article Management

Vendor Management

Purchase Orders

Conditions

GRANDE System

Store Structure Mgmt.

Store Master File

AgreementDocuments GRANDE System

Receipts &Distribution

Module

Vendor Object

Receipts Management

Services

2.1.3Manage External

Sourcing

2.1.2Create/Maintain

PurchaseRequisition

PAID Diagrams

Copyright © 2004. Deloitte Development LLC. ConfidentialThis document may contain confidential information. Do not disclose outside The Home Depot without prior written approval.

Recurring Theme – Abstraction layers

Recurring Theme – Abstraction layers

Reuse Services

Why make the change? �  To deliver applications faster!

“Shifts focus to application assembly rather than implementation details” 1

Benefits to SOA (Rob High, IBM Chief Arch SOA) 1.  Customer Satisfaction 2.  Business Operations

Efficiency 3.  Process Efficiency 4.  Reuse

1 “ARTS Retail Enterprise Architecture & Services Oriented Architecture” - Copyright © 2006

Hardware

Data Model

User Interface

Application Logic

Business Logic

SQL

E F

F o

r t

Hardware

Data Model

Data Access

Business Logic

E f f o r t

Application Logic

User Experience

App Development Today

App Development Tomorrow

Drive Innovation

What’s the Return?

Incremental ROI along the way •  Granular Configurable Services •  Graphical development •  Extensibility (regression

testing) •  Loose Coupling (upgrades,

flexibility) •  Consistency, Standards

Big Payoffs at the End •  Dynamic application development (composites/

mashups) •  Dynamic Business Processes •  Organizational Speed/responsiveness •  Customer Service

Significant cost savings will not happen without discipline

Document the Architecture

Near-real time (Event Server)

ENTERPRISE FINANCE INITIATIVEInterface Architecture

SAP@THD (EFI & Mexico)

CPHPFIN1

PR02

SAP R/3

Mer

cato

rAd

apte

r

Outbound

File

ALE

Bat

ch

MercatorMapping

File

Inbound

SAP BW

ALE

ALE

Bat

ch

User Upload

FDQ(DI)

CPWIFI06

PPRCA001

MQSeries

EPR Tables

Mainframe

CPHPFIN2

FTP

FTP

FDQ

FTP

DB2

FDQ

MQ

EAI Hub (cpaiss50/53) B2B Gateway

FDQ

MQ

Credit/Debit Processing

Sales Audit - Flash

FDQ

MQ

Tool Rental

CPHPLM01

Labor Management

FDQ

MQ

PPRPO008

Ariba

FDQ

MQ

CPHPMDR1

NT Applications

FDQ

MQ

PTS,FastTax,

TMS,CorpTax

File

ExcelAccess

PC Data / Users

BPI

BPI AuditorDB2

Web Browser - BPI/FDQ Processes

MercatorAudit logs(Non-BPI)

Interface Audit Data

BatchProcessing

CPAITA33

EDW

FDQ

MQ

DB2Con

nect

Dirmon

Bloomberg (Exchange Rate)

FTPwebsite

E

ncry

pted

THD Portal - WAS

Fire wall

Citrix

SAPWAS

Taxware

TCP/IPCPHPFI04

Monterrey SSC (Mexico)

FDQ

MQ

Finance HR

Firewall

E

ncry

pted

Firewall

ENTERPRISE APPLICATION INTEGRATIONInterface Architecture

SOSI Release 1

SSC WAS Cluster

Siebel Mer

cato

rAd

apte

r

Outbound

EA

I

MercatorMapping

File

Inbound

Event Server (CADQ)

FDQ(DI)

CPNTFI07

PPRCA001

MerchandisingSystems;

Central OrderDatabase

MQSeries

Base TablesPII Tables

EPR Tables

Mainframe

CPHPSRV1

FTP(FTA)

Win MappedDrives

MQ

DB2

FDQ

MQ

EAI Hub (cphpta71/72) B2B Gateway

FDQ

MQ

Special Services Store Hub

SRV1 - Customer / Zip Codes

FTA

Call Center Workstations

EJ Trickle

Ab Initio GDE

MQ

EDW

Ab Initio Base Tables

FDQ

MQ

Windows OS

NT Applications

FDQ

MQ

FlooringConfigurator

Server

BPI

BPI AuditorDB2

Web Browser - BPI/FDQand Log Processes

EAI Audit logs(all interfaces)Interface Audits

Event Server (GDE - CoBrand) Event Server (C3)

CPNTFI07

Connect3 - NT Server

SQLServer

MercatorMapping

MercatorMapping

MercatorMapping

FDQ

DB2Connect

Mer

cato

r - D

ataD

irect

Near-real time (Event Servers - SOSI1, SOSI2, HTTP)

FDQRouterMap

Mainframe

FDQ

MQ

Ab Initio

App A

xxxFDQ

MQ

App B

xxxxFDQ

MQ

App C

xxxFDQ

MQ

Future EAI Projects

Build a Conceptual View

StoresExternal

EAI / ESB

AribaSAP EFI

SAP Hub

SOSI/MM Hub

Legacy AP Sales Audit (FIN1)Credit (FIN2)

Labor Mgmt

SVSSiebel SOSI BaseTables (GDE)

Connect3 SVR1 PC (Call Center, FC)

B2BGateway

MQ Gateway

Citi

BigHammer

Trane

All Other Partners

and Vendors

Firewall

TMSUCM WCSWMS

SAP Core Retail

SAP WFMSAP CTI

Dist

MQ Hub Channel Workload

Mgmt

Affiliate/International

HD Supply

Mexico

Store nnn

ISP

Call Box

PSP Base Int

Other

Inv UsrPO

Cust

WPC

GDSN

FI MD

COD

Tool Rental PC (FastTax, TMS, CorpTax)

Ship

Other

Rally Projects Around Your Vision

ESB

Reuse

July2009

Mar2009

Today

Prepare Governance Standards

Begin dev on ES API – define THD standard

BPM Go-live (4 new services)

2011

Sept2009

Nov2009

Jan2010

Mar2010

May2009

May2010

July2010

Technology Standards Adoption

Industry Standards Adoption

THD Standards Adoption

·∙ Loosely Coupled (LC) Applications

·∙ Some LC Services·∙ Initial services/reuse·∙ Messaging Stds·∙ Some XML Stds·∙ Interface Visibility

Begin dev on highly reused service list

·∙ Rollout Enterprise Service Repository

·∙ Service Governance

SC Visibility (new services)

MSR Go-live (new services) User Service Discovery

Tool Live ·∙ Store Integration Standards

·∙ Managed Services

·∙ Capability for Service Oriented Processes ·∙ Dynamic

Service Discovery

·∙ Service enabled Processes reused

·∙ New apps built from service composition & Mashups

All interfaces for enterprise

registered (IA)

New Integration Capability live in all

storesBusiness

Becomes SOA Driver?

$RO

I

$$$

RO

I

$$ RO

I

$$ROI

$ROI

·∙ Dynamic Services

$$$ROI

Select ESR

Next phase BPMServices Live

Planogram Services Live

Establish SOA Governance Panel

Implement Standards

�  Know and plan your layers of reuse: code, modules/objects and services

�  Certify that all service interfaces are extensible

�  Spend time up front on API/Interface design

�  Develop Middleware/SOA Cost Model

�  Ask COTS package providers to supply “out-of-the-box” industry standard formats to exchange data

�  Don’t get caught in the Web Service guise of loose coupling, broker synchronous calls unless they are true Web Services

Challenges In the Enterprise

Principles Should Chart the Course Forget the acronyms and stick to the principles

•  Low total cost of ownership �  Repeatability �  Ease of maintenance �  Object Reuse �  Reliable performance �  Rapid development

•  Adherence to architectural principles �  Design that is flexible and versatile (abstraction between applications) �  Application Decoupling (Loose coupling reduces assumptions two

parties make about each other, while tightly coupled architectures tend to result in brittle, hard-to-maintain, and poorly scalable solutions)

�  Ease in importing application metadata •  Highly Scalable, Secure and Reliable

�  High volume processing capability �  Scalability

Always ask yourself, “why is this better than point-to-point?” •  If it is not cheaper, then don’t do it •  If it’s not providing a desired end-state, then don’t do it (be careful with

this one) •  If it can’t support your volumes and growth, then don’t do it

Graphical Mapping – Cost Savings

Development Estimates Developer Skill Level Map Complexity

Basic – Small to medium size data structure with simplistic mapping logic

Average – medium to large size data structure with a moderate amount of logic and complexity

Difficult – large data structures, advanced logic, multiple inputs and outputs and possibly, disparate data cardinality

*Percent of hours to add if type trees cannot be imported

Novice 35 hrs 80 hrs 160 hrs 50% Beginner (a few maps) 20 hrs 56 hrs 120 hrs 50% Proficient 8 hrs 24 hrs 80 hrs 30% Advanced 4 hrs 16 hrs 40 hrs 20%

Typical Interface estimates (Days)**

Low Med High Complex

Interface Development (batch)

10 15 20 30

Interface Development (real time)

10 15 25 35

56% less 33-43%

less

** Estimates provided byTCS

Consider Performance We needed

250-300/sec Invoice record to

Invoice IDoc

Rationalize the Cost Model

Over 120 standard functions

182 Interfaces 0 lines of code

�  Are your tools ready for XML (extended character sets, parsing, etc. These can become costly)?

�  Are you paying a premium for middleware development?

�  Do your middleware tools provide rapid (or graphical) development?

�  Consider maintenance cost (e.g. scripts and complex logic)