Services Based Architectures - Defining The Issues For Data Professionals

1

Services Based ArchitecturesServices Based ArchitecturesDefining The Issues For Data DevelopersDefining The Issues For Data Developers

Robert Abate Robert Abate Ruben TuRuben TuññgolgolManaging Partner/Principal Consultant Senior Architect

Intellisys, Inc.Intellisys, Inc. Novell, Inc.Novell, Inc.

INTELLIgentSYStems

CorporatIoNINTELLISYS, Inc.

Robert AbateRuben Tuñgol

INTELLIgentSYStems


2

Table Of Contents

�� ServiceService --Based Architecture (SBA) OverviewBased Architecture (SBA) Overview

�� SBA Case Studies SBA Case Studies –– Foundations and Foundations and PracticesPractices

�� Polo Ralph Lauren & NPD GroupPolo Ralph Lauren & NPD Group

�� Global Bank, Leasing Company, ConglomerateGlobal Bank, Leasing Company, Conglomerate

�� SBA and Web ServicesSBA and Web Services

�� Questions & AnswersQuestions & Answers

3

Service-Based Architecture

Overview

The Business Dilemma,The Business Dilemma,IT Implications,IT Implications,

And Key Characteristics Of SBA.And Key Characteristics Of SBA.


INTELLIgentSYStems


4

The Business Dilemma

Provide access to Provide access to DataDataServicesServicesProcesses Processes

Anytime, anywhere, on Anytime, anywhere, on any deviceany device

Without forgetting Without forgetting security and privacysecurity and privacy

Existing Systems

Transactions PackagesUtilitiesCollaboration Docs/ContentNetwork

Business Requirements

Employees BusinessesPartners Customers

Services Based Architecture Overview


INTELLIgentSYStems


5

Solution Requirements

Leverage existing systems

Protect systems

Build new applications

Security & Identity Management

Simplified sign-on

Identity management

Security

Multiple Levels of IntegrationProcess management

Integration Services

Web Services Foundation

Device support

Personalization

Presentation

Resources,

Existing Systems Transactions PackagesUtilitiesCollaboration Docs/ContentNetwork


Employees B2BPartners Customers

Services Based Architecture Overview


INTELLIgentSYStems


6

Business Drivers For SBA / SOAServices Based Architecture Overview

�� Interconnection Of Enterprise ApplicationsInterconnection Of Enterprise Applications�� Replacement Of Legacy SystemsReplacement Of Legacy Systems�� Removal Of Hard To Support, Complex InterfacesRemoval Of Hard To Support, Complex Interfaces

�� Foundation For Web ServicesFoundation For Web Services�� Increased Value from Information Technology:Increased Value from Information Technology:

�� Lower Total Cost to Change Lower Total Cost to Change (Shared Integration Infrastructure)(Shared Integration Infrastructure)

�� Lower Total Cost of Ownership Lower Total Cost of Ownership (Shared App., Network & Semantic Infrastructure)(Shared App., Network & Semantic Infrastructure)

�� FutureFuture --Proof Information Facilities Proof Information Facilities (Flexibility & Utilization of Assets)(Flexibility & Utilization of Assets)

�� Improved Business Productivity:Improved Business Productivity:�� Higher Throughput Per Managed Process Higher Throughput Per Managed Process (Strait Thru Processing)(Strait Thru Processing)

�� The RealThe Real --Time, Dynamic Enterprise Time, Dynamic Enterprise (Zero(Zero --Latency and Highly Agile)Latency and Highly Agile)

�� Enterprise Data Quality, Architecture & IntegrationEnterprise Data Quality, Architecture & Integration�� Standardization Of Corporate MetadataStandardization Of Corporate Metadata


INTELLIgentSYStems


7

IT Implications Of An SOA

�� Business processes need to span organization Business processes need to span organization boundaries. boundaries. Businesses produce and consume services. Businesses produce and consume services. They need to exchange data, They need to exchange data, ““ plugplug --inin ”” to each otherto each other ’’s s processes without compromising security and privacy an d processes without compromising security and privacy an d confidentiality.confidentiality.

�� Businesses need to measure the relative value of Businesses need to measure the relative value of the services they offer or consume. the services they offer or consume. Services that Services that become discrete elements of business processes provide an become discrete elements of business processes provide an opportunity to factor them into KPIopportunity to factor them into KPI ’’s.s.

�� Risk mitigation. Risk mitigation. Incremental implementations provide Incremental implementations provide measurement (and exit) points minimizing risks.measurement (and exit) points minimizing risks.

Service-Based Architecture Overview

The The ““ realreal --timetime ”” enterprise requires services that can enterprise requires services that can be recomposed, repurposed and delivered to any be recomposed, repurposed and delivered to any device, anytime from anywhere.device, anytime from anywhere.


INTELLIgentSYStems


8

IT Implications Of An SBA / SOA

�� Adaptability. Adaptability. IT solutions need to be more adaptable to a IT solutions need to be more adaptable to a changing landscape of standards, technologies, produc ts and changing landscape of standards, technologies, produc ts and service producers and consumers.service producers and consumers.

�� Past, current & future technology investments need Past, current & future technology investments need to leveraged. to leveraged. Rewrites or replacement of legacy systems Rewrites or replacement of legacy systems (i.e., anything in production) is deemed as too expe nsive.(i.e., anything in production) is deemed as too expe nsive.

�� Multiple levels of Integration. Multiple levels of Integration. Integration points must be Integration points must be at the data, application logic and business process levels.at the data, application logic and business process levels.

�� Security. Security. Ensure that the Ensure that the rightright people are participating in the people are participating in the rightright process and accessing process and accessing rightright data.data.



INTELLIgentSYStems


9

What Is A Service …Service-Based Architecture Overview

A Service Is An Application And / Or Function That:A Service Is An Application And / Or Function That:

�� Is Designed To Receive Requests From Any Source Is Designed To Receive Requests From Any Source –– Making No Making No Assumptions As To The Functional Correctness (Synta ctic Or Assumptions As To The Functional Correctness (Synta ctic Or Semantic) Of An Incoming RequestSemantic) Of An Incoming Request

�� Within Each Request, Encompasses A Complete & Indep endent Within Each Request, Encompasses A Complete & Indep endent Unit Of Work (Business Or Technical)Unit Of Work (Business Or Technical)

�� May Stand On ItMay Stand On It ’’s Own Or Be Part Of A Larger Set Of Functions s Own Or Be Part Of A Larger Set Of Functions That Constitute A Larger Service; But ItThat Constitute A Larger Service; But It ’’s Scope Is Such That s Scope Is Such That Each Request Leaves The System In A Long Term Stead y StateEach Request Leaves The System In A Long Term Stead y State

�� Is Designed For And Provides For A NetworkIs Designed For And Provides For A Network --Accessible InterfaceAccessible Interface

�� Keep Units Of Work Together That Change Together (H igh Keep Units Of Work Together That Change Together (H igh Cohesion) & Build Separation Between Units That Cha nge Cohesion) & Build Separation Between Units That Cha nge Independently (Low Coupling).Independently (Low Coupling).


INTELLIgentSYStems


10

Key SOA Characteristics #1

Services as published contracts between service provide rs / Services as published contracts between service provide rs / consumers. This impliesconsumers. This implies

�� WellWell--defined, computer defined, computer ““understandableunderstandable”” producer/consumer responsibilities producer/consumer responsibilities and interfacesand interfaces

�� Services descriptions that permit automated discovery and invocaServices descriptions that permit automated discovery and invocationtion�� Loose coupling and high cohesion. Services do one thing and one Loose coupling and high cohesion. Services do one thing and one thing well thing well

and should have minimal interdependencies. Hard to produce and should have minimal interdependencies. Hard to produce --> Hard to > Hard to consume!consume!

�� A service, like a contract, once published and consumed is very A service, like a contract, once published and consumed is very difficult to difficult to amend amend –– a challenge that architects need to address.a challenge that architects need to address.

Standards based. ConsiderStandards based. Consider�� Implementation technologies (.NET, J2EE, Web Services, XML) Implementation technologies (.NET, J2EE, Web Services, XML) �� Business facing standards (OASIS, Rosetta, Business facing standards (OASIS, Rosetta, ebXMLebXML, etc), etc)

ProcessProcess --oriented, potentially longoriented, potentially long --runningrunning�� Many services are actually front ends to discrete business proceMany services are actually front ends to discrete business processes.sses.�� Services, specially, those in the B2Bi space may take hours or dServices, specially, those in the B2Bi space may take hours or days to ays to

complete.complete.Secure, trustworthySecure, trustworthy

�� Services should be calibrated to the consumerServices should be calibrated to the consumer’’s identity and authorizations.s identity and authorizations.�� Services should provide guarantees certain guarantees (guaranteeServices should provide guarantees certain guarantees (guaranteed delivery, d delivery,

guaranteed action, nonguaranteed action, non--repudiation, etc)repudiation, etc)



INTELLIgentSYStems


11

Key SOA Characteristics #2Service-Based Architecture Overview

A SBA Consists Of:A SBA Consists Of:

�� Data Cleansed Data Cleansed FoundationFoundation

�� Integrated MetadataIntegrated Metadata

�� Defined Units Of Defined Units Of WorkWork

�� Standards BasedStandards Based

�� Shared ServicesShared Services

�� Units Of WorkUnits Of Work

�� Loosely CoupledLoosely Coupled

�� Federated ControlFederated Control

12

SBA Best Practices (Learned)

““ Best practices are those strategies, activities, or Best practices are those strategies, activities, or approaches which have been shown through approaches which have been shown through implementation / evaluation or industry usage to be implementation / evaluation or industry usage to be effective at managing issues thru use of these effective at managing issues thru use of these practices, or proactively limiting their effect"practices, or proactively limiting their effect"


INTELLIgentSYStems


13

Organizational Challenge

CEO

CFO CIO COOBusiness Line

Presidents

Finance Legal BusinessLine #1

BusinessLine #nIT Marketing

CompetitorInformation Contracts

StoresSales

Claims Customers

LogisticsInventory Inventory

Vendors

Business Business Issues:Issues:�� Multiple, Multiple,

Views & Views & Definitions Definitions Of DataOf Data

�� Inconsistent Inconsistent SemanticsSemantics

�� No Single No Single Dictionary / Dictionary / MetadataMetadata

�� Businesses Businesses Are StoveAre Stove --Piped In The Piped In The Organization, Organization, Why Should Why Should The Data Be The Data Be Different?Different?

�� ““ I Own That I Own That DataData””



INTELLIgentSYStems


14

Organizational Data Solution

Sales Data

External Data

Vendor Data

Product Data

Customer Data

CEO

CFO CIO COO Business LinePresidents

Finance LegalBusinessLine #1

BusinessLine #nIT Marketing

Operational Data Store CorporateData

Warehouse

CorporateMetadata

Exisiting TDSSystem

Exisiting TDSSystem

Exisiting TDSSystem

Exisiting TDSSystem

Exisiting TDSSystem

Web Solutions

Supply Chain /CRM / Add-Ons

"Single Version Of The Truth"

Business Business Solutions:Solutions:�� Single View & Single View &

Definition Of Definition Of Metadata Metadata (Single Version (Single Version Of Truth)Of Truth)

�� Semantics Semantics From Single From Single DictionaryDictionary

�� Data Data Warehousing Warehousing And Other And Other Application Application Additions Are Additions Are Easily Added / Easily Added / Connected Connected Since Metadata Since Metadata DefinedDefined



INTELLIgentSYStems


15

Metadata As A Glue

�� Metadata Metadata promotes agreement on nature / content of datapromotes agreement on nature / content of data�� Comprehensive metadata capabilities Comprehensive metadata capabilities for all tools in the for all tools in the

integration environment is required (modeling, IDE, etc .)integration environment is required (modeling, IDE, etc .)�� A Single dictionary A Single dictionary fosters common understanding and clear fosters common understanding and clear

communications; communications; two or more breed confusiontwo or more breed confusion�� A centralized metadata management architecture, A centralized metadata management architecture, allows users allows users

to assess the downstream impact of change, before th e to assess the downstream impact of change, before th e changes are made (Crosschanges are made (Cross --Impact Analysis)Impact Analysis)

�� This importance is best conveyed though example: This importance is best conveyed though example: responsibilities for taking data from acquisition, th ough responsibilities for taking data from acquisition, th ough integration, to analysis and to delivery to business users may integration, to analysis and to delivery to business users may rest with multiple, and sometimes disconnected groups . rest with multiple, and sometimes disconnected groups . The The data however is not disconnected.data however is not disconnected. If one group makes a If one group makes a change to a data model / process, that change will h ave a change to a data model / process, that change will h ave a ripple effect throughout the entire data lifecycleripple effect throughout the entire data lifecycle



INTELLIgentSYStems


16

Data Integration Issues

Integration Of Enterprise Into An SBA / SOA Integration Of Enterprise Into An SBA / SOA Requires That Foundation Supply Data:Requires That Foundation Supply Data:

�� Consistency (Semantic, etc.)Consistency (Semantic, etc.)

�� Quality (Element, Groupings, etc.)Quality (Element, Groupings, etc.)

�� Cleanliness (Nulls, Bad Records)Cleanliness (Nulls, Bad Records)

�� Integrity (Referential Integrity, etc.)Integrity (Referential Integrity, etc.)

�� Standards Based (XML, SOAP, J2EE, etc.)Standards Based (XML, SOAP, J2EE, etc.)

�� Lowest Level Of Granularity (Elements, Rows, ...)Lowest Level Of Granularity (Elements, Rows, ...)

�� Timely Delivery At A Reasonable Cost !Timely Delivery At A Reasonable Cost !



INTELLIgentSYStems


17

Semantic Consistency

Semantic Consistency Semantic Consistency Applies To Most Applies To Most Artifacts:Artifacts:��MetadataMetadata

��Domain ObjectsDomain Objects

��Promotes Business Promotes Business Understanding That Understanding That The Underlying Data The Underlying Data Is The Same, View Is Is The Same, View Is DifferentDifferent

��Promotes A Common Promotes A Common Language For Language For Developers When Developers When Combined With A Combined With A Common DictionaryCommon Dictionary

Semantic Consistency

Business Domain

Domain Objects

Metadata

yields

Executables

Database Tables

Messages, Transactions

Implemented as

XML Schemas, DTDs

Service Descriptions

Dictionaries

Taxonomies

Takes the form of



INTELLIgentSYStems


18

Promoting Common Understanding

Integration Of Enterprise Into An SBA / SOA Integration Of Enterprise Into An SBA / SOA Requires That Foundation Provide Data In A Requires That Foundation Provide Data In A Common Form / Model:Common Form / Model:

�� High Level UML ModelHigh Level UML Model

��Business Process With Data Integration PointsBusiness Process With Data Integration Points

�� Mid Level Data ModelMid Level Data Model

��Logical Problem Space With Connectors, KeysLogical Problem Space With Connectors, Keys

�� Low Level Data ModelLow Level Data Model

��Physical Characteristics (Block Size, Rows, Physical Characteristics (Block Size, Rows, ……))



INTELLIgentSYStems


19

Iterative Development

Iterative Development Requires:Iterative Development Requires:

�� Business Process Model ProvidesBusiness Process Model Provides

��Stability To IterationsStability To Iterations

��Defines Subject Area And InterconnectionsDefines Subject Area And Interconnections

�� Logical Data Model ProvidesLogical Data Model Provides

��Hooks And InterHooks And Inter --System CommonalitySystem Commonality

��Allows For Patterns And ReAllows For Patterns And Re --UseUse



INTELLIgentSYStems


20

Best Practices Observed #1

Phased Development Approach:Phased Development Approach:Phase IPhase I –– Centralize, Document & Cleanse Data Centralize, Document & Cleanse Data -- ODSODS

Phase IIPhase II –– Data Layer IntegrationData Layer Integration

Phase IIIPhase III –– Integrated Data PresentationIntegrated Data Presentation

Phase IVPhase IV –– Component IntegrationComponent Integration

�� Publish & Subscribe Publish & Subscribe –– Centralized DBCentralized DB ’’s (Customer)s (Customer)

�� Real Time MessagingReal Time Messaging

Phase VPhase V –– Enterprise Applications Integration Enterprise Applications Integration



INTELLIgentSYStems


21


1.1. Define the longDefine the long --term vision in the form of an enterprise term vision in the form of an enterprise integration architecture that utilizes standardsintegration architecture that utilizes standards --based tools based tools that are proven to work togetherthat are proven to work together

2.2. Always start with a Common Data Model [CDM] that in tegrates Always start with a Common Data Model [CDM] that in tegrates the data between systems (ODS, CDW, etc.)the data between systems (ODS, CDW, etc.)

3.3. Create a hubCreate a hub --andand --spoke architecture using a data integration spoke architecture using a data integration platform (ETL or Integration Broker) to access data sourc es platform (ETL or Integration Broker) to access data sourc es and then populate the ODS and the CDW. Buy, not buil d and then populate the ODS and the CDW. Buy, not buil d components. Todaycomponents. Today ’’s tools integrate at the metadata level s tools integrate at the metadata level with powerful connectors and parallel interwith powerful connectors and parallel inter --operabilityoperability

4.4. Build bottomBuild bottom --up and not topup and not top --down for minimization of riskdown for minimization of risk5.5. Services must be designed with a business process con text. Services must be designed with a business process con text.

Therefore, start with business document context to de fine.Therefore, start with business document context to de fine.



INTELLIgentSYStems


22


6.6. Avoid building or buying any solution that does not integrate Avoid building or buying any solution that does not integrate at the metadata level with a central metadata reposit ory. The at the metadata level with a central metadata reposit ory. The metadata can be generated, maintained and reported on by metadata can be generated, maintained and reported on by the data integration platform (either ETL or Integrati on the data integration platform (either ETL or Integrati on BrokerBroker ’’s provide this functionality).s provide this functionality).

7.7. Use a second generation data integration platform or on e that Use a second generation data integration platform or on e that supports both native interfaces into the source and target supports both native interfaces into the source and target databases (not ODBC) and one that support both batch and databases (not ODBC) and one that support both batch and messaging based transfers using todaymessaging based transfers using today ’’s industry accepted s industry accepted standards (J2EE, SOAP, etc.). standards (J2EE, SOAP, etc.).

8.8. When selecting XML standards, try to stay within you r When selecting XML standards, try to stay within you r corporations trade area (i.e.: Mortgage companies use corporations trade area (i.e.: Mortgage companies use MISMO, Retailers use EDIFACT, etc.)MISMO, Retailers use EDIFACT, etc.)



INTELLIgentSYStems


23


9.9. Integrate ALL components of the architecture using th e Integrate ALL components of the architecture using th e metadata including:metadata including:

* Modeling, Application Development, Reporting, OLAP, etc. should all connect to the repository* Use the data integration platform to both generate an d maintain central metadata / change control records* Generate extensible, XML-compliant metadata and LDAP-compliant directories (as a minimum)* Derive ALL of the local metadata using the central metadata as the source of record.

10.10. Integrate different or multiple, networked, data integra tion Integrate different or multiple, networked, data integra tion platform engines with a common global metadata. BEW ARE platform engines with a common global metadata. BEW ARE OF MULTIPLE VERSIONS OF THE TRUTH!OF MULTIPLE VERSIONS OF THE TRUTH!



INTELLIgentSYStems


24


11.11. Use the realUse the real --time features of the data integration platform to time features of the data integration platform to populate realpopulate real --time data into the ODStime data into the ODS

12.12. Do not load dirty source data into the ODS. Use the d ata Do not load dirty source data into the ODS. Use the d ata integration platformintegration platform ’’s cleansing function to resolve logical s cleansing function to resolve logical inconsistencies and provide for referential data integ rity.inconsistencies and provide for referential data integ rity.

13.13. Limit sources with access to the ODS, this will elimi nate Limit sources with access to the ODS, this will elimi nate contention issues in the foundation.contention issues in the foundation.* * Deliver the information from the metadata tools usin g the existing portal to the user community* Use a business intelligence oriented corporate portal as the gateway to both structured and unstructured informati on* It is also easy to add data warehousing functional ity through this interface



INTELLIgentSYStems


25


14.14. Anticipate scalability and performance issues by ensu ring Anticipate scalability and performance issues by ensu ring that bottlethat bottle --necks are avoidednecks are avoided* * Parallel queries* Distributed Tables* Indices On Key Fields* ODS Optimized For Speed Of Write (Third Normal Form)* DW Optimized For Speed Of Read (Star Schema, …)

~ Pre-built Aggregates For Common Queries~ In-Line Aggregation During Loading Process

15. Use a staging area, it will help in debugging loadin g issues, and will provide a starting point from the same machi ne removing network issues.



INTELLIgentSYStems


26

Data Cleansing And Integration

�� What Is Cleansed Data And Why Is It What Is Cleansed Data And Why Is It ““ DirtyDirty ”” ??�� Cleansed data refers to the transformation of data so that each Cleansed data refers to the transformation of data so that each

disparate system inputs itdisparate system inputs it ’’s information into the ODS with the s information into the ODS with the appropriate values & correct formatappropriate values & correct format (PC: data is integrity challenged).(PC: data is integrity challenged).

�� ““ DirtyDirty ”” refers to data where relationships and/or business rules are refers to data where relationships and/or business rules are lacking or are inconsistently applied.lacking or are inconsistently applied.

�� What Does This Mean To I.S. Developers?What Does This Mean To I.S. Developers?�� Part of the cleansing process to understand the way the data is Part of the cleansing process to understand the way the data is

currently captured in its currently captured in its ““ nativenative ”” or legacy system.or legacy system.�� Second, is to understand the format of the Second, is to understand the format of the ““ newnew ”” model and transform model and transform

the data appropriately (data vs. transaction).the data appropriately (data vs. transaction).�� Next, business rules are used to govern data behavi or.Next, business rules are used to govern data behavi or.

�� Finally, the data must be validated in itFinally, the data must be validated in it ’’s new form against the s new form against the operational, legacy system for accuracy and busines s rules (someoperational, legacy system for accuracy and busines s rules (somebusiness rules may not exist in legacy systems).business rules may not exist in legacy systems).



INTELLIgentSYStems


27

Data Modeling Best Practices

�� Definition Of CDM?Definition Of CDM?�� A Common Data Model [CDM] is a set of standard fiel d names, A Common Data Model [CDM] is a set of standard fiel d names,

descriptions and usage across the different divisio ns & systems descriptions and usage across the different divisio ns & systems organized around business processes.organized around business processes.

�� A resultant output of the CDM is that it allows for the integratA resultant output of the CDM is that it allows for the integrat ed ed business processes.business processes.

�� Implementation Considerations?Implementation Considerations?�� First, a model must be built to represent the busin ess process mFirst, a model must be built to represent the busin ess process m odel odel

including the including the ““ actorsactors ”” , data and activities., data and activities.�� Second, identify the core entities, their relations hips and the Second, identify the core entities, their relations hips and the unifying unifying

fibers and repeating patterns across the whole mode l.fibers and repeating patterns across the whole mode l.�� Third, the representations used by business areas m ust be validaThird, the representations used by business areas m ust be valida ted ted

to ensure accuracy (checks with CFO, divisions, dep artments, etcto ensure accuracy (checks with CFO, divisions, dep artments, etc ).).�� Fourth, the CDM must be logically modeled and norma lized.Fourth, the CDM must be logically modeled and norma lized.�� Fifth, the CDM must be physically modeled & impleme nted.Fifth, the CDM must be physically modeled & impleme nted.


28

SBA Case Studies

Polo Ralph LaurenPolo Ralph LaurenAndAnd

The NPD GroupThe NPD Group


INTELLIgentSYStems


29

Data Integration Challenge

Wholesale

OLTP

System

WEB /eCommerce

System

Licensing

OLTP

System FinancialSystem

[AP/AR/GL]

Business

Planning

System

Warehouse

System

Shipping

System

Data

Warehouse

Starting Point Starting Point Issues:Issues:�� For Every For Every

Added Added System, Must System, Must Code ( N Code ( N -- 1 ) 1 ) InterfacesInterfaces

�� Multiple, Multiple, Disparate Disparate Copies Of Copies Of DataData

�� Inconsistent Inconsistent SemanticsSemantics

�� No Single No Single Dictionary / Dictionary / MetadataMetadata

�� Multiple Multiple Security Security MechanismsMechanisms

SBA Case Studies – Polo & NPD Group


INTELLIgentSYStems


30

Data Layer Integration Solution

WholesaleWholesaleOLTPOLTP

SystemSystem

Web /Web /eCommerceeCommerce

SystemSystem

LicensingLicensingOLTPOLTP

SystemSystem

FinancialFinancialSystemSystem

[AP/AR/GL][AP/AR/GL]

BusinessBusinessPlanningPlanningSystemSystem

WarehouseWarehouseSystemSystem

ShippingShippingSystemSystem

DataDataWarehouseWarehouse

Centralized ODSCentralized ODS

Common Data Model

= Wrapper-Facade

Asynchronous Asynchronous communicationscommunications

Transactional Transactional System IntegrationSystem Integration

Async. comm. Async. comm. via a messagevia a message --oriented oriented middleware middleware [MOM] added [MOM] added the following the following capabilities:capabilities:�� Just in time Just in time

propagation propagation of business of business eventsevents

�� Trickle in Trickle in data data integrationintegration



INTELLIgentSYStems


31

Polo Ralph Lauren and NPD GroupSBA Case Studies – Polo & NPD Group

Security and Identity Management

Employee Partner Business Customer

Services

Users and Devices

Resources,

Existing Systems

Java ClientPortlets .NETServletsJSP



INTELLIgentSYStems


32

Polo Ralph Lauren Corporation

Administration &Runtime

MonitoringEnvironment

DevelopmentEnvironment

Runtime Environment

Testing Environment

Security Environment

Polo Realitiesand


Constrain

Constrain

�� MultiMulti --Faceted Faceted ArchitectureArchitecture�� Development & Development &

RunRun --Time Time EnvironmentEnvironment

�� Testing Testing EnvironmentEnvironment

�� Scalable Features Scalable Features (Rapid Growth)(Rapid Growth)

�� Administration & Administration & Monitoring Of Monitoring Of EventsEvents



INTELLIgentSYStems


33

Polo Ralph Lauren Corporation

�� Stylized Data Flow Diagram Stylized Data Flow Diagram (DFD) Denotes Informatica(DFD) Denotes Informatica ’’s s Power Center As The Hub.Power Center As The Hub.

�� Power Center Pumps Data To Power Center Pumps Data To And From Spokes (Polo And From Spokes (Polo Legacy Systems, Data Legacy Systems, Data Warehouses, Etc.)Warehouses, Etc.)

�� It Automatically Generates It Automatically Generates The Metadata And Stores The Metadata And Stores Consistent With The Sterling Consistent With The Sterling KnowledgeBase.KnowledgeBase.

�� It Has BiIt Has Bi --Directional Links To Directional Links To Erwin, PoloErwin, Polo ’’s Physical Data s Physical Data Modeling ToolModeling Tool

�� Power Center was entirely Power Center was entirely confined to the Data Layerconfined to the Data Layer

IA/IQ's Reporting Tools

Data Integration Layer

Informatica's Power CenterExtraction, Transformation and Load Tool

Polo Legacy Applications

PCIS is Men'sWholesale's home

grown "ERP"application

PCIS to ODS

PCISRetail

Island Pacific isRetail's off-the-shelf

grown "ERP"application

IP to ODS

Retail Sales(DB2/400 Files)

Wholesale Sales(DB2/400 Files)

Polo ODS

i2's Active DataWarehouse feedsi2's SCM package

ODS to i2 ADW

i2 ADW

Informatica'sDynamic DataStore (DDS)

Metadata Services

? ? ?

ODS to Polo DW

Polo DataWarehouse

?

IQ KnowledgeBase

IQ Objects My Eureka

SBA Stylized Data Layer Flow



INTELLIgentSYStems


34

Integrated Environments

CAPABILITIES:CAPABILITIES:

�� Development Development

EnvironmentEnvironment

�� Runtime Runtime


�� Administrate Administrate


�� Best Best

PracticesPractices

�� Purchase Purchase

Before Build Before Build

(Where (Where

Possible)Possible)

Development Run Time Administration Best PracticesDevelopmentDevelopment Run TimeRun Time AdministrationAdministration Best PracticesBest Practices

Code:Jbuilder, SynerJ

RPG CLDreamweaver

Code:Code:Jbuilder, SynerJJbuilder, SynerJ

RPG CLRPG CLDreamweaverDreamweaver

Design:Rational Rose

Visio Professional

Design:Design:Rational RoseRational Rose

Visio ProfessionalVisio Professional

Data Model:ERwin

Data Model:Data Model:ERwinERwin

Requirements:Rational Rose

Visio ProfessionalMS Word

Requirements:Requirements:Rational RoseRational Rose

Visio ProfessionalVisio ProfessionalMS WordMS Word

LEGACYDEVELOPMENT

LEGACYLEGACYDEVELOPMENTDEVELOPMENT

Common Bus.Components:

EJB’s

Common Bus.Common Bus.Components:Components:

EJBEJB ’’ss

Wrappers:Native Compiler

Wrappers:Wrappers:Native CompilerNative Compiler

ApplicationServers:

Forte’ iPlanetBEA Weblogics

ApplicationApplicationServers:Servers:

ForteForte ’’ iPlanetiPlanetBEA WeblogicsBEA Weblogics

Messaging:CORBA

BEA Tuxedo

Messaging:Messaging:CORBACORBA

BEA TuxedoBEA Tuxedo

Databases:ORACLEDB2/400

Databases:Databases:ORACLEORACLEDB2/400DB2/400

DevelopmentMethodology:Rational Unified

Process

DevelopmentDevelopmentMethodology:Methodology:Rational UnifiedRational Unified

ProcessProcess

QAUser Signoff

QAQAUser SignoffUser Signoff

Testing:Winrunner

Alpha & Beta

Testing:Testing:WinrunnerWinrunner

Alpha & BetaAlpha & Beta

Documentation:Component

Oriented Design,Implementation

Guide

Documentation:Documentation:ComponentComponent

Oriented Design,Oriented Design,ImplementationImplementation

GuideGuide

Standards:Transactions, DB

Standards:Standards:Transactions, DBTransactions, DB


Consoles:iPlanet Console

WebLogics ConsoleBEA Manager

Consoles:Consoles:iPlanet ConsoleiPlanet Console

WebLogics ConsoleWebLogics ConsoleBEA ManagerBEA Manager

Operations:

HP OpenView

Operations:Operations:

HP OpenViewHP OpenView

Data Movement:

InformaticaPowerCenter

Data Movement:Data Movement:

InformaticaInformaticaPowerCenterPowerCenter

35

SBA Case Studies

Retail Retail –– Executive DashboardExecutive DashboardGlobal Bank Global Bank –– Corporate BankingCorporate Banking

Leasing Company Leasing Company –– Lease SyndicationLease SyndicationConglomerate Conglomerate –– ProvisioningProvisioning


INTELLIgentSYStems


36


Services

ForecastsOrdersCustomerDocs/ContentInventoryProducts

Retail - Executive Dashboard

Inventory

StatusProducts Forecasts

Order

Management

Planning

Engines

Customer

ManagementWarehouse

ManagementDocs/Content

Product Design

& Development

Resources, Existing Systems

Executive

Customer

DataOrders

SBA Case Studies –Retail, Bank, Leasing & Provisioning


INTELLIgentSYStems


37


Global Bank - Service and Content Aggregation

Services

Users and Devices


Employee Partner BusinessCustomer

Employee

Resources,

Existing Systems




INTELLIgentSYStems


38

Leasing Company - Business Process Integration

Services

Users and Devices


Lease Applicant

Employee Investor Vendor / Supplier

Resources,

Existing Systems





INTELLIgentSYStems


39

J2EE Application Server

Identity

NetworkeDirectory

Active Directory

TransactionsOracle, MySQL

exteNd

PackagesPeopleSoft, Baan

PortalexteNd

CollaborationGroupWise, Notes

Workflow / BPMexteNd

ID Synchronization

Sync. Sync. Sync.Sync.

Sync.

Sync.

Firewall

Metadirectory

Web Simplified Sign-on

Resource/Services Simplified Sign-on

Identity

Web Applications Web Applications

Content Management

Workflow Search

Portal

Rules

Personalization

Web

ID Synchronization

Identity

Business Process

Integration

Web Services

Identity

Web Services IntegrationWeb Services Integration

Forecast Track HistoryPurchasingInventory

Conglomerate - Provisioning and Identity Management



INTELLIgentSYStems


40

Runtime Environment

Red Hat Linux AS Red Hat Linux AS 2.1 on Intel2.1 on Intel

AIX on IBM HW, AIX on IBM HW, Windows 2000 on Windows 2000 on Intel, MainframeIntel, Mainframe

Windows 2000 on Windows 2000 on IntelIntel

Solaris 2.6 on Sun Solaris 2.6 on Sun HWHW

Hardware / OSHardware / OS

PeopleSoft, Baan, PeopleSoft, Baan, AD, eDirectoryAD, eDirectory

Mainframe Apps, Mainframe Apps, C++ AppsC++ Apps

AS400 Apps, VB AS400 Apps, VB AppsApps

AS400 Apps, I2 AS400 Apps, I2 Planning EnginesPlanning Engines

Backend SystemsBackend Systems

Oracle 9i, Oracle 9i, MySQLMySQLOracle 9iOracle 9iSQL Server 2000SQL Server 2000Oracle 8iOracle 8iDatabaseDatabase

exteNdexteNd JBrokerJBrokerMQ SeriesMQ SeriesMSMQ, MQ MSMQ, MQ SeriesSeries

Sun (Forte) Fusion, Sun (Forte) Fusion, PowerMartPowerMart

MiddlewareMiddleware

exteNdexteNd App ServerApp ServerOracle 9i App Oracle 9i App ServerServer

COM+COM+Sun (Forte) App Sun (Forte) App ServerServer

Application ServerApplication Server

exteNdexteNd Composer Composer BPMBPM

NoneNoneBizTalkBizTalkSun (Forte) FusionSun (Forte) FusionProcess Process IntegrationIntegration

exteNdexteNd DirectorDirectorexteNdexteNd DirectorDirectorIISIISEureka PortalEureka PortalPresentationPresentation

Conglomerate Conglomerate (Identity Mgmt & (Identity Mgmt & ProvisioningProvisioning

Global Bank Global Bank (Corporate (Corporate Banking)Banking)

Leasing Leasing Company (Lease Company (Lease Syndication)Syndication)

Polo (Executive Polo (Executive Dashboard)Dashboard)



INTELLIgentSYStems


41

Design & Development Environment

Custom developedCustom developedCustom developedCustom developedCustom developedCustom developedCustom developedCustom developedUnit Testing Unit Testing ToolTool



Windows 2000 on IntelWindows 2000 on IntelWindows 2000 on Windows 2000 on IntelIntel

Developer Developer PlatformPlatform

CVSCVSPVCSPVCSMS Visual SourceSafeMS Visual SourceSafePVCSPVCSVersion Version ControlControl

JEditJEdit, XML Spy, , XML Spy, exteNdexteNd Workbench, Workbench, Composer Composer DesignerDesigner

JDeveloperJDeveloper, , InteliJInteliJIDEA, IDEA, SunONESunONE(Forte for Java)(Forte for Java)

MS Visual Studio, XML MS Visual Studio, XML SpySpy

JBuilderJBuilder, Visual , Visual SlickEditSlickEdit

IDEIDE

Enterprise Enterprise Architect, Architect, exteNdexteNdComposer BPM Composer BPM DesignerDesigner

Together Together ControlCenterControlCenter, , ErwinErwin

Rational Rose, Erwin, Rational Rose, Erwin, XML Spy, BizTalk XML Spy, BizTalk Orchestration DesignerOrchestration Designer

Rational Rose, Erwin, Rational Rose, Erwin, Sun (Forte) Fusion Sun (Forte) Fusion Process DesignerProcess Designer

Design ToolDesign Tool

Conglomerate Conglomerate (Identity Mgmt & (Identity Mgmt & Provisioning)Provisioning)

Global Bank Global Bank (Corporate (Corporate Banking)Banking)

Leasing Company Leasing Company (Lease Syndication)(Lease Syndication)

Polo (Executive Polo (Executive Dashboard)Dashboard)



INTELLIgentSYStems


42

Key Patterns Used

�� Layers, Service LayerLayers, Service Layer�� HalfHalf --Sync, HalfSync, Half --AsyncAsync�� Domain ObjectsDomain Objects�� Wrapper FaWrapper Fa ççadeade�� ObserverObserver�� MediatorMediator�� Abstract FactoryAbstract Factory�� Composite, CompositionComposite, Composition



INTELLIgentSYStems


43

Layers, Service Layers

Business ProcessBusiness Process

ApplicationApplication

DataData

Data

Layer

(sam

ple

Com

ponents

)

Business Process Layer (sample activities)

Applic

ation L

ayer

(sam

ple

Com

ponents

)

Get Request Data

approval not required

Perform Unit of Work #2


Get Approval Apply Ruleapproval required

Approver

invoke

output

input

Rule Processor

Database(ODS, Data Warehouse)

XML Document

Data

Serv

ice

Layer

Request Data Transfer Object

Approval Request Entry

invoke

invoke

Data Set Rule

delegate some work

Domain Object Data Transfer

Object

Domain Object

Data Mart

Data Mart

XML Document

Identity

invoke invoke



INTELLIgentSYStems


44

Half-Sync, Half-Async

Data

Layer

(sam

ple

Com

ponents

)

Business Process Layer (sample activities)

Applic

ation L

ayer

(sam

ple

Com

ponents

)

Get Request Data

approval not required



Get Approval Apply Ruleasync

Approver

sync

output

input

Rule Processor

Database(ODS, Data Warehouse)

XML Document

Data

Serv

ice

Layer

Request Data Transfer Object

Approval Request Entry

synch

sync

Data Set RuleDomain Object Data Transfer

Object

Domain Object

Data Mart

Data Mart

XML Document

sync async

sync sync

Mix of synchronous Mix of synchronous (blocking calls) and (blocking calls) and asynchronous (non asynchronous (non blocking call)blocking call)

Sync and Sync and asyncasync mix mix is usually required in is usually required in long running long running interactions. interactions. Examples:Examples:�� Lease application Lease application

processingprocessing�� Wire transferWire transfer�� ProvisioningProvisioning�� Approval Approval

processingprocessing

Identity



INTELLIgentSYStems


45

Domain Objects

Leasing Company (Lease Leasing Company (Lease Syndication)Syndication)�� Lease Application, Equipment, Lease Application, Equipment,

Offer, Investor, Customer, Offer, Investor, Customer, Salability RuleSalability Rule

Global Bank (Corporate Banking)Global Bank (Corporate Banking)�� Account, Account Portfolio, Bank Account, Account Portfolio, Bank

Statement, Transaction HistoryStatement, Transaction HistoryConglomerate (Identity Conglomerate (Identity Management & Provisioning)Management & Provisioning)�� Identity, Provision, Provisioning Identity, Provision, Provisioning

Task, Event Task, Event

Multiple implementations: Classes, Multiple implementations: Classes, XML Schemas/documents, Data XML Schemas/documents, Data Transfer Objects, TablesTransfer Objects, Tables



INTELLIgentSYStems


46

Adapter, Wrapper Facade

Resources,

Existing Systems

Databases PackagesCustom AppsProcesses Docs/ContentDirectories

Services��““ WrapWrap ”” , , repurpose, repurpose, integrate integrate existing systemsexisting systems

��Expose Expose backend backend functionality as functionality as Web ServicesWeb Services

Services



INTELLIgentSYStems


47

Wrapper Façade – Global Bank

�� AccountPortfolioAccountPortfolio and and AccountPortfolioServiceAccountPortfolioService are are part of a Wrapper Fapart of a Wrapper Fa ççade ade that aggregates banking that aggregates banking services to a customer services to a customer portalportal

�� Some FaSome Fa ççades evolved into ades evolved into Domain ObjectsDomain Objects

�� Backend Wrappers and Backend Wrappers and Object Adapters Object Adapters homogenize access to homogenize access to various backvarious back --end systemsend systems

�� Enabled incremental roll out Enabled incremental roll out of servicesof services



INTELLIgentSYStems


48

Observer, Mediator, Abstract Factory

�� Initiate processing Initiate processing

based on business based on business

eventsevents

�� Abstract Factory and Abstract Factory and

Mediator simplify the Mediator simplify the

ObserverObserver ’’s s

responsibility to that of responsibility to that of

a reporter of eventsa reporter of events



INTELLIgentSYStems


49

Observer, Mediator, Abstract Factory

�� Leasing Company, Conglomerate Observers and MediatorsLeasing Company, Conglomerate Observers and Mediators�� Observer (Polo, Leasing Company, Global Bank, Conglomera te)Observer (Polo, Leasing Company, Global Bank, Conglomera te)

�� Implemented as message queue listeners (receivers)Implemented as message queue listeners (receivers)�� Messages in queue contain events or work requestsMessages in queue contain events or work requests�� Initiated business processes directlyInitiated business processes directly

�� Mediator (Global Bank)Mediator (Global Bank)�� Invoked by ObserverInvoked by Observer�� Orchestrated delegated tasks in the absence of a pr ocess managerOrchestrated delegated tasks in the absence of a pr ocess manager

(e.g., wire transfer)(e.g., wire transfer)

�� Abstract Factory (Leasing Company, Conglomerate)Abstract Factory (Leasing Company, Conglomerate)�� Invoked by observerInvoked by observer�� Apply rules before instantiating a process, e.g., Apply rules before instantiating a process, e.g.,

�� Determine suitability of Lease Application for synd icationDetermine suitability of Lease Application for synd ication�� Determine availability of Provision for a given Ide ntityDetermine availability of Provision for a given Ide ntity



INTELLIgentSYStems


50

Composite, Composition

NOTE: Composition is not a design patternNOTE: Composition is not a design pattern

�� More of a design approach: favor More of a design approach: favor composition over inheritancecomposition over inheritance

�� Differences (at the risk of oversimplifying): Differences (at the risk of oversimplifying): �� Composite aggregates entities with a Composite aggregates entities with a

common interface or supercommon interface or super --classclass�� Composition aggregates dissimilar Composition aggregates dissimilar

entitiesentities

Common usage Common usage �� PartPart --whole situationswhole situations�� XML Schema XML Schema –– simpleTypes, simpleTypes,

complexTypes as building blockscomplexTypes as building blocks�� Data Transfer Object and Domain Object Data Transfer Object and Domain Object

build upon finer grained objects.build upon finer grained objects.ExamplesExamples�� Transactions, Balances, StatementTransactions, Balances, Statement�� Lease Application, Lessor, Lessee, Lease Lease Application, Lessor, Lessee, Lease

ObjectObject�� Provision, Provisioning Task, Provisioner, Provision, Provisioning Task, Provisioner,

IdentityIdentity�� Shirt, collar, sleeve, trim, trim setShirt, collar, sleeve, trim, trim set



INTELLIgentSYStems


51

Composite, Composition

Provisioning and Lease Syndication: XML elements an d complexTypeProvisioning and Lease Syndication: XML elements an d complexType s s consistent with Domain Objects, Data Transfer Objec ts and databaconsistent with Domain Objects, Data Transfer Objec ts and databa se tablesse tables



INTELLIgentSYStems


52

Questions & AnswersOpen Discussion …



[email protected]@att.net [email protected]@novell.com

(201) 745(201) 745--76807680

INTELLIgentSYStems


53

Services Based ArchitecturesServices Based ArchitecturesDefining The Issues For Data DevelopersDefining The Issues For Data Developers



[email protected]@att.net [email protected]@novell.com

(201) 745(201) 745--76807680

INTELLIgentSYStems


Documents

Services Based Architectures - Defining The Issues For Data Professionals