SOA & Integration

13 décembre 2007Eric SAMSON, CTO - XCALIAStève SFARTZ, MICROSOFT France

The innovation cycle

Do « more »FeaturesQuality

Do « better »FasterCheaper

Co

mp

lexi

ty

Eff

icie

ncy

Fea

ture

s

Ease

Breakthrough

Standards

?

Cumulative innovation

1990 1995 2000 2005 2010 2015

Mainframe

Client-Server

J2EE

SOA

3

Innovation rarely replaces

but rather complements

older technologies!

Innovation & IT Architectures

In terms of IT architecture, innovation consists of being more flexible while not decreasing quality of service

Do more, faster, more cheaply, more easilyKeep scalability, security

Based on the recurrent notion of « decoupling »Identify and separate concerns, skills Introduce well-defined layers, tiers, with replaceable solutions

New layersIncrease flexibility, butDecrease agility as well (new interfaces to manage)Jeopardize « Quality of Service »

New interfaces

Network cost

1970: Mainframes

1-tier architectures, everything in a single placeComponent model: formsProcedural languagesHierarchical and network databasesNo networkGreen screens

Business

Logic

Business

Model

GUI

1990: Client-server

2-tiers architectures, the DB in a separate serverComponent model: tablesProcedural languages, 4GLRelational databasesThe downsizing

Unix, local networksWindows clients

Business

Logic

Business

Model

GUIWindows Unix

1993: Distributed systems

CORBA2*-tier architectures, distributed applicationsComponent model: distributed objectsObject languagesUnix, IIOP

Business

Logic

Business

Model

GUI

Business

Logic

Business

Model

GUI

1998: Application Servers

J2EEComponent model: EJB entities3-tier architectures, the application in a separate serverJava, then .NetInternet, XML, thin clients

Business

Logic

Business

ModelGUI

2005: Services

SOAComponent model: Services4-tier architectures, the application in 2 separate layersESB, WSDLInternet, rich clients

Business

Logic

Business

Model

GUI Services

SOA Emerges as New Foundation

Increase business agility and decrease IT costsRapidly respond to changing business needs

Reduce development effortReuse of business knowledge

Manage heterogenous information systems

Get the most of our legacy and mainframe assets

Easily scale IT infrastructure

Flexible application architecture

Enable incremental deploymentEase of maintenance

10

Services are Not New

Same architecture as used in typical mainframe environments

• Save mainframe resources

• Re-use / publish business logic

Common mistakes in architectures

Services won’t replace objects

Components and objects are different thingsComponents needs more than an architecture

Culture, repository, semantic metadata, brokers

The network bandwidthInternet is still slowIf services are just components over the Internet

Then SOA is already dead

How to reconcile SOA and integration?

Common mistakes around SOA

Forgetting dataThere is always a database behind a service

Transactions, security, scalabilityWe will never have 100% of data accessed only through services

Mix services and databasesUnified view of database and service required

Forgetting ScalabilityThousands of services (internal /external, legacy / new)

Metadata, Repository, Dynamic compositionTransactions

Forgetting FlexibilityBAM, optimize cost of execution AdaptabilityDo not hardcode in BPELDiscover new services

Forgetting HeterogeneityDifferent services technologies, not only Web ServicesNot only one service for a form

Composite applications, Complex M-N relationships between service consumers and providers

Migrating to SOAIntegration of databases & Data Services faces many challenges

Ease of development• Automatic mapping between

model and database• Transparent manipulation• Transparent navigation• High Performance• Caching• Dynamic Optimization

Pain ! Facing exactly the same issues as

with manual DB programming before O/RM

Encapsulation of database access via services

The real SOA layers

15

GUI, Presentation

Applications, Processes

Services

Data sources

GUI, Presentation

Applications, Processes

Services

Data sources

Xcalia Intermediation:

Transparent, unified mapping

Threats for SOA (1)

Heavy development (manual and tedious calls to services)

When thousands of data services have to be accessedPoor software design (regression to procedural coding)Hard-coded business processes (BPEL…)

Performance issuesDifficult tuning (services are black boxes)Gartner's hype curve

Tech Trigger inflatedExpectations

Disillusion Enlightment Productivity

maturity

visi

bilit

y

16

SOA

Threats for SOA (2)

See for instanceExploring the Dark Side of SOAs, Robbie Clark, Dec. 2004, http://www.ftponline.com/weblogicpro/2005_01/magazine/features/rclark/ Is SOA really just the same old architecture?, Ben Ellingston, Mar. 2005, http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1065615,00.html and http://www.theserverside.com/news/thread.tss?thread_id=32417 Is SOA Really The Future or Just So Much Hype?, Paul Ballard, Jan. 2005, http://theserverside.net/news/thread.tss?thread_id=31222 Should SOAs be developed top down or bottom up?, Nitin Bharti, Mars 2005, http://www.theserverside.com/news/thread.tss?thread_id=32807 Beyond SOA: Principles of Service Engineering, Mark Davydov, Aug. 2004, http://www.ftponline.com/javapro/2004_09/magazine/features/mdavydov/ Troubleshoot Your SOA, Robbie Clark, July 2004, http://www.ftponline.com/javapro/2004_07/magazine/features/rclark/ Avoid Dead-End SOAs, Franck Martinez, Sept. 2004, http://www.ftponline.com/ea/magazine/fall2004/columns/soainsights/ SOA: debunking 3 common myths, Tarak Modi, Sept. 2004, http://www.ftponline.com/ea/magazine/fall2004/features/tmodi/

What Is Required?

A new way to access and integrate heterogeneous data sources in a service-oriented way to compose new business applications

Key requirements:

Transactional capabilities and performance must be preserved

End-to-end mapping of all resources with the business layer

No impact into application source code

No impact on data sources

Metadata to describe services and how they interact at runtime

Dynamic composition of data services including transactional updates

Databases and data services resources are easily made available for reuse

Data access is configured, not coded, to save time and cost

18

Requirements for effective SOA

In order to be successful a move to SOA imposes Best Practices

Methodology

Agility is keyCo-existence and integration with previous architectures is importantContinuous move and architecture flexibility

New productsService Metadata standard and repository

Topology, cartography, taxonomy of SI

Orchestration, CompositionScalable and transactional heterogeneous data access / integration

Data Integration, the future of EIM*

O/R mapping is necessary but not sufficientSOA plays a fast-growing role

Disconnected mode is imposed by Web ServicesDynamic mapping (mapping of unknown models)Emerging SDO standard

Mainframes are still there for longService-oriented versus Database-oriented data sources

Production constraints are importantNeed for optimization featuresNeed for dedicated & configurable administration console

Enterprise Information ManagementNew tools to access Services are requiredIntegration of Heterogeneous Data SourcesTransactional and ScalableVirtual Federation of Data, Metadata Hub Towards a Data Access Server

(*) Ted Friedman, Gartner, January 2005

The need for a Data Bus

InfoQ: Incorporating Enterprise data into SOA (Nov. 2006)The 451 Group study (Dec. 2006)Integration has to re-invent itself

Metadata is seen as more and more criticalProprietary EAIs evolves into standardized ESB EII (XML or SQL) is nice for reporting

Mostly read-only, does not fit with requirements of transactional Composite Applications

Services bring flexibility but won’t replace databasesMix SOA and J2EE/.Net, databases and servicesServices as business interfaces in front of databases are new data providersManually calling thousands of services will quickly become unmanageable

Composition/aggregation of fine-grained data servicesStatic versus dynamic (coded versus configured)

SOA, Integration and Persistence are merging Composite Applications and BPM require intermediation

21

Expenses in the SOA market

ESB; Expenses; 24; 24%

Data Services; Expenses; 19;

19%

BAM; Expenses; 12; 12%

Security; Expenses; 12; 12%

Portal; Expenses; 16; 16%

BPM; Expenses; 12; 12%

Repository; Expenses; 5; 5%

22 Source: GCR 2006

Intermediation: more mature SOA

SOA introduces a new layer Bringing more Flexibility and allowing to do more

At a cost: Complexity & ScalabilityDecreasing Agility

Intermediation makes SOA easier and betterF

eatu

res

Ease

3-tiers

SOAXIC

2006: Intermediation

XIC DASComponent model: Services & Objects (POJOs)5-tier architectures, the data & service intermediation serverVirtual federated data sourcesMetadata, repository, dynamic adaptive composition

Business

Logic

GUI

Business

Model

Intermediation

ServicesServices

Business

Process

Xcalia intermediationWhat it is, how it works… Data Access Service, connected and disconnected transactions, mapping of data services

25

Xcalia IC

Synchronous

Transactions

• Metadata

• Cache

• Mapping

• Dynamic Composition

• Dynamic tuning

ES

B, E

AI

The complexity of integration

26

Composite

Application

odbms

xml

rdbms

Middlewares

Business

Process

Applications

Web servicesMainframe

Reporting

toolDisconnectedData-oriented

Business Model(XML, SDO…)

ConnectedObject-orientedBusiness Model

EII

ORM

Intermediation: how it works.

27

BusinessApplication Xcalia IC

• Transactions• Queries• Cache• Metadata• Transformations• Business semantic

RDBMS,XML…

Web Service,mainframe,

applications… … cust = order.getCustomer(); cust.setStockSymbol(“C1”); …

SQL…

SOAP, JCA…

Customer ?

Transparent delegation to Xcalia intermediation

Dynamic selection of the best sequence of services to call

Connected object model (POJO): Transparent, unified & standardized access to databases and data services.

order

XIC & SDO / DAS

Client applications (Java, .Net, BPEL…) are manipulating enterprise information through the Xcalia Data Access Service (DAS). This is a generic Web service, giving dynamic access to enterprise information (SDO, XML…) in a disconnected model through mapping with a published business model.

28

Java / .Net

Application

Xcalia DAS RDBMS,XML…

Web Service,mainframe,

applications…

do { … das.fetch(« customer1 »);

… das.applyChanges();

}

SQL…

SOAP, JCA…

Explicit calls to the remote Xcalia Data Access Service Generic Xcalia Data

Access Web ServiceStandard Xcalia IC mapping

XMLSDOs

Xcalia Intermediation

• Non-intrusive

• Composite application connectivity

• App-to-data

• App-to-services

• Service-to-service

• Service-to-data

• Data-to-data

• App to-data + services + applications

• Transactional, dynamic, adaptive

EII (Information Integration)

• Data-to-data connectivity

• Distributed query, XML or SQL

based• No programming APIs

• Mostly for read-only

applications and reporting

EAI (Application

Integration)

• App-to-app messaging

connectivity

• Static, Point-to-point

• Dependent on custom

adapters

• Evolves into ESB

From Integration to Intermediation

29

ORM (Object relational

mapping)

• Limited to RDBMS

• Limited to POJO

• Limited to connected models

Connecto

rs

Persiste

nce

Business

Application

Intermediation & ESB

30

EAI

RDBMSXML

ODBMS

Serv

iceServ

ice

Components

Serv

ice

mapping

MainframeSAP

Connecto

rsServ

ice

Siebel

ESB

Asynchronous

XML messages

Xcalia IC

Synchronous

transactions

Send

DetectTransform

?

Map

UpdateAccess?

WS

ESB

Data

source

s Data

Servi

ces

Xcalia

Interm

ediation

•Cach

e, metadata, m

apping

•Dyn

amic co

mposition

XIC into an Application Platform Suite

31

Business

Laye

r

• Presentation, P

ortals

• Busin

ess Lo

gic

• Busin

ess Se

rvice

s

• Busin

ess M

odels

Applic.

Intermediation, integration & SOA

32

Inte

gratio

n

EAI, EII,

JCA…

SOA

Web se

rvice

s,

XML, B

PEL

Persist

ence

EJB3, J

DO

Xcalia

intermediationService metadata

Composition of Data & Services

Convergence ofIntegration and Persistencefor Composite Applications in SOA ESB

SD

O

Mappers

High-level coarse-grained Business Services

MngtApproval

Intermediation & BPM

33

CreateAccount

Validate

ComputeLoan

Web

Service

Legacy

Service

Xcalia IC

Fine-grainedData Services

Orchestration

Dynamic composition

Intermediation

A unique approach to integration in a SOA worldDynamic and adaptiveMaps the application business model to all data sources, support for MDA

Provides unified, standardized and transactional access to heterogeneous data sources

DBMS, XML, Web services, mainframes transactions, packaged applications…

Dynamic composition provides transactions (updates) at runtimeExtended metadata defines data services, where they are and what they are doingLess coding, easily create composite applications

Fits easily into existing infrastructureNon-intrusive into systems, applications or resources

Extends traditional integration solutions to SOAMix SOA and non-SOA, J2EE and .NetMigrate incrementally

34

Business Logic

Xcalia intermediationProduct Offer

35

Xcalia Offer

Xcalia Intermediation Core 5Universal Mapping Engine, Engine Trigger, Custom MappersTransaction Management, distributed Queries, CachesJMX administrationPOJO client (JDO2 now, JPA2 later)Default DBMS mapper: RDBMS, XML, ODBMS (Versant), Jalisto

Xcalia DASDAS Server implementation: local and remote protocols (Axis2 or JAX-WS 2)SDO / DAS clients

Java and .Net (including LINQ), static and dynamic APIs

MappersData Services: Web Services, mainframe transactions, packaged applications, JMS, Components (EJB), JavaFramework for mapping non-structured data (XML, COMMAREA, Stored Procs)

Xcalia Studio 5Navilis (testing tool)

JalistoA fast & light 100% Java data store for embedded applicationsContributed to ObjectWeb as open source

36

Xcalia Intermediation Core Components

37

Xcalia Studio

Java

.Net

Xcalia DAS

Xcalia Intermediation Core Components

38

Web Service

Xcalia IC

Conn

ectio

ns

Dat

a Ca

che

Tran

sacti

ons

Dis

t. Q

uerie

s

Map

ping

XIC

Serv

er

laye

r

XIC

Clie

nt

Net

wor

k la

yer

JDO

SDO

Map

pers

Invo

kers

Met

adat

a

DAS

Administration (JMX)

Mainframes, Stored Procs,Applications, Components

Dyn

. Com

pos.

Xcalia Studio: screenshots

Xcalia Studio: ease of use

Wizards for the 3 typical approachesTop-down, Bottom-up and Meet-in-the-middle WizardsMake-persistent wizardAuto-mapping feature (based on lowest vocal distance)

Mix of 3 configurable algorithmsReverse engineering of existing database schema

Configurable generation of Java source codeOptional enrichment of incomplete database models

Define Schema tools with schema evolution, SQL output, etc.

NavilisTesting toolPersistent object model browsing (CRUD, queries, business methods)

Multiple mapping filesWorks for ORM and OSM (object to service mapping)

© 2007 Microsoft France

Votre potentiel, notre passion TM