Enterprise Service Bus Deep, Deep Dive Burr Sutter October 2007

Enterprise Service Bus Deep, Deep Dive

Burr Sutter

October 2007

Questions: [email protected]

A Real World Scenario

SourceSystem

ASCII

File SystemFile System

ASCII


ASCII FTPFTP

FTPGET

MSG MQMQ

JMSListener

JDBC/ODBCSelectUpdate

SQLSQL

DataData SocketSocket

TCP/IPSocket


ReceivingSystem N

ASCII


ReceivingSystem N

ASCII

FTPFTP

ReceivingSystem N

JMSJMS

The ESB Solution

ASCII


SourceSystem

ReceivingSystem 1

ReceivingSystem 2

ASCII


ReceivingSystem 3

XML

JMSJMS

XML

SOAPSOAP

JBoss ESB


Enterprise Service Bus

Declarative Transformation includes:Message Normalization,Validation, Enrichment,

Via XSLT, Groovy, Java, StringTemplate

ServiceRegistry

UDDI

ESB

MessageStore

Transformation Routing

Infrastructure Services

Security Management

DeclarativeOrchestration

Engine

BPELjPDL

ProcessStore

EventNotification

Pluggable Architecturefor integrating infrastructure services

Web Services

EJB

POJOs

Java EE 5

Business ServiceComponents

BusinessData

JCA

RDBMSLegacyCOTS

Business Servicesruns within a container or standalone

Service Data Object

Service Component Architecture

HTTP(S)

FTP

File

JMS

Email

SOAPExcel

Browser

Java

Native

ASCIIXML

Binary

Transports

Web

SQL

Event Listeners and Actions provide transport mediation

Socket

Hibernate

JCA/Inflow

CustomAction

Seam

Drools

Spring

Groovy


ESBs Across The Enterprise

HTTP/S

FTP

File

JMS

Email

SOAP

Portal

Seam

ServiceRegistry

JBoss ESB

EventStore

JSR 181 POJOs

EJB3

POJO

BusinessServiceTransformation Routing Security

BusinessData

Management

CompositionEngine

BPELjPDL

Scripting

JCA

WebProcess

Store

Notification

FTP

File

Email

SOAP

ServiceRegistry

JBoss ESB

EventStore

Routing Security

CompositionEngine

BPEL ProcessStoreNotification

File

SOAP

ServiceRegistry

ES

B

CompositionEngine

BPELProcess

Store

Invoicing/Order Entry

Web Order EntryAccounting

SharedDistributed

Registry

Plus Lightweight ESBArchitecture


Service Federation

Registry& Messaging

Transformation Services Cluster

Rules Services Cluster

Order Validation

Credit Check

Process Store

Customer Inquiry

Customer Update

Accounting

Order Services

Routing Services

Customer Services

* JVM

Order Validation

Credit Check

Shipping Calc

Inventory Check

Inventory Update

Warehouse Services

Shipping Calc

Inventory Check

Inventory Update

Shipping Calc

Inventory Check

Inventory Update

Customer Inquiry

Customer Update

Accounting


Enterprise Service Bus

Declarative Transformation includes:Message Normalization,Validation, Enrichment,

Via XSLT, Groovy, Java, StringTemplate

ServiceRegistry

UDDI

ESB

MessageStore

Transformation Routing

Infrastructure Services

Security Management

DeclarativeOrchestration

Engine

BPELjPDL

ProcessStore

EventNotification

Pluggable Architecturefor integrating infrastructure services

Web Services

EJB

POJOs

Java EE 5

Business ServiceComponents

BusinessData

JCA

RDBMSLegacyCOTS

Business Servicesruns within a container or standalone

Service Data Object

Service Component Architecture

HTTP(S)

FTP

File

JMS

Email

SOAPExcel

Browser

Java

Native

ASCIIXML

Binary

Transports

Web

SQL

Event Listeners and Actions provide transport mediation

Socket

Hibernate

JCA/Inflow

CustomAction

Seam

Drools

Spring

Groovy

Mid 2007 Future Partners


Where’s the Logic?

• As a scriptlet in your JSP• Behind your Swing Button Click Action Handler• In your Struts Action• In your DAOs• In your Stored Procedures


The Modern Silo

DB

App

Web Pages: Struts, JSF, Tapestry

DAO/JPA/SLSB/Spring BeanHibernate

Query/Stored Procedure

Browser


Silos Across the EnterpriseClient

Client Client

Client

CRMOrdering

Accounting Manufacturing

Warehouse


Integrating Silos (Dump & Load)Client

Client Client

Client

CRMOrdering

Accounting Manufacturing

Warehouse

PHP

MySQL

Web5250

RPG

DB/2 400

VisualBasic

Stored Proc

SQL Server

Struts

DAO

Oracle

DUMPASCII

SOAP

FTP

ASCII

ASCII

VisualBasic

ASCII

FTP

FTP


Intermediate those Silos

DB

App DAO/JPA/SLSB/Spring BeanHibernate

Query/Stored Procedure

Service 1 Service 2 Service 3

Browser Rich B2B Legacy

Mediation

Service 4 Service 5 Service 6Mediation


Splice Technique

Web Order Entry

Oracle

Web

JSF

HibernateEJB3

Seam

Web Order Entry 2

DB/2

Web

Struts

HibernateiBatis

SpringInterceptors

Callback Listeners

AOP

JBoss ESB

queuequeue

queuequeue

Splice Demo


ESB Definition

• There is NO J2EE-like specification with a TCK that can define and validate an ESB like there is for App Servers.

• ESBs are categorized by their capabilities – some come from a heritage of EAI and others of MOM.

• Capabilities:– Message delivery– Transport protocol mediation– Event capture– Intelligent routing– Orchestration– Transformation– BPM– Adapters– And much, much more…


JBossESB 4.2 provides

• Message delivery– JMS (JBossMQ, JBoss Messaging, ActiveMQ, MQSeries, Oracle AQ)– (S)FTP, SOAP, HTTP, email, database, shared file system

• Action chaining/pipeline – For custom service enablement & mediation

• Transformation– XSLT, Java, Groovy, StringTemplate

• Content-based routing– JBoss Rules, XPath DSL– Recipient List/Static Routing is also supported

• Orchestration – BPM, Intelligent routing, EIPs and hot deployment

• Registry– JAX-R based– UDDI support out-of-the-box (Scout and jUDDI)

• Message Store• Gateways• Extensive examples and documentation• And more…


JBoss ESB 4.2

• JMX Microkernel

• Federated Clustering

• WS-*– SOAP

– WSDL

– 181 Annotated POJO Endpoints

– EJB Endpoints

– JAXB 2.0

• BPEL 2.0

• New Deployment Architecture

• .esb Archive

• Hot Deployable Core Services

• Content Based Filter

• Business Rules Services

• Native Process Orchestration

• Hibernate Gateway/Listener

• HTTP Gateway/Listener

• Groovy Scripting

• Declarative Message Store

• JCA Container

• Spring Action AOP & POJO

• Monitoring & Mgmt Tools

• Additional EIP examples


ESB Transformation Engine

• Validation• Enrichment• Inline transformations (Fragment based Processing)• Profile based XML/non-XML processing• Anything to Anything (XML, CSV, binary, X12, POJO)• Mix/Match transformers on a single message:

– Java– Groovy– StringTemplate – XSLT

• 4X performance boost over plain XSLT (Xalan): http://milyn.codehaus.org/Chiba+Integration


jboss-esb.xml

• Configuration for one or more services which show up in the registry.

• Deployed as the key element of a .esb archive in 4.2+• Structure of a .esb archive

META-INF

jboss-esb.xml

com.mycompany.soa.mycustom.actions

MyCustomAction.class

Tip: Use one of the quickstarts as your template for your new project. We doassume that you are comfortable with Ant. An Eclipse Project template is underDevelopment and will be available in SVN soon (as of May 16, 2007)


Hello World – jboss-esb.xml 1

<?xml version = "1.0" encoding = "UTF-8"?><jbossesb xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/

schemas/xml/jbossesb-1.0.1.xsd" parameterReloadSecs="5">

<providers> <jms-provider name="JBossMQ"

connection-factory="ConnectionFactory"> <jms-bus busid="quickstartGwChannel"> <jms-message-filter dest-type="QUEUE" dest-name="queue/quickstart_helloworld_Request_GW"/> </jms-bus> <jms-bus busid="quickstartEsbChannel"> <jms-message-filter dest-type="QUEUE" dest-name=" queue/quickstart_helloworld_Request_ESB"/> </jms-bus>

</jms-provider> </providers>


Hello World – jboss-esb.xml 2

<services> <service category="FirstServiceESB" name="SimpleListener" description="Hello World"> <listeners> <jms-listener name="JMS-Gateway" busidref="quickstartGwChannel" maxThreads="1" is-gateway="true"/> <jms-listener name="helloWorld" busidref="quickstartEsbChannel" maxThreads="1"/> </listeners> <actions> <action name="action1" class="quickstart.helloworld.MyListenerAction" process="displayMessage"/> </actions> </service> </services> </jbossesb>


Hello World - action

package org.jboss.soa.esb.samples.quickstart.helloworld;

import org.jboss.soa.esb.message.Message;

public class MyListenerAction

{

public Message displayMessage(Message message) {

System.out.println("Body: " + new String(message.getBody().getContents()));

// modify the contents, do anything Java can do…

return message; // always return the message for the “chain”

}

public MyListenerAction(ConfigTree config) {

// this constructor will be used to handle any jboss-esb.xml

} // custom property configuration

}


Action Pipeline (in a Service)

FTP Action Action Action ActionBusiness

LogicComponent

TransformationCSV to XML

TransformationXML to POJO

RoutingSplitter

orBPM

LoggingAuditTrail

My_Registered_Service

Business Logic:• POJO• EJB • WS• Rules• JCA with Adapters• Anything with Java API

Actions are reusable mediation components that can be chained together to form the capabilities of a registered service. Actions can be dynamically added/removed at runtime.

Action

BusinessLogic

Invocation

This box can also represent the ESB’s JVM – actions are “in VM”

Basic Service Demo


Enterprise Integration Patterns

Aggregator

File Transfer

Translator/Transformer

Router/CBRChannel

Message

Recipient List

Splitter

Wire Tap

Enricher

Content Filter

Process Manager

Message Store

Message Filter

More information atwww.enterpriseintegrationpatterns.com


Routing, Services & Actions

FTPASCIIXML

Binary

Action Action

SOAP

JMS

Action

Action

DB

DBPOJO

EJB

A_Service

B_Service

C_Service

JMS Action DBWS

D_Service


Content Based Routing

SOAP Action

DBPOJO

Platinum_Service

FTP

ASCIIXML

Binary

Action

Ordering_Service

Randomly grab orders for review – tracer andKeep a total of all orders flowing through the system

SOAP Action

Special_Handling

EJB

Orders over 12K or from Platinum customers

DB

Orders from customers in certain states/countries


Content Based Routing Engine

• Currently based on JBoss Rules – supports XPATH DSL

• Access to the complete Header and Body/Contents for evaluation

• Zero to many destinations

rule "Routing Rule using XPATH Greater" when xpathGreaterThan "/Order/@totalAmount", "50.0" then Log : "Really It's EXPRESS Shipping"; Destination : “express";end

rule "Routing Rule using XPATH Less" when xpathLessThan "/Order/@totalAmount", "50.0" then Log : "Really It's NORMAL Shipping"; Destination : “normal";end

Destination is determined by the jboss-esb.xml


Splitter/Aggregator Quickstart

JMS Action

Inbound Splitter

JMS Action

Red

Also demonstrates, federated ESB, multi-JVM capabilities

JMS Action

Green

JMS Action

Blue

JMS Action

Aggregator_Service

EIP Demo


Load Balancing & Fail-Over

The same discount_service.esb can be deployed to multiple ESB instances/nodes. Each one is a new EPRs in the registry. The routing engine is clever enough to load-balance and fail-over between the various EPRs. Should no EPR be available, it will scheduled the message for redelivery, waiting N seconds and retrying all possible EPRs found at that time.


Deployment Options

• 1) ESB “within” an Application Server – Users have existing investments in the J2EE/Java EE App Servers and wish to run the ESB as a set of components in that App Server.

– JBoss ESB can run “within” the JBoss Enterprise Application Platform 4.2

• 2) ESB “beside” an Application Server – Users wish to leave their existing application server environments untouched but still add mediation, SOA & integration capabilities.

– JBoss ESB has its own lightweight engine/container for the hosting mediation services. It can run “beside” your existing middleware from your vendor of choice.

• 3) ESB “standalone” in a J2SE environment – this option is primarily for development & testing purposes, it is not recommended for production as of the 4.2 release.


Management & Monitoring

• 4.2 Release is focused on JMX instrumentation, the ability to monitor the health and load of every distributed service as well as start, stop & restart them remotely.


Management & Monitoring Architecture

Orchestration


Orchestration (Got Flow)

Customer

CustomerService

Warehouse

Shipper

Submit Order

Receive Order Validate Order

Credit Check

Inventory Check

Handle Shipment

Send Shipment

CreditAgency

CSR Teams

Issues

Customers submit orders via web browser, FTP and Web Services.The system receives and validates the order, errors are sent to the appropriate CSR TeamA credit check request is made, any areas of concern are sent to the appropriate CSR TeamAn inventory check request across multiple warehouses is sent, lack of availability goes to the CSRs.Assuming the order is valid, checks credit and available inventory, pick tickets are printed, shipping labels are printed, someone throws the parts into a crate or places on a pallet and puts the package by the loading dock. Various shipping companies are employed.

Simple Order Handling Process


Service Oriented Orchestration

Customer

CustomerService

Warehouse

Shipper

Submit Order


Credit Check

Inventory Check

Handle Shipment

Send Shipment

CreditAgency

CSR Teams

Issues

a Parse XMLb Transformc Apply Business Rules

Validate Order

a Create Outbound Msgb Handle Responsec Apply Business Rules

Credit Check

a Send to N Warehousesb Handle N Responsesc Determine Best WHSEsd Handle Drop-Ships

Inventory Check

ESB Mediates& Provides Services

a Determine Shipper(s)b Print Labelsc Print Pick Ticketsd Create & Send ASNs

Send Shipment


Service Orchestration & BPM

Submit Order


Credit Check

Inventory Check

Handle Shipment

Send Shipment

CSR Teams

Issues

Customer CSR Sales Mgmt WHSE MgmtHumans

J2EE AppServer

…

FTP

SOAP

File

ValidateOrder

CreditCheck

InventoryCheck

SendShipment

External Credit/Bank Legacy Partners

Various Transports

An ESB Service is visible in the UDDI registry automatically. A single ESB Service may or may not expose a WSDL. It may use out-of-the-box actions, custom actions and/or consume external services from various sources. Focus on the reuse of existing infrastructure and assets.


Orchestrated/Composed Services

ExternalActor

ExternalActor

ExternalActor

Action DBPOJO

Pricing_Service

Action DBEJB

Order_Service

Action

Action DBDAO

Validation_Service

BPMJVM


Orchestration Designer

Orchestration Demo

Questions?

Documents

Enterprise Service Bus Deep, Deep Dive Burr Sutter October 2007