Using an ESB to Build a SOA SOA/ESB/Legacy Integration

© 2007 The MITRE Corporation. All rights reservedFor Internal MITRE Use

E54BAdvanced Architecture

Technologies & Solutions

Using an ESB to Build a SOA SOA/ESB/Legacy Integration

Bob Sadler

April 2008

© 2007 The MITRE Corporation. All rights reserved

Agenda

What is an ESB?Standards supportedWhat are the benefits?What do you do with it?Notional ESB architectureESB FeaturesHow does an ESB work?Pros and Cons of using an ESB to build a SOA


What is an ESB?

ESB – Acronym for Enterprise Service Bus Middleware/integration backbone that ties applications,

services, and resources together within a business area Connects Web services, J2EE, .NET, B2B, Portals, and

Legacy applications (i.e., databases, ERPs, CRM, programs, etc.)

Defines service mediation (i.e., intelligent message routing, transformation, and monitoring)

Allows publishing and discovering of services (WSDL/UDDI) Enables the connection of software running in parallel on

different platforms, written in different programming languages, and using different programming/data models

Provides service governance, security, and error handling


Generic ESB Architecture

Enterprise Service Bus

B2B(Partner

Systems)

Legacy

Applications

Portal

J2EEApps

.NETApps

WebServices


What are the benefits?

Simplifies the integration process and the reuse of business components

Adapts easily to new business requirementsEasy to use, lower-cost, and standards-based

integrationEnables incremental application development and

deployment, thus reducing risk and up-front investments

Centrally managed


Standards Supported

WSDL, SOAP, UDDI, WS-Addressing, WS-ReliableMessage, WS-Security, WS-Policy

XML, XML Schema, XSLT, XPath, and XQueryJMS, JCA, J2EE


What do we do with it?ANSWER: Integrate ApplicationsMost organizations have legacy systems, and new

applications that they want to buildMany of the legacy systems need to be integrated

along with the new applicationsMany of the legacies are in a client/server

configurations (tightly coupled)Some of the new applications could be SOA

DSNYDER

Is the pith of this bullet that legacy systems need to be interated with new applications?

DSNYDER

Need to delete the "a" in this bullet...

DSNYDER

What does "tightly coupled" mean in this instance?

DSNYDER

May want to be consistent with the use of terminology i.e., systems versus applications. Seem to be mixing these terms and interchanging them...May want to consider using a consistent term...

DSNYDER

Again, just for clarity, what does it mean to say that an application could be SOA?


Notional ESB Architecture

Enterprise Service Bus

DB DBDB

Service Management

Service Registry

Portal

Web ServiceApps

Business Process

Mgmt (BPM)Test

Manager

Data Service Layer

DSNYDER

How does this chart differ from chart #5? Chart #5 is titled "Generic ESB Architecture". May want to further distinguish these 2 charts from one another....


User Capabilities

Can input requests, services, and commands through client application or portal

Can view/output responses, services, and commands through client or portal

Can define business processes/services through Business Process Management (BPM)

Can orchestrate business processes with Business Process Execution Language (BPEL)

Can create/maintain/execute policies (i.e., governance) through service registry/repository

Can have service and transport-level security Can develop/implement/test through test management tools Can integrate databases through Data Service Platform


Vendor Specific Products “My Experience”

ESB – BEA AquaLogic Service Bus (ALSB)Data Service Layer – Composite Data Service

PlatformDatabases – Oracle 10gTest Manager – iTKO LISAPortal – BEA AquaLogic Portal


ALSB Features (1)

Intelligent routing– Message routing according to XQuery-based policies or call-

outs to external Web services– Support for both point-to-point and one-to-many routing

scenarios to support request-response and publish-subscribe models

– Dynamic routing destinations that enable service destinations to be set dynamically in the proxy pipeline. External rules can be configured to feed the actual business service to be routed at run time

Message brokering– Support for multiple messaging models including

synchronous, asynchronous, and publish-subscribe– Support for multiple message formats including SOAP, SOAP

with attachments, XML, structured non-XML data, raw data, text, and email with attachments

– WS-I compliance for SOAP/HTTP messages


ALSB Features (2)

Message transformation– Validates incoming messages against schemas– Dynamic service selection, based on message content or

headers with the ability to transform messages based on the target service

Message transformation based on XQuery or XSLT– Multiple formats: XML and structured non-XML data

Service bus security– Supports authentication, encryption and decryption, and digital

signatures as defined in the Web service security (WS-Security) specification

– Uses SSL to support traditional transport-level security for HTTP and JMS transport protocols

– Supports one-way and two-way certificate based authentication– Supports message-level security– Supports HTTP basic authentication


ALSB Features (3)

Service Level Agreement (SLA)

– Administrators can set SLAs on a variety of attributes including throughput times, processing volumes, success/failure ratios of messages processed, number of errors, security violations, and schema validation issues

– Administrators can configure alerts for SLA rules violations and trigger automated actions, and send alerts to the console, or email

Error handling

– Allows you to configure your system to format and send error messages, and return messages for consumers of services who expect a synchronous response

Testing

– Provides a built-in test console in the development environment

– Allows you to test inline XQuery expressions used in the message flow

Logging and monitoring

– You can gather statistics about message invocations, errors, performance characteristics, messages passed, SLA violations, etc.


How does ALSB Work?

Application client– Service consumer

ESB– Proxy service – Intermediary Web service that exchange

messages with client; Proxy services are published by ALSB; defined in terms of WSDLs, pipelines, routes, and branches; sends and formats messages to appropriate business services

– Business service – Enterprise services that ALSB will route messages to; any service not implemented in pipeline; represent services external to ALSB; client stub

Existing service– Back-end services


Intelligent Message Brokering

ProxyService

BusinessService

ApplClient

BackendService


SOA Development


Cons of using an ESBto Build a SOA

Don’t use ESB-oriented architecture, if it does not have “business value”. SOA must be based on business requirements

An ESB is a means to an end, not the end itself SOA approach consists of several components

such as service producers, service consumers, service registries and repositories. These components must be defined and developed first, then integrated with an ESB

Integrating ESB with other software may be difficult, particularly legacies

Organization may have to develop new skills, techniques, and technologies


Benefits of ESB Prototype

Do an ESB prototype firstStarting point for developing and evaluating tangible

and reusable SOA services and productsEnables a means of testing services with portals,

databases, and other applicationsProvides a means of training staff on SOA/ESB

concepts, development (i.e., SOA/ESB analysis, design, implementation, and testing) and tools

Provides “lessons learned” before you build a production SOA integrated environment

Production cost estimates and development schedules can become more accurate and realistic because of the experience gained from building an ESB prototype


SOA Development


SOA Guiding Principles

Gain buy-in to service-oriented design from senior executives and stakeholders, based on the benefits they will receive from a SOA approach.

Share “lessons learned” among SOA development and implementation teams to avoid duplicating earlier mistakes

Create a SOA roadmap/development plan based on enterprise business strategy

Define IT design and development policies, best practices, and standards

Educate staff about SOA policies, practices, products, and tools Survey SOA range of technologies and techniques Implement SOA governance process that ensures teams are

following design and development policies, architecture, and best practices

Regularly measure the results of the SOA adoption in terms of added business values to the organization

Leverage your legacy systems (e.g., existing architecture, infrastructure, and investments)

Define a step-by-step procedure detailing how the SOA will be tested


Service Lifecycle Phases

Planning

Define/Develop services

Discover services

Integrate services

Orchestrate services

Secure services

Manage services

Test services

Deployment

Access services

Analyze SOA/services and environment

Monitor services


Planning

Define the business value upfront (i.e. make a business case)

Define the scope of SOA (i.e., business processes, number of services, organizations, external users, etc.)

Establish boundaries and alignments with current and future business and IT initiatives

Use a phased approach, starting with a single, low-risk application or a pilot program that spans business functions and eventually expand to an enterprise SOA

Use architecture guidelines, requirements, and policies Define SOA standards Define roles and responsibilities Assess the maturity of the current and desired target state,

followed by a roadmap for transformation toward SOA Define SOA tasks Plan task schedule and resources Start the governance process


Define/Develop Services Create service candidates by defining roles and collaboration

as stated in business processes Identify candidates for potential business services from

business model Potential business services should be reusable Design and build services that correspond to specific steps

within a business process Identify services that can be combined into a composite

service or application for carrying out a specific business function

Use a Business Process Management (BPM) tool


Discover Services Define the service registry/directory Service directory acts as service broker between service provider

and service consumer. It registers the services through its metadata or service contracts (i.e., it hold data about the services).

Define service providers and consumers Service provider is a person, department, organization, or application

that offers the service. It registers service contracts to the appropriate service –brokering nodes. Define Service Level Agreements/Contracts and metadata.

Service consumer is a person, department, organization, or application that finds services with which its client would like to bind and interact. The service consumer may query service broker to find contracts and the metadata associated with services (i.e., service location, methods or remote services, message input/output formats, policies, and other parameters that a client uses to bind to a service)

Service broker is an entity that facilitates registration, location, discovery, retrieval, synchronization, replication, and/or delivery of service


Integrate Services

Integrate services with other services, applications or IT systems such as databases, data transformation services, reliable message delivery with routing, monitoring, and management (i.e., could be ESB and Service Management tools )

Integration often requires transformation of data to map between different data schemas, as well as dynamic routing for connecting the appropriate services at run time. This can be done in a Data Service Platform (DSP).


Orchestrate Services

Orchestration is the combining of services into seamless, reliable process flows

It differs from integration, because orchestration is the sequencing of services to fulfill a business task or process

Orchestration is also called “gluing” services together into flow logic

Can be done in Business Process Execution Language (BPEL) and Business Process Management (BPM)


Secure Services

Define service security policies Define user access in SOA environment (i.e., authorizing

and authenticating users, as well as provisioning them and managing their identities) must be mapped out before potentially sensitive information is exposed as a service

Identify services and data with security requirements Provide information security controls to security

requirements May be implemented in the Service Registry and/or ESB ESB provides message and transport level security Web service security standards can be applied (i.e., WS-

Security, SAML, SSL, XML-Signature, XML-Encryption, etc.)


Manage Services

Define Service Level Agreements (SLA)s for services, and how to enforce them

Create SOA operational policies for daily operations, maintenance, auditing and billing (if appropriate) of services

Define metrics for service performanceMay be implemented and enforced in Service

Registry and Service Management


Deployment

Deployment Considerations:– Cross domain requirements– Physical facilities/power– Does services require 24/7 availability– What are the security and scalability requirements– Maximum number of concurrent users– Service transaction time requirements– Tools, training, and technologies– Configuration and version control– C&A and COOP/DR requirements


Service Access

Services are typically exposed to users through a portal or a composite Web application

May also be exposed through wireless devices such as cell phones and handheld devices

SOA environments support multi-channel access to services and enable organizations to adapt user interfaces without modifying underlying services


Analyze Services

Periodically analyze services, events, and business processes involved in the business operations

Define how operational managers and users can effectively monitor, analyze, and respond to time-sensitive issues

Identify bottlenecks and other problems in the process and alert the appropriate personnel when a particular event warrants attention

Analyze if all requirements are being met


Monitor Services

Monitor the service execution and results Ensure that the services are reliable, available,

and constantly monitored for exceptions or failures

Services can be monitored through the ESB, and performance tested with a Service Test Manager


SOA/ESB Lesson Learned 1

Do an ESB prototype first ESB as an integration and SOA solution is still a new

technology– Getting trained and skilled people may be a problem– Integrating ESB with other software may be difficult

Standards and best practices are still maturing Organization may have to develop new skills, techniques, and

technologies Management must endorse and enforce governance from the

beginning Security must be considered from the beginning (i.e., SOA

Security, Web Service security, Defense-in-Depth, DoD compliance, etc.), and you can never start the C&A process too early


SOA/ESB Lesson Learned 2

Performance requirements (i.e., reliability, availability, scalability, transaction time, response time, etc.) should be defined and tested, performance may be an issue

SOA may not be the answer for all legacy integrations

Demonstrate the ESB capabilities to stakeholdersUse vendor experts as much as possible to assist

with developmentProvide training to users


The End

Documents

Using an ESB to Build a SOA SOA/ESB/Legacy Integration