Dynamic Messaging with Microsoft BizTalk Enterprise Service Bus (ESB) Guidance v2 Brian Loesgen

Principal SOA ArchitectMicrosoft Corporationhttp://blog.BrianLoesgen.comSOA317

Session Objectives

Have an understanding of the ESB architectural style and its place within the overall Application PlatformUnderstand architecture of the ESB GuidanceHave seen the capabilities and features of V2, and how they can accelerate ESB deployments

Agenda

Service-orientation; ESB: why it mattersArchitectural OverviewTechnical DrilldownDemos, demos, demosService compositionGovernance

Common Requirements

MessagingService InvocationDynamic TransformationDynamic Routing Protocol Mediation Endpoint ResolutionLoosely-Coupled & Event-DrivenWS* SupportQuality of Service (QoS) Management

Two Views of BizTalk?

BizTalk was positioned as a Hub-and-Spoke…

Now it’s an Enterprise Service Bus?

Mapping ServiceRouting Process

Orchestration

ProtocolAdaptation

End Point Resolution

Pub/Sub Service Service ProvidersService Consumers

Re-thinking the Solution as a Set of Capabilities

SOA Pattern Implementation

PolicyCentralization

RulesCentralization

Event-DrivenMessaging

Data Format Transformation

Protocol Bridge

Message Router

Message Broker

Legacy Wrapper

Architectural Design PatternsMessage Routing

Scatter-Gather

Routing Slip

Recipient List

Message Transformation

Content Enricher

Data Model Transformation

ESB Usage Patterns

Repair andResubmit

VETO/VETRO

Gateway

Reply Forward

MetadataCentralization

Perimeter Service Router

Content-BasedRouter

ESB Guidance

BizTalk... BizTalk+ESB Guidance...

BizTalk is all about providing solutions based on configurationConfiguration happens at dev time or post-deploymentESB Guidance is all about runtime resolution, it interacts with external stores (e.g., services registry) to get operational configuration in a JIT mannerESB Guidance adds a set of runtime resolution capabilities that BizTalk developers would need to create from scratch

Enter Microsoft ESB Guidance…

From Patterns and PracticesProvides architectural guidance, patterns and practicesDelivers reusable BizTalk Server ESB and .NET components Enables construction of large and small-scale ESB solutions

The ESB Stack

BizTalk Server

Adapter Providers

Resolvers

Mediation Components

Mediation Policies

ESB Guidance 2.0 Changes

Built on BizTalk Server 2009Provides greatly enhanced tooling, on top of an optimized coreProvides even more extensibility pointsProvide even more prescriptive guidance about enterprise integration patternsStreamlined installation experience (Powershell, configuration tool, etc)

ESB Guidance Core EngineComponents

Adapters Dynamic Ports

Pub Sub Engine

TransformationEngine

Business Rules Engine

HostEnvironment

ItineraryServices Resolvers

Adapter ProviderFramework

ASMX On-Ramps

BizTalkComponents

ESBComponents

Custom Web Components

WCF On-Ramps

ManagementPortal

Other Servers& Components

UDDI2.0

Governance Tools Databases

OrchestrationEngine

Frameworks ResolverFramework

Adapter Providers

Enterprise Library4

ExceptionManagement

Core WebServices

UDDI3.0

DevelopmentTools

Management (or custom) PortalException Management

Core ESB Services

External Services

Resolver Web Service

Transformation Web Service

Exception Web Service

Exception Logger

Exception Handler 1..nGeneric Custom Application

Transformation Agent

Generic Delivery Agent

On-ramps Off-ramps

Generic SOAPSend

Generic WCFSend

Generic JMSSend

Generic CustomSend

Pipeline Components

Pipeline Components

Pipeline Components

Pipeline Components

Generic SOAPReceive

Generic WCFReceive

Generic JMSReceive

Generic CustomReceive

Pipeline Components

Pipeline Components

JMSComponents

Namespace Components

Architectural Overview

Custom & Business Processes

Create New

Order

Scatter Gather Service

Update ERP

Systems

ItineraryProcessing

Resolvers AdapterProviders

The Core ESBG Concepts

Itinerary Concept

Heart of the ESB guidanceItineraries help provide the runtime flexibility that BizTalk doesn’t have by defaultItineraries provide a service composition mechanism

ItineraryProcessing

Developing Itineraries with V2

Leverages VS 2008 and its capabilities for Domain Specific LanguagesWe now have a Visual Itinerary Designer

Avoid creating itineraries in plain old XML (no more Notepad)Validate itinerary model at design-timeExport capabilities to the Itinerary repository (SQL) and fileMigration tool to move between environments

ItineraryProcessing

Developing Itineraries ItineraryProcessing

Using ItinerariesDesign Time

ItineraryProcessing

XML File

ItineraryDatabase

Run Time

Itinerary

OnRamp

Itinerary Selector

ESB Processing

Itinerary &Message

Message

Three Ways to Assign Itineraries ItineraryProcessing

Client resolves itinerary via resolver service and then sends request to On-Ramp with itinerary header

Client sends request to an Itinerary On-Ramp passing itinerary SOAP or WCF Header

Client sends message to Itinerary Generic On-Ramp, pipeline component selects an itinerary using configurable resolver

AdvancedServiceClient

AdaptiveServiceClient

ServiceProxy

Resolvers

For runtime flexibility ESB Services are not hard-coded to specific endpoints or maps

This metadata is determined at runtimeResolver mechanism can locate and retrieve this metadataItineraries define which ESB services execute and in which orderResolvers define how ESB services execute

Resolvers

Resolvers Shipped with V2

Endpoint Resolution

UDDI 2.0

Artifact Resolution

ESB Resolvers

Resolvers

Static Static Map - BRE

WS-MEX BRE SQL Itinerary - Static

XPATH Composite Itinerary - BREUDDI 3.0

ESB Guidance V1

ESB Guidance V2

Adapter Providers

The Itinerary and Resolver mechanisms are .NET components. The endpoint information produced by the resolvers is stored in .NET based Dictionary objectsWe leverage BizTalk dynamic ports to handle the actual routing of messages. However, BizTalk isn’t built to work with Dictionary objectsAdapter providers act as a bridge between the .NET based ESB components and the BizTalk based ESB components

AdapterProviders

AdapterProviders

FTP

MQSeries

ESB Guidance V1

ESB Guidance V2

FILE

Custom

WCF-BasicHTTP

WCF-WsHTTP

SQL

WCF-Custom

SMTP

ESB Adapter Providers

Adapter Providers

Using ESB Itineraries and Resolversdemo

Service CompositionComposing, Choreographing or “Chaining” services is a key capability for an ESB

Physical Service Physical Service Physical Service

Composed Service

Service Composition with

We have two optionsMessaging-based: Lightweight composition using only Itineraries and BizTalk ports (using ItineraryForward pipeline component)Orchestration-based: Orchestrations available for more complex business processes

ESB Itineraries: Composing Servicesdemo

OrchestrationOnRamp

Any Transport

Off-Ramp

Any Transport

Scope

Compensation

ESB Audit Off-Ramp

SQL Transport

Filter {BTS.FaultCode Exists}

BizTalk Failed Message Routing Processing ESB Failed Orchestration Message Routing Processing

Filter {BTS.FaultCode Exists Or ESB.FaultCode Exists}

ESB Exception Service Processing

Unified Exception Management

Exception On-Ramp

SOAP Transport

ExceptionDatabase

ESB Administration PortalOverview

Sample application, used for an IT Pro type view into the ESBLocated at http://localhost/esb.portal/ASP.NET-basedThis is a sample of how ESB operational metrics and data can be surfacedProcess or group-specific portals could be created in SharePoint, leveraging ESBG assets such as BAM tracking and exception handling

ESB Administration PortalFeatures

The Portal providesGraphical metricsRepair and resubmit functionalityAlerting based on exception eventsAuditing trails for repair and resubmitUnified view of the .NET Exception data + BizTalk message + BizTalk context propertiesHistorical views of exception dataRemote web-based accessFiltering of exceptions based on application

Management Portaldemo

SOA Governance Integration

Integrated with leading governance solutions from AmberPoint and SOA SoftwareSpecific functionality varies by governance provider, but typically include:

Service metricsPolicy-driven securityPolicy-driven service-level agreementAlerting

Governance

SOA Governance Architecture

Publish/SubscribeInfrastructure

ESB Core Services Virtualized ServiceOff-ramp

IIS

Web ServiceOn-ramp

ManagementPoint

Governance Portal

Governance Server

Security Enforcement

SLA Enforcement

Service Registry

Service Metrics Collection

Service Configuration Metrics Reporting

Metrics Reporting Service Lifecycle Management

PhysicalService

Governance

Announcing the BizTalk ESB Toolkit

Name change From “ESB Guidance” to “BizTalk ESB Toolkit”

General Availability Mid June 09

Packaging & Distribution

Signed binaries and samples code (MSDN -> Download Center)Documentation (MSDN)Private Fixes (Microsoft Connect site)

License free for BizTalk Customers

Support & Bug Reporting

Managed MSDN forums with Microsoft Customer Support Services* in the loop to fix issues as required

Online Community Managed MSDN forums

* Ramping up through the summer

Summary

Provide an understanding of the ESB architectural style and it's place within the overall Application PlatformArchitectural overview of the ESB GuidanceDemonstrated the capabilities and new features of V2 that accelerate ESB deployments

question & answer

Call to Action:Get ready and involved!

Attend BizTalk Sessions @TechEdExplore the existing resources on our websiteUpgrade to BizTalk Server 2009, or downloadthe Evaluation version

Figure out how your organization can save today with BizTalk!

Related BizTalk Content at TechEdBreakout SessionsSOA304 Introducing the Microsoft Integration Server: BizTalk Server 2009SOA318 Microsoft BizTalk Goes Mobile: Collecting Physical World Events from Mobile DevicesSOA317 Dynamic Messaging with Microsoft BizTalk Enterprise Service Bus (ESB) Guidance v2ARC306 Connecting Enterprise Applications to Existing DataSOA301 Achieving Success with Integration in the Enterprise Using Microsoft BizTalk Server 2009SOA321 Best Practices for Virtualization of Microsoft BizTalk Server 2009 with Hyper-VSOA305 Enhancing the SAP User Experience: Building Rich Composite Applications in Microsoft Office SharePoint Server 2007 Using the BizTalk Adapter PackSOA312 Microsoft BizTalk: Application Lifecycle Management and Productivity EnhancementsSOA205 Radio Frequency Identification: Affecting End Users, Online, ERP to Plant Floor

Interactive SessionsSOA04-INT Deep Dive with Microsoft BizTalk Server 2009 Development PlatformSOA06-INT Integrating with Partners: Learn to Develop with EDI/AS2 in Microsoft BizTalk Server 2009

Hands on LabsSOA05-HOL Microsoft BizTalk Server: Building and Deploying Your First BizTalk Server SolutionSOA01-HOL Microsoft BizTalk Server: Analyzing Business Activity Data with Business Activity Monitoring (BAM)SOA02-HOL Extending BizTalk Server 2009 BAM Capabilities through Representation State Transfer (REST)SOA03-HOL Microsoft BizTalk Server: Capturing Data with Business Activity Monitoring (BAM)SOA04-HOL Connecting Microsoft .NET Applications to Legacy IBM Mainframe and Midrange Systems Using Microsoft Host Integration Server 2009

Additional ResourcesBizTalk Website

BizTalk Server RoadmapIntroducing Microsoft BizTalk ServerMicrosoft BizTalk Server Technical OverviewBizTalk Technical Posters (including interactive “posters”)BizTalk MSDN Developer Center

Microsoft SOAReal World SOA Customer Stories

Microsoft BPMMicrosoft's People-Ready Process visionIntegrated architecture for process-centric applications

Track ResourcesSOA

www.microsoft.com/soa

msdn.microsoft.com/biztalkwww.microsoft.com/biztalk

“Dublin”www.microsoft.com/net/dublin

“Oslo”msdn.microsoft.com/oslowww.microsoft.com/net/oslo.aspx

msdn.microsoft.com/wcfmsdn.microsoft.com/wfwww.microsoft.com/net

msdn.microsoft.com/azure/netserviceswww.microsoft.com/azure/netservicestwitter.com/dotnetservices

www.microsoft.com/teched Sessions On-Demand & Community

http://microsoft.com/technet Resources for IT Professionals

http://microsoft.com/msdn Resources for Developers

www.microsoft.com/learning Microsoft Certification & Training Resources

Resources

www.microsoft.com/learningMicrosoft Certification and Training Resources

Complete an evaluation on CommNet and enter to win!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.