1 Message Queuing with BizTalk 2006 R2: MSMQ, IBM WebSphere MQ and Ordered Delivery Thomas Abraham...

1

Message Queuing with BizTalk 2006 R2:

MSMQ, IBM WebSphere MQand Ordered Delivery

Thomas AbrahamEnterprise ConsultantTS: BizTalk 2004 & 2006blogs.digineer.com/blogs/tabraham

Randall McClureBizTalk ConsultantTS: BizTalk 2006blogs.digineer.com/blogs/randallm

2

Agenda• Why Message Queuing?• Using MSMQ• Using WebSphere MQ• A Solution for Ordered Delivery based on MSMQ

3

• Guaranteed and/or ordered delivery• Cross-platform communication• Transactions• Asynchronous data transmission

(a.k.a. fire and forget)• Buffering/batching• Loose coupling• Load balancing

Why Message Queuing?

4

• Dead letter queue management• Poison message management• Consistency of message format per

queue• Latency• Licensing and support costs

Queuing Considerations

5

• Included with almost every OS edition

• No extra licensing fees• Supports Network Load Balancing

and Cluster Services• Can be bridged to WebSphere MQ

with Host Integration Server 2006• Publish queue locations in Active

Directory and use Windows authentication

Microsoft Message Queuing

6

• Limited management tools• 4 MB message size limit• Shipping V4.0 with Windows Vista

and Windows Server 2008– Transactional remote receive– Subqueues– Poison message handling

MSMQ Considerations

7

DEMO

Using MSMQ with theBTS 2006 R2 WCF

Adapter

Demo

8

• Formerly MQSeries• Available for dozens of platforms• Highly configurable, widely adopted• Includes extensible management

tools hosted in Eclipse (Java-based)• Two BizTalk adapters:

– Server-Based Adapter (w/ BizTalk)– Client-Based (w/ Adapters for Host

Systems)

WebSphere MQ

9

Server-Based Adapter

10

Client-Based Adapter

11

• Interacts with Active Directory for authorization of queue access rights

• Domain policy may disallow non-domain users from reading domain account group memberships

• Path of least resistance and hassle: always set up a domain account for the MQ service

WebSphere MQ Security

12

• Requires COM+ and DTC running on WebSphere MQ for Windows server

• Enable Network DTC and Network COM+

• Allow In- & Outbound for Transaction Mgr

• Enable Mutual Authentication• Enable XA Transactions• Enable DCOM with Packet Privacy

Installing MQAgent

13

DEMO

Using WebSphere MQ for Windows 6.0

Demo

14

Ordered Delivery Solutionwith MSMQ Queuing

15

Agenda

• Patterns, Reloaded• Business Problem• Initial Solution

– Implementation Details– Demo

• Better Solution– More Details– Demo

• Take-aways

16

Patterns, Reloaded

• Uniform Sequential Convoy– Multiple identical messages, over time– First receive initializes a correlation set– Second receive is in a Listen, inside a Loop, using the same logical receive port and operation– Later messages that match and follow the correlation set are received in the loop

• Ordered Delivery = uniform seq. convoy ++

– Enforce order in orchestration receive shape and physical ports (limited types, 2006 only)

17

Uniform Sequential Convoy

Activate, init correlation set

First and followers in same receive shape

Loop until Delay path fires before receiving following message

Listen shape with Receive and process branch, and Delay branch

Process first message

Following correlation set

18

Ordered Delivery

Ordered Delivery = true

Ordered delivery on physical ports as appropriate

19

Business Problem

• Consume business events• Filter and transform messages• Deliver to three unique consumers, one of

which affects people’s lives adversely if not …

• In-order!• Low volume – 300/day• 6am-6pm operation window

20

Business Problem

21

Initial Solution

• Generate WSE 2.0 web service from acknowledgement orchestration

• Ordered delivery orchestration pattern

• Called orchestration chain to preserve order

• Web service facades and data access layer to call SQL Server stored procedures

• Oracle adapter to call stored procedures

22

Initial Solution – current mncis

23

Initial Solution – MNCIS XML

<CaseNotification schemaVersion="3:1" …> <NotificationAdminHeader

caseNumberKey="62J007553366" caseKey="1568853">

<CourtFileNumber> <ID>62-J0-07-553366-madeup</ID> </CourtFileNumber> … </NotificationAdminHeader> <HearingTrialSettingNotification>…(0:n)<\CaseNotification>

24

Initial Solution – current RCCC

25

Initial Solution

DEMO

26

Oops!

• What if…– Need to one or all consumer applications? – A message is suspended, breaking ordered

delivery?– Find a bug that requires messages to be

replayed, in-order?• Not quite there…

– Had incoming message file wire-tap, but not in order, and no means to replay them

– Tedious manual process to re-order and replay

27

Better Solution

• Solution– Might have implemented a resequencer

pattern athttp://www.microsoft.com/downloads/

details.aspx?FamilyID=02c5fd53-fee9-44fc-a780-5d1d34ee8754&DisplayLang=en

– Decided to insert MSMQ queues in main path and in each consumer path

– Decided to have the web service write directly to the main path queue, to have replay capability separately from BizTalk

– Decided to decouple the web service from BizTalk

28

Queuing Solution – planned mncis

29

Queuing Solution – planned RCCC

For replay

30

Queuing Solution

DEMO

31

Take-aways

• Ordered delivery easy to implement, tougher to support

• MSMQ very helpful for isolation, replay and fault tolerance

• Watch out for subscription loops• Third-party components help but

they may not be robust enough

32

Take-aways

• Third-party tools are very helpful• Consider fault tolerance and replay

EARLY • File SEND (but not receive) can do

ordered delivery; use %datetime% in filename

33

Resources

• DebugViewhttp://www.microsoft.com/technet/sysinternals/

utilities/debugview.mspx• Wfetch (in IIS 6.0 resource kit)

http://www.microsoft.com/downloads/details.aspx?familyid=B134A806-D50E-4664-8348-DA5C17129210&displaylang=en

• ContextAdder – Jon Flandershttp://www.masteringbiztalk.com/blogs/jon/

CommentView,guid,25768f43-c0b6-4f9d-bb7e-636d52dcd7eb.aspx#commentstart

– Modified here by 12/15http://blogs.digineer.com/blogs/randallm/

default.aspx

34

Resources

• Cogin QueueExplorerhttp://www.cogin.com/

• CreateMSMQQueues.Tools (by 12/15)http://blogs.digineer.com/blogs/randallm/

default.aspx• SSO tools by BizTalk Samples, Scott

Colestock and others…

35

Message Queuing with BizTalk 2006 R2:

MSMQ, IBM WebSphere MQand Ordered Delivery

Thank you for attending!Please visit us at www.digineer.com