Embed Size (px)
Citation preview
Page 1
Issue 8 Q4 2009
Issue 8 Q4 2009 ................................................................................................................................. 1
Preface .............................................................................................................................................. 4
Sending Messages from ESB Toolkit to MSMQ ................................................................................... 5
Introduction .................................................................................................................................. 5
Resolvers ....................................................................................................................................... 5
Adapter Providers.......................................................................................................................... 5
Itinerary ........................................................................................................................................ 6
Sending messages to non Transactional MSMQ using WCF-Custom Adapter Provider ................... 6
Sending messages to Transactional MSMQ using WCF-Custom Adapter Provider ........................ 10
Creating Custom Adapter Provider for MSMQ ............................................................................. 11
Testing MSMQ Adapter Provider ................................................................................................. 13
Non Transactional Queue ........................................................................................................ 13
Transactional Queue ................................................................................................................ 14
Conclusion ................................................................................................................................... 15
Simplified BAM tracking of human intervention in a process ........................................................... 16
Working with BizTalk Adapter Pack 2.0 ............................................................................................ 20
Background ................................................................................................................................. 20
Architecture ................................................................................................................................ 24
Installation and Configuration...................................................................................................... 27
The BizTalk Adapter Pack 2.0 in action ......................................................................................... 31
Scenario and walkthroughs .......................................................................................................... 31
Adapter with BizTalk Server Walkthrough ................................................................................ 32
Adapter with .NET Walkthrough .............................................................................................. 38
Conclusion ................................................................................................................................... 40
Page 2
Resources .................................................................................................................................... 41
Acknowledgements ..................................................................................................................... 41
Empowering BizTalk management ................................................................................................... 42
Introduction ................................................................................................................................ 42
What is PowerShell ...................................................................................................................... 42
Why a provider for BizTalk ........................................................................................................... 43
Architecture ................................................................................................................................ 44
Management Automation Layer .............................................................................................. 44
Provider ................................................................................................................................... 46
Usage sample .............................................................................................................................. 47
Scenario Overview ................................................................................................................... 47
A CmdLet template .................................................................................................................. 48
Backing-up an application ........................................................................................................ 49
Uninstalling an application ....................................................................................................... 49
Removing assemblies from the GAC ......................................................................................... 50
Installing an application ........................................................................................................... 51
Upgrading an application ......................................................................................................... 52
Future plans ................................................................................................................................ 52
More information ........................................................................................................................ 52
About the authors ....................................................................................................................... 52
HL7 Accelerator v2.0 [BTAHL7] ........................................................................................................ 54
General ....................................................................................................................................... 54
HL7 Standard ............................................................................................................................... 54
BTAHL7 Components ................................................................................................................... 55
Schema Generator ................................................................................................................... 55
HL7 Disassembler & HL7 Assembler ......................................................................................... 55
MLLP Adapter .......................................................................................................................... 55
Batch Orchestration ................................................................................................................. 55
HL7 Configuration Explorer ...................................................................................................... 55
HL7 Schemas ............................................................................................................................... 55
Adding schemas to your solution ............................................................................................. 56
Schema name and namespace ................................................................................................. 56
Z-Segments .............................................................................................................................. 57
Custom Trigger Event ............................................................................................................... 58
Page 3
MLLP Adapter .............................................................................................................................. 58
Configuration ........................................................................................................................... 58
MLLP Test Tools ....................................................................................................................... 59
Performance ............................................................................................................................ 60
Acknowledgements ..................................................................................................................... 60
General .................................................................................................................................... 60
Custom ACK pipeline component ............................................................................................. 61
End-to-End Acknowledgements ............................................................................................... 62
How to Create a Custom Receive Pipeline Decoder in BizTalk Server 2009 ....................................... 63
Summary ..................................................................................................................................... 63
Install Microsoft BizTalk Server Pipeline Component Wizard ........................................................ 63
Use BizTalk Server Pipeline Component Wizard ........................................................................... 64
Add C# code for project ........................................................................................................... 67
Using Custom Component Created .......................................................................................... 68
Testing Receive Pipeline........................................................................................................... 70
Automating Business Processes with SharePoint, Workflow and BizTalk .......................................... 72
BizTalk Management Tools .............................................................................................................. 79
Operations Manager 2007 ........................................................................................................... 79
CODit SOA Dashboard.................................................................................................................. 79
FRENDS Helium ........................................................................................................................... 79
Page 4
Preface This is the 8th issue of BizTalk HotRod magazine. BizTalk Server 2009 R2 was anncounced recently.
We also published a new roadmap for it at http://www.microsoft.com/biztalk/en/us/roadmap.aspx.
BizTalk has come a long way since it was released in 2000. It now has over 10500 customers. These
customers are based in all 6 inhabitated continents. This issue has articles written by residents of
North America, South America, Europe and Africa. I want to thank the folks who submitted articles
for this issue. I also want to thank Andreas Erben for his help in reviewing these articles. Feel free to
submit articles in the next edition or let us know if you have any suggestions for improvement.
Page 5
Sending Messages from ESB Toolkit to MSMQ By Rajinder Singh, Technical Solutions Professional, Microsoft, http://rajsinghblog.wordpress.com
Introduction The BizTalk ESB Toolkit 2.0 is a collection of tools and libraries that extend BizTalk Server 2009
capabilities of supporting a loosely coupled and dynamic messaging architecture. You can learn more
about ESB Toolkit 2.0 at http://msdn.microsoft.com/en-us/biztalk/dd876606.aspx. This article will
show you how you can use ESB Toolkit to communicate with MSMQ.
ESB Toolkit uses dynamic send ports called Off Ramps to send messages to various BizTalk adapters.
It relies on .Net components called Resolvers and Adapter Providers.
Resolvers For runtime flexibility ESB Services are not hard-coded to specific endpoints or maps. This metadata
is determined at runtime. Resolvers can locate and retrieve this metadata. Itineraries define which
ESB services execute and in which order. Resolvers define how ESB Services execute. List of
Resolvers included in ESB Toolkit 2.0 are shown below.
Adapter Providers Information retrieved by a Resolver is stored in a .Net based Dictionary object. BizTalk isn’t built to
work with Dictionary objects. Adapter provider act as a bridge between .Net based ESB components
and the BizTalk based components. Here is a list of adapter providers that are included in ESB Toolkit
2.0.
Page 6
As you can see from the above list MSMQ Adapter provider is not included in ESB Toolkit. This is
where WCF-Custom adapter provider comes in handy. WCF-Custom adapter is able to invoke any
WCF binding. WCF includes netMsmqBinding which can be used to send messages to MSMQ.
Itinerary Any message that is sent to the BizTalk Server goes through a series of steps before its processing is
complete. In messaging only scenarios a BizTalk receive location might pick up a message and send it
to the MessageBox. One or more Send Ports might be subscribed to this message so they will get
this message and send it to the destination system. In case of an Orchestration the message will be
subscribed by an Orchestration where it may be decomposed, transformed and sent to one of more
system. All this can be easily accomplished by BizTalk without ESB Toolkit. However Itineraries
provide the runtime flexibility that BizTalk does not have by default. They also provide more flexible
Service Composition. Itineraries are designed in the Itinerary designer in Visual Studio. The designer
allows you do to design time validation of the itinerary. When the design activity is complete the
Itinerary is exported to an Itinerary Repository.
Sending messages to non Transactional MSMQ using WCF-Custom Adapter
Provider 1. Create a private MSMQ queue called test
2. Start Visual Studio 2008 and create a new project of type Class Library.
3. Add a new Itinerary to this project.
4. Set the name of the Itinerary to wcfmsmq
5. Select the Model Exporter to have a value “Database Itinerary Export”. This will enable
export of the itinerary to database.
6. Set the Itinerary Status to “Deployed”. This is similar to a business rules engine policy
deployment where a policy can be in published or deployed status.
Page 7
7. Set the Export Mode to “Strict”. This mode captures more details about the itinerary and
should be used for any new itinerary. The other mode is called “default” and it is there for
backward compatibility with ESB Guidance 1.0 Itineraries.
8. Set the Encryption Certificate Property if you want to encrypt your itinerary. If you want to
disable encryption you can see the instructions here
(http://blogs.msdn.com/pkelcey/archive/2009/06/08/disabling-itinerary-encryption-in-the-
esb-toolkit-2-0.aspx).
9. Other values in the properties windows can be left unchanged.
10. Drag On-Ramp to the design surface.
a. Update the name property of the OnRamp.
b. Select “On-Ramp ESB Extender” as the value of Extender
c. Set the value of BizTalk Application to “Microsoft.Practices.ESB”. This value should
be set to the application where OnRamp exists. In this example we are going to use
an OnRamp that exists in “Microsoft.Practices.ESB” application.
d. Select “OnRamp.Itinerary” receive port as the value of Receive Port. This port is a
one way receive port.
Page 8
11. Drag Itinerary Service to the design surface.
a. Set the name of this service to “RoutingService”
b. Set the value of Itinerary Service Extender to “Messaging Extender”. This instructs
ESB Toolkit to execute this setup in a pipeline.
c. Set the value of Service Name to be “Microsoft.Practices.ESB.Services.Routing”. This
service is a part of the ESB Toolkit. You can create your own messaging services and
register them in ESB.Config and they will become available to you in Visual Studio
Itinerary Designer.
d. Set the Container to “OnRamp1->ReceivePort”. This instructs ESB Toolkit to execute
the routing service in the pipeline for the Receive Port.
e. Tracking Enabled property determine if steps of the itinerary will be tracked in BAM.
f. Right click on the service and add a new Resolver.
g. Set the name of the resolver to “MsmqTest”
h. Set the value of Resolver Implementation to “STATIC”. This implies that we will hard
code the settings in the itinerary
i. Set the Transport Name to “WCF-Custom”
j. Set the Transport Location to “net.msmq://localhost/private/test”. This tells the
WCF Custom adapter to use netMsmqBinding of WCF to send message to MSMQ
k. Set the value of Action to any string. This value is not used in case but its value still
needs to be set.
l. Press the … button for Endpoint Configuration to bring up the configuration dialog
box.
i. Set the BindingType to be “netMsmqBinding”
ii. Set the BindingConfiguration to <binding
name="netMsmqBinding" durable="false"
exactlyOnce="false"><security mode="None"
/></binding>
This setting implies that MSMQ message will be not be transactional or
recoverable.
Page 9
12. Right click on the design surface and select “Validate”
13. If there are no errors you should right click and select “Export” Model
14. If there are no errors you can use test application
C:\Projects\Microsoft.Practices.ESB\Source\Samples\Itinerary\Source\ESB.Itinerary.Test\bin
\Debug\ESB.Itinerary.Test.exe
a. Set the Itinerary Name to be “wcfmsmq”
b. Set the Version to 1.0
c. Select the appropriate file in the Load Message section
d. Press the “Submit Request”
15. Verify that MSMQ message was written to the Test Queue.
Page 10
Sending messages to Transactional MSMQ using WCF-Custom Adapter
Provider Most of the steps required to accomplish this task are similar to the steps in the previous section.
The only differences are:
1. Create a new private Transaction MSMQ Queue called “trantest”
2. When you create a new resolver it should be named “MsmqTranTest”
3. Transport Location should be “net.msmq://localhost/private/trantest”
4. Binding Configuration should be set to
<binding name="netMsmqBinding" durable="true"
exactlyOnce="true"><security mode="None" /></binding>
Page 11
Creating Custom Adapter Provider for MSMQ We can easily extend ESB Toolkit to create a Custom Adapter Provider for MSMQ. The only
requirement for this is that the adapter which you are invoking has to support dynamic
behavior.
1. Create a new Class Library project in Visual Studio called
“Microsoft.Practices.ESB.Adapter.MSMQ”
2. Set a reference to C:\Program Files\Microsoft BizTalk ESB Toolkit
2.0\Bin\Microsoft.Practices.ESB.Adapter.dll
3. Add a strong name key to this project.
4. Add a new class called “AdapterProvider”
Page 12
5. Add a statement using Microsoft.Practices.ESB.Adapter;
6. Inherit the AdapterProvider class from BaseAdapterProvider class.
7. Override the properties AdapterName andAdapterContextPropertyNamespace as shown
below.
8. Add a new XML file to the project called MSMQPropertyManifest.xml. This file has to be
named in AdapterNamePropertyManifest.xml format. This file contains all the MSMQ
Adapter schema properties (http://technet.microsoft.com/en-
us/library/aa577593(BTS.10).aspx ). These are the properties that are used by BizTalk to
send a message to MSMQ. Partial list of these properties is shown below.
9. Build the class library and add it to Global Assembly Cache.
10. Copy the file MSMQPropertyManifest.xml to C:\Program Files\Microsoft BizTalk ESB
Toolkit 2.0\Tools\Itinerary Designer
11. Register the adapter provider in esb.config file by adding the line to the section where
other adapter providers have been registered.
<adapterProvider name="MSMQ"
type="Microsoft.Practices.ESB.Adapter.MSMQ.AdapterProvider,
Microsoft.Practices.ESB.Adapter.MSMQ, Version=1.0.0.0,
Culture=neutral, PublicKeyToken=d3afc588de1b9f91" moniker="MSMQ"
adapterAssembly="Microsoft.BizTalk.Adapter.MSMQ.MsmqAdapterProper
ties, Version=3.0.1.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35" />
Page 13
Here the name is the name of the adapter, Type is the .Net type of custom adapter
provider you just created, moniker is Transport Name in the Resolver configuration and
adapter assembly is the BizTalk assembly that contains all the context properties related
to the MSMQ Adapter.
Testing MSMQ Adapter Provider Most of the steps required to accomplish this task are similar to the steps in the previous section.
Non Transactional Queue
The only differences are:
Page 14
1. Name of the Itinerary should be ExpressMsmq.
2. After creating Itinerary Service called RoutingService, you need to add a new Resolver.
3. Resolver Implementation is still STATIC.
4. If your MSMQ Adapter provider registered correctly you should be able to select “MSMQ” as
the Transport Name.
5. Set Transport Location to FORMATNAME:DIRECT=OS:ESBWIN2008\PRIVATE$\TEST
6. You can once again use
C:\Projects\Microsoft.Practices.ESB\Source\Samples\Itinerary\Source\ESB.Itinerary.Test\bin
\Debug\ESB.Itinerary.Test.exe to test this itinerary.
Transactional Queue
The only differences are:
1. Name of the Itinerary should be TranMsmq.
2. After creating Itinerary Service called RoutingService, you need to add a new Resolver.
3. Resolver Implementation is still STATIC.
4. If your MSMQ Adapter provider registered correctly you should be able to select “MSMQ” as
the Transport Name.
5. Set Transport Location to FORMATNAME:DIRECT=OS:ESBWIN2008\PRIVATE$\TRANTEST
6. Press the … button for Endpoint Configuration to bring up the configuration dialog box and
change the value of Transactional Property to True.
7. You can once again use
C:\Projects\Microsoft.Practices.ESB\Source\Samples\Itinerary\Source\ESB.Itinerary.Test\bin
\Debug\ESB.Itinerary.Test.exe to test this itinerary.
Page 15
Conclusion ESB Toolkit provides Adapter Providers to allow you to communicate with various BizTalk adapters.
WCF-Custom adapter offers a lot of flexibility because it can use any WCF binding. For example WCF-
Custom can be used to communicate with any of the WCF based adapters that are included in
BizTalk Adapter Pack 2.0. When the existing Adapter Providers don’t meet your need you can easily
extend ESB Toolkit to create new Adapter Providers. You can download code samples used in this
article from my blog at http://rajsinghblog.wordpress.com
Page 16
Simplified BAM tracking of human intervention in a process
By Brian Loesgen, Principal SOA Architect, Microsoft Corporation, http://blog.brianloesgen.com
Earlier this year I created a demo that highlighted integration between BizTalk, ESB Toolkit, InfoPath and SharePoint. Although I was quite happy with my first implementation of my Order Demo (see prior post here that points at videos of completed solution), I wanted to incorporate some BAM tracking in my second version of the demo.
The scenario around the demo is very common and simple: if an order total amount is below a threshold amount, then it flows straight through. If the order total is above the threshold, then it is sent to a SharePoint process site where a human needs to approve/deny it. The BizTalk BRE is what’s making the decision: as an order is received it looks at the total order amount, and applies the appropriate ESB Toolkit itinerary to the order.
The data I wanted to track was simple: when orders needed to be approved or denied, I wanted to track how long it took someone to act on it, the disposition, etc.
The first way I thought of to implement this was go back to my order schema, and add in some new elements like InterventionStart, InterventionEnd and InterventionDisposition to capture the data. That would have worked, but I really didn’t want to change my source schemas just for this, particularly as there were multiple InfoPath forms built off those schemas that would need to be refreshed. Plus, it would mean carrying around content in a message that may only be used for a specialized activity tracking report. So, I started thinking…. there must be a better way. And, I was right.
The way I solved this was by looking at and leveraging the properties surfaced by the SharePoint adapter. For example, timing the duration of an intervention would be a simple matter of tracking “document creation time” (when it was written to the doclist, intervention completed would be the time I picked it up. As a bonus, I could also get other properties, such as what account last modified the document.
The beauty of this approach was that it was completely “touch free”. I didn’t have to go in and modify the document structure, any of the artifacts, nor the InfoPath forms. In fact, I deployed my tracking solution WITHOUT TOUCHING THE ALREADY DEPLOYED AND RUNNING BIZTALK SOLUTION. Pretty cool!
Here’s how the BAM portion of the solution is assembled.
My activity definition looks like this:
Page 17
My view definition looks like this:
Page 18
I used the Tracking Profile Editor to do the mapping from the tracking model to the physical data items and their respective collection points. Note that there is no orchestration in the mix, we are doing messaging-only BAM, and the screen shot below shows that I have set the port mapping to be the “ReceiveIntervenedOrders” port, and that the data I am picking up is “InCreated” from the SharePoint adapter properties.
And, that’s all there is to it from the collection standpoint. With the tracking profile deployed, any messages coming in through that receive port get the properties tracked.
Finally, now that we are capturing metrics, I needed to render them, and used my favorite tooling for this: SQL Server Reporting Services hosted in SharePoint. Although I rarely work with SSRS, this is what I was able to quickly cobble together:
Page 19
A couple of important points to note:
in case you missed it above, I implemented tracking intervention duration with no changes to the existing solution
Both human intervention and the interval tracking is completely de-coupled from the process. There is no orchestration running to time intervention and no corresponding correlation that has to happen. This simplified design means not needing to worry about items deleted from SharePoint, answers that never come, etc…
because I used Reporting Services to define the report, it would be trivial to set up a subscription to do something else with the report, such as run it every Monday morning and email a PDF to an alias
Elapsed time from the point of having the idea to having something working, only about 5 hours. I then spent a couple of hours tweaking the reports. It would have been faster if I were more familiar with SharePoint and Reporting Services.
I heard someone say once that good developers are lazy, as laziness makes them strive for the easiest way to solve problems. In this case, my laziness led me to what I see as a very clean and elegant solution pattern that I can see lots of people taking advantage of.
Page 20
Working with BizTalk Adapter Pack 2.0 By Steef-Jan Wiggers, Consultant, motion10
Microsoft’s BizTalk Server offering contains the BizTalk Adapter Pack. The Adapter Pack provides interoperability with multiple Line-Of-Business (LOB) applications like SAP, Siebel or Oracle. Currently version 2.0 of the adapter pack is available supporting BizTalk Server 2006 R2 and BizTalk Server 2009 for Developer, Standard, and Enterprise Editions. The Adapter Pack can be purchased separately and is licensed on a per processor basis and BizTalk Server is not required. BizTalk Adapter Pack 2.0 is a collection of key Line-of-Business (LOB) application adapters that enables any Windows application to integrate with LOB applications using the Windows Communication Foundation (WCF) programming model. These adapters are written using the WCF LOB Adapter SDK, a prerequisite to be able to use the pack. This article will dive into the background of the adapter pack, its architecture, how to install and configure the pack and describes a scenario of how the pack could be used.
Background As BizTalk Server evolved the number of adapters grew the Enterprise Adapters became first class citizen of BizTalk adapter’s collection. Out of the box BizTalk offers a lot of adapters that support numerous databases, protocols and applications. The adapter pack extended the number of adapters to provide more integration capabilities. Interoperability to SAP has been supported by Microsoft since 2000 with connectivity to SAP R/3. Adapters for SAP were based initially on the BizTalk Server Adapter Framework and later on the Adapter Pack 1.0 release which makes use of the Windows Communication Framework (WCF). Besides SAP Microsoft in August 2005 acquired eight .NET based application adapters from iWay. These adapters allowed BizTalk Server customers to easily connect to third-party applications like Amdocs, JD Edwards, Oracle Corp., PeopleSoft, Siebel Systems Inc. and TIBCO Software Inc. This purchase of the iWay adapters took place before the major release of BizTalk Server (2006), SQL Server (2005) and Visual Studio (2005). These adapters, which were based on the BizTalk Server Adapter Framework (.NET Framework 2.0), were bundled into the “Microsoft BizTalk Adapters for Enterprise Applications” offering which targeted at BizTalk 2006 and later BizTalk Server 2006 R2. The Adapter Framework is a set of extensible APIs. The APIs enable developers to take advantage of relevant services within BizTalk Server when building adapters and connectors that integrate third-party applications. Therefore also custom adapters could be made using this framework. Table 1 lists the adapters that are included in ‘Microsoft BizTalk Adapters for Enterprise Applications’.
Table 1: Adapters included in ‘Microsoft BizTalk Adapters for Enterprise
Applications’
Adapter Description Supported Version
J.D. Edwards Enterprise
One
8.10,8.11, and 8.12
Page 21
J.D. Edwards OneWorld XE B7.3.3.3 with Service Pack 23, and B7.3.3.4 with Service Pack 23
Oracle Database Provides access to Oracle Database tables, views, stored procedures, functions, and PL/SQL packages and is based on ODBC.
8i, 9i, and 10g
Oracle E-Business Suite Provides access to Oracle eBusiness Suite applications and is based on ODBC.
Oracle E-Business Suite Application 11.5.10.2
PeopleSoft Enterprise People Tools 8.17.02, 8.43, 8.45, 8.46, and 8.48
Siebel eBusiness
Applications
Provides access to Siebel eBusiness Applications via Business Objects, Business Components, and Business Services.
Version 6.2.1 with patch 110 or later, Version 7.0.*, Version 7.5.*, Version 7.7, and Version 7.8
TIBCO Enterprise Message
Service
TIBCO Enterprise Message Service (EMS) version 4.2
TIBCO Rendezvous TIBCO Rendezvous version 7.3
The LOB adapters provided through Microsoft BizTalk Adapters for Enterprise Applications consist of several components. Once these adapters are deployed you might find the following processes running in your environment as you use the design-time and run-time components of the adapters: • Adapter Framework – The adapters are built on the BizTalk Server Adapter Framework. The
Adapter Framework is a set of extensible APIs that enable developers to take advantage of relevant services of BizTalk Server when building adapters and connectors that integrate third-party applications.
• Java Runtime – The Adapter Framework, which is used by the LOB adapters, is dependent upon the Java Runtime (version 1.4.3 or later).
• Browser Agent – This process is used by the Adapter Framework and invoked by Visual Studio .NET. It retrieves and formats metadata from the LOB systems into XSD files that the BizTalk Server development environment can then consume and use. After five minutes of inactivity, it automatically shuts down to release any resources, such as memory, that may have been used. This process appears in Task Manager as browsingagent.exe.
• Runtime Agent – This process is used by the Adapter Framework and is invoked by the BizTalk Server runtime (BtsNtSvc.exe) to call business functions on the LOB systems. Unlike the Browser Agent, the Runtime Agent does not shut down after a period of inactivity. This process appears in Task Manager as runtimeagent.exe.
• Connectors, JAR files – JAR files contain the set of Java classes specific to the LOB system. The Adapter Framework exposes these classes as Java interfaces, which are used to communicate with the BizTalk Server design-time and run-time executables.
The figure below shows the high-level architecture of both the design-time and run-time
components and how they interface with the LOB adapters and the LOB system.
Page 22
Figure 1: LOB Adapter process flow.
Microsoft continued to work on improvements of the LOB adapters and in March 2008 the BizTalk Adapter Pack 1.0 was made available. This pack contained three adapters for Siebel, SAP and Oracle Database based on WCF LOB Adapter SDK. It offered the possibility to connect to a variety of clients (such as Microsoft Office, BizTalk Server, and custom .NET applications). The SDK inherently relies on the core WCF concepts, principles, and classes implemented in the .NET Framework 3.0 and later. It provides for a simplified programming model on top of WCF Channel Model to enable adapter developers in creating service-oriented interfaces to existing systems and applications. Microsoft delivered this Pack to enable a flexible integration and application infrastructure connecting LOB systems with a unified open approach on the technology of the WCF programming model. The WCF LOB Adapter SDK provides a set of run-time and design-time components, a .NET object model, and support components including:
Component Description WCF LOB Adapter Development Wizard Provides step-by-step guidance in the creation of
WCF LOB Adapter SDK projects within Visual Studio .NET.
WCF LOB Adapter Service Development Wizard Provides step-by-step guidance in creating a Web project to host an adapter in Internet Information Services (IIS).
WCF LOB Adapter SDK run-time system Supports the WCF LOB Adapter SDK by extending the WCF channel architecture and providing other run-time services.
WCF LOB Adapter SDK object model A collection of classes, types, and interfaces that support common adapter tasks such as metadata normalization, caching, connection management and pooling, and messaging inspection.
Add Adapter Service Reference Plug-in Gives custom .NET applications the ability to consume adapters developed using the WCF LOB Adapter SDK.
Consume Adapter Service Add-in Gives BizTalk Server the ability to consume adapters developed using the WCF LOB Adapter SDK.
Page 23
The architecture of WCF LOB Adapter SDK will be described in following architecture paragraph. Picture below will give some insight how adapters interfacing with LOB systems changed with BizTalk Adapter Pack.
Figure 2: BizTalk Adapter Pack 1.0 Architecture
With BizTalk 2009 Microsoft continued their commitment to simplify interoperability. In Server Adapter Pack 2.0, the BizTalk Server Adapter Pack 1.0 was extended with Oracle E-Business Adapter and SQL Server Adapter. This SQL Server adapter replaces the native SQL adapter offered out-of-the-box from BizTalk Server and offers many more features, to name a few:
Support for new data types in SQL2005 and SQL2008 (notably FILESTREAM, varbinary (max))
Insert/Update/Delete/Select operations on tables and views
Invoke (SQL and CLR) Stored procedures, scalar valued functions , table value functions
Supports execution of generic T-SQL statements, via the ExecuteReader(), ExecuteNonQuery() and ExecuteScalar() operations
Configurable polling – polling statement, polling interval, receive multiple result sets
Support for x86 and x64 platforms What is included in the Adapter Pack 2.0 is displayed in table 2 below.
Table 2: Adapter Pack 2.0 features
Adapter Description Supported Version
SAP Supports execution of BAPIs, RFCs, and tRFCs, data exchange using IDOCs, and receiving events from SAP with a generic RFC Server.
SAP R/3 4.x and R/3 6.20
(Enterprise) SAP ECC 6.0
Siebel
eBusiness
Applications
Provides access to Siebel eBusiness Applications via Business Objects, Business Components, and Business Services.
7.5.3, 7.7, 7.8, and 8.0
Page 24
Oracle
eBusiness Suite
Provides access to Oracle eBusiness Suite applications and is based on ADO.NET.
ODP.NET 11.1.0.7, Oracle
EBS 11.5.9, 11.5.10 and
12
Oracle
Database
Provides access to Oracle Database tables, views, stored procedures, functions, and PL/SQL packages and is based on ADO.NET. Timed polling is also provided, allowing applications to periodically look for changes in the database.
Oracle 9i (9.2.0.2), 10g
(10.1.0.2.0 & 10.2.0.1.0)
SQL Server Provides access to SQL Server databases. 2000, 2005, and 2008
WCF LOB
Adapter SDK
Provides a simple programming model for developing adapters that are based on Windows Communication Foundation. This is a separate download available to everyone using the .NET Framework 3.0.
1.1
What is very important to note here are the supported versions; if for instance you need
compatibility with earlier versions of LOB applications, you will need to use older versions of
adapters like ‘Microsoft BizTalk Adapters for Enterprise Applications’ (non-WCF LOB Adapters).
These non-WCF LOB adapters are deprecated in BizTalk Server 2009 and to achieve a better
supportability you might need to use an older version of BizTalk.
Architecture Adapters in the Adapter Pack 2.0 are based on WCF LOB Adapter SDK, which consists of a runtime, a
collection of APIs, and design-time tools for creating adapters that expose data and operations from
line-of-business systems. Adapters manage messages between the adapter consumer and the line-
of-business system and can consist of metadata, data, or other information. Figure 3 shows the
internal architecture and main components of WCF LOB Adapter SDK.
Page 25
Figure 3: Architecture of WCF-Based Adapters.
Picture above shows to types of developers:
1. Solution developer consuming an adapter.
2. Adapter developer implementing a custom adapter.
Adapter development will not be discussed in this article and focus is on the solution developer
working with the Adapter Pack 2.0. Channel implementation is an important part of architecture. An
adapter built using the WCF LOB Adapter SDK is basically a transport channel
(System.ServiceModel.Channels.IServiceListner). It is surfaced to the consumer as a WCF binding,
where the binding is used to create the channel stack. This binding can be considered a peer to other
predefined WCF bindings such as BasicHttpBinding, WsHttpBinding, and NetTcpBinding, and can be
set via app.config or in code by the client application when calling a service. This binding contains an
ordered set of binding elements, with the adapter being the key binding element that derives from
Page 26
the T:System.ServiceModel.Channels.TransportBindingElement class. In an outbound scenario, the
WCF LOB Adapter SDK runtime uses a channel factory to create the adapter (that is, the transport
channel). This scenario will be used to walkthrough usage of adapter pack 2.0 later on in this article.
In an inbound scenario, the WCF LOB Adapter SDK runtime makes use of channel listeners for
incoming channels in a service application. Both run-time as well as design-time messages pass
through this component.
Other important parts of WCF-based architecture for an adapter consumer are:
The Connection Uri Builder allows adapter consumers to programmatically build connection URIs without specific knowledge of syntax.
The Metadata Exchanger is responsible to handles the client’s metadata messages including Adapter SDK’s Browse / Search.
The WSDL Builder provides automatic WSDL generation from the WCF LOB Adapter SDK's internal metadata object model (it can be overridden for scenarios that require custom WSDL generation).
The Metadata Browse/Search allows for browsing and searching all LOB metadata.
The Metadata management is responsible for the object-oriented representation of the caching of the metadata for the target system. Metadata can be held in a common cache accessible across all credentials, or it can be cached per credential basis.
The Connection Pool Management is responsible for the lifetime management of the connections of an adapter. It internally keeps a pool of the connections ready for use. This connection pool is credential and URI-based. The credential contains a user name and password that define the security context the connection runs under.
And important tool for solution developers is the Adapter Metadata Utility (through Add Adapter
Service Reference Plug-in, Consume Adapter Service Add-in) available within Visual Studio, once the
WCF LOB Adapter SDK is installed in your development environment. This will be discussed later on
in the next paragraph. This utility allows the solution developer to generating code for the client (for
outbound scenarios) and for the service (for inbound scenarios) based on operations selected by the
solution developer.
The goal of the WCF LOB Adapter SDK is to facilitate uniform development of reusable metadata-
oriented WCF-based adapters that enable enterprise applications, databases and messaging
platforms to integrate with each other. The picture below will show this with BizTalk Adapter Pack
2.0 in mind which differs with Adapter Pack 1.0 (see picture 2).
Page 27
Figure 3: Adapter Landscape of the BizTalk Adapter Pack 2.0.
Installation and Configuration To install the BizTalk Adapter Pack 2.0 you will need an environment or a virtual machine with for instance the following components installed:
Operating System like Windows Server 2008 Standard or Enterprise;
SQL Server 2008 Developer Edition;
Microsoft .NET Framework 3.5 SP1;
Microsoft Visual Studio 2008 SP1;
BizTalk Server 2009.
The required environment above will be used as a starting point for usage scenarios for BizTalk Adapter Pack 2.0. Other environments are possible having BizTalk Server 2006R2 and Visual Studio 2005. To install the adapter pack 2.0 you will first need to install WCF LOB Adapter SDK SP2. Installation of the SDK is straight forward, but you may run into a pop window telling you to wait for installer to determine disk space. You can get around this problem by executing the following line (run the installation with reduced UI): msiexec /i “<directoryname>\AdapterFramework.msi”/qr
An important note here is that you have to assure that with WCF LOB Adapter SDK you install Runtime, Tools, BizTalk Server Add in and Samples. You can do it running following command line: msiexec /i “<directoryname>\AdapterFramework.msi”/
You probably will not be bothered by pop-window and you can choose/change and add Runtime, and other components as you can see below in the picture.
Page 28
Picture 4: WCF LOB Adapter SDK Setup features to install. The last step to get the Adapter Pack 2.0 installed is the pack itself, available as a 120-day evaluation edition or an officially licensed one through Microsoft. Again a pop-window can ask you to wait for installer to determine disk space and you can use command line if this takes too much time as described above. Installation of Adapter Pack 2.0 is one thing the other will be an enterprise application supported by the pack (see figure 3). To support the walkthroughs with Adapter Pack and Oracle in coming paragraphs I will shortly describe the installation of Oracle 10g Expression that easily be installed on VM having described configuration. Oracle Express Edition can be easily obtained through Oracle Technology Network (OTN) after you have registered yourself. For Oracle Database 10g Express Edition installation you can do the following:
1. Password for both SYS and SYSTEM database you can set for instance: welcome1 2. Below will be shown in next screen and you can leave it as is:
Destination Folder: C:\oraclexe\ Port for 'Oracle Database Listener': 1521 Port for 'Oracle Services for Microsoft Transaction Server': 2030 Port for HTTP Listener: 8080
The above is basically all you have to do for the installation of Oracle 10g Express Edition. The installation will finish with the option to launch the database home page, which you can do to log into the database through SYSTEM account. You then need to unlock the sample user necessary for this scenario. To unlock the sample user account for HR Schema you need to do the following:
Page 29
1. Click the Administration icon, and then click Database Users. 2. Click the HR schema icon to display the user information for HR.
Figure 5: Screenshot of Oracle Database Express Edition: Manage Database Users. 3. Under Manage Database User, enter the following settings:
•Password and Confirm Password: Enter hr for the password. •Account Status: Select Unlocked. •Roles: Ensure that both CONNECT and RESOURCE is enabled.
Page 30
Figure 6: Screenshot of Oracle Database Express Edition: Manage Database User.
4. Click Alter User. Once you have set this up you can check HR Schema by logging in with HR account and password. Go to Browser Objects (1) and select for instance Employees table (2) and then select data (3) and you will see what is shown below.
Figure 7: Screenshot of Object Browser showing data.
Page 31
To work with the Adapter Pack 2.0 and Oracle you will need Oracle.DataAccess.dll version 2.111.7.0. This is included in Oracle 11g ODAC and Oracle Developer Tools for Visual Studio 11.1.0.7.20 and available through OTN. ODAC is a zip file you will need to extract. After extraction you are able to execute the setup. In setup you will need to select Oracle Data Access Components for Oracle Client 11.1.0.7.20. In next screen leave everything as is and proceed to available product components. You only have to select Oracle Data Provider component as shown below, and Oracle Services For Microsoft Transaction Server (additional Oracle Instant Client is selected by Wizard and is required!). To start installation click next and install and check GAC afterwards to verify is correct Oracle.DataAccess.dll is present like picture below.
Figure 8: Screenshot of Assembly Cache showing Oracle.DataAccess assembly version.
Having done all the preparations above us we can now dive into how to build an application that retrieves data from backend system in this case Oracle. Through provided Oracle adapter interoperability can be achieved.
The BizTalk Adapter Pack 2.0 in action
Scenario and walkthroughs We will walkthrough how to retrieve data from Oracle’s test DEPARTMENTS table; firstly from a BizTalk perspective using an orchestration, schema and query message and then from a windows application perspective using a .NET client proxy code and a Windows form. Microsoft offers a wide variety of examples based on scenario’s shown in picture 9.
Page 32
Figure 9: Usage and Hosting Scenario’s. Through this location: http://msdn.microsoft.com/en-us/biztalk/dd796258.aspx, the examples of scenario’s depicted in picture 9 can be obtained. These samples are built with Visual Studio 2005 and need to be migrated if you want to apply them in a BizTalk Server 2009 environment. Walkthroughs described following paragraphs will provide for an example of integration with Oracle 10g Express Edition with:
1. Microsoft BizTalk Server using Metadata Utility tool to generate XML Schema’s. 2. Microsoft .NET using Meta Utility tool to generate .NET Client Proxy.
Adapter with BizTalk Server Walkthrough
The following tasks need to be performed in Visual Studio:
1. Create a new Empty BizTalk project and Name it: HR_Messaging. 2. Go to your project and right-click, navigate to Add Add Generated Items … and click on it 3. Select Add Adapter Metadata, click Add 4. Select WCF-OracleDB from list of registered adapters and click Next 5. The Consume Adapter Service Window will pop up (Adapter Metadata Utility Tool) and you
can select OracleDB binding and click on configure and pop up window will appear called configure adapter.
Page 33
Figure 10: Screenshot of Configure Adapter Window.
6. Within Configure Window select on security tab Username for client credential type. 7. Fill in user name HR and password hr. 8. Select next Tab URI Properties and fill in the Server Address: that is your computer name and
Service Name: XE and then click OK. 9. Click Connect and contract type and category should appear. 10. Select HR in category and click, and navigate to the DEPARTMENTS table.
Page 34
Figure 11: Screenshot of Consume Adapter Service Add In.
11. Choose Select in available categories and operations and click Add. 12. Click Ok and a schema will be generated. 13. Rename BizTalk Orchestration to Department_Query.odx. 14. Delete multi-message parts in orchestration viewer. 15. Place four shapes in the orchestration designer: Receive, Send, Receive, Send and name
them accordingly: a. ReceiveMessage b. SendMessage c. ReceiveResponse d. SendResponse
16. In the Orchestration View create two messages by right clicking on message and selecting add new message:
a. First message is a request message: i. Identifier: Request ii. Message Type: HR_Messaging.OracleDBBinding_DEPARTMENTS.Select
b. Second message is a response message: i. Identifier: Response
Page 35
ii. Message Type: HR_Messaging.OracleDBBinding_DEPARTMENTS.SelectResponse
17. Next create three ports: a. Click new configured port:
i. Name: FileIn ii. PortTypeName: FileInType iii. Communication Pattern: One-Way iv. Access Restriction: Internal – limited to this project v. Port direction of communication: I’ll always be receiving messages on this
port vi. Port Binding: Specify later vii. Operation Identifier: Select
b. Click new configured port: i. Name: LOBPort ii. Use Exsisting Port: HR_Messaging.HR_Table_DEPARTMENTS iii. Port direction of communication: I’ll be a sending a request and receiving a
response iv. Port Binding: Specify later
c. Click new configured port: i. Name: SaveResponse ii. PortTypeName: SaveResponse Type iii. Communication Pattern: One-Way iv. Access Restriction: Internal – limited to this project v. Port direction of communication: I’ll always be sending messages on this
port vi. Port Binding: Specify later vii. Operation Identifier: Select
18. Configure the shapes accordingly: a. ReceiveMessage Shape
i. Message: Request ii. Activate: True iii. Connect to FileIn Port (Request)
b. SendMessage i. Message: Request ii. Connect to LOBPort (Request)
c. ReceiveResponse i. Message: Response ii. Connect to LOBPort (Response)
d. SendResponse i. Message: Response ii. Connect to SaveResponse (Response)
19. Your orchestration should look like picture below.
Page 36
Figure 12: Screenshot of Department Query Orchestration.
20. Next build your solution. 21. Right-click your project in solution explorer and go to signing 22. Select sign assembly and click new
a. Name: HR_Messaging b. Unselect Protect my key file with a password c. Click Ok
23. Go to deployment tab and fill in Application Name: HR_Messaging 24. Right click on project and click Deploy. 25. Create an instance of Select Message by right clicking on your generated schema and click
generate instance. Message: <ns0:Select xmlns:ns0="http://Microsoft.LobServices.OracleDB/2007/03/HR/Table/DEPARTMENTS">
<ns0:COLUMN_NAMES>COLUMN_NAMES_0</ns0:COLUMN_NAMES> <ns0:FILTER>FILTER_0</ns0:FILTER>
</ns0:Select>
26. Click on URL in Output Window and copy message to notepad 27. Save it as TestInstance.xml, with UTF-8 encoding in the folder where your solution resides. 28. Create in that folder two new folders called IN and RESPONSE. 29. Go to BizTalk Administration Console and open BizTalk Group. 30. Navigate to HR_Messaging Application and right-click to Configure.
Page 37
31. Select Orchestration and select BizTalk Host. 32. Next configure ports:
a. New Receive Port: i. Name: ReceivePort FileIn ii. Receive Location: new
1. Name: Receive Location FileIn 2. Type: File
a. Configure file location (folder in solution directory called IN) 3. Receive Pipeline: XML Receive
b. New Send Port i. Name: Send Port LOB ii. Type: WCF-Custom
1. Configure select on Binding Tab the Binding Type OracleDBBinding
Figure 13: Screenshot of WCF-Custom Transport Properties Window.
iii. On General tab fill in URI: oracledb://<your machine name> iv. SOAP Action Header:
Page 38
Action: http://Microsoft.LobServices.OracleDB/2007/03/HR/Table/DEPARTMENTS/SELECT
c. New Send Port: i. Name: SendPort Response ii. Type: File
1. Configure file location (folder in solution directory called RESPONSE) iii. Receive Pipeline: PassThruTransmit
33. Finally right click application and click Start. 34. Check if proper access rights are applied to folders, if not do so by right click folders IN and
RESPONSE and add BizTalk Application Users in Security Tab. 35. Modify Test Instance message by opening it and change as displayed below:
<Select xmlns="http://Microsoft.LobServices.OracleDB/2007/03/HR/Table/DEPARTMENTS">
<COLUMN_NAMES>*</COLUMN_NAMES> <FILTER>DEPARTMENT_NAME=’IT’</FILTER>
</Select>
36. Save the message as depicted above. 37. Copy TestInstance.Xml to IN folder 38. Browse to the RESPONSE folder and a message similar to the one below should have been
received as a response:
Figure 14: Screenshot of response message.
Adapter with .NET Walkthrough
The following tasks need to be performed in Visual Studio:
1. Create a new Windows Forms Application project and Name it: HRApplication. 2. Navigate to Form1 in your project and rename it to DEPARMENT SEARCH. 3. In property windows of form rename Text to Department Search 4. On form place the following controls:
a. Label i. Name: lblDepartment ii. Text: Department
Page 39
b. Textbox i. Name: txtDepartment
c. Button i. Name: btnSearch ii. Text: Search
d. Label i. Name: lblResult ii. Text: Result
e. Textbox i. Name:txtResult ii.
5. Form should look like something below
Figure 15: Screenshot of Windows form.
6. Next right click you project and click Add Adapter Service Reference … 7. The Add Adapter Reference Window will pop up (Adapter Metadata Utility Tool) and you can
select OracleDB binding and click on configure and pop up window will appear called configure adapter.
8. Within Configure Window select on security tab Username for client credential type. 9. Fill in user name HR and password hr. 10. Select next Tab URI Properties and fill in Server Address: that is your computer name and
Service Name: XE and then click OK. 11. Click Connect and contract type and category should appear. 12. Select HR in category and click, and navigate to tables to DEPARTMENTS. 13. Choose Select in available categories and operations and click Add. 14. Click Ok and a .NET class OracleDBBindingClient.cs will be generated. 15. The generated class acts as a .NET Client Proxy. Now go to Windows Form and click on
Search button. 16. Paste the following code in the code behind:
using microsoft.lobservices.oracledb._2007._03.HR.Table.DEPARTMENTS;
17. Paste the following code inside the brackets of bth_Search_Click():
Page 40
18. Build solution by right clicking project in solution explorer and click build. 19. Click F5 to run. 20. Fill in ‘IT’ in Department and Click Search. 21. Result should look like screen below:
Figure 16: Screenshot of Windows form performing a search.
Conclusion In this article I gave some historical background on BizTalk’s LOB Adapters while focusing on the current BizTalk Adapter Pack 2.0 and its WCF programming model. Two walkthroughs provided Adapter Pack usage scenarios; one using BizTalk Server and the other using Microsoft .NET. The Adapter Pack provides versatile and consistent interoperability with LOB systems like SAP, Siebel, Oracle, and TIBCO. Data can be exposed through adapters to BizTalk Server, .NET, MOSS, SQL Server
Page 41
or IIS depending on usage/hosting scenarios. Migration of solutions between these different usage scenarios is made easier due to the shared use of the WCF programming model. Do bear in mind that the version of the LOB system you want to integrate with depends on supported versions of Adapter Pack 2.0. If you require interoperability with older versions of a LOB system you may have to turn to a non-WCF LOB Adapter as mentioned in this article.
Resources Microsoft Adapter Pack 1.0:
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=58bd17c8-8efa-4b69-
ab51-9f5b11a962dc
Microsoft BizTalk Adapter Pack 2.0 Installation Guide:
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=ff0b19a6-72c1-43b3-
9d38-b874303590ce
BizTalk Adapter Pack 2.0 now Released!!!:
http://blogs.breezetraining.com.au/mickb/2009/05/06/BizTalkAdapterPack20NowReleased.aspx
WCF LOB Adapter SDK: http://www.microsoft.com/downloads/details.aspx?familyid=56278FDE-
B708-469C-987E-DED9C6C5E580&displaylang=en
BizTalk Adapter Pack 2.0: http://www.microsoft.com/biztalk/en/us/adapter-pack.aspx
Oracle 10g Express Edition: http://www.oracle.com/technology/products/database/xe/index.html
BizTalk Adapter Pack 2.0 Oracle Binding: http://soa-thoughts.blogspot.com/2009/08/biztalk-
adapter-pack-20-oraclebinding.html
Microsoft Adds Eight Application Adapters for BizTalk Server:
http://www.microsoft.com/presspass/press/2005/aug05/08-02applicationadapterspr.mspx
WCF LOB Adapter SDK and BizTalk Adapter Pack: http://blogs.msdn.com/adapters/default.aspx
Whitepaper: WCF LOB Adapter SDK and the BizTalk Adapter Pack:
http://www.pluralsight.com/community/blogs/aaron/archive/2009/04/27/whitepaper-on-wcf-lob-
adapter-sdk-and-the-biztalk-adapter-pack.aspx
Acknowledgements I would like to thank my colleagues Sander Nefs, Gary Daly and Tomasso Groenendijk from motion10
helping through auditing and editing this article. Thanks a lot guys!
Page 42
Empowering BizTalk management Introducing the PowerShell provider for BizTalk
By: Maxime Labelle, Michel Hubert and Randal van Splunteren
Introduction Management is a fundamental key for a successful BizTalk platform. In fact, because of BizTalk
typically addressing integration scenarios managing flow of data, the availability of a BizTalk
platform depends on upstream and downstream systems. Moreover, some flows are near real time.
So, robust management capabilities are crucial to run those often mission critical systems. By
default, Microsoft provides some useful tools to administrate a BizTalk platform. For example, we
can notice the Administration Console, a command-line tool and APIs. But the provided tools are not
easy-to-use by IT Operations. The APIs are complex and heterogeneous, indeed, to administrate
BizTalk, you have deal with various technologies like WMI, runtime Dlls, registry keys, some
information on the management database and more.
The purpose of this project is to provide more useful and easy tools mainly for IT Operations in order
to :
Automate administrative tasks
Automate deployment of BizTalk applications avoiding human errors
This article describes how Windows PowerShell can be utilized to achieve those goals.
What is PowerShell Windows PowerShell is Microsoft’s main shell and scripting technology. The first version (v 1.0) was
released in 2006, in October 2009 the second version (v 2.0) came out.
Every administrator or developer knows it is quite cumbersome to automate tasks using batch files,
VBScripts or Windows Scripting Host. There is no uniformity, it is very hard to debug and lacks easy
ways to work with variables across different processes to name only a couple of disadvantages. In
fact PowerShell is Microsoft’s effort to overcome the limitations of the old ‘cmd.exe’ shell and
accompanying scripting technologies.
PowerShell has been built from the ground-up with four principles in mind:
Object orientation – PowerShell itself is a .Net application, hence leverages the .Net
Framework. Everything in PowerShell is an object. While other shells exchange information
between commands using text strings, PowerShell exchanges object instances.
Extension mechanism – PowerShell comes with a consistent, well-documented API that
makes it easy to extend it using a .Net language like (C# or Visual Basic). The API provides
interfaces for building new commands, hosts, providers, etc.
Backward compatibility – Existing scripts and batch files run in PowerShell. Users don’t have
to throw away scripts they currently use. This makes it possible to migrate gradually and use
heterogeneous scripts to do tasks.
Page 43
Consistency and uniformity – PowerShell makes it possible to manage and control different
technologies. Examples are SQL Server, Exchange, file system, the registry, etc. The great
thing is that you have to learn only one scripting language and syntax to control it all.
In the above diagram the architecture of PowerShell is shown. Its modular structure makes it easy to
extend. You can build for example your own graphical user interface (GUI) to host the runtime.
PowerShell (version 2.0) comes with two host implementations out of the box, a command line shell
and Integrated Scripting Environment (ISE). If this does not meet the requirements you can build
your own host.
Features like remoting (in version 2.0) can make this even more powerful.
At the bottom of the diagram we see the cmdlet providers and cmdlets. cmdlets represent the
implementation of commands. A cmdlet provider is the implementation of a set of related cmdlets
used to browse and manage a certain technology or product. PowerShell comes out of the box with
providers for the file system, the registry and environment variables.
Also we will probably see more and more products have their management GUI run on top of
PowerShell. Microsoft Exchange Server 2007 for example has its own PowerShell provider to manage
the server. The Exchange management console is built on top of this provider and uses it to process
management commands. When the user needs scripting, the provider can be “accessed directly” by
using PowerShell shell.
Why a provider for BizTalk As we’ve seen, Windows PowerShell relies on a set of additional software components, known as
providers, the purpose of which is to abstract away the intricacies and specificities of a given server
platform while exposing its underlying functionality in a consistent manner.
Fortunately, Windows PowerShell has been designed from the ground up as a highly open and
extensible platform. Therefore it is capable of supporting an arbitrary number of third-party
providers that help administer otherwise unsupported platforms.
Page 44
Today, many server software components, even originating from third-party vendors, come with
PowerShell bindings, either built-in or in addition. But, this is still not the case for BizTalk. Even as of
BizTalk Server 2009, no news of a potential provider came from Redmond. With all the effort
Microsoft is devoting towards making Windows PowerShell a key piece of its strategy with regards
to the next generation of its server platforms, it only made sense to build a provider for BizTalk.
Incidentally, as of this writing, Microsoft BizTalk Server 2009 R2 has been officially announced and,
pursuant to this policy, Microsoft has announced the support of PowerShell for the most common
used administration and management tasks. It is still unclear though, as to what the real scope of
the future provider will be.
Anyhow, with our provider, a user can now automate a large spectrum of administrative tasks in a
single consistent way. In fact, the most common deployment activities with regards to BizTalk
solutions are supported, with more to come in future releases.
Thanks to its respectable set of built-in cmdlets, the provider for BizTalk allows to quickly interact
with a BizTalk solution from the command-line. Combined with the flexibility of the command-line
pipeline, the provider makes it easy to perform various tasks on many BizTalk artifacts
simultaneously with previously unmatched feedback and speed.
Equally as important, the hierarchical nature of the underlying object model allows designing a rich
library of reusable scripts for systems administrators and developers. This object model supports a
virtual data store that has been modeled after the BizTalk Server Administration MMC Snap-In, so as
to make it familiar to any seasoned “BizTalker”.
Coupled with other existing providers – e.g. Internet Information Services, SQL Server – or cmdlets –
e.g. Event Log - the PowerShell provider for BizTalk adds another powerful tool to the systems
administrator’s toolbox.
Architecture The project is divided into two parts: a core object model and a PowerShell provider for BizTalk.
The object model must be considered as the main part of the project and we can consider the
PowerShell provider as an example of the implementation of the object model.
The object model provides a unique and uniform way to administrate a BizTalk Platform. It can be
used by any kind of application. The PowerShell provider allows a better integration into System
Center Operations Manager for example and gives IT Operations the capabilities to manage the
platform efficiently and to automate by scripting administrative tasks.
Management Automation Layer
The BizTalk Management Automation layer is the fundamental building block on which the provider
is actually built. It provides a façade over the numerous and heterogeneous libraries supporting the
automation of a BizTalk group.
Page 45
This layer serves several purposes:
First, it helps unify the various technologies one needs to use in order to effectively program against
a BizTalk group.
Indeed, the BizTalk runtime on which the automation layer is built spans a wide range of different
assemblies and programming paradigms. The Microsoft.BizTalk.ExplorerOM assembly, used for
starting and stopping applications for instance, exposes a neat hierarchical object model that
supports transactional access to the underlying management database. Whereas the
Microsoft.BizTalk.ApplicationDeployment assembly, used for exporting an application and creating a
Windows Installer package, makes direct use of the SQL Server data access client libraries and
commits changes immediately to the database. Other tasks, such as programmatically creating a
new BizTalk host or host instance, can only be carried out through the use of the BizTalk WMI
provider. Finally, working with the Business Rules Engine requires a totally different API and object
model.
That’s why the automation layer tries very hard to iron out all these differences. This is an important
goal by itself. Without the automation layer, all those differences would ultimately trickle down to
the end user. Because PowerShell lets you work with objects, it would force script authors to know
and understand these differences when using the provider.
By building an abstraction over the BizTalk runtime libraries, the automation layer exposes a
complete, stable and hierarchical object model over BizTalk artifacts. It enables you to perform
various tasks on BizTalk groups, including administrating and deploying BizTalk solutions. In the same
way that Windows Powershell aims to provide a consistent way to administer several heterogeneous
platforms, the goal of the automation layer is to provide a façade over various distinct underlying
technologies in a single unified object library.
Another purpose of this layer is to provide a strong foundation on which to build additional tools
that target the BizTalk programming model. Although the boundary between dynamic and static
languages is increasingly becoming somewhat fuzzy, the scenarios for using scripts versus statically
Page 46
compiled programs do not completely overlap. If you need a more long-term solution than what
scripts can provide, you may want to create a full blown .Net application or framework to
accommodate your needs.
It is our aim for the BizTalk Management Automation layer that it is adopted by a wider audience of
software developers and architects in order to help them support any number of unanticipated
scenarios.
The primary role for the BizTalk Management Automation Layer is to be a supporting library for the
provider, but it can be used as a standalone project that does not in fact incur any dependencies on
PowerShell. This makes it an ideal candidate to use in your own programming projects around
administrating and manipulating BizTalk artifacts.
Provider
It is obvious that the PowerShell provider for BizTalk lets the user manage BizTalk. Of course this is
done in the uniform PowerShell way. Basically it provides two things:
a way of navigating through the BizTalk artifacts
a set of BizTalk specific cmdlets (acting upon the fore mentioned BizTalk artifacts)
As far as navigation is concerned the provider has the same structure of artifacts that we’re used to
in the BizTalk administration console. This means we start at the top level with the BizTalk group,
which in PowerShell terms is called a drive. One level below we find the applications and platform
settings container, etc. The figure below shows a screenshot of how we can navigate through this
structure.
Page 47
The BizTalk cmdlets are used to instruct BizTalk to perform a certain task. In the current release of
the provider there are about 27 cmdlets implemented. Some samples are: Start-Application, Add-
Reference and Restart-HostInstance.
As mentioned before PowerShell works with objects. This means all commands that return data
actually return objects and all commands that process data need at least one object as input. The
BizTalk PowerShell provider does this by providing and returning objects from the BizTalk
Management Automation Layer. For example when we get a list of all BizTalk applications for a
certain BizTalk group we can use the Get-ChildItem command on the applications container. This
command returns an array of BtsApplication objects from the Management Automation Layer. Also
when we for example want to start a host instance we would use the Start-HostInstance cmdlet. This
cmdlet expects a parameter of type BtsHostInstance object or a path pointing to a BtsHostInstance
object.
Usage sample One of the main tasks of a BizTalk administrator is to help to install incremental versions of an
application during the quality assurance phase. As part of this work, one has to perform a set of
operations on one or several BizTalk applications in order to ensure a high level of availability. This is
particularly true if this solution is part of a greater project in the enterprise.
In order to assess the quality and conformance of an upgraded application, the following tasks are
required:
Backup an existing application, including the deployed assemblies and associated bindings,
Potentially remove an existing application, including removing assemblies from the GAC,
Import the updated application and its bindings.
Scenario Overview
This section demonstrates, in the scope of a very simple application, how the provider for BizTalk
makes it easy to address this scenario. For the purposes of this article, let’s focus on an application
that receives, as part of a nightly batch, a set of “items” from a back-end system. Each “item”
consists of a record in the incoming flat file, and can be represented by a unique identifier and a
textual description.
The application uses a Receive pipeline with a FlatFileDisassembler component in order to split the
incoming batch into individual records and map them to a corresponding XML messages, a process
known as debatching. Each message is treated by a dedicated instance of a processing orchestration.
As part of its processing, information about the message needs to be logged to an external
monitoring system for legal purposes. The orchestration makes use of a simple C# helper class to do
this job.
Finally, each processed item needs to be made available to a legacy application that listens on a
shared directory.
The application consists in the following components:
Page 48
HotRod.Samples.Messaging.dll: This BizTalk assembly holds the XML schemas and
associated maps.
HotRod.Samples.Staging.dll: This BizTalk assembly holds a custom pipeline with a
FlatFileDisassembler used to disassemble incoming flat files.
HotRod.Samples.Processing.dll: a simple orchestration that processes each item as they are
received.
HotRod.Samples.Helper.dll: a simple C# library, with a helper class that consists of a single
static method.
The following illustration shows our BizTalk application up and running on the quality assurance
platform as it appears from the BizTalk Administration Console.
The following tutorial will walk you through the steps required to implement a set of custom
CmdLets that leverage the PowerShell provider for BizTalk. In the same way as one can combine
existing CmdLets to perform complex operations, we will combine our own CmdLets as well. Note
that error handling will be omitted as it falls outside the scope of this article. So, let’s get started!
A CmdLet template
Before we dive into the actual work, we need to plan for the CmdLet that we’ll be writing. As of the
latest release of Windows PowerShell, the syntax for authoring custom CmdLets from scripts has
been slightly modified. The following code shows our new template for writing a custom CmdLet.
Page 49
Function Backup-Application
{
[CmdletBinding(DefaultParameterSetName = "Path")]
param (
[Parameter(ParameterSetName = "Path")]
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[Parameter(ValueFromPipelineByPropertyName = $true)
[Alias("PsPath")]
[String]
$path,
)
if (!$inputObject) { $inputObject = (Get-Item $path) }
# do something with $inputObject
}
Since we would like our functions to accept input from a pipeline, we make use of the
ValueFromPipeline and ValueFromPipelineByPropertyName attributes. The first one is useful when
we pass objects down from the pipeline. The second one helps for the cases where objects from the
pipeline expose a property named “Path” or “PSPath”. Fortunately, most PowerShell objects are
“decorated” this way.
Thus, the following syntaxes will be possible:
PS:\> Backup-Application –Path 'BizTalk:\Applications\BizTalk HotRod Application'
PS:\> (Get-ChildItem BizTalk:\Applications) | Backup-Application
Before PowerShell 2.0, such functions written directly within Powershell used to be known as script-
CmdLets. They are now officially called advanced functions.
Backing-up an application
In case our upgrade does not go well, we need to make sure we do not disrupt the activity of the
quality assurance lab. Therefore, we need to backup the existing application before we upgrade. The
backup strategy goes like this:
Export the application to a Windows Installer .MSI package in order to be able to rollback in
case of a problem, as well as the associated bindings and transport options, using the
“Export- Application” CmdLet.
It’s nice to also have the bindings backed-up because in case we need to have a look at the
configuration of old bindings without resorting to import the full application back to BizTalk.
We will use the “Export-Bindings” CmdLets here.
…
Export-Application –Path $path –Package $package
Export-Bindings –Path $path –Destination $bindings
…
Uninstalling an application
As a matter of policy for preventing potential conflicts caused by an upgrade, we prefer that new
versions of an application be installed from a clean state. Therefore we want to completely remove
the application from BizTalk as well as the file system.
Page 50
In order to remove an application from BizTalk, we must first completely stop the
application. This disables all receive locations, and sets all send ports and orchestrations into
the unlisted state. The “Stop-Application” CmdLet is available for this task. The added benefit
of completely stopping an application is that potentially suspended service instances are
also removed.
Removing the application from BizTalk leverages the standard “Remove-Item” CmdLet that is
overridden by the provider for BizTalk on its own artifacts. The provider exposes functions
to support item removal in a standard and consistent way.
Finally the application is then uninstalled from the operating system using Windows
Installer’s msiexec.exe command-line utility.
…
Stop-Application –Path $path –StopOption StopAll
Remove-Item –Path $path -Recurse
# Get Package CLSID
$clsid = Get-ChildItem $targetDir |
Where-Object -FilterScript {
$_ -match "^[A-F0-9]{8}\-([A-F0-9]{4}\-){3}[A-F0-9]{12}$" } |
ForEach-Object -Process { $_.Name }
$expression = "msiexec /x `"{$clsid}`" /qn"
Invoke-Expression $expression
…
In order to uninstall the application using the Windows Installer utility, we make use of the
knowledge that when BizTalk solutions are installed on disk, there is an additional folder in its
directory hierarchy. This folder is conveniently named after the unique identifier of the Windows
Installer package used to install the application in the first place.
Therefore, uninstalling is a simple matter of calling the following command-line:
C:\> msiexec /x <package identifier>
However, you’ll notice that for the call to the msiexec.exe utility, we need to create a representation
of the intended command-line as a string which is passed on to the “Invoke-Expression” CmdLet.
Indeed, for reasons that pertain to the way legacy command-line utilities are usually written, and the
peculiar parsing engine of Windows PowerShell, it can sometimes be very difficult to call command-
line tools from script. In effect, we need to combine a bunch of escape characters and double
quotation marks in just about the right sequence in order to achieve seemingly simple results.
Removing assemblies from the GAC
One curious omission of the uninstall process is that assemblies that are registered into the Global
Assembly Cache (GAC) are not actually removed. Our goal is to start installation from a clean state.
Therefore, we need to remove the assemblies that are part of our solution ourselves.
Page 51
One possibility would be to use the GacUtil.exe command-line utility that ships with the Microsoft
.Net Framework SDK. However, this would require us to package this utility because we are almost
certain that it is not already present on the target platform. However, GacUtil is not redistributable.
Besides, calling GacUtil from a custom action as part of an installation is not recommended.
That leaves us, for our simple scenario, with an intermediate solution that consists in calling into
internal classes from the framework to perform the operation. It is not particularly clean, but at
least, we sure stay away from legal proceedings!
…
[Reflection.Assembly]::Load($assembly) | Out-Null
$gacutil = New-Object System.EnterpriseServices.Internal.Publish
if (Test-Path $path) {
$gacutil.GacRemove($path)
}
…
Installing an application
Installing an application is a two way operation that consists of first importing the contents of the
.MSI package into the BizTalk Management database, and then installing the assemblies on the file
system.
These are the exact reversed operations that we performed during uninstall.
Importing the application into BizTalk is best done with the “Import-Application” CmdLet.
Since in our scenario we are actually upgrading an existing version of our BizTalk solution,
we would like the bindings to be automatically re-applied. This is done by calling the
“Import-Bindings” CmdLet.
…
Import-Application -Path . -Package (Get-Item $package)
if ($bindings) {
$path = "$((Resolve-Path .).Drive.Root)\Applications\$name"
Import-Bindings -Path $path -Source $bindings
}
…
Notice that we hit a kind of a conundrum here since that, in order to import the bindings, we need
to know the path of the application that has just been installed. So a quick hack is to record the root
of the current drive and combine it with the relative path to the application, given its name.
Future versions of the provider will probably return the application object from the
“Import-Application” CmdLet to prevent this situation.
Finally, installing the application is performed by calling the msiexec.exe utility with the path
to the .MSI installation package with which to upgrade. In a similar way than before, we first
create a string representation of the command-line with proper escape-sequences before
calling the “Invoke-Expression” CmdLet.
Page 52
…
$property = "TARGETDIR=```"$targetDir```""
$expression = "msiexec /i `"$package`" /qn $property"
Invoke-Expression $expression
…
Upgrading an application
Since we now have a (somewhat) useful set of dedicated functions, let’s combine them together
with other CmdLets from the provider in order to automatically backup an application before
installing a new version.
Not surprisingly, we’ll call this function “Upgrade-Application”.
…
if (Test-Path $path)
{
Backup-Application -Path $path -Destination $backupDir
Uninstall-Application -Path $path -targetDir $targetDir
}
Install-Application -Name $name -Package $package -TargetDir $targetDir
…
The above example shows how BizTalk can be easily managed in a flexible way from within
PowerShell. Administrators can create their own script library so repetitive tasks can be executed
over and over again.
Future plans As explained earlier, the PowerShell provider must be considered as a first and complete example of
a management tool for BizTalk Server. Based on the object model, the second step is to provide
other complementary tools. We can quote:
- WCF services for interoperability
- A new concept of web-based Management Dashboard
More information More information on the provider can be found at http://psbiztalk.codeplex.com. A getting started
guide can be downloaded from here: http://cid-
2b880e94db699632.skydrive.live.com/self.aspx/PowerShell.BizTalk/v1.0.0/PSBizTalk%20getting%20
started%20guide.pdf?sa=636405841
More information on PowerShell at: http://technet.microsoft.com/en-
us/scriptcenter/dd742419.aspx
An article to get started with PowerShell can be found at:
http://www.microsoft.com/technet/scriptcenter/topics/winpsh/manual/default.mspx
About the authors Maxime Labelle, SOA Architect (http://maxime-labelle.spaces.live.com)
Maxime currently works as a Software Architect for Logica Management Consulting and is involved
in the design and implementation of Software Oriented Architectures. He is essentially interested in
Page 53
the various platforms and technologies that Microsoft has developed over time and is currently
bringing his attention to the Microsoft BizTalk Server platform.
Michel HUBERT, MVP BizTalk (http://michelhubert.spaces.live.com/blog)
Michel works for Logica Management Consulting as SOA Architect specialized on Microsoft
technologies. He is expert on SQL Server, SharePoint, BizTalk Server, System Center Operations
Manager and .Net in general speaking. He is a speaker for Microsoft events such as TechDays and
animates a French community about SOA : www.soa-factory.org
Randal van Splunteren, Consultant (http://biztalkmessages.vansplunteren.net)
Randal works as a consultant for Aviva Solutions in the Netherlands. He focuses on EAI, BPM and
SOA using the Microsoft .Net platform especially Microsoft BizTalk Server. Randal has worked with
all BizTalk versions and is a BizTalk Server MCTS.
Page 54
HL7 Accelerator v2.0 [BTAHL7] By Toon Vanhoutte [[email protected]],
BizTalk Consultant @ Delaware Consulting
Audience: BizTalk Developers
Technologies: BizTalk Server 2006/2009, VS.NET 2005/2008
Skill level: Beginner to Expert
General The BizTalk Accelerator for HL7 is part of the BizTalk Accelerators package. The accelerator offers
the BizTalk developer a wide range of extra tools/components that make the implementation for
health care integration scenarios a lot easier and faster.
HL7 Standard HL7 (Health Level 7) is a messaging standard that enables clinical applications to exchange data.
The radiology information system (RIS), lab information system (LIS), hospital information system
(HIS) and electronic medical record (EMR) need to communicate with one another seamlessly. The
HL7 messaging standard is used worldwide and is still being modified to meet the changing data
needs of the healthcare world.
The structure of a HL7 message resembles that of EDI. A HL7 message consists of segments. Each
segment consists of one or more composites, also known as fields. The MSH (Message Header)
segment includes the sender and receiver of the message, the message type, and the date it was
sent. Every HL7 message specifies MSH as the first segment. Over 120 different segments are
available for use in HL7 messages. The PID segment, for example, contains the patient information.
MSH|^~\&|EPIC|EPICADT|SMS|SMSADT|199912271408|CHARRIS|ADT^A04|1817457|D|2.5| PID||0493575^^^2^ID 1|454721||DOE^JOHN^^^^|DOE^JOHN^^^^|19480203|M||B|254 8ST^^EUCLID^OH^44123^USA||(216) |||M NK1||CONROY^MARI^^^^|SPO||(216)731-4359||EC||||||||||||||||||||||||||| PV1||O|168 ~219~C~PMA^^^^^^^^^||||277^ALLEN FADZL^BONNIE^^^^|||||||||| ||264|||||||||||||||||||||||||198||||||005
Some special characters separate one composite in a segment from another, or separate one sub-
composite from another. Here's a quick overview of those delimiter characters:
Character Purpose
0x0D Marks the end of each segment
| Composite delimiter
^ Sub-composite delimiter
& Sub-sub-composite delimiter
~ Separates repeating fields
\ Escape character
Page 55
BTAHL7 Components
Schema Generator
A HL7 message can have a complex structure. I would take many hours to develop a schema by
yourself. The BizTalk HL7 Accelerator offers you an enormous range of standard HL7 schemas. You
can add them to your Visual Studio project, by a simple schema generation tool.
HL7 Disassembler & HL7 Assembler
The HL7 disassembler is a pipeline component that parses incoming HL7-encoded messages into
XML segments for processing. It performs validation of the message header and basic validation of
the body. It determines the schema that it uses to parse the HL7 message, determines the source
party for the message, and performs additional validation of the body (if enabled for the party). The
HL7 disassembler can also generate ACK messages.
The HL7 assembler serializes XML segments into an outgoing HL7 (flat file) message. It determines
the schema that it uses to serialize the HL7 message, determines the destination party for the
message, and performs validation of the body (if enabled for the party).
These important pipeline components are available in Visual Studio, so you can use these
components in your own custom pipelines. BizTalk also has four default HL7 pipelines that contain
these components.
MLLP Adapter
This adapter is a TCP/IP socket adapter that uses the Minimal Lower Layer Protocol (MLLP). That
protocol is often used in healthcare solutions. MLLP wraps messages inside a <SB> Start Block
character and a <EB> End Block character. The adapter supports ordered delivery, which is often a
requirement.
Batch Orchestration
The HL7 Accelerator provides an orchestration (BatchOrchestration.dll) that is capable to process
batches of HL7-encoded (2.X) messages and/or acknowledgments (ACKs). The orchestration works
with batch activation, batch termination, and batch timer messages that BTAHL7 uses to control
batches.
HL7 Configuration Explorer
This management console can be accessed in this way: Start > All Programs > Microsoft BizTalk
Accelerator for HL7 2.0 > BTAHL7 Configuration Explorer. You can configure some extra settings for
each HL7 party. For example settings about validation, ACK’s, batching, MSH map,…
HL7 Schemas A ‘normal’ XSD-Schema exists of one schema file. Due to the complexity of HL7 and the different
interpretations of the HL7 standard, a HL7 schema file should be very complex. BizTalk covers this
problem by having the data fields, segments and tables - that identify looping structures - defined in
separate schemas per version. Those version-specific schemas are used in the individual schema for
each message type. These references give the schema an object-oriented way of developing.
Page 56
The benefit of this approach is that schemas can be both globally and locally modified.
Modifications that must apply for each message type have to be done in the definitional schema.
Changes, for a specific message type only, must occur in the message schema.
Adding schemas to your solution
After installation of the accelerator, Visual Studio offers you a range of BTAHL7 projects. These
projects contain the predefined definitional HL7 schemas, per version. First, add all definitional
BTAHL7 projects to your solution. Then, add a project for the HL7 message schemas. This project
needs references to the definitional projects, because it uses these segments/datatypes/values.
A quick overview of the solution structure:
BTAHL7V2X: contains the standard Message Header en Acknowledgement schemas
Delaware.HL7.BTAHL7V2X: contains the customized Acknowledgement schemas (custom namespace)
Delaware.HL7.BTAHL7V23: contains the customized definitional schemas for all V 2.3 message schemas (custom namespace)
Delaware.HL7.Schemas: contains the customized message schemas (custom namespace)
Schema name and namespace
In BizTalk a schema is uniquely defined by the following combination:
SchemaNamespace#RootNodeName.
The RootNodeName of a HL7 schema has the following fixed structure:
MessageType_TriggerEvent_Version_GLO_DEF
Because the RootNodeName must be fixed, the way that customized schemas are differentiated
from the standard schemas is by altering the SchemaNamespace on the customized schemas.
The standard BizTalk HL7 SchemaNamespace is:
http://microsoft.com/HealthCare/HL7/2X
Page 57
The only way to change the namespace of a standard HL7 schema to your own custom namespace is
by executing a Find/Replace so that for example all http://microsoft.com/HealthCare/HL7/2X are
replaced by http://Delaware/HL7/2X. Do not forget to change also the namespace of the referenced
definitional schema.
In the BTAHL7 Configuration Explorer you can specify for each party the corresponding namespace:
Z-Segments
It is important to know how BizTalk handles Z-Segments. Z-Segments are used to add some extra
information to a message, which is not described in the standard. Standard HL7 allows the use of Z-
Segments, but Z-Segments are not predefined. So they are 100% custom segments.
It's important to be aware of how BizTalk processes a HL7 message. In the receive pipeline, a
message is transformed into the XML representation of the message. This XML message is a multi-
part message. It consists of 3 parts:
MSHSegment [Type = System.Xml.XmlDocument]
BodySegments [Type = HL7 message schema]
ZSegments [Type = System.String]
BizTalk assumes that Z-Segments are always at the end of a message. So when the HL7 Disassembler
parses the HL7 message and an unknown Z-Segment is found; it writes all the next segments to the
ZSegments part of the multi-part message, as a string. Why as a string? Because BizTalk does not
know the definition of those segments, so a Z-Segment cannot be serialized to XML.
Page 58
This means that standard segments that are placed behind an unknown Z-Segment will be treated as
Z-Segments. This can lead to a validation error, because mandatory segments are missing. To solve
this problem, you must customize the message schemas. Add all expected Z-Segments
(optional/mandatory) to the message schema. Now BizTalk will not threat these segments as
unknown Z-Segments, but as any other known body segment.
Custom Trigger Event
Some hospitals make use of a custom trigger event, for example an event called ZAP for the MFN message type. Just like Z-Segments, you will not find this in the HL7 standard. The first thing you must do to address this, is adjusting the definitional message header schema, to allow an event called ZAP.
Open the MSH_25_GLO_DEF schema and find in which table the enumerations are stored Table3.
Adding an enumeration to a message header schema is not as simple as adding one to a definitional schema. You must perform the next steps:
Right-click the MSH node and add Child Field Element.
In the Properties window, click the Data Type node and select Table3. Add ZAP to the enumeration.
Delete the created element.
Now it is time to add the MFN^ZAP schema to our project. Only one problem: this schema doesn’t exist. The fastest way is adding a schema that resembles to our schema: MFN^M01:
Change RootNodeName, FileName and TypeName to MFN_ZAP_23_GLO_DEF
The only thing to do now, is customizing the schema.
MLLP Adapter
Configuration
The MLLP adapter can be configured as a one-way/two-way receive/send port.
Page 59
Block Characters. Messages that BizTalk receives or sends on an MLLP adapter require the following
wrappers:
<SB> Start Block character
<EB> End Block character
<CR> Carriage Return Byte (optional)
Network Connections Parameters.
Unique connection name
For send specify the server name as host (def=localhost)
Always specify TCP port ID
Persistent Connection and Timeout handle the way
the connection closes.
Ordered Delivery. Is often a requirement in healthcare integration:
For receive: DeliveryMode
For send: Transport Advanced Options
Solicit-Response for One-Way Send Port. You can enable receive ACK's on the same TCP connection.
If enabled, specify the URI of the receive location that's configured to receive the ACK's.
MLLP Test Tools
There are two test tools that are shipped with the HL7 Accelerator. You can find these command
prompt apps at C:\Program Files\Microsoft BizTalk 2009 Accelerator for HL7\SDK\MLLP Utilities.
MLLP Receive Test Tool
This application runs an MLLP listener. This is a very handy tool to make sure that a message is sent
correctly, during development.
Open command prompt
Navigate to the correct location in the command prompt
In the command, specify the port ID, ES/SB/CR characters, the directory where to store the
received messages,...
Listen to port 2000 and save the messages to separate files on C:\Delaware
>> mllpreceive.exe /P 2000 /SPLIT /SB 11 /EB 28 /CR 13 /D C:\Delaware
MLLP Send Test Tool
This application sends a message to a specific TCP port. This is a very handy tool to test receive
locations and to make sure that ACK's are returned.
Page 60
Open command prompt
Navigate to the correct location in the command prompt
In the command, specify the server, the port ID, ES/SB/CR characters, the file to send, how
many times to send the message, if you expect an ACK,...
Send 10 times the message C:\Delaware.hl7 to server 10.10.0.10 on port 2000
>> mllpsend.exe /I 10.10.0.10 /P 2000 /REPEAT 10 /F C:\Delaware.hl7
Performance
When using a request-response port for sending HL7 messages and receiving their corresponding
ACK, you can run into performance issues. This problem occurs because the timeout setting of the
MLLP adapter, for an asynchronous call of messages to the BizTalk Server MessageBox database, is
by default 10.000 milliseconds. You can change this value, through the following registry key:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Accelerator for HL7\Version 2.0]
"MLLPSendFlushTimeout"=dword:00000500 (Value = compromise between latency and throughput)
To decrease end-to-end latency, another performance setting can be changed through the Microsoft
SQL Server Management Studio. Connect to the BizTalkMgmtDb database and open the table
adm_ServiceClass. Change the value of MaxReceiveInterval to (for example) 50 ms corresponding to
the Name = Messaging InProcess. Initially it is 500 ms. This setting specifies the maximum
polling interval at which the messaging agent polls the MessageBox.
Acknowledgements
General
The configuration of the Acknowledgements per party, must be done in the HL7 Configuration
Explorer:
When keeping all ACK fields blank, you take the default values (like switching sender and receiver). You can overwrite the default values, by filling in these fields. When you want a default field not to be displayed, you can override this by filling in: \.
For each Acknowledgement type you can select if you never/always/only on success/only on error want to create the (N)ACK.
A common used option is to Route ACK to send pipeline on request-response receive port. When this option is turned on, the ACK (generated by the HL7 disassembler) is automatically returned to the sender of the message on the same connection.
Page 61
It's important to keep this in mind when developing custom pipelines. For example: if your pipeline
contains a custom routing pipeline component , that promotes properties based on an advanced
query in a preconfigured routing database, you have to take into account the fact that this
component will also receive the ACK. This component can use the MessageType property to
determine which message it is processing.
Custom ACK pipeline component
In some cases, the standard BizTalk HL7 Acknowledgement functionality might be insufficient. Then
you can consider developing your own ACK pipeline component. Creating the ACK message, based
on the content of the incoming HL7 message, is not that difficult. But which properties of the ACK
do you need to write/promote in order to route back the ACK to the send pipeline of the receive
port? Let's take a closer look. (Note that no error handling is added to this simple example)
public Microsoft.BizTalk.Message.Interop.IBaseMessage
Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc,
Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
return inmsg;
}
public void Disassemble(IPipelineContext pContext, IBaseMessage inMsg)
{
//Retrieve content of the incoming HL7 message
Stream inputStream = inMsg.BodyPart.GetOriginalDataStream();
inputStream.Seek(0, SeekOrigin.Begin);
//Create ACK Stream, based on the incoming HL7 message
MemoryStream ackStream = new MemoryStream();
StreamWriter ackWriter = new StreamWriter(ackStream);
//TODO: Add logic to build the ACK Stream based on inputStream
//Flush the content to the ACK Stream
ackWriter.Flush();
ackStream.Seek(0, SeekOrigin.Begin);
//Create a new pipelinemessage.
IBaseMessage ackMsg;
ackMsg = pContext.GetMessageFactory().CreateMessage();
//Set the Bodypart = ACK Stream.
ackMsg.AddPart("Body",
pContext.GetMessageFactory().CreateMessagePart(), true);
ackMsg.BodyPart.Data = ackStream;
//Create an empty Message Context
ackMsg.Context = pContext.GetMessageFactory().CreateMessageContext();
//Add all necessary properties to the context. This is the
minimum to route back the ACK to the sendpipeline of the
receiveport
Page 62
const string NSPACE =
"http://schemas.microsoft.com/BizTalk/2003/system-properties";
//Promoted Properties
ackMsg.Context.Promote("RouteDirectToTP", NSPACE,
inMsg.Context.Read("IsRequestResponse", NSPACE));
ackMsg.Context.Promote("EpmRRCorrelationToken", NSPACE,
inMsg.Context.Read("EpmRRCorrelationToken", NSPACE));
//Not Promoted Properties
ackMsg.Context.Write("ReceivePipelineResponseConfig", NSPACE,
inMsg.Context.Read("ReceivePipelineResponseConfig", NSPACE));
ackMsg.Context.Write("CorrelationToken", NSPACE,
inMsg.Context.Read("CorrelationToken", NSPACE));
ackMsg.Context.Write("ReqRespTransmitPipelineID", NSPACE,
inMsg.Context.Read("ReqRespTransmitPipelineID", NSPACE));
//Write the input and the ACK message to the outqueue
qOutputMsgs.Enqueue(ackMsg);
qOutputMsgs.Enqueue(inMsg);
//Now the GetNext method will be called
}
public IBaseMessage GetNext(IPipelineContext pContext)
{
if (qOutputMsgs.Count > 0)
{
return (IBaseMessage)qOutputMsgs.Dequeue();
}
else
{
return null;
}
}
These are the properties that the sendpipeline of the receiveport subscribes on, or needs to process the message correctly: RouteDirectToTP (Promoted) EmpRRCorrelationToken (Promoted) ReceivePipelineResponseConfig (Not Promoted) CorrelationToken (Not Promoted) ReqRespTransmitPipelineID (Not Promoted)
End-to-End Acknowledgements
End-to-End ACK’s are possible with BizTalk. First of all, you have to disable the option of routing the ACK back to the send pipeline of a request-response receive port. Do this for a certain party in the HL7 Configuration Explorer.
There are two options to create an End-to-End scenario:
In a pure messaging HL7 scenario you can subscribe with a request-response send port on the incoming request-response receive port.
When more logic is necessary, you can use an orchestration to subscribe with a request –response receive port on the incoming request-response receive port.
Page 63
How to Create a Custom Receive Pipeline Decoder in BizTalk Server
2009 By Ruth Resende, Biztalk Consultant, ITGroup - MS GOLD Certified Partner
This article assumes you are familiar with BizTalk Server Pipelines, C# Code and Visual Studio 2008.
Audience: BizTalk Developers
Technologies: BizTalk Server 2009, VS.NET 2008
Skill level: Intermediate
The source code used in this article can be downloaded from the following location:
http://custompipeline.codeplex.com/
Summary Out-of-the-box solutions unfortunately sometimes are not sufficient to solve specific problems, so
we must build a custom solution to address those.
In this article, I will try to show you how to build a Custom Receive Pipeline Decoder that handles
special characters (like HTTP and XML reserved chars, e.g.: &, quotes and diacritical marks as a
cedilla, tilde, circumflex, or macron) of a message, using as parameters a Regular Expression and its
replacing value.
In this tutorial we will use Regular Expressions, since is a very flexible form to identify characters in a
text. To learn more about regular expressions visit http://www.regular-expressions.info/.
Install Microsoft BizTalk Server Pipeline Component Wizard The easiest way to get started is downloading the BizTalk Server Pipeline Component Wizard, an
add-in for Visual Studio that in a few steps will create our Decode Component project with all
necessary external references and interface implementations.
The wizard can be found in http://btsplcw.codeplex.com, in this article we will be using version 2.10.
To install the wizard, unzip the file and at the extract folder go to directory
\PipelineComponentWizard Setup\Debug and execute Setup.exe.
Page 64
Follow the steps in the Setup Wizard.
Use BizTalk Server Pipeline Component Wizard 1. Open Visual Studio 2008 and click on File -> New -> Project.
2. In Biztalk Projects choose the option Biztalk Server Pipeline Component Project.
3. Choose a directory and project name. Click OK.
4. The Setup Wizard should run automatically, and the Welcome screen “Pipeline Component
Project Wizard” should appear. Click Next.
Page 65
5. In the next screen you must fill out some properties like class name and namespace. Since
we are creating a Decode Component we need to select “Receive” as Pipeline Type and
"Decoder” as Component Type. Click Next.
6. Fill out component name (that will be the assembly and Visual Studio project name), version
and description. Click Next.
Page 66
7. In the next screen, add two string properties, so we can configure our Receive Decoder. The
first one is a Regular Expression and the second one the replacing text. Click Next and Finish.
The wizard finishes and our BizTalk project is ready to go. It should appear:
8. Click twice on ReplaceString.resx and change Component’s name to “ReplaceString
decoder”.
9. In Solution Explorer right-click on Project Name/Properties, change the name from
“Assembly Name” to the same name as “Default Namespace”.
Page 67
Add C# code for project
When you open the project, this structure should appear:
I will not drill down into all required interfaces; you can find further information at
http://msdn.microsoft.com/en-us/library/aa560021%28BTS.20%29.aspx.
1. The pipeline component entry point is the IComponent.Execute
method, so expand the IComponent region and in the Execute
method, copy and paste the code below.
public Microsoft.BizTalk.Message.Interop.IBaseMessage
Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc,
Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
try
{
if (inmsg.BodyPart != null) //Verify the message is different from null
{
MemoryStream memStrm = new MemoryStream();
Stream originalStrm = inmsg.BodyPart.GetOriginalDataStream();
StreamReader sReader = new StreamReader(originalStrm,
System.Text.Encoding.Default);
if (_RegularExpression != null)
{
if (_ReplacedValue == null || _ReplacedValue == "")
_ReplacedValue = " ";
//Replace the regular expression with the value specified
string strReplace =
System.Text.RegularExpressions.Regex.Replace(sReader.ReadToEnd(),
_RegularExpression, _ReplacedValue);
Page 68
StreamWriter swriter = new StreamWriter(memStrm,
System.Text.Encoding.Default); //Retrieve modified message
swriter.Write(strReplace);
swriter.Flush();
memStrm.Seek(0, System.IO.SeekOrigin.Begin);
inmsg.BodyPart.Data = memStrm;
pc.ResourceTracker.AddResource(memStrm);
swriter.Dispose();
}
sReader.Dispose();
}
}
catch (Exception ex)
{
System.Diagnostics.EventLog.WriteEntry("ITGroup.Biztalk.Pipeline.Components",
"Occurred an error on modify the message: " + ex.Message + "\n" + ex.StackTrace,
System.Diagnostics.EventLogEntryType.Error);
throw (ex);
}
return inmsg;
}
This code will load the message the receive pipeline is processing and use the
System.Text.RegularExpressions namespace to replace the desired expression for the replacing text.
This code sample is not suitable for large messages. If you are processing large messages you should
use streams as they are more efficient.
2. Build the project and copy the generated DLL to the folder \Microsoft
BizTalk Server 2009\Pipeline Components
Using Custom Component Created
1. CREATE AN EMPTY BIZTALK SERVER PROJECT.
2. Right-click on the toolbox and select “CHOOSE ITEMS”.
Page 69
3. In the “Choose Toolbox Items” dialog box, click the BizTalk Pipeline Components tab. Click
“ReplaceString decoder”, and then click OK.
4. ADD A NEW “RECEIVE PIPELINE”.
5. Drag the “ReplaceString decoder” pipeline component from the Toolbox into the Decode
stage of the custom pipeline.
6. CLICK ON “REPLACESTRING DECODER”, FILL OUT THE REGULAREXPRESSION AND REPLACEDVALUE
PROPERTIES. YOU CAN USE ANY REGULAR EXPRESSION.
Page 70
THIS EXAMPLE WILL CHANGE THE SPECIAL CHARACTER “&” TO “E”.
7. YOU MUST CREATE A STRONG NAME KEY FILE, CHOOSE A NAME FOR APPLICATION NAME AND DEPLOY
YOUR PROJECT.
Testing Receive Pipeline
1. OPEN NOTEPAD, place the code below AND SAVE IT AS A XML FILE AT C:\TEST.XML.
BIZTALK PROVIDES THE HANDY TESTING UTILITY AT \MICROSOFT BIZTALK SERVER
2009\SDK\UTILITIES\PIPELINETOOLS\PIPELINE.EXE THAT ALLOW you to test if a pipeline is functioning
correctly without having to configure the BizTalk Server environment, such as send/receive ports.
2. THEN, OPEN THE COMMAND PROMPT AND TYPE:
PIPELINE -PT "[Fully Qualified Assembly Name created at ‘Using Custom Component Created’]" -D
C:\TEST.XML -C
The result must be:
Page 71
Tools like the Custom Pipeline Component Wizard really help us in creating our domain specific
solutions like this string replacing component, doing the hard job of implementing all the required
interfaces and leaving us only the Execute method to fill. I hope you could use this tutorial in your
BizTalk solutions so it will be easier to develop and test your custom pipelines.
The code at http://custompipeline.codeplex.com/ contains this sample and a more complete
scenario with orchestration, Send and Receive Ports.
Page 72
Automating Business Processes with SharePoint, Workflow and
BizTalk Paul Somers, Microsoft MVP
Business Processes are the core of an organization. These processes can be made more efficient by
implementing human based workflow in SharePoint and system based workflows in BizTalk. If you
look at the desired end result for the business process, the end result may be that the approval of
the workflow is not where the process finishes or there may be further steps required that can only
be manually performed. This is where BizTalk can take the workflow output and fully automate the
Business process, giving real benefits to an organization.
BizTalk and SharePoint can work together to seamlessly automate business processes that need
human intervention and system to system integration.
Explore SharePoint; it is a good central web based user interface with dashboards, document
libraries and lots of other functionality, it is a great host for human based workflow and it’s an even
better place to allow users to provide the feedback required to advance a workflow to the next step.
It can draw users in and get them to fill in forms and answer the questions that a human based
workflow might require, such as approvals, get users to fill in a Purchase Order…etc. These can all be
stored in a document library in SharePoint. They are placed there perhaps as part of a workflow, or
part of a larger business process. The possibilities for scenarios are only limited by your level of
business process automation (BPA).
Most business processes need to end somewhere. Where is that end point? Is it when the manager
clicks approve, or the requested document is uploaded.... then someone manually prints these off
and acts on them? This occurs more often that you can imagine; the fact that the document or the
approval is recorded in SharePoint and we printed this off is thought to be enough.
To implement an “approval workflow” in SharePoint is very simple these days and you get some
benefits by doing this, but what about the rest of the business process?
Follow the business process after approval - what happens to the printed document? It is manually
entered into a system, used to update items in a second system, and then something else is printed
from a third system, and then faxed. This entire process is manual; it is time consuming and totally
unnecessary.
We have the technology! Better still we have proven technology to take the request which was
approved in SharePoint, act on it electronically, and perform the additional steps automatically.
BizTalk can read from SharePoint, it can be configured to wait for approved documents in the
document library, it can communicate with your internal systems, update details, and it can even
orchestrate a back end process for this, which would follow your current process. It can FAX the
order to the destination company, thereby removing all of the manual steps needed to process the
workflow.
It might not seem like much, but for a basic request where there can be as little as 100,000 requests
a year, it can very easily add up. For example: Take the cost of the Fax machine, a telephone call, the
Page 73
cost of the staff member who re-keys, then faxes and files the document, the electricity, paper,
office space…etc. Let’s very conservatively say around $20 per request is saved.
100,000 requests per year * $20 = wait for it....
You could save 2 million dollars a year, by removing this small step.......?
The question is put to you, why wouldn't you do this? Would you like an extra $2 million in your budget?
The cost of the Technology to do this is much less than the amount you would save in most cases
and this is only for one process. What about all of the others? A company can save a great deal if
they start to automate a few of their processes from start to finish, removing as many manual steps
as possible, and removing all re-keying of data.
How do we do this? Take a very common scenario of purchasing from a vendor, as most
organizations purchase something from another company.
The PO Requester
Fills in a form, prints it for signature, and hands it to the PO Approver
The PO Approver
May eventually sign and approve it. It is then manually handed to the purchasing manager
Purchasing Manager
Would manually generate an order, and then phone, fax or email an order to the Vendor.
The Vendor
Page 74
Provide an invoice which would kick off a different process inside the organization, again
another manual process.
This process is time consuming, requires several people in the loop, and requires re-keying of
information which could lead to errors. The result is that it costs the company money it need not
spend.
If you start to add workflow to this scenario you get something that looks like the diagram below.
Instead of doing this manually we leverage SharePoint and InfoPath documents to submit a PO
Request and route it to the PO Approver and Manager.
You can see that the purchase order approval process is automated - no more paper - and you can
now start to track PO approval status. You can have multiple approvers and multiple people who can
submit a PO. It simplifies the process as well as automating it.
You will note that the process for the purchasing manager has not changed that much; they can now
see a list of approved PO’s with which they need to manually work, perhaps they have an ERP
system into which these need to be re-keyed, and the Order numbers generated from, then
manually generate the order and manually receive the invoices as before.
You will achieve efficacies by adding workflow; there are ways to make this process much more
efficient.
A workflow does not end once it is approved by a human. This is where the real power and benefits
of this solution are demonstrated in the diagram below:
Page 75
Looking at where the previous diagram left off, we can see now that once the PO’s are approved:
BizTalk
Takes the PO’s directly from SharePoint, the ERP system is sent the order, BizTalk generates
the PO for the vendor.
ERP
In an automated process, from interfacing to the ERP system, would receive the details of
the order, and generate an order number along with enough information for the vendor to
process the order, and then surface this to BizTalk.
Vendor
The communication with the vendor can occur in several ways depending on the vendor.
The most efficient communication method is a vendor web service that the Vendor would
expose via the internet to allow secure automated submission of PO’s which would flow to
the internal systems of the vendor.
Page 76
Should the vendor not have a service, they can receive the order in PDF Format via an email
in order to confirm the order.
If the vendor is unable to or does not have email, we can even revert to sending via faxes to
the vendor’s fax machine. In reality, any industry standard protocol is supported as a means
of communication with a vendor, e.g.: FTP, HTTP
SharePoint
As the interaction to the vendor occurs, the status of the PO can be updated, so the business
can see where the PO is, and the original PO that the vendor is sent can be archived in the
SharePoint document library - no need for manual filing of a piece of paper or printing the
paper for that matter.
The key here is that there is no human interaction required. The entire task of the purchasing
manager for this process is removed, the purchasing process is not only automated, it is made more
efficient, no more waiting for someone to action the request once it is approved. The efficiency gains
to the organization can start to be seen.
Vendor Invoicing
The vendor would eventually send an invoice for the goods; this can be received in several ways as
well:
Page 77
Vendor
The vendor can send the invoice, electronically and securely, to an invoice service that has
been identified on the internet. They could of course also email or fax their invoices.
BizTalk
The service would communicate with BizTalk directly. The email can be interpreted and read
from the local mail server or the fax can be scanned using Optical Character Recognition
(OCR).
SharePoint
The invoices could be stored centrally in SharePoint, where a workflow can be started to
process these invoices. This would lead to an invoicing process, where the vendor’s invoice is
matched with a PO, interfaced to the ERP and accounting system, and payment made on the
invoice once the payment is approved. This is yet another area for improving the business
process.
What is the organization saving for just this one business process? Let’s look at another conservative
example. Take an organization that has 100 purchase orders a month (that’s 100 x 12 = 1200 per
year), with a conservative estimate saving of $100 per PO, and we are talking a saving of $120,000
for this process alone for a small number of PO’s.
For a larger organization that sends say 1000 purchase orders a month, we are looking at 12 x 1000 =
12000 per year - an estimated saving of $ 1 200 000.
The cost of implementing such a system can be a fraction of the saving; this means you get the
entire system for the cost of the first process with extra money to spare, while you build the rest of
the processes on top of the same system. Your savings increase for every business process you
automate.
The question remains: the technology exists to implement these kinds of scenarios, and you can save
money from this - why wouldn’t you?
Many thanks to the Microsoft and Quicklearn please have a look at this demonstration to dig deeper into the scenario.
http://www.microsoft.com/biztalk/en/us/click-through-demo/hol.htm
Page 78
Paul Somers [email protected] http://blog.paul.somers.com
Paul is a Microsoft MVP specializing in driving Business Process and Integration for all organizations,
no matter the vertical. Every organization can benefit from automating their business processes and
making the organization more efficient, thereby saving money.
Page 79
BizTalk Management Tools I will use this page to document BizTalk Management Tools.
Operations Manager 2007 http://www.microsoft.com/downloads/details.aspx?FamilyId=389FCB89-F4CF-46D7-BC6E-
57830D234F91&displaylang=en&displaylang=en
CODit SOA Dashboard http://www.codit.eu/inside.aspx?sec=integrationdashboard&subsec=EAIB2Bmonitoring
FRENDS Helium http://frends.com/blog/frends-helium-tutorial/
