IMS DataPower - IMS UG Phoenix 12-2013

© 2012 IBM Corporation

IMS and DataPower

Shyh-Mei F. Ho IBM Distinguished Engineer

[email protected] IMS On Demand SOA Chief Architect

SVL, San Jose, CA. USA


Public Cloud

IBM Cast Iron

Synching data with SaaS apps to leverage new cloud economy

Private Cloud

BPM

WAS DB

DBCICS

DB2

SAP

.JCAPs

.NETPattern

Pattern

IMS

ODM

DB

IBM Integration portfolio

Mobile

Internet of Things

Trading partner communities

DeveloperCommunities

IBM Message Broker

Integration Bus provides universal connectivity for heterogeneous environments across enterprise processes, applications, and data

IBM MQ messaging

Messaging backbone provides reliable transport and data delivery across data center

IBM PureApplication System

Enterprises looking to achieve “more with less” by better managing IT resources as collectives

IBM Worklight

Productive multi-device development and management

IBM Web API Management

Manage your APIs to open up access encouraging innovation from App Developers

IBM MQTT

Reliable, efficient, scalable messaging for mobiles and sensors

DMZ DMZ

IBM DataPower Gateway Appliance

B2B Integration Gateway for secure collaboration with communities of trading partners

IBM Caching Appliance

Cache grids improve scale and performance

IBM DataPower Gateway Appliance

Integration Gateway for secure & controlled access to enterprise resources, while optimizing workload delivery


WebSphere DataPower Gateway Appliances

Security & Integration Gateway Appliances

Internet Trusted Domain

Consumer

Application or Service

DMZ

DataPower DataPower


DataPower Appliances Benefits

� Reduce Complexity: Replace software servers functionality with DataPower

Appliances, reduce infrastructure footprint, and off-load systems intensive processes.

� Lower TCO: DataPower Appliances have demonstrated reducing operational costs by

as much as 50%

� Reduce Time to Market: DataPower Appliances dramatically decrease the testing time

and amount of development required to upgrade your environment, most policy are

configuration driven as opposed to development driven

� Reduce Risk: DataPower Appliances provide the communication layer without requiring

application modification, and deliver improved security and audit

� Flexibility & Security: DataPower Appliances shield business applications from security

requirements, protocol changes and service versioning - no application modifications

needed


Configuration-driven approach speeds time to market

� Uses intuitive pipeline message processing to secure, control, optimize, & integrate

services and application traffic

� Enforce security standards with zero coding

� Import/export configurations between environments

� Transaction probe shows message content between actions for debugging


DataPower: Mainframe integrationOffload processing for reduced MIPS

Web Services Enablement for IMS, DB2, CICS


Premier System z Services enablement through DataPower as Ubiquitous Gateway

IBM cross-brand initiative

� Corporative synergy between DataPower, System z, including IMS, CICS and DB2, to support DataPower as the premier System z gateway

–Position DataPower for mainframe with a complete range of connectivity and integration to all databases, in addition to all transactions

DataPower Processing Flow

Protocol transformation

WS Monitoring

Native z transports

Transaction distribution

Network-level HA and distribution

RACF-centric security

IMS

DB/2

CICS

© 2012 IBM Corporation8 IBM Confidential5 December 2013

IMS Integration with WebSphere DataPower

ServiceOriginator

IMS

Connect

Client

IMS

OTMA

IMS Application

IMS

Connect

SOAP/HTTP

Z Service Provider

� MQBridge to drive IMS transactions

� MQ client is embedded in DataPower

� IMS Connect to drive IMS transactions (inbound requests)

� IMS Connect client in DataPower natively connects to IMS Connect

• Inbound support only

• Commit mode 1, Sync Level NONE or Confirm

TCP wire protocol

Cobol/MQMQ Client

MQ Server

MQ

Brdg


IMS Integration with WebSphere DataPower +

WebSphere DataPower 6.0 (GAed on 6/28/2013)

IMS Database integration

IMS Transaction integration

IMS Callout to DataPower is a leadership project

Many WW customers express interest in IMS Callout



WS Monitoring

Native z transports

Transaction distribution



IMS

New

New


Recent DataPower deliverables for System z

1. IMS Callout – Allows applications running in IMS to make service calls to external web service providers using the existing IMS ICAL interface [DP v6.0]

2. IMS DB – Provides a new information-as-a-service interface for service consumers to access IMS databases using standard SQL (requires DataPower’s Database Connectivity feature) [DP v6.0]

3. DB2 WLM – Provides support for real-time load distribution of SQL calls from DataPower to Sysplex-aware DB2 instances running on z/OS [DP v6.0]

4. XI50z firmware upgrade – Provide support for DataPower firmware level v5.0 [1Q13]

IMS

CICS

DB2

1

2

3

4

Business Integration


IBM WebSphere DataPower Family

Integration Appliance XI52� High density 2U form

� “Any-to-Any” conversion at wire-speed

� Bridges multiple transport protocols

� Mainframe integration & enablement

Service Gateway XG45� Entry-level device, slim footprint (1U)

� Security gateway (AAA, XML threat, etc)

� Service level management and monitoring

� Intelligent load distribution & dynamic routing

� Lightweight integration functions (optional module)

B2B Appliance XB62� High density 2U form

� B2B Messaging (AS1/AS2/AS3/ebMS)

� Trading Partner Profile Management

� B2B Transaction Viewer

Integration Blade XI50B/XI50z� Functionally equivalent to XI52

� Form factor flexibility

� XI50B: BladeCenter form factor

� XI50z: zEnterprise BladeCenter Extension (zBX) form factor


DataPower Models for IMS solutions

XI52, XI50B, XB62, XG45 IMS DB support

XI52, XI50B, XI50Z, XB62IMS TM provider support

XI52, XI50B, XB62IMS TM synchronous callout support

DataPower Models supporting 6.0 release

WebSphere DataPower V6.0:

More capabilities planned for future DataPower releases

New

New

New




WS Monitoring



IMS

IMS

Connect

Direct Access IMS database via DataPower

ODBM

ODBM

IMSDB

� IMS Open Database offers direct access to IMS database resources anywhere in the

IMSplex from z/OS and distributed environments– Support different APIs to leverage Distributed Relational Database Architecture (DRDA)

• IMS universal DB resource adapter to support J2EE, e.g. WebSphere

• IMS universal JDBC driver to make SQL calls

• IMS universal DL/I driver

– Open Database Manger (ODBM) works together with IMS Connect as a DRDA server for IMS

data

� DataPower to access IMS database directly via the Open Database capability, i.e.

via IMS Connect and ODBM– An IMS database is defined to DataPower as an SQL data source. For each IMS database that

you will access, you need to configure a separate SQL data source

New


IMS Synchronous Callout with DataPower 6.0

� IMS synchronously go outbound to external server via DataPower− Implement IMS synchronous callout protocol− Support the common container model architecture with IMS Callout Front-side Handlers that retrieve IMS callout messages and send response data

� Using existing WTX Design Studio tooling and/or XSLT stylesheet for data transformation

� The handler internally creates one or more IMS Connect dedicated persistent socket connections to the host system, using Enterprise Suite V2.2 IMS Connect API in Java.

� The handler communicates with IMS Connect via a new DataPower dedicated user message exit, HWSDPWR1. – For shared queue environment, user can choose to create multiple IMS Callout connections, one for each IMS datastore.

Core Engine Front-side

Handler

IMS IMS Application

IMS

Connect

Front-side

New

© 2012 IBM Corporation15

Prerequisites for IMS synchronous callout

� Software requirements

� IMS V12 (IMS V13 is recommended)

– IMS Connect

– OTMA

� IBM WebSphere DataPower Firmware 6.0.0.0 or higher

� Hardware requirements

- IBM WebSphere DataPower appliance XI52, XI50B, XB62

� Tooling

� WebSphere Transformation Extender (WTX)

Provides mapping between different data formats.

� WTX maps can be built as deployable artifact for DataPower, providing data

transformation between IMS callout bytes and XML data for web services.

� A WTX map can be set using a DataPower-specified variable, then called

within XSL code in a DataPower policy.

IBM Confidential


IMS Synchronous Callout DataPower & IMS Versions

� IMS V12

� IMS Connect

� DataPower User Exit Installation - Object Code Only user exit HWSDPWR1(new)Specified in the EXIT= parameter of the TCP/IP statement in the IMS Connect configuration file (HWSCFGxx).

� V13 PTF UK97704 & PTF UK97704� V12 PTF UK91544

� OTMA

� IMS Synch Callout user can specify a 1-to-8 byte mapname as the first 8 bytes in AIBUTKN so that this ID can be included in the OTMA state data in the callout message. The ID can be used as a unique service identifier for data transformation mapping and service routing

� V13 (available as base code)

� V12 PTF UK82636 (PM73135) AIB MAP name field

� IMS V13

� IMS Connect: Socket listening redesign


Prerequisites for IMS DB

� Software requirements

� IMS V12, IMS Catalog, ODBM and SCI

– IMS Catalog to access to metadata of IMS programs and databases

resources .

– IMS Connect

� ODACCESS statement in HWSCFGxx member of a concatenated PROCLIB data set

� IBM WebSphere DataPower Firmware 6.0.0.0 or higher

� Hardware requirements

� IBM WebSphere DataPower appliance XI52, XI50B, XB62, XG45

IBM Confidential


External Details

IBM Confidential


Synchronous Callout Solution Highlights

• The IMS callout connection is a DataPower “Front SideHandler” that can retrieve IMS callout messages and sendresponse data.

• The handler internally creates one or more IMS Connect dedicatedpersistent socket connections to the host system, using Enterprise Suite V2.2 IMS Connect API in Java.

• The handler communicates with IMS Connect via a new DataPowerdedicated user message exit, HWSDPWR1.

• For shared queue environment, user can choose to create multipleIMS Callout connections, one for each IMS datastore.

IBM Confidential


IMS Synchrouns Callout - DataPower Solution Highlights

IMS ApplicationIMS Application

ICAL A (Xid*)

ICAL B (Yid*)

ICAL C (Zid*)

ICAL A (Xid*)

ICAL B (Yid*)

ICAL C (Zid*)

* 8 bytes unique service id* 8 bytes unique service id

Each descriptor:

- Member, TPIPE- Adapter, Converter

- Timeout

Each descriptor:

- Member, TPIPE- Adapter, Converter

- Timeout

Destination

Routing

Descriptor

Destination

Routing

Descriptor

OTMAOTMA

IMS

Connect

IMS

Connect

DataPowerDataPower

resume

TPIPE

bytes

IMS

External

Service

X

Y

Z

External

Service

X

Y

Z

request bytes

with service id

response

bytes

COBOL copybook/

PLI import

for

ICAL request/response

A, B, C

COBOL copybook/

PLI import

for

ICAL request/response

A, B, C

WebSphere

Transformation

Extender (WTX)

WebSphere

Transformation

Extender (WTX)

Schema

from

services

X, Y, Z

Schema

from

services

X, Y, Z

deploy

import /

generate

type

tree

request/

response

In XML

IMS Callout

front side

handler

IMS Callout

front side

handler

WTX generated

Maps for

A-X, B-Y, C-Z

WTX generated

Maps for

A-X, B-Y, C-Z

WTX generated

Maps for

A-X, B-Y, C-Z

WTX generated

Maps for

A-X, B-Y, C-Z

WTX generated

Maps for

A-X, B-Y, C-Z

WTX generated

Maps for

A-X, B-Y, C-Z

Bottom-up or meet-in-middle

mapping

Transform rule: (user

defined XSL can

select a map based

on service id)

Transform rule: (user

defined XSL can

select a map based

on service id)


DataPower: most policy are configuration driven via browser-based Web GUI


DataPower Browser-based Web GUI

DP Developer / Admin

Create Service Gateways


Configure DataPower for IMS Synchronous Callout Requests

1. Configure the Multi-Protocol Gateway (MPG)2. Configure IMS Callout Front Side Handler.3. Configure the connection between DataPower and the backend external service provider.

4. Define a MPG “Policy”. Create one or more “Rule”(s) for the Policy

– Define the MPG processing policies and rules that determine the actions that DataPower takes the callout requests and responses that it handles.

5. Apply the changes, and save the configuration

IBM Confidential


Multi-Protocol Gateway

� A Multi-Protocol Gateway (MPG) connects client requests that are transported over one

or more protocols to a remote destination that uses the same or a different protocol.

– MPG supports the FTP, HTTP, HTTPS, IMS™, MQ, NFS, SFTP, TIBCO EMS, and

WebSphere® JMS protocols.

– However, MPG cannot use a WSDL to determine the configuration.

� A MPG includes following capabilities

– Implement Reliable Messaging policies

– Implement WS-Addressing protocol enforcement

– Accept and send SOAP, raw XML, or unprocessed (binary) documents

– Transform XML to binary format documents and binary format documents to ML

– Filter, validate, transform, encrypt, or decrypt XML documents

– Route XML documents

– Sign documents or verify signatures

– Process large documents in the streaming mode

– Implement document-level security or service-level security

– Communicate with clients, servers, and peers with SSL encryption

– Monitor and control data traffic based on request sources and requested resources

– Allow, reject, strip, or process attachments (MIME, DIME, MTOM)


DataPower Web GUI Control Panel

IBM Confidential


Configure Multi-Protocol Gateway

IBM Confidential


Configure IMS Callout Front Side Handler

� One or more IMS Callout Front Side Handlers can be specified in a single MPG.– For each handler, one or more TPIPEs can be specified on the same IMS Connect host and port, and same

IMS data store.

� Specify IMS system parameters to configure the handler

IBM Confidential


Define a Processing Policy for MPG

� A processing policy defines many, if not all, of the actions that are taken against the messages that

pass through the Multi-Protocol Gateway service.

– A processing policy consists of one or more rules.

– A rule consists of a matching rule and a processing rule.

– A matching rule defines the criteria to determine whether incoming traffic is processed by its

processing rule.

– A processing rule identifies the actions to perform against the incoming traffic.

� To access the configuration panel for defining processing policies, from the Configure Multi-Protocol

Gateway panel, click on the “+” button under Multi-Protocol Gateway Policy.


Configure a Matching Rule & Match Actions

� In the Configure Multi-Protocol Gateway Style Policy panel, a rule is depicted as a line with symbols

on it.

– Each rule consists of a Matching Rule, which determines whether or not to process the incoming

data, a Results Action, and one or more processing actions in between.

– Each rule can be configured to flow from client to server, vice-versa, or in both directions.

� A matching rule determines whether and how to process incoming data.

– A matching rule is represented by a Match Action icon, .

– A Match Action icon is automatically placed on the rule line when you create your policy rule

� Double click on the Match Action icon to define the match rule. Click on the “+” button to add a new

Matching Rule; click on “Y” to edit an existing one.


Configure a Transform Action (a map or XSL Stylesheet-driven Action)

� A Transform Action transforms a message from one format to another format

– For example: from COBOL byte arrays of the copybook of an IMS application program to XML

schema used by external service provider

� The Transform Action requires either a WTX map artifact or a stylesheet that maps the data

between the two formats.

– A stylesheet can also be used to select between multiple WTX maps.

� To add a Transform Action to the processing rule in the MPG Style Policy panel, drag the

“Transform action” icon, , onto the rule line right after the matching action .

� To configure a Results Action, drag the Results action icon, , onto the end of the line that

represents your processing rule.

� Double click on the Action icon on the line to configure the Rule.


Configure Multi-Protocol Gateway Style Policy

IBM Confidential


Data Power Configuration in supporting IMS Callout


Operational Considerations

� Operational Characteristics

– DataPower administrator can configure an IMS Callout front side handler with the

following properties: IMSHost, IMSPort, DataStoreName, TPipe(s), UserID, Password,

Group, RetryErrorLimit, RetryInterval, Connection Timeout

– DataPower administrator can enable/disable an IMS Callout front side handler

– IMS Callout Message Header

IMS Callout Front side handler sets the two headers in the request to DataPower:

ims-callout-service-id: IMS ICAL AIBMAPNM field

ims-callout-correlation-token: Hex representation of ICAL correlation token

DataPower administrator can define the XSL in the transform policy to access the header fields in the MPG policy, e.g.

� service ID as the request identifier to select input/output transformation map � correlation token as the message ID in the outbound HTTP/SOAP request.

IBM Confidential


� Operational Considerations

–An IMS Callout front side handler has the following “opstates”:

• Up: resume tpipe processing is in operation

• Down: no active processing

• Pending: in recovery mode (detected IMS Connect and/or IMS outage)

� Operational Recommendations

–When not in use, the administrator should disable the IMS Callout front side handler

Operational Considerations +


Error Handling

� Error can occur during different stages of DataPower processing:– DataPower MPG inbound/outbound policy (including WTX map transformation)– Backend service

� Error response sent to IMS application– Return code and Reason code: X’0100’– Extended reason code: 2000-3000 (set as 2000+HTTP error code from DataPower)– Error message: REQUEST PROCESSING FAILED, CHECK EXTENDED REASON CODE.“

� Error retries– When encounters error during resume tpipe, administrator can configure the following to retry before the front side handler stops processing:• RetryErrorLimit: Number of time to attempt to resume a TPIP. Default 5.• RetryInterval: Interval to wait before attempting to resume the TPIPE. Default 3 seconds

� Self-Recovery support:– Once the IMS Callout front side handler goes down after reaching the max retries, if the error is due to network problem, the front side handler attempts to self-recover. A /DISPLAY OTMA command is issued every min to verify network connectivity and IMS availability. Once the command is successful, DataPower brings up the front side handler to resume operation.

� Non-operative Auto Detection support– DataPower detects when an IMS front side handler becomes non-operative and restarts the processing automatically.


Performance Considerations

� Performance Characteristics

– New IMS Callout code has no impact on IMS or IMS Connect performance

– Initial studies in a simplified environment show that the solution is capable of

processing 8k bytes request/response messages, passing through 10 OTMA

TPIPEs, with ICAL Response Time around 30 msec.

The workload generated for this particular test case was 800-1200 TPS

� Performance Considerations

– Adding TPIPEs can improve throughputY up to a certain point.

IBM Confidential

© 2012 IBM Corporation37 IBM Confidential

High Level Diagnostics

� Documentation to collect to diagnose a problem

� IMS/ICON Side

– IMS Recorder trace

– /DIS TMEMBER hws1 TPIPE tpipe1 SYNC.

– F ICON1,QUERY PORT NAME(5555) SHOW(ALL)

� DataPower Side

– Object Status Page

– FSH and MPG System Log

– Create a Log Target “IMS” is recommended

– IMS Callout Trace messages

Note: If an error occurs, make sure the problem is recreated with the DP

trace/logs on and make them available to the tech support.


High Level Diagnostics: Documentation to collect to diagnose a problem

� Object Status ExampleI locating



� Object Status Example



� Example of IMS Front Side Handler log



� IMS Callout Trace in DataPower. Example:


Troubleshooting


Configure a Log Target for IMS Callout

� Set up a Log Target for “IMS” – Add a Log Category


Configure a Log Target for IMS Callout +

� Configure a Log Target for “IMS” callout


Configure a Log Target for IMS Callout +

� Configure a Log Target for “IMS” callout – Event subscription


Thanks

Technology

IMS DataPower - IMS UG Phoenix 12-2013