IMS Synchronous Callout

© 2015 IBM Corporation

®

IMS

Using IMS Synchronous Callout

[email protected]

mailto:[email protected]

IMS

2

Customers running IMS have a variety of computer hardware and network configurations installed, typically employing IMS as a repository for information.

Some customers also use IMS as one of their application platforms in Enterprise resource planning (ERP) systems that integrate internal and external management information.

Why calling out?

IMS

3

Topics

Asynchronous callout vs synchronous callout

Overview of 6 IMS callout methods

Deep-dive of the DL/I ICAL call for callout

Best practices of using DL/I ICAL call

V13 synchronous program switch using DL/I ICAL call

V14 DL/I ICAL call with control data

V14 Parallel callout using DL/I ICAL calls


®

IMS

Asynchronous callout vs Synchronous callout

IMS

5

Synchronous Model

IMS Application

GU IOPCB

SEND_DATA Message

READ_with_WAIT RECEIVE REPLY

ISRT IOPCB to CLIENT

Message READ_DATA Message PROCESS_DATA

SEND_DATA Reply

CLIENT

Reply

MSG-Q CLIENT IMS/non-IMS

MSG-Q

ONE IMS Unit of Work (UOW)UOW1

IMS

6

Asynchronous Model

READ_DATAMessage+appl-token

PROCESS_DATA

SEND_DATA Message+appl-token

IMS Application

GU IOPCB

SEND_DATA Message+Token

Message+Token

CLIENT

IMS Application

GU IOPCB Reply+Token

ISRT ALTPCB to CLIENT

Reply+Token

MSG-Q

MSG-Q

CLIENT IMS/non-IMS

READ_DATA Message+Token

PROCESS_DATA

SEND_DATA Reply+Token

MSG-Q

MSG-Q

TWO IMS Units of Work (UOW)

UOW1

UOW2


®

IMS

Overview of 6 IMS callout methods

IMS

8

IMSA-PGMAGU IOPCB

ISRT ALTPCB

CPI-C TP-ACMRCVCMSEND

CMDEAL APPC/IMSCMALLC Allocate TP-AType=NONE/CONFIRM CMSEND SEND_DATACMDEAL DeAllocate

ConversationA

CLIENT

ISRT IOPCB

UOW1

IMS LU6.2Descriptor

UOW2

CPI-C/APPC Synchronous

CMALLC Allocate TP-BType=NONE/CONFIRM/SYNCPT CMSEND SEND_DATACMRCV RECEIVE_and_WAITCMDEAL DeAllocate

ConversationB

UOW1

CPI-C TP-BCMRCV CMSEND CMDEAL

#1 Callout using APPC

APPC Asynchronous

IMS

9

IMSA-PGMAGU IOPCB

MQCONN MQ-BMQPUT NO-SYNCPOINT

MQGET NO-SYNCPOINTMQDISC

MBMQGETMQPUT

MQDISC

Queue

messages

#2 Callout with WebSphere MQ API

CLIENT

ISRT IOPCB

IMS Managed

UOW1

IMS

10

Service

IMSA-PGMAGU IOPCBEXEC SQL:

CALL PROC ( :aaa : bbb … DB2 stored procedure

User-defined function (UDF)

data

#3 DB2 Stored Procedure via ESAF

CLIENT

ISRT IOPCB

IMS Managed

UOW1

IMS

11

READ_DATAWRITE_DATA

CLOSE

Socket Connect - TCP/IP-BWRITE_DATAREAD_DATA

CLOSE

TCP/IP-BREAD_DATASocketConnect -TCP/IP-C

Connection

datagrams

#4 User-Written TCP/IP Synchronous callout

CLIENT IMSA-PGMAGU IOPCB

ISRT IOPCB

UOW1

UOW2

IMS

12

IMS-ApplGU IOPCBISRT ALTPCB

OTMA Descriptor TMEMBER TPIPE IMS Connect

Socket

Message

#5 Asynchronous callout via OTMA

TCP/IP-BResume_TPIPEREAD_DATAACK

CLIENT

ISRT IOPCB

UOW1

UOW2

IMS

13

IMS-ApplGU IOPCBICAL OTMA

Descriptor TMEMBER TPIPE IMS Connect

Socket

Message

#6 Synchronous callout using DL/I ICAL

TCP/IP-BResume_TPIPEREAD_DATA(COR_ID)ACKSEND_DATA(COR_ID)

CLIENT

ISRT IOPCB

UOW1

UOW2

SEND_DATA

RECEIVE_DATA


®

IMS

Deep-dive of the DL/I ICAL call for callout

IMS

15

DL/I ICAL call provides the capability to callout to resources outside IMS synchronously

IMS SOAP GATEWAY

z/OS

IMS

Database

DB Services

OT

MA

TM/CTLServices

IMS

Connect

TCP/IP

user-written Client

WebSphere

IMS TM resource adapter Application

Program

ICAL

MPP/JMP/IFP/BMP/JBP

OTMADescriptor

OTMADescriptor

DataPower

IMS

16

Overview - IMS Synchronous Callout using DL/I ICAL

IMS listener in WAS or distributed application

It is a SYNCHRONOUS call in your IMS application.

–Need to change your existing IMS application or create a new IMS application

– Leverage asynchronous flow, although appears synchronous to IMS application. Based on OTMA Resume TPIPE/Send Only protocol function for retrieving synchronous callout requests and sending responses.

Does not go through IMS Message Queue, and the synchronous callout message/response are not recovered.

Does not support 2 phase commit support using RRS

Callout message needs to go through IMS Connect

– It does not go through MQSeries support

IMS

17

Overview - IMS Synchronous Callout using DL/I ICAL…

Supports messages larger than 32K without segmentation– No need to build data in LLZZ format

Timeout support to optionally terminate callout request and free the dependent region

IMS commands enhanced to view synchronous callout status and stop the callout processing

Limited security credentials propagated by IMS to external server

If a shared queues back-end IMS does not have an IMS Connect connection, the back-end IMS cannot process DL/I ICAL for callout

IMS

18

Protocol of Sync Callout using ICAL

IMS

19

Steps to have a synchronous callout application using ICAL

STEP 1:

STEP 2:

STEP 3:

STEP 4:

Determine the external application (WebSphere TMRA, TCP/IP roll-your-own application, TCP/IP application using IMS Connect API, DataPower or IMS Soap Gateway application)

Setup IMS (OTMA destination descriptor)

Setup IMS application (COBOL, PL/I, REXX, C, JAVA…..)

Setup the external application or server for callout messages

IMS

20

Determine the external application

Database Management

A light-weight Web Service solution for IMS without the need of a Java EE server

Java EE apps (MDB or EJB) running in WebSphere Application Server (WAS) using IMS TM Resource Adapter (IMS TM RA)

Transaction ManagementWebSphere MDB/EJB appl.

TCP/IP application connects to IMS Connect

Need to understand IMS Connect and OTMA protocols

SOAP Gateway Appl.

SystemRoll-your-own Appl.

Supports the high performance IMS Callout solution, and DataPower’s hardware acceleration for XML processing.

Roll-your-own w/ DataPower

A simplified callable interface for interaction with IMS Connect

Addresses the complexities of writing RYO applications

Roll-your-own w/ ICON API

IMS

21

Setting up OTMA destination descriptor in IMS PROCLIB

D destname keywordsWhere: destname is destination names and can be masked by ending in an * keywords are: TYPE=IMSCON TMEMBER=name TPIPE-name SMEM=YES|NO ADAPTER=adapname CONVERTR=convname SYNTIMER=timeout (If both ICAL & Descriptor specify timeout, the lower value is used)

D OTMDEST1 TYPE=IMSCON TMEMBER=HWS1 TPIPE=TPIPE1 SYNTIMER=500

The OTMA Destination Routing Descriptors were introduced in IMS 10. They externalize the routing definitions and specifications for callout messages without IMS user exits. Descriptors are read and initialized at IMS startup.

– Use ‘D’ descriptor type in DFSYDTx member of IMS.PROCLIB.

For example:

D SOAPGW1 TYPE=IMSCON TMEMBER=HWS1 TPIPE=HWS2SOAPD SOAPGW1 ADAPTER=XMLADPTR CONVERTR=XMLCNVTR SYNTIMER=200 D SANDY01 TYPE=IMSCON TMEMBER=HWS2 TPIPE=TPIPE2

IMS

22

IMS type-2 commands for OTMA destination routing descriptors

Type-2 commands can also be used to update, create, delete, or query the descriptor entries

– CREATE OTMADESC

– UPDATE OTMADESC

– QUERY OTMADESC

– DELETE OTMADESC

For example: UPDATE OTMADESC NAME(OTMDEST1) SET(SYNTIMER(800))

QRY OTMADESC NAME(OTMDEST1) SHOW(ALL)

IMS

23

Prepare your IMS application for the DL/I ICAL

>>-ICAL--aib---request_area---response_area----><

The format of the ICAL call for a synchronous callout request:

IMS

24

COBOL Application with ICAL

Descriptor name

Timeout

Request Area LengthResponse Area Length

Output return codes

ICAL

AIB

IOAreas

IMS

25

BMP with DDLT0 tool

25-32 a destination name in OTMA destination descriptor name, 8–byte character field (ICAL). Our example is JOHNSON1

34-39 The wait time for the synchronous call to be processed, 6–byte character field with a range from 1 to 999999 (ICAL), our example is 100 seconds

41-45 The input message length nnnnn The length of the input data in the request area (ICAL). Our example is 50 bytes

47-51 The response area length nnnnn The length of the response area for the output message (ICAL). Our example is 1000 bytes

//JACK EXEC DDLT0,PSBNAME=PSAPP99,TYPE=BMP,IMSID=TEST, // QUAL=IMSTSYS,NBA=50 //SYSIN DD * S1111 1 1 1 1IOPCB AIB L ICAL SENDRECV JOHNSON1 010000 00050 01000 L DATA HELLO FROM IMS E OK

--------------------------------------------------------------------------------SEGMENT =(HELLO FROM IMS ) ................................................................................CALL=ICAL SEGMENT =(HELLO FROM WS ) COMP RET CODE=OK IOPCB LTERM= RET CODE= RACFUSER=PSAPP99 . 0003 0003 0000 AIB BLOCK AIBOALEN = 0002, AIBOAUSE = 0006, RETURN CODE = 0000, REASON


®

IMS

Best practices for using ICAL DL/I call

IMS

27

Best practices for using ICAL DL/I call from IMS applications

Do not ignore the extended reason code of the ICAL call

You can issue “/DISPLAY TMEMBER xxx TPIPE yyy SYNC” to see the callout message

IMS

28


Specifying the destination descriptor info correctly in the AIB block for callout

IMS

29


If needed, specify a 8-byte unique service identifier in AIB for data transformation and service routing for call-out to DataPower.

IMS

30


Use a wild card character if possible for OTMA destination descriptor

– IMS V13 can support up to 510 entries in OTMA destination descriptor

– IMS V14 supports up to 4095 entries

– For IMS Soap Gateway callout messages, we may have thousands of different “converter names” to be specified in the OTMA destination descriptors.

– If the destination names are similar, application programs can use a wild card character when specifying the descriptor name so that its callout request can be routed to any destination that matches the portion of the name that is not masked by the wildcard character.

IMS

31


Design synchronous callout applications to minimize resource contention

– After an IMS application issues an ICAL call for a synchronous callout, it waits in the IMS dependent region until either the response is received from the external application or the timeout interval expires. While waiting, all of the resources that the application program has accumulated during the current unit of work are held.

– To avoid excessive resource contention, have the application program issue the ICAL DL/I call as early as possible, before too many resources are obtained.

IMS

32

Best practices for using ICAL DL/I call from IMS applications…

Define an appropriate number of IMS dependent regions

– When many IMS applications use synchronous callout, especially with slow external service providers, it may become necessary to increase the number of IMS dependent regions. This comes with the concerns of more resources and greater chances for resource contention.

IMS

33


Define appropriate timeout values on the ICAL call

– The synchronous callout request also allows for setting a timeout value so if the callout response is delayed the IMS application will receive return and reason codes indicating the timeout and the application can decide whether to proceed without the callout information or to retry the callout request. The timeout value must be balanced between allowing sufficient time for the callout request to be processed and protecting the IMS dependent region from being held up.

IMS

34


ICAL timeout value might need to greater than OTMA ACK time-out value. This is because IMS OTMA will cancel a Resume Tpipe request due to ACK time-out.

IMS

35


Manage high volume and multiple destinations with OTMA destination descriptors

– If high volumes of callout requests are anticipated then the service provider should make retrieval requests to different destinations and separate descriptors can be used to route callout requests to these separate destinations. This can improve throughput, but might add complexity to the IMS applications.

– The client application might want to use a dedicated Resume TPIPE task with AUTO option and a separate processing & response thread using a second socket. And allow for multiple worker threads using separate sockets for the responses to get more parallelism.


®

IMS

Synchronous program switch in IMS V13

IMS

37

Synchronous Program Switch

Additional capability that enhances the DL/I ICAL support

– Allows an IMS application program to synchronously call and wait for a reply from another IMS application program

• Within the calling program’s UOW

Database

IMS DB services

IMS dependent region

IMS TM services

MPP, JMPIFP, BMP,

JBP

OTMA

Application Program

ICALIMS SOAP GATEWAY IMS

Connect

TCP/IP

user-written Client

WebSphere

IMS TM resource adapter

IMS Version 13

38

The DL/I ICAL callSame Format

>>-ICAL--aib--request_area--response_area----------------------><

Call Name DB/DC DBCTL DCCTL DB Batch TM Batch

ICAL X X

LLZZ+Trancode+Data

LLZZ+Data

LLZZ+Data

Request Data (example of multi-segment):

LLZZ+Data

LLZZ+Data

LLZZ+Data

Response Data in multi-segment:

IMS Version 13

39

The DL/I ICAL Call …

Continues to use OTMA Destination Routing Descriptors – Which externalize the routing definitions and specifications for callout messages

and synchronous program switch messages– Allowing up to 510 destination routing descriptor entries defined in DFSYDTx

member of IMS.PROCLIB

– With new TYPE= IMSTRAN for synchronous program switches

D entry_name keywords Where entry_name is descriptor entry name and can be masked by ending in an * keywords are: TYPE=IMSTRAN

LTERMOVR=name TMEMBER=name TPIPE-name SMEM=NO|YES EXIT= NO|YES REPLYCHK=YES|NO SYNCTP=NO|YES SYNTIMER=timeout value

For example: D OTMDEST1 TYPE=IMSTRAN SYNTIMER=500

D OTMDEST2 TYPE=IMSTRAN TMEMBER=SCOTTHWS1 TPIPE=BRYCE EXIT=YES


®

IMS

ICAL call with Control Data in IMS V14

© 2015 IBM Corporation41

Syntax of the ICAL DL/I call before IMS V14

>>---ICAL---aib---request_area---response_area-----><

The parameter aib specifies the application interface block (AIB) that is used for the call.

The request_area specifies the request area to use for this call.

– This request area contains the request message data that is sent from the IMS application program to the application that is specified in the OTMA destination descriptor.

The response_area specifies the response area to use for this call.


V14: Enhanced ICAL syntax with control data

>>--ICAL--aib---request_area---response_area---control_area><

The parameter aib specifies the application interface block (AIB) that is used for the call.

The request_area specifies the request area to use for this call.

The response_area specifies the response area to use for this call.

The control_area specifies the optional control data to use for this call.


How to specify the control data for an ICAL call

The control data can consist of 1 to many control data items.

The format of control data item(s) in the ICAL control data are as follows:

LLLL | <tag> | data | </tag> {LLLL | <tag> | data | </tag>…}

The tag name and data contents will be treated as binary and passed "as is" to the target client.

The AIBOPLEN field in AIB must specify the total length of the control data.


®

IMS

V14 Parallel Callout

IMSIMS Connect

TPIPE1Has 3 messagesWhich will be Delivered one byOne. If one is blocked,The others cannotBe delivered. It is FIFO.

MPP1

ICAL to Tpipe1

MPP2

MPP3

ICAL to Tpipe1

Problem: Bottleneck for ICAL Output

TPIPE1

Msg1

Msg3

Msg2

Our problem: A bottleneck of throughput when there are many Outbound messages sent one by one via 1 tpipe/link

ICAL to Tpipe1

© 2014 IBM Corporation46 IMS 14 T3 Class April 28 – May 1, 2014 | IBM Confidential

Problems being Addressed

Multiple IMS Applications issuing ICAL to a single TPIPE – TPIPE serialization makes them wait longer, holding the dependent regions,

increasing region occupancy and transaction response times

IMS sending callout message require ACK before next message can be sent – Ensures message integrity (good) but serializes the message flow (bad)

Only one Resume TPIPE can be active for any given TPIPE– Other Resume TPIPEs will be queued and have to wait for the active RT to terminate – This provides for failover but not throughput or workload balancing

IMS

47

Parallel Callout (MULTIRTP=YES) Illustration

Client1

Resume TPIPEALTCID=ICONTP1

Client2


Client3


TPIPE=ICONTP1

Callout msg 1Callout msg 2Callout msg 3

IMS

48

MULTIRTP for OTMA

can set MULTIRTP using OTMA Member descriptors in DFSYDTx PROCLIB member

– ‘M’ descriptors for specific members MULTIRTP={NO|YES}

– Special DFSOTMA to set global value

– Can be used to set default to MULTIRTP=YES ***

----+----1----+----2----+----3----+----4----+----5M HWS1 DRU=HWSYDRU0,MULTIRTP=YESM HWS2 INPT=4500M HWS3 MULTIRTP=NOM DFSOTMA MULTIRTP=YES

HWS1 is MULTIRTP=YESHWS2 uses DFSOTMA default of MULTIRTP=YESHWS3 is MULTIRTP=NOHWS4 uses DFSOTMA default of MULTIRTP=YES

Technology

IMS Synchronous Callout