IMS SOAP Gateway Overview

© 2016 IBM Corporation

IMS Customer InternshipSOAP Gateway Overview, April 2016

2016 Customer Internship

Bryant Panyarachun, Sandy Sun

IBM Confidential



2

Prerequisites Software requirements

– Enterprise Suite V3.2 SOAP Gateway

– IMS V14

Tooling– IBM Rational® Developer for System z™ Version 9.0.1 or later

– Installation Manager V1.8.1 or later

IBM Confidential



IMS Enterprise Suite SOAP Gateway Overview

A Light-weight Web Service solution for IMS without the need of a Java EE server

Integrates IMS assets into the Service-Oriented Architecture (SOA)

Provides end-to-end interactions between IMS transactions and web services clients in an On Demand environment

Enables IMS as Provider and Consumer of Web Services

Secure

Excellent tooling support–Utilizes Rational Developer for System z tooling to create converters

for transforming XML messages to IMS messages and vice versa–No need to change existing IMS application code

3



WW Customers using SOAP Gateway

Production– A bank in South Africa uses SOAP Gateway in Production; used it also during the 2010 World Cup

Soccer– A large commercial bank in Japan– US Defense– US State Governments– A large bank in Europe– A car and equipment manufacturing company in Europe– An equipment manufacturing company in US– A global bank in Germany– A large bank in Canada

Customers expressed interest and are investigating or doing PoCs/PoTs – World's leading aerospace company and the largest manufacturer of commercial jetliners and military

aircraft combined – A leading financial institution in USA – A global leader in providing IT-enabled business solutions and services – A large U.S. based health insurance company – A logistics services and packaging/shipping company – One of the biggest European airline company – Systems software company with emphasis in systems management, mainframes – Global insurance companies group – A financial institution in China– A large US telecomm Co.– A IT consulting firm in USA– more

4


Installation using IBM Installation Manager for z/OS

Presenters Name: Bryant Panyarachun


IBM Confidential



6 IBM Confidential

Outline

The installation architecture in IMS Enterprise Suite V3.2– Benefits

Installation using IBM Installation Manager for z/OS– How does it work?

Overall process flow Installing SOAP Gateway

JCL CLI WEB UI JCL generation script



7 IBM Confidential

The challenges before

The server files could not be mounted in READ only to prevent alterations.

Web service artifacts and log files consume critical server space, thus halting the server.

Difficult to install SOAP Gateway across multiple data centers from one managed copy. Must use MVS DUMP, copytree, or

PAX to install SOAP Gateway multiple times.

Prior to IMS Enterprise Suite V2.2, SOAP Gateway is installed under one big directory:



8 IBM Confidential

The new installation architecture

In Version 2.2+, SOAP Gateway installation is divided into three components:



9 IBM Confidential

The new installation architecture

Each component can be mounted separately.



10 IBM Confidential

What is IBM Installation Manager for z/OS?

IBM Installation Manager has been used on distributed platforms for many years for installing IBM products and applying updates.

It installs from a repository. – A repository contains a repository.config file that

stores the metadata on how to install and lay out the selected packages on the system.

More and more IBM z/OS products are using IBM Installation Manager for z/OS, including WebSphere Application Server for z/OS.



11 IBM Confidential

Installing multiple copies of SOAP Gateway

The SOAP Gateway repository is a compressed file (.zip file) that you obtain after you go through the standard SMP/E process.

– Accessible through FTP, HTTP or shared disk

The Installation Manager for z/OS will connect to the repository and install SOAP Gateway

You then use the Installation Manager for z/OS to install from this repository file.

LP AR 1 LP AR 2 LP AR 3 LP AR 4

Installation ManagerInstallation Manager

S O APS O AP S O AP S O APS O AP S O APS O AP

S O AP G atew ay repository

HTTP, FTP, shared disk

Sysplex



12 IBM Confidential

The new installation process



Installing SOAP Gateway

Page 13



SMP/e receive & apply – Will allocate and mount an HFS for the SOAP Gateway repository– Result: IMSES_SOAPGateway_V3.1.0.x.zip file in an HFS


Page 14



Allocate filesystem(s) for SGW to be installed– Use sample job AEWTSCFS


Page 15



Use Installation Manager to install SOAP Gateway– Use sample job AEWTSINS– Specify:

• SOAP component to be installed (imsserver, imsbase, imssoap)• Installation Directory• Shared resourced directory for IM• SOAP Gateway repository location

– Must run once for each SOAP Gateway component (total of 3)


Page 16



Run post installation JCL AEWPOSIN to configure SOAP Gateway– Specify:

• Directory paths where SOAP Gateway components are installed.• Java Home• Specific encoding if MVS and OMVS encoding is different. Default will use

MVS codepage. Ex. IBM-1047, IBM-500...


Page 17



Edit the configuration member AEWIOGCF for SOAP Gateway– Specify:

• Directory paths where the imsserver component is installed.


Page 18



Edit the proc AEWIOGPR to start SOAP Gateway– Specify:

• Procname• Hlqual where java load module is located• Load module version. 70 for 31-bit, 76 for 64-bit• Dataset location of SOAP Gateway config member


Page 19



Installing using Installation Manager CLI– Use ./imcl under /InstallationManager/bin/eclipse/tools


Page 20



Installing using Installation Manager Web UI– Available with IM version 1.8+– Provides same GUI as seen on distributed platforms– Use ./ibmim-web under /InstallationManager/bin/eclipse/web– Can use https with -secure flag

• Enter username and password


Page 21



Installing using Installation Manager Web UI


Page 22





Page 23





Page 24





Page 25





Page 26



Installing using sample JCL generation script


Page 27

- Define variables in a single properties file- Generates 9 files

- 7 JCL for installation/configuration- 1 procedure for starting SOAP Gateway- 1 SOAP Gateway config file

- Copies generated files to user defined dataset via ftp- Generated JCL can be submitted without additional modification





Page 28





Page 29

- Generate JCL using command line or batch job





Page 30

- Generated JCL located in HFS and uploaded to MVS dataset via FTP



Applying maintenance to SOAP Gateway– Receive & apply to get updated zip file– Modify AEWTSINS to update previous installations


Page 31



Applying maintenance to SOAP Gateway– Use ./imcl listInstalledPackages -verbose to see installation details


Page 32



Applying maintenance to SOAP Gateway– Use Web UI update


Page 33



Applying maintenance to SOAP Gateway– Use Web UI to see installation details


Page 34



Applying maintenance to SOAP Gateway– Use Web UI to see installation details


Page 35


SOAP Gateway OverviewSOAP Gateway Overview

Presenters Name: Sandy Sun


IBM Confidential



37 IBM Confidential37

Features at a glance Platforms Supported in ES 3.2 SGW

– z/OS

64bit support on z/OS Send Only with Acknowledgement for

Synchronous Callout Web Services standards

– HTTP 1.1, SOAP 1.1, WSDL 1.1, UTF-8 encoding, WS-I BP 1.0, WS-Security 1.1, 1.2

– Web Service Provider and Consumer

Management Utility and runtime cache– Supports task automation– Hot-update of cache– View properties

IMS app interaction modes– Web Service Provider

• Non-Conversational COBOL/PL/I applications

• Commit mode 1 with Synclevel none• Multi-segment messages• Execution timeouts

– Web Service Consumer• Process async and sync callout request

from IMS COBOL/PL/I applications• Top-Down Sync Callout w/ COBOL

Graceful shutdown modes–Immediate–Graceful

Connection management–Enhanced connection pooling with persistent sockets with stale connection refresh–Purge idle connections between the server and IMS Connect–Multi-Datastore support to support failover

Security–RACF userid/pwd authentication and authorization via IMS Connect and OTMA –Mutual Authentication, SSL, HTTPS, UNTP 1.0 (provider), SAML 1.1, 2.0 (provider and callout)– AT-TLS, SAF keyring support on z/OS

XML transformation and Tooling–Leverages RD/z (Rational Developer for System z)–Creates WSDL for IMS transactions as providers–Generates converters to handle XML transformation in IMS Connect–Multi-segment support

Advanced Installation Support–using IBM Installation Manager

Inbound monitoringTransaction Tracking/Logging for Inbound/ProviderTransaction logging for Outbound/Callout




Inbound support ...

Execution

Generation

<SOAP><LL><ZZ><DATA>

SOAP clientsXML

AdapterFor COBOL

AdapterTask

Manager

COBOL/PL1Converters

IMS ConnectIMS

IMSApp

LLZZTRCDDATA

LLZZDATA

TCP/IP

<LL><ZZ><TRCD><DATA>

<LL><ZZ>DATA>

SOAP Gateway

HTTP SOAP

endpoint

Gatewayconnector

SOAPprocessor

<SOAP><LL><ZZ><TRCD><DATA>

HTTP/SOAP

SOAP Gateway Management UtilityDeploy services

WSDL

CorrelatorFile

Publish

RD/zGenerate

COBOL/PL1Copybook

XMLDocument

Log

…/server/logs/imssoap.log

Development environment

Runtime environment

Java Client

.NET Client




SSL Support

- SSL Server Authentication supported

- SSL Client Authentication supported

- SSL support between Client and SOAP Gateway and between SOAP Gateway and IMS Connect (using AT-TLS)

- Supports SAF keyring

- Supports RACF configuration in IMS Connect

- AT-TLS support (on z/OS) AT-TLS stands for Application Transparent Transport Layer

Security




WS-Security Support Dynamic message based security UNTP 1.0 (Username Token Profile)

– UserID and Password map to RACF– No SSL required but suggest to have at least “Server

Authentication” SAML 1.1, 2.0 (Security Assertion Markup Language)

– We support Unsigned or Signed SAML tokens– Sender-Vouches Confirmation Method– UserID maps to RACF; SOAP Gateway extracts ID

and passes it to IMS (there is no password for SAML tokens)

– SAML 2.0 is a newer standard that introduces features such as session management, attribute profiles, encryption, metadata specifications, and pseudonyms

– Needs SSL “Client Authentication” support




WS-Security Support OTMA Security setup in relation to WS-Security support in SOAP

Gateway– If OTMA security is enabled (OTMASE={CHECK|PROFILE|

FULL}), OTMA authorizes the user to access transactions or OTMA commands.

– If OTMA security is set to OTMASE=NONE, then no authorization check is performed.

– For further information on this topic refer to SOAP Gateway documentation at:• http://www-01.ibm.com/support/knowledgecenter/SS9NWR_3

.1.0/com.ibm.ims.soap31.doc/sgw_wss_iconotma.htm?lang=en

Custom Authentication Module– Intercept the SOAP request and do more

authentication/validation on the security token before continuing (or aborting)

– Pre-reqs WS-Security

http://www-01.ibm.com/support/knowledgecenter/SS9NWR_3.1.0/com.ibm.ims.soap31.doc/sgw_wss_iconotma.htm?lang=en






SOAP Gateway Callout Overview Enable IMS as a Web Services consumer

– Enable IMS applications as clients or Web Service requesters– Interoperate with business logic outside the IMS environment

Callout IVP support Supports two types of IMS callout

– Asynchronous callout• IMS application invokes external applications without waiting for

response. Response can be received by another IMS application.– Synchronous callout

• IMS application invokes external application and synchronously wait for the response.

• A new DL/I call, ICAL, is added for synchronous callout with timeout capability and support large messages

Invokes external Web Service using SOAP– Request-Response or One-way SOAP invocation– Correlates responses back to IMS for Synchronous callout– Provides Thread pool to concurrently retrieve and process IMS callout

request to provide maximize performance– Auto reconnect to IMS Connect– Transform SOAP and XML messages




Synchronous Callout Flow

IMS

IMS App1

z/OSz/OS, zLinux, Win

DL/I ICALDescriptor nameTimeout

IMS Enterprise Suite SOAP Gateway Server

IMS Connect

Receive Callout Request in XML

Send Callout Response in XML

WebService

A

XMLAdapter

Msg1Msg2

TPIPE

Send ACK

XMLConverter

1

2

3

5

6 7

Development & Runtime Environment

4

Execution

Generation

RD/z Meet-In-The-MiddleWizard

Existing WSDL

CorrelatorFile

RD/zGenerateTask

Existing COBOLCopybook

Development environment




Asynchronous Callout Flow

IMS

IMS App1

z/OS

z/OS, zLinux, Win

:ISRT ALTPCB IMSSOAP1

SYNCPT Starts..

IMS Enterprise Suite SOAP Gateway Server

IMS Connect

IMS App2

WebService

A

InitiatingClient

XMLAdapter

Msg1Msg2

TPIPE

DFSYDTx IMSSOAP1 TYPE = IMSCON TMEMBER=SM01 TPIPE=TP3 SMEM=Y ADAPTER = HWSXMLA0 CONVERTR=SOAPIT

Send ACK

SOAPITXMLConverter

Connection Bundle(CBA)TPIPE = TP1,TP3…

1

2

3

46

78

Runtime Environment

5

Receive Callout Request in XML

Send Callout Response in XML




Allow overrides of WSDL

RFE# 53400, RTC ID: 31525 Customer Description: Allow for overriding the

destination in the WSDL and overriding the program being used as the converter.

Business Justification: bring the base level of functionality of the IMS SOAP Gateway up to par with other SOAP products

Use Case: Currently, the other products in our environment that support SOAP service calls allow for overriding the destination in the WSDL, and CICS allows overriding the program being used as the converter routine. This would also relieve the issue we have based on the limited number of OTMA descriptors




ICAL Control Data SOAP Gateway (SGW) will be enhanced to support ICAL Control Data so that

customers can send specific data elements to SGW or to the external server via SGW, and to act, based on those elements

These ICAL Control Data elements would be part of the ICAL invocation originating from an IMS application

ICAL Control Data elements are optional

Example a PORTNAME in a WSDL– SGW would route the request to the endpoint location pointed to by the

incoming value in the ICAL Control Data instead of using the endpoint location that is part of the WSDL or static service configuration done as part of service deployment

Each ICAL Control Data element is enclosed in XML style tags along with its content/value

There are 2 types of XML tags– Tags that start with the prefix "DFS"; these are defined by IBM/IMS and are

consumed by (other) IBM/IMS components– Custom tags defined by customers that are non-DFS tags and cannot start

with the prefix "DFS"




ICAL Control Data– DFSCVTNR– DFSPORTNAME

• HELLOPort– DFSHOSTPORT

• lnxec497.svl.ibm.com:8088– DFSENDPOINTURL

• http://lnxec497.svl.ibm.com:8088/axis2/services/HELLOService<wsdl:port binding="tns:HELLOBinding" name="HELLOPort">

– <soap:address– location="http://lnxec497.svl.ibm.com:8088/axis2/services/HELLOService" />– </wsdl:port>

– DFSCSHEADERS

<DFSCSHEADERS>

<DFSCSHEADER> <customHeader1> ... </customHeader1> </DFSCSHEADER>

<DFSCSHEADER> <customHeader2> ... </customHeader2> </DFSCSHEADER>

</DFSCSHEADERS>




Performance - provider

Compares the performance results of ES 3.1 release with the prior ES 2.2 release

The Non Secure PL/I provider scenario using 1.5M message, ES 3.1 SOAP achieved:– 1700 trans/sec with 16KB response message size with PL/I Top-

Down application– Over 35.6 transactions per second– 0.5% increase in transaction rate over ES 2.2 SOAP– 31% decrease in average CPU time used per transaction in the

SOAP Gateway address space when compared with ES 2.2 SOAP

– A 38% improvement in ITR (CPU efficiency) when compared to ES 2.2 SOAP




Performance - Provider

Compares ES 3.1 release performance to the prior ES 2.2 release

For the COBOL Provider Scenario with 600 byte message size, ES 3.1 SOAP achieved:– Over 6000 transactions per second– 4.2% improvement in ES 3.1 tran rate as compared to

ES 2.2– 0.8% increase in Avg. CPU time used/tran in SOAP in

ES 3.1 as compared to ES 2.2– 1.3% improvement in ITR in ES 3.1 as compared to

ES 2.2




Performance - callout

Compares the performance of ES 3.1 release to the prior ES 2.2 release

For the Non Secure Callout Scenario with 5 TPIPEs, 32 Worker Threads, 20 MPPS and 60 clients, ES 3.1 SOAP achieved:– Over 5000 ICAL requests per second– 6% improvement in ICAL request per second when

compared to ES 2.2 SOAP– 0.3%% reduction in average CPU time used per ICAL

within the SOAP Gateway address space compared to ES 2.2

– 0.9% improvement in ITR (CPU efficiency) compared to ES 2.2




IMS application development and modernization: Rational Developer for System z (RDz)

Eclipse-based application development tool for modernizing and developing System z applications

– COBOL, PL/I, C, C++, HL Assembler, Java– Supports IMS, CICS, Batch, USS, etc..– Interactive access to z/OS system– Access PDS and run JCL from your workstation

Premier IBM Integrated Development Environment for development and test of IMS applications

– RDz 7.6 adds drag-and-drop code snippet function for IMS DLI calls

Also supports IMS SOA enablement– Enables CICS and IMS applications for Web Services and SOA– Built-in wizard for SOAP Gateway and Web 2.0– Generate XML COBOL/PLI converters for XML transformation– Generate WSDL, correlator files for Web Services access




Rational Developer for System z (RDz) Tooling Support

Generate

ExistingBusiness Apps

New Business

Service

Bottom-up(Inbound only)

Map andGenerate

Meet in the middle(outbound only)

ExistingBusiness Apps

Existing service description WSDL

Transform existing applications as services

Generate

Existing service description WSDL

Top-down(Inbound & outbound)

NewBusiness App

Enable New workloads




RDz tooling




Callout with SOAP Gateway – Artifact Generation Use Rational Developer for System z (RDz) to generate callout artifacts

– Meet-in-the-middle• Maps WSDL with COBOL data structures

– Generates XML Converters • Runs in IMS Connect• Converts callout request from bytes to XML and vice versa

– Generates Correlator file• Used by SOAP Gateway to identify which Web Service to invoke at runtime and

specify callout properties– Deploy WSDL

• Contains URL address for the outbound web service




Management Utility

SOAP Deployment using Management Utility (MU)

- used to manage, deploy, and configure the SOAP Gateway server

- customers can now automate route configuration tasks

- refer to the Management Utility Command Syntax in the SOAP Gateway documentation for more details

MU Web Service deployment (example) iogmgmt –deploy –w HELLO.wsdl –r HELLO.xml

External WSDL Correlator




References

Product web site:– http://www-01.ibm.com/software/data/ims/enterprise-suite-soap-

gateway/

Product download site:– https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?

source=swg-imsentersuite Samples Website with step-by-step instructions covering many

scenarios including provider, callout, secure, and non-secure etc.– https://www.ibm.com/developerworks/mydeveloperworks/blogs/

8a337b1c-3c0c-48a5-b7cc-7f805884dbb9/tags/soap?lang=en

The latest IMS™ Enterprise Suite SOAP Gateway information is available at:– http://www.ibm.com/support/knowledgecenter/SS9NWR_3.2.0/

com.ibm.ims.es32.doc/imses_product_landing_v32.html?lang=en




Thank You!

Questions?

Technology

IMS SOAP Gateway Overview