INNOV-11: An Introduction to DataDirect Shadow Products

INNOV-11: An Introduction to DataDirect Shadow Products

Brian AndersonDirector of Product Strategy,

DataDirect

© 2006 Progress Software Corporation2INNOV-11: An Introduction to the DataDirect Shadow Products

Agenda

History of DataDirect’s Shadow/RTE Mainframe Primer Accidental Mainframe Integration

Architectures Mainframe Operational Challenges Technical Architecture Customer Use Cases Questions

Shadow/RTE – A Unified Platform for Mainframe Integration


History of DataDirect’s Shadow Products

Progress acquired NEON Systems in Jan. 2006 NEON Systems was leading mainframe integration

vendor• Data• Events• Services

NEON Systems was a market consolidator in mainframe integration space• Acquired InnerAccess Technologies in July 2004• Acquired ClientSoft in December 2004

Products in existence back to 1987


Leader in Gartner MQ

NEON Systems


Gartner Group on Mainframe Integration

““The future of the programmatic integration market is one of fewer vendors, more focused on SOA or event-driven architectures. They are the leading strategies selected by organizations as part of their legacy modernization

initiatives...””

Dale Vecchio, Research VP, GartnerMagic Quadrant for Programmatic Integration Servers, 2005

Gartner Research, February 25, 2005


Mainframe Primer Introduction

What is a Mainframe• Proprietary hardware• Proprietary OS• Data throughput• High availability• Perceived Expensive• Legacy code• Backward compatibility

Mainframe Manufacturers

• Stratus• Bull• Siemens• ICL• Unisys• Hitachi• Tandem (HP)• Fujitsu• IBM

– iSeries (formerly AS/400)– z/Series

VM VSE z/OS


Evolution of the IBM Mainframe from 1965

1

10

100

1000

10000

1970 1975 1980 1985 1990 1995 2000 2005

MIPS

s/360s/360 MFTMFT

s/370s/370 MVSMVS 24 bit24 bit

30333033 MVS/SP3MVS/SP3 24 bit24 bit

3084Q3084Q MVS/XAMVS/XA 31bit31bit

30903090 MVS/ESAMVS/ESA 31bit31bit

ES/9000ES/9000 OS/390OS/390 31bit31bit

z/Seriesz/Series z/OSz/OS 64bit64bit

z/Seriesz/Series z/OSz/OS 64bit64bit


State of the Art

IBM z/Series 2094/754• $22m• Known as z/9• 17802 MIPS across multiple z/OS images• 11687 MIPS in any single z/OS image• 54 General Purpose CPUs• 512GB main memory• 1024 I/O Channel Processors• 10gb/s Data Transfer rate• Java Co-Processor (zAAP)• Information Management Co-Processor (zIIP)• Crypto Co-Processor

Usage• 14000 z/Series machines worldwide• 50% z/OS (50% VSE, OS/390, VM, z/Linux)


Mainframe Subsystems & Application Portfolio Subsystems

• Data– Flat Files

QSAM,VSAM

– Hierachical IMS/DB, Adabas, Datacom, IDMS, M204

– Relational DB2, Supra, IDMS/R, Oracle

• TP Monitors (App Servers)– IBM

CICS, IMS/TM

– Other IDMS/DC, Com-Plete

• Languages– 2GL

Assembler

– 3GL Cobol, PL/1, Fortran

– 4GL Natural, Mantis, IDEAL, CSP

• Batch/Online

Business Applications (25-30 yeas old)• Inventory management• Credit card authorization• Retail banking• Risk management• Parts tracking• ATM• Billing• Supply chain• General ledger• Student registrations• Logistics• Payroll


Mainframe Integration Drivers

Mainframe Relevance

• Greater than 60% of the world “system of record” data on mainframe

• More than 450 of Fortune 500 rely upon mainframes

• More commercial transactions processed on mainframe than any other platform

Business Pressures

• Regulatory Compliance

• Real-time Enterprise

• Increased Agility

• Total Economic Impact

Technological Initiatives

• Business Intelligence

• Business Process Optimization

• Business Activity Monitoring

• SOA & EDA


ReplicationServer

CICS Integration

IBM DB2WORF

IBM DB2 ConnectEnterprise Edition

3270Screen Scraping

SQL AccessGateway

IBM CICSTransaction Gateway

Oracle TransparentGateway

TIBCO/webMethodsSeeBeyond Adapters

Management Management consolesconsoles

Typical Mainframe Accidental Architecture


Significant Operational Issues

Quality of Service (QoS) • End-to-end solutions as strong as the weakest link• Fragile implementations often based on gateways• Lack of consolidated diagnostics• Latency due to increased complexity

Agility• Complexity slows support for new applications• Additional hardware deployment often required• Lack of clarity on which option to choose – analysis paralysis

Cost• Redundant hardware gateways for resilience• Software licensing fees for multiple products• Multiple vendor management• Training costs for administration of multiple product• Additional employee costs for effective management

of all products• Cost of systems outage due to QoS issues


Typical Mainframe Accidental Architecture

Management Management consolesconsoles

IBM DB2WORF

SQL AccessGateway

IBM CICSTransaction Gateway

Oracle TransparentGateway

IBM DB2 ConnectEnterprise Edition

ReplicationServer

CICS Integration

3270Screen Scraping

Wouldn’t it Be Nice?Shadow RTE – A Single Unified Platform

Unified Management Console


Quality of Service

Scalability• Sysplex routing and load balancing• WLM support• Proven to support thousands of

transactions per second

Security• SAF interface for all interactions• Optimizations for session-less

integration• SSL encryption of conversation• Client SSL certificate based

authentication• Audit/chargeback

Transactions• Exposes XA protocol to JTS via

Shadow JDBC/J2CA adapters• Server Distributed Syncpoint Resource

Manager (SDSRM)• Full Resource Recovery Services

(RRS) integration• In Doubt transaction management

Reliable messaging• Support for WebSphere/MQ transport

for SOAP additions in w/MQ 6• RFH2 header support for URI• Supports ErrorQ processing• In conjunction with SDSRM enables

Async Transactional Messaging for Web Services

• SC34-6651-00 (May 2005) details the implementation

• NEON implementation using MQMD for service/action and MQSTR structure

Instrumentation• Extensive. Low-overhead event tracing

for all mainframe integrated workloads• All nodes in the Sysplex log to the

coupling facility• Central correlation point for Sysplex

transactions• Eclipse viewing tool


Shadow RTE Pluggable Features

• z/Direct: SQL access to mainframe resources Data and Applications via ODBC, JDBC or J2CA (JCA).

• z/Services: M Mainframe ainframe Web services (Publish and Consume)

• z/Events: R Real-time capture, enrichment and publishing of critical mainframe business events.

• z/Presentation: M Mainframe Web enablement

• Shadow Studio: Full featured,Full featured, Eclipse-based IDE


Shadow z/Direct

Direct SQL/Data access to:• DB2, IDMS/DB, IMS/DB, VSAM and Adabas

Supports programs using industry standard ODBC, JDBC or J2CA (JCA) without any intermediate hardware gateways

Makes mainframe applications appear as database stored procedures, supporting:• CICS, IMS/TM, Natural and IDMS/DC programs and

presentation layer

Scales to thousands of transactions per second Supports two-phase commit

(2PC) for data integrity


Web services - SOAP-based mainframe integration

• SLI, BLI, DLI

• Sources - CICS, IMS, IDEAL, IDMS, Natural, databases

Supports both Publish and Consumption

Simplifies mainframe integration

Flexible deployments

• Mainframe– Shadow RTE

– CICS

• Distributed

Shadow z/Services


SOA – Mainframe Perspective

Specific Relevance to Mainframe SOASpecific Relevance to Mainframe SOA

• Arcane systemsArcane systems

• Original tightly coupled developmentOriginal tightly coupled development

• ““Fancy” programming for technical expediency of a bygone Fancy” programming for technical expediency of a bygone eraera

• Existing mainframe integration methods force tight couplingExisting mainframe integration methods force tight coupling Web ServicesWeb Services

• Not a solution to organizational loose couplingNot a solution to organizational loose coupling

• Without SOA, Web Services are just 21Without SOA, Web Services are just 21stst century RPCs century RPCs

Web Services with SOA Web Services without SOA<SOAP-ENV:Body <SOAP-ENV:Body xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01">xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01"><s0:QSTAFF><s0:QSTAFF><s0:QSTAFFInput><s0:QSTAFFInput><s0:ID xsi:type="xsd:short">10</s0:ID><s0:ID xsi:type="xsd:short">10</s0:ID></s0:QSTAFFInput></s0:QSTAFFInput></s0:QSTAFF></s0:QSTAFF></SOAP-ENV:Body></SOAP-ENV:Body>

<SOAP-ENV:Body <SOAP-ENV:Body xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01">xmlns:SI="HTTP://DEV1.NEONSYS.COM:8093/MJC/MARK01"><s0:QSTAFF><s0:QSTAFF><s0:QSTAFFInput><s0:QSTAFFInput><s0:ID xsi:type="xsd:short">10</s0:ID><s0:ID xsi:type="xsd:short">10</s0:ID><s0:FLAG xsi:type="xsd:char">X</s0:FLAG><s0:FLAG xsi:type="xsd:char">X</s0:FLAG><s0:ACCODE xsi:type="xsd:short">8</s0:ACCODE><s0:ACCODE xsi:type="xsd:short">8</s0:ACCODE><s0:STAT xsi:type="xsd:short">5</s0:STAT><s0:STAT xsi:type="xsd:short">5</s0:STAT></s0:QSTAFFInput></s0:QSTAFFInput></s0:QSTAFF></s0:QSTAFF></SOAP-ENV:Body></SOAP-ENV:Body>


Mainframe Business Event Management Real-time Change Data Capture Maintain data consistency with mainframe system of record Automate heterogeneous business processes Non-invasive event capture

• ADABAS, DB2, Native VSAM, CICS VSAM, IDMS and IMS DB Messaging support

• WMQ, HTTP/HTTPS,

as well as inbound

J2CA communications

Shadow z/Events


Mainframe screen access via Web browser Development of “web” channels Access mainframe screen-based applications Repurpose/Rejuvenation user interface Screen aggregration for process reengineering Non-invasive Integrate with other Web

components

Shadow z/Presentation


Success Story - ABN/AMROShadow/RTE

Challenges• Due to compliance issues, ABN’s

commercial banking unit required a replacement of MDp

• Essentially all business applications are embedded into MDp, MQ, Gideon and Sybase servers, including mortgage, teller, commercial loans, small business, VRU

• Over 1M request/day flow through Gideon

• Wanted to eliminate need for MDp developer and reduce development time by 50%

• More importantly, they required a solution that would preserve and use much of the existing business logic and client code

Why DataDirect?

• Shadow RTE selected through exhaustive RFI/RFP process

• Shadow RTE achieved “technical win” during POC

• ABN/AMRO advised that early metrics show >50% development time reduction and a 30% reduction in mainframe MIPS

• Shadow RTE enables ABN/AMRO systems to achieve application agility – escaping the limitations of their previous technologies

• PERFORMANCE PERFORMANCE PERFORMANCE!


Success Story - Her Majesty’s Land Registry (UK) – z/Direct

Challenges

• Needed an easy-to-use, industrial-strength mechanism to deliver access to mainframe data in a mission-critical application from a Java environment

• Required support for both ODBC and JDBC applications accessing DB2 without a middle-tier

• Performance was an absolute requirement

• 4th largest DB2 shop in the world

Results• Shadow provided enterprise-class

integration for business-critical, distributed applications with the company’s DB2 mainframe database.

• Faster, reliable access to DB2 data resulted in higher quality of service.

– Performance metrics exceed 30,000,000 SQL calls per hour (peak), 300,000,000 SQL calls per day – every day

• Reduced management complexity


Success Story - Sempra UtilitiesShadow z/Services

Challenges• Facilitate development of

mainframe Web services for eServices customer portal (self-service) and new business contractor

• Key problem was sheer complexity and time requirements of hand-coding of mainframe Web services.

• No mainframe-based in-house Web services talent

Why DataDirect?• Deal signed on 9/29/05• Initially whittled vendors down to

InnerAccess and ClientSoft in 2004• Decision suspended for a few months• Sempra called us back in April 2005

to move forward with ClientSoft due to superior development environment

• When customer found out ClientSoft had merged with NEON, they were extremely pleased

• Sempra views Shadow RTE as a truly strategic offering – a unified platform for mainframe integration, not just a tactical Web services generator


Success Story – Wells Fargo

Challenges• Open up existing legacy

databases and applications• Facilitate development of

mainframe customer portal• Expand channels to service

customer• No mainframe-based in-house

Web services talent

Results?• Customer since 1996• Leveraged Shadow RTE to create

customer portal driven primarily from existing mainframe functionality

• Contains an assortment of Web services for servicing customer accounts through a variety of channels e.g. internet banking, customer services, etc.

• Accesses primarily legacy data and transactions in DB2 and VSAM/CICS environment via XML, SOAP, and proprietary messaging.

• Over 7mm invocations of mainframe Web services per day


Success Story - Miami-Dade County, Florida

Challenges• Needed to create a service-

oriented architecture that would standardize access to multiple disparate mainframe systems

• In an effort to build new Web & wireless applications, needed to enable effective reuse of mainframe functionality via Web services to build new apps

• Wanted to eliminate need for expensive point-to-point integration

Results• Standardizing access to core

mainframe functionality across 40 county departments

• Easily incorporated mainframe into new J2EE applications via Web services

• Reduced IT development and maintenance costs moving forward

• Enhanced services to county employees and taxpayers


Success Story - Electric Insurance Company

Challenges

• Needed to enhance revenues by bringing pricing changes to market in a more efficient manner

• Wanted to eliminate the need for redundant policy rating functionality

• Needed to enable the mainframe to directly consume rating data from an external, Web-based rating engine via Web services

• Required interoperability within Microsoft .NET

Results• Projected revenue increases of

30% by enabling more efficient pricing changes

• Eliminated need for redundant COBOL rating engine

• Enabled mainframe to consume .NET Web service from more modern Web-based rating engine

• By incorporating mainframe into new SOA, now have ability to offer umbrella coverage nationwide via the Internet


For More Information, go to…

PSDN• www.shadowrte.com

• www.datadirect.com

A complete series of detailed product webinars including demonstrations are available at the above URLs.


Questions?


Thank you foryour time


Documents

INNOV-11: An Introduction to DataDirect Shadow Products