2011 06 evolution of the ims database_nyc_toronto_ug

© 2011 IBM Corporation

Evolution of the IMS DatabaseFrom the Moon to the Cloud

Marilene Roder – Software Engineer

[email protected]


Data Base

Concurrent Update

From Multiple

User/

System

Multiple Systems

Coupling (MSC)

Batch

DBMS

Exploit MP

Architecture

Two

Phase

Commit

Deadlock

Detection

> 2500 Transactions

Per Second

1968

2011+

Concurrent

Image Copy

Parallel

Systems

N-Way

Data Sharing2-way Data

Sharing

JDBC Access

Remote Site

Recovery

APPC/IMS

Distributed

Access

...

Web Services

> 1000 Transactions

Per Second

~28000 Transactions

Per Second

IMS Database Progress

Java

Dependent

Regions


Agenda

� IMS Databases: overview

� Database Quiesce (IMS v11)

� Open Database (IMS v11)

� XML DB (IMS v9) and XQuery (IMS v10)

� Integration with other Tools


IMS/DB is a Database Management System

� A Database Management System – is a collection of programs for storing, organizing, selecting, modifying, and

extracting data from a database.

� IMS DB is organized hierarchically– To optimize storage and retrieval – To ensure integrity and recovery– Segments are implicitly joined

A

B C DINSTRUCT

I 11

STUDENT

S 11

COURSE

C 1

CLASS

L 11

1213

1415

1617

12

13


IMS Database Access Methods

Sequential Access

HSAM

SHSAM

GSAM

Indexed Sequential Access

HISAM

SHISAM

Direct Access Indexed Direct Access

HDAM

PHDAM

DEDB

MSDB

HIDAM

PHIDAM


HALDB - What Is It?

� Large Databases– Databases are partitioned

• Up to 1001 partitions per database• Partitions have up to 10 data set groups• Partition definition separated from database structure definition

� High Availability Databases– Partition Independence

• Allocation, authorization, reorganization and recovery by partition– Self-Healing pointers

• Reorganization of partition does not require changes to indexes or logically related databases which point to the partition

� Partitioned Database Data Sets– PSINDEX

• 1 data set - secondary index data set– PHDAM

• 2 minimum/11 maximum - 1 for data, 1 for indirect list data set– PHIDAM

• 3 minimum/ 12 maximum - 1 for data, 1 for indirect data set, 1 for primary index data set


IMS HALDB Highlights

� Database types

– PHDAM - partitioned HDAM

– PHIDAM - partitioned HIDAM• Index is also partitioned

– PSINDEX - partitioned secondary index

� Hierarchic structure is maintained

– A database record resides in one partition

� Partition selection

– By key range or by user exit routine

...


Availability

Compatibility

Usability

Performance

IMS HALDB Benefits

� Greater database capacity

– "Unlimited" capacity

– Partitions may be modified as requirements change

� Increased database availability

– Shortened reorganization process

– Option of Online Reorg (IMS V9+)

– Batch window may be shortened with concurrent processing

� Improved performance– Concurrent processing against partitions

� Application program changes are not required


Customer Examples –

� German Bank

– 600 GB total capacity of all

databeses, therefrom– 512 GB for all HALDBs

•Largest HALDB: 340 GB

� German Customer

– 17 HALDB

– Largest HALDB: 630 GB with 90

partitions and 2 secindex with 55

partitionen

� …

� France

– Avant 01/2004 en France•AGF, Euroinformation, Natixis,

Renault, Pole Emploi

– Après 2004

•GCA, …

� UK Insurance

– Largest HALDB: 140GB


IMS HALDB Information

� Redbook:

– The Complete IMS HALDB Guide, All You Need to Know to Manage HALDBs

• SG24-6945 – June 2003

• http://www.redbooks.ibm.com/abstracts/sg246945.html?Open

� Presentations and articles on the Web

– IMS High Availability Large Database (HALDB)

• http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS185

– Migrating to IMS HALDB – Refreshed 2009


– Application Design and Programming with HALDB


– Using GENJCL.USER to Allocate IMS HALDB Data Sets

• http://www-1.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD100491


�Integrated OnLine Reorganization of HALDBs

ƒ HALDB OLR provides 100% availability of the largest databases in the world!

ƒ OLR provides non-disruptive reorganization of HALDB PHDAM and PHIDAM partitions

�Concurrent IMS updates are allowed while OLR

is active

ƒ Planned data outage not required

Online HALDB Reorganization


Relationship between Input and Output Datasets


Agenda







Database Quiesce

� Offers the ability to stop accessstop access to a Database, HALDB Partition, DEDBDatabase, HALDB Partition, DEDB or Area,Area, or

Datagroup,Datagroup, allowing a singlesingle coordinated recovery pointcoordinated recovery point to be established

–– WithoutWithout taking the resource offline! No /DBR is required

• Type-2 UPDATE DB|AREA|DATAGRP command is used to start and stop quiesce

– The recovery point will be coordinated across the IMSplexacross the IMSplex to create one common

recovery point

– Allows an image copy to be taken while the database or area is online

�� Reduces the complexityReduces the complexity in establishing a recovery point for a database

Improved Usability

Improved Availability

ReducedComplexity


Database Quiesce…

� Creating a recovery point withoutwithout Database Quiesce…– Issue /DBR on each IMS to establish a recovery point

•Each IMS will record a DEALLOC time in the RECON•Database resource is closed and data sets are deallocated

– Check to make sure the database or area was successfully taken offline on each IMS– Issue /STA on each IMS to allow database to be used

•OPEN option on /STA command is required to allocate and open the data sets

Otherwise first access will open the data sets

Improved Usability


ReducedComplexity

IMS

RECONIMS

IMS


Database Quiesce…

� Creating a recovery point withwith Database Quiesce…– Issue UPDATE DB|AREA|DATAGRP START(QUIESCE) command

•Command is processed by one IMS (command master) in the IMSplex

Databases and data sets are not closed and deallocated

Command waits for active use of the resource to reach a commit point and then quiesces the resource

Coordinated across the IMSplex, OLDS are switched on each IMS by default

RECON updated by one IMS with common DEALLOC timestamp on behalf of all the IMS systems

» New allocations for FP areas done on local IMS

» For FF databases, a new ALLOC is created at first database update after releasing the quiesce

Command master communicates with IMS systems to make resource available again

Improved Usability


ReducedComplexity

IMS

RECONIMS

IMS


Agenda







IMS Open Database

� Solution Statement– Open new avenues to IMS

• Offer scalable, distributed, and high-speed local access to IMS database resources

� Value– Business growth

• Allow more flexibility in accessing IMS data to meet growth challenges

– Market positioning• Allow IMS databases to be processed as a standards-based data

server

� Key differentiators– Standards-based approach (Java Connector Architecture, JDBC, SQL, DRDA)– Solution packaged with IMS

� Enables new application design frameworks and patterns– JCA 1.5 (Java EE)– JDBC

Innovative ImprovedUsability

OpenStandards


Integrating IMS DB in a composite application

1. Channel: Web User Interface

3. Master Data Management Services

4. Operational Data Store: IMS DB

2. Process Services

ACCOUNT

ACCTID

ACCTNAME

BALANCE

CONTRACTID

TERMS

PRODID


IMSIMSIMS

IMS

Mainframe

IMSIMSIMS

IMS

Mainframe

IMS Open Database

J2EE

JDBC Universal drivers

DLI

TCP/IP

TCP/IP

J2EE

JDBC Universal drivers

DLIDirect

z/OS

Distributed (e.g.; LUW)


OpenStandards


Solution highlights – connectivity

� Type 4 drivers– Universal drivers support type 4 connectivity to IMS databases from TCP/IP

enabled platforms and runtimes• WebSphere Application Server

• Windows

• zLinux

• z/OS

• Standalone Java SE

� Type 2 drivers– Universal drivers will support type 2 connectivity to IMS databases from z/OS

runtimes• WebSphere Application Server for z/OS

• IMS Java dependent regions

• CICS

– All environments zAAP eligible


OpenStandards


Solution highlights – Universal drivers

� Three Universal drivers

–Universal DB resource adapter• J2EE deployment

• Standard Java SE deployment

–Universal JDBC driver• Standard Java SE deployment

–Universal DLI driver• Standard Java SE deployment


OpenStandards


Solution highlights – J2EE deployment

� Universal DB resource adapter

–JCA 1.5• XA transaction support

Manage multiple datasource connections in a single UOW

• Local transaction support

Manage multiple datasource connections each in their own UOW

• Connection pooling

Pool released connections for future use

• Connection sharing

• Multiple programming models available

JDBC (Universal JDBC driver incorporated)

CCI with SQL interactions

CCI with DLI interactions


OpenStandards


Solution highlights – JDBC

� Universal JDBC driver– Significant enhancements to classic JDBC offered in IMS 9 and IMS 10

• Standardized SQL support

• XA transaction support (type 4)

• Local transaction support (type 4)

• Concurrency control

Control release of distributed locks

• Updatable result set support

• Batching support

Fetch multiple rows in a single network call

• JDBC metadata discovery support

� Standard SQL and metadata discovery enables significant integration opportunities for IMS


OpenStandards


Solution highlights – DLI

� Universal DLI driver

–Java implementation of DL/I API

–Complete DL/I support for database access

–All IMS command codes supported

–Can mix usage of JDBC and DLI drivers in the same application• SQL cannot always express what DLI offers


OpenStandards


Solution highlights – Java dependent region deployment

� Java dependent region resource adapter– Java framework for applications operating in an IMS container

• Message queue processing

• Program switching

Deferred and immediate

• Transaction demarcation

• GSAM support

• Additional IMS call support necessary for IMS transactions

INQY

INIT

LOG

Etc

– Shipped with type 2 Universal drivers


Open Database Environment

LPAR Az/OS

IMS DBCTL

IMS

PCDistributed

UniversalDB

Resource

Adapter

J

CA

1.5

T

C

P

I

P

WebSphere

S

O

A

P

Evolution

LPAR B

LPAR C

XCF

TCP/IP


OpenStandards

IMS Connect

T

C

P

I

P

SCI

SCI

O

D

B

A

ODBM


Open Database EnvironmentInnovative Improved

UsabilityOpen

Standards

Universal

DB Resource

Adapter

J

C

A

1.5

T

C

P

I

P

WAS on Any Platform

S

O

A

P

LPAR Az/OS

Distributed

LPAR C

IMS DBCTL

IMS

PCSCI

O

D

B

A

IMS 11 Solutions

DLI

TCP/IP

JDBCTCP/IP

RYO DRDA Appl.

IMS DBCTL

IMS

PCSCI

O

D

B

A

IMS Universal Drivers

LPAR A

IMS DBCTL

IMS

PCSCI

O

D

B

A

ODBM

IMS Connect

T

C

P

I

P

SCI

ODBM

ODBM

LPAR B

XCF

z/OS WAS Universal

DB

Resource

Adapter

J

C

A

1.5

T

C

P

I

P

OD

BA

PCSCI

Traditional ODBA

DLI

JDBC

IMS Universal DriversTraditional ODBA

JEE

Non-JEE


IMS Open Database – Performance Considerations

The JDBC and SQL

layer only adds

approximately 5%

overhead

4000 TPS

vs.

3800 TPS

Same language,

Same platform,

Different interface

C) DIST JAVA DLI

vs.

DIST JAVA SQL

System Z was

approximately 2 times

faster

4000 TPS

vs.

7892 TPS

Same call,

Same language,

Different platform

B) DIST JAVA DLI

vs.

SysZ JAVA DLI

Both COBOL & Java

meets business

requirements.

7892 TPS

vs.

9980 TPS

Same call,

Same platform,

Different language

A) SysZ Java DLI

vs.

SysZ COBOL DLI

Notes IBM SVL LabDescriptionScenario


Agenda







IMS XML Database

� View/map native IMS hierarchical data as XML documents

� Allows the retrieval and storage of IMS Records as XML documents with no

change to existing IMS databases

� XML Evolution over XML Revolution

– Run XML transactions concurrently with traditional workload

XML Documents

IMS

Data

book

@year

title

seq

pricepublisherchoice

author

last first

seq

editor

last first

seq

affiliation

xs:date

xs:string

xs:string xs:string xs:string xs:string xs:string

xs:string xs:decimal

XML Schema

TITLE PUBLISH

FIRSTLAST FIRST

0:oo0:oo

AUTH EDIT

BOOK

YEAR PRICE

LAST AFFIL

PCB: BIB21

IMS DBD


IMS XML-DB Metadata

� “Natural” mapping between hierarchic XML data and hierarchic IMS

database definitions.

PSB

DBD

IMS DB definition

<?xml version="1.0" encoding="UTF-8"?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:ims="http://www.ibm.com/ims"

xmlns="http://www.ibm.com/ims/PSBName/PCBName"

targetNamespace="http://www.ibm.com/ims/PSBName/PCBName"

elementFormDefault="qualified">

<xsd:annotation>

<xsd:appinfo>

<ims:DLI mode="store" PSB="AUTPSB11" PCB="AUTOLPCB"

dsg="DATASETG" meanLength="1000" numDocs="100"/>

</xsd:appinfo>

</xsd:annotation>

<xsd:element name=“A”>

<xsd:complexType>

<xsd:sequence>

<xsd:element name=“field1" type="xsd:int"/>

<xsd:element name=“field2">

<xsd:simpleType>

<xsd:restriction base="xsd:string">

<xsd:maxLength value="30"/>

</xsd:restriction>

…

XML Schema

mapping

XML view of IMS

data

XML document definition


Agenda




� XML DB (IMS v9)

� Integration


Integration

� IBM Mashup Center

– Web 2.0, Data as a service

� Data Source Explorer– IMS connectivity via Generic JDBC

– Database exploration

– View/Edit data

� Data Project Explorer– SQL scripting

• Graphical

• Content assist

� Cognos– Operational business intelligence and reporting

– IMS Connectivity via Generic JDBC

Offered by the

IMS Explorer


IBM Mashup Center - Data as a service

� Web 2.0 support for IMS data

– Develop an IMS RESTful service/feed

• Generate meta files

from IMS DB

• IMS Web 2.0 Editor in InfoSphere MashupHub


Web 2.0 support for IMS Data (Mashup Center)

� Enterprise Database Plugin– Allows users to create feeds from

any Database that supports a JDBC driver

� Supports the following databases:– DB2 – Derby– IDS– IMS

– Microsoft SQL– MySQL– Oracle



� Features:

– Support both managed and

unmanaged connections

– Can store connection profile

information for reuse



� Supports generating SQL queries via a GUI interface


Operational Business Intelligence and reporting

� Type of BI that helps drive and optimize business operations on a daily basis

– Extends the use of BI to a much wider user audience

– Enables more timely business decisions; in relation to real time or right-time BI

Processing

– Allows business users to report on, analyze and optimize business operations

– Reduces the time between the discovery of problems or opportunities and taking action

on them

� Main pillars of BI are querying, reporting, OLAP, and business analytics


The Three Levels of Business Intelligence

© Claudia Imhoff


Reporting and OLAP ■ Enterprise Reporting

• Supports multiple report types: Production, Managed, Ad-hoc, Financial, etc

• Operates from a single metadata layer

• Can be personalized and targeted

• Can be distributed via email, portal, MS-Office, search application and mobile device

• Can assist in the management and optimization of daily business operations

• Capable of reporting on real-time data

• Presents complex data in a business way so it is easy to understand

■ Analysis

• Enables the guided exploration of information that pertains to all dimensions of your business

• Moves from summary level to detail levels of information effortlessly

• Analyzes and reports against online analytical processing (OLAP) and dimensionally aware relational sources

• Gets to the “why” behind an event or action to improve business performance.

• Identifies potential problems and recognizes business trends


Cognos and IMS

� Ability to leverage IMS assets within the Cognos product family can be a tremendous

competitive advantage for any organization

� IMS 11 Open Database can enable this integration

– Removes inhibitors

• Prior to IMS 11, no built-in ad hoc query capabilities in IMS

• Standard tools often did not support hierarchical data; left to write own interfaces and

reports or had to purchase bridging tools

• Must be skilled in IMS to integrate with IMS and write reports


IMS Value

� IMS is Efficient: 22K trans/sec on a single eight way z9. Remarkable performance that translates to the most cost efficient run-time environment. Reusing IMS transactions and data saves money.

� IMS enables Growth: IMS SOA provides Open, standard interfaces allowing ‘any-to-any’connectivity and access for new applications.

� IMS supports Growth: with IMS scalability you can just keep on growing. Customers have routinely handled peaks of 100 million transactions in a day.

� IMS is Resilient: Customers report going thousands of days without an unplanned outage. One bank reports availability of 99.998% over 10 years, 1.5 hours for planned outages.

� IMS and z/OS are secure

� Most efficient run-time, maintains application investment protection

� Fully integrated into today’s AD toolsets

� Fully integrates with WebSphere Application Server/ ESB / Process Server

� Fully supports SOA

� Improved efficiency and reduced skill requirements for systems and development staff

� Flexibility: JAVA, COBOL, C, PL/I, XML, Web 2.0…

� Continuous improvement to Rock solid availability

� Continuous improvement to operations and serviceability tools

� Fully supported by IMS & Tivoli tools

ResilienceResilience

GrowthGrowth

EfficiencyEfficiency

IMS “Value Proposition” 40+ years of Continuous Core Systems ImprovementIMS “Value Proposition” 40+ years of Continuous Core Systems Improvement


The Message

� IMS continues to be a premier server with architected standard interfaces

– New products and tools from a variety of vendors provide access to IMS transactions

and data

� SOA is revolutionizing the way businesses are being designed and run. For it to make

sense:

– All assets must be easily accessible in a standard way

– All data must be represented and manipulated in a standard way

� Our goal is to leverage IMS Database Manager as an integral part of the enterprise in the

evolving business world through

– Addition of support for complimentary standards surrounding IMS data connectivity,

data representation, and application development

� And to allow you to realize the promises of building a Service Oriented Architecture:

– Simplify the business environment

– Respond to market changes more quickly and cheaply


Technology

2011 06 evolution of the ims database_nyc_toronto_ug