Information Management System z - IBM Management System z 2008 ... DB2 9 for zOS and DB2 tools Seminar ... DB2 Optimization Expert, DB2 Optim solution, IBM Dataquant,

© 2008 IBM Corporation

Information Management System z 2008

Information ManagementSystem z

Louis HollaInformation Management zSeries Software

2

Information Management 2008


Agenda

• Why Data Warehousing on System z

• Data Warehousing Market Directions

• Target Segments for Data Warehousing on System z

• What exists today for Data Warehousing on System z

• What is coming for Data Warehousing on System z

3



Understanding the terminology

� Business Intelligence (BI) and Data Warehousing (DW) are sometimes used interchangeably

– Typically BI includes end user tools for query, reporting, analysis,

dashboarding etc.

– Both concepts depend on each other

– BI almost always assumes a Warehouse (WH), Operational Data Store

(ODS) or Data Mart (DM) exists with timely, trusted information

– A DW depends on end user tools that turn data into information.

� Both terms (DW and BI) address desire for timely, accurate, available data delivered when, where and how the end users want it

4



What is Data Warehousing?

� Data Warehousing provides the underlying data storage facilities required to support any BI Solution.

� An Operational Data Store (ODS) is a subject oriented database organized by business area. It is up to date (vs. historical) and detailed (vs. summarized).

� A Data Warehouse (DW or sometimes WH) is a multi-subject oriented database populated from operational systems and/or ODS’s. It is historical (vs. point-in-time) in nature and typically contains detailed data. It is often looked upon as the single source of corporate “truth”.

� A Data Mart (DM) is a database designed to support the analysis of a particular business subject area. Data has usually been transformed and aggregated from the source DW or operational system. Data Marts can be relational, multidimensional (OLAP) or statistical in nature.

5



How customers use WH/BI to manage their business better

� Financial intelligence and business performance management– The ability to provide accurate information to shareholders and regulators

– The ability to monitor and correct business performance as quickly as possible

� Marketing intelligence– Insights about their customers

– A single view of their customers

– Market and competitive intelligence

– Information on the profitability of their customers

� Operations intelligence– The ability to monitor business activities

– Insights into their suppliers

– A single view of their suppliers

– Insights on their production processes that enable greater efficiencies

� Human resources intelligence

– Insights into their workforce

� IT intelligence– The ability to analyze information systems and performance

6



Why is Warehousing important to IBM – and to you

� Warehousing/Business Intelligence/Analytics is a high growth(6-8%), lightly penetrated opportunity

– Estimated 85 percent of users who need some form of reporting/analytic capability do NOT have it yet

� GTS survey results:– Information Management is largest and fastest growing middleware services area

– Business Intelligence Services was #2 priority for surveyed customers (Information Management was #1) – 35% rated highly important/top budget item

� A top priority for Data Services Segment– Grow share among SAP clients– Grow volume of Dynamic Warehousing solutions

– Extend Data Warehousing value to Information as a Service– Grow Dynamic Warehousing reach and share via channel partners

� A top priority for STG: Focus on new workloads– Business Intelligence

– Mission critical enterprise management”,

– Business integration

7



Why Talk about Data Warehousing on z/OS in 2007

� System z customers are asking for increasing support and capabilities for DWH on z– Protects their investment in System z

– Keep up or lead in BI and DWH technologies

– Significant amount of enterprise data is on System z

� Many z customers have a DWH on DB2 z/OS (?)

� System z can capture DW consolidation and modernization opportunities

� Demand for real-time access to operational data for BI analytics is growing

� The BI operational needs for reliability, availability and security are growing

– The z/OS platform is a secured system second to none.

� Significant DW capabilities have been added to DB2 V8 with even more DB2 9

� The new zIIP processor provides TCO and IT optimization

– Particularly useful for BI queries (star schema) and parallel query processing

� Newly acquired data integration and analytic components combined with partner

capabilities make best of breed integrated solution.

8



A Single View of the Enterprise Drives the requirement for Data Warehousing

� Data volumes rise� Amount of historical

Information grows� Number of users

grows � Questions are more

numerous and sophisticated

� Schema complexity grows

� Workload mixturechanges

� Development time shrinks

9



DW Compliance � System z Supports Compliance

– Single version of the truth

– Data confined to one location regardless of where the application is running

– Single process to manage compliance

– Fewer people to manage compliance process

– DB2 Audit Management Expert

– Tivoli Compliance Insight Manager

– Single place for auditors to look for both operational and warehouse data

– Industry specific data models assist DW compliance

10



Factors that Influence Data Warehouse on System z Platform Selection

� Existing infrastructure� Platform preferences and skills in-house� Budget/Cost

� Server Consolidation� Market trends in warehousing solutions and vendors� Complexity

� Competitive influences� Mergers and acquisitions or consolidations� Data or application sources� Business processes required and business drivers � Reliability, Extremely High Availability, Security

11



When is System z the Preferred Platform?

�Need true real-time Operational Data Store (ODS)– Operational data is on System z

– Data must virtually be in sync with the operational data

– Availability, Security and Resiliency needs are high

– Meet auditable data warehouse requirements

�Specialty engine allows for IT optimization– zIIPs, zAAPs, IFLs.

�Want to leverage & optimize existing z skills and investment– Can balance workload based on business policies/needs (WLM), separate OLTP and DW

workloads

� Consolidation of distributed marts or DW to anexisting System z Data Serving platform

And when NOT………..

12



Commercial

IBM - ForumBourgetlaan, 42

B-1130 Brussel

Phone : +32 2 655 54 40

Wednesday April 23th, 2008:DB2 9 for zOS and DB2 tools Seminar

DB2 9 for zOS and tools workshopThis one-day free seminar provides a unique opportunity for technical professionals,

architects and managers to gain insight into DB2 9 for z/OS and optimizing its use by getting

the latest information on the IBM DB2 tools. We also provide you with the latest industry

trends and hot topics like Information on demand, Information integration and Data

governance on System Z.

You'll hear a brief overview of the IBM's Information as a Service initiative, learn about the

latest IBM acquisitions (Princeton Softech, Datamirror and Cognos) and see some upcoming

releases of key DB2 tools (DB2 9 for z/OS Value Unit Edition, IBM Data Studio, IBM

Information Server for System Z, DB2 Optimization Expert, DB2 Optim solution, IBM

Dataquant, DB2 Audit Management Expert, DB2 Recovery Expert, DB2 Cloning tool, DB2

Accessories Suite).

You can register on :https://www-926.ibm.com/events/wwe/swgp/swgpbe.nsf/agenda?openform&seminar=kwo98DJNWES&lang=be

13



Market Directions

14



Dynamic WarehousingA New Approach to Leveraging Information

Dynamic Warehousing

Traditional Data Warehousing

OLAP & Data Mining

to Understand Why and

Recommend Future Action

Query & Reporting

to Understand

What Happened

Information On Demand

to Optimize Real-Time

Processes

15



Warehousing strategic pillarsGuiding principles for innovation

Extended InsightBeyond traditional capabilitiesFurther leverage informationExtended business insight

Support broader usage

SimplicityEasy to deploy and integrate

Easy to useEasy to manage

Easy to start and grow as needed

Reliability & PerformanceReliable access to information

Highly availableReal-time performance

Maximized resource efficiency

16



IBM DB2 Warehouse software A complete, integrated platform

Mo

deli

ng

an

d d

esig

n

Ad

min

istr

ati

on

an

d c

on

tro

l

Data movement and transformation

Database management

Performance optimization

Workload

control

Data

partitioning

Deep

compression

Embedded analytics

Data mining and

visualization

In-line

analytics

IBM DB2 Warehouse

17



Recent IBM Investments Further Contributing to Adoption of System z for New Workloads [WIP]

Application-Driven Projects Information-Driven Projects

Application Development and Delivery Infrastructure Incorporating System z

Enterprise IT Management Infrastructure Integrating System z

� DB2 VUE provides cost effective solution for

new workload implementations through an OTC

pricing model

� DB2 Warehouse offering for critical

performance for dynamic warehousing and

OLTP requirements

� Information Server for System z enables

System z to be the trusted information hub

� DB2 9 optimized for SAP

� IMS 10 for critical on-line operational app’s

� Content Manager v8.4 and Content Manager On

Demand v8.4 support a strategic ECM platform

� Web 2.0 apps enabled using WS Portal

� Workload consolidation benefits through

common infrastructures (e.g., WebSphere

Application Server)

� Rapid time to market & low business risk with

CICS Service Flow Feature & WebSphere

Adapters

� Ability to integrate existing & new assets via

WebSphere MQ messaging backbone and

federated ESBs

� Faster business innovation built through

WebSphere Process Server

� Asset Management

� Architecture Management� Skills

Management� Quality

Management� Change and Release

Management

• Interdependent Applications • Linux and SOA • Composite Applications

• Core Applications • Financial and Service • Storage• Federated Identity • Workload and Availability

• Distributed Environments

18



The Information Challenge

Information Challenges

Accuracy

Timeliness

Relevance

Accessibility

Security

. . .

Business Challenges

Globalization

M&As

Supply chain

Risk & compliance

Customer loyalty

Operational costs

Business velocity

. . .

People can spend up to 70% of their time looking for

information

60%+ of CEOs: Need to do a better job leveraging

information

5X more value creation by organizations using

information effectively

Sources: IBM Attributes & Capabilities Study, 2005; Client Interviews 2004; IBM CFO Study, 2006

INFORMATION

IS A

STRATEGIC

ASSET

INFORMATION

IS A

STRATEGIC

ASSET

19



Addressing Key

Customer Requirements

� Workload consolidation

� Integrating data via SOA

� Reduced total cost of

ownership

� Next generation warehousing

IT Projects Driving New Business Initiatives

People, Processes, Applications

Trusted InformationIn Context

Data Governance & Accountability

• Automated archiving to manage continuous data growth

SAP Information Infrastructure

Dynamic Data Warehousing• Co-located OLTP & DW • Move to System z decreased reporting times 80%

Master Data Management

• Significant IT savings• Improved customer service• Business growth

SOA Data Serving• Exposed legacy IMS through Web services

• Guaranteed integrity on over 4 billion IMS transactions per month

• consolidated customer profile to enable customer centric processing for sales and service

20



Information On Demand and SOATwo sides of the same coin…

SOA provides business flexibility

IOD provides the trusted information needed by your services

“You will waste your investment in SOA unless you have enterprise

information that SOA can exploit.”– Gartner Research

21



Gartner perspective on BI and DWH market

System z holds a viable place in the Leaders quadrant

The emergence of issues based on a mix of 4 DW workloads

• Continuous (near-real-time) data loading – similar to an OLTP workload

• Large numbers of standard reports • An increasing number of true ad hoc query users • An increasing level of analytics and BI-oriented functionality in OLTP

The mixed workload performance will become the single most important performance issue in DW

The transactional DBMSs have an edge that challenges the DW DBMSs (such as Teradata) to stay ahead

22



IBM System z Data Servers

Trusted Platform for SOA

� IMS 10 XQuery and Web services enhancements ease data service delivery

�DB2 9 pureXML for seamless and efficient integration of XML & relational data

Industry’s Highest Level of Availability & Scalability

�Unmatched availability for “utility class” information access

�Rapid and non-disruptive scalability for variable mission critical requirements

Unmatched Risk and Compliance Management

�New security capabilities for greater control and business flexibility

� Improved auditing and accountability

Lowest Operating Cost

�Reduced operating costs through z/OS and zIIP engine exploitation

�Superior environmental efficiency – “the green machine” power, cooling, and space

23



Commercial

24



What Exists Today for Data Warehousing on System z

25



ODSs, Staging, detail, denormalized, Raw Source

Duration: 60, 120, 180, etc days

Near 3rd Normal Form, Subject Area, code and

Reference tables. Duration: Years

Summarized DataPerformance and rolled up Data

Duration: Years

Star Schema, Data Mart, Cubes, Duration: Years

Dashboard

Static reports,

Fixed Periods

Floor 1

Floor 5

Floor 4

Floor 3

Floor 2

Near RealTime, MQ, LOAD, multi-row insert…

Mining, scale, growth, Merge…

ELT, MQTs, SQL Function…

Logical DM using Views for flexibility wherever possible, otherwise Physical DM

Alphablox, DataQuant, …

Enterprise Data Warehouse using layered data architecture

26



Solution Architecture

Member A

OLTP

CEC One

Member B

OLTP

CEC Two

DB2 Data

SharingGroup

CFCF

CEC One CEC TwoCFCF

DB2 Data

Sharing

Group

Member C

DWH

Member D

DWH

Within a data sharing environment, the data warehouse resides in the same group as the transactional data.

27



Initial load of the Data Warehouse

Member A

OLTP

Member B

OLTP

CEC One CEC TwoCFCF

Member C

DWH

Member D

DWH

ETL Accelerator

pSeries

xSeries

zLinux

ETL is done, by using an ETL Accelerator which

in it’s current implementation is represented by

DataStage (IBM Information Server) on a

secondary system. Until end of 2007, pSeries

and xSeries are the only supported systems.

Later on the ETL Accelerator will move to

zLinux as soon as the Information Server is

available there.

Extract

Load

Extract

JDBC/ODBC

Software AG

Adabas

VSAM,

IAM &

sequential

CA

IDMS

CA

Datacom

IMS

Classic Federation Server

z/OS

DB2 UDB

for LUWOracle SQL Server

Distributed Data Sources- Integration -

Extract

JDBC/ODBC

The data is extracted from the OLTP information of the data sharing group, transformed by DataStage and then loaded into the

data warehouse tables again.

Legacy data sources are integrated through the Classic

Federation Server. This way data

is extracted from DataStage

directly out of the data sources

like IMS, VSAM…

Distributed data sources are directly integrated

through the IBM

Information server.

Da

taS

tage

Pa

ralle

l E

ngin

eD

ata

Sta

ge

Pa

ralle

l E

ngin

e

28



At runtime, the Data Warehouse is updated incrementally

Member B

OLTP

CEC One CEC TwoCFCF

Member C

DWH

Member D

DWH

ETL Accelerator

pSeries

xSeries

zLinux

DB2

VSAMSoftware AG

Adabas

CaptureIMS

Data Event Publisher

CA-

IDMS

z/OS

DB2 UDB

for LUWOracle SQL Server

Data Event Publisher

Capture

Distributed

MQ

MQ

MQ

Update & Insert

Member A

OLTP

Chan

ge

Cap

ture

29



In Database ELT is triggered by DataStage

Member A

OLTP

Member B

OLTP

CEC One CEC TwoCFCF

Member C

DWH

Member D

DWH

ETL Accelerator

pSeries

xSeries

zLinux

Simple example:

-- Aggregate by salary by department into AGGRSALARY

INSERT INTO AGGRSALARY ( DEPTCODE, AVGBAND, AVGSALARY )

SELECT DEPTCODE, AVG( BAND ) AS AVGBAND, AVG( SALARY ) AS AVGSALARY

FROM STAFF

GROUP BY DEPTCODE

Wherever possible, “In Database” transformations (ELT) are used to

spare the transport of the data to the accelerator. But the used SQL is still

sent from the ETL Accelerator to the database to have one place of

documentation for all ETL steps.

This can also be used to shift the data up the hierarchy within the Layered

Data Architecture.

SQLELT

30



What DW Capabilities Exist Today for System z

�DB2 z/OS V8 & V9 DW Base

–Functional and performance enhancements

–Easier online reporting and data management capabilities

–DB2 family compatibility

�DW ETL

–WS DataStage EE, WS Classic Federation Server, WS Data Event Publisher, WS Classic Event Publishers, Distributed DBMS Event Publishers, BatchPipes for OS/390, DB2 Unload/Load Utilities, DataMirror Transformation Server

�Analyze/Report

–From IBM: Alphablox, QMF and DataQuant, Cognos

– From partners: Hyperion, Business Objects, SAS, IBI

�Performance Management

–IBM Tivoli Omegamon XE for DB2 Performance Expert on z/OS, DB2 Query Monitor, Optimization Service Center (V9)

�Other IBM Information Server Components

–WS Information Analyzer, WS QualityStage

31



DB2 for z/OS Enhancements for Data Warehousing� DB2 Version 8

– Usability, Availability & Scalability

– Online Partitioning Changes, More Partitions,

Rotate

– Schema Evolution:

– VSCR with System z 64-bit Architecture

– Data Warehousing Support

– Star Join Improvements

– Materialized Query Tables

– Overall System & Query Performance

– Locking Improvements

– Multi-row INSERT & FETCH

– More index Access for VARCHAR, data type mismatch, …

– DDF Performance Improvements

– zIIP for improved cost of ownership

� DB2 Version 9– Usability, Availability & Scalability

– Universal & Partition by Growth Table

Space

– Clone Tables (Online Load Replace)

– XML

– Data Warehousing Support

– Star Join Improvements (Dynamic Index ANDing)

– SQL Enhancements (RANK, DENSE_RANK, Text, XML, INTERSECT, …)

– Overall System & Query Performance

– TRUNCATE Statement for fast delete, MERGE for insert or update

– Index Compression

– Significant CPU Reduction for DB2 Utilities & many queries

– Insert and update scaling

32



DB2 9 for z/OS Update:

� Simplification, Reduced TCO

– Index Compression, Partition By Growth & Cloned tables

� Workload Consolidation– Trusted context and ROLES, Online REBUILD INDEX

� SOA Enablement– pureXML, Optimistic locking for WebSphere

� Dynamic Warehousing– Optimization Service Center, New built-n OLAP expressions

Rapid Adoption Already!

33



Specialty engines work together

z/OS LPAR

DWApplicationData

Warehouse Application

not on System z

TCP/IP

The IBM System z9 specialty

engines can run independently or complement each

other

(shown at left, instances where specialty engines can be employed)

Linux on System z LPARDW Application

IFL

TCP/IPHiperSockets

DRDA

DB2

CP

z/OS LPARWAS JAVA DW Application zAAP

TCP/IPHiperSockets

WASApplication

zAAP

zIIP

DRDA

DRDA

StarSchema// query

34



System Z Green Credentials

� Environmental Issues are becoming a concern

– Power

– Proliferation of servers increase demand on the external infrastructure

– Cooling

– Heat generated by increase number of servers has to be considered

– Space

– Will the expanding number of servers fit into the available space

35



Commercial

� Customer visit of Guogen (Gene)

Zhang,

Database Technology Institute for

z/OS, IBM Silicon Valley Lab

36



What is DataQuant

� DataQuant provides a comprehensive query, reporting and data visualization platform for both web and workstation-based

environments.

� DataQuant introduces a wide variety of powerful business intelligence capabilities, from executive dashboards and interactive visual

applications to information-rich graphical reports and ad-hoc querying

and data analysis

.� DataQuant provides 2 components

– DataQuant for Workstation – An Eclipse-based environment for the development of query, report and dashboard solutions

– DataQuant for WebSphere – A runtime environment capable of displaying DataQuant content using a “thin client” model

37



IBM DATA QUANT FOR Z/OS� Adds compelling new Warehouse/Business Intelligence component to WH on z� Visual Dashboards, Enhanced Graphical Reporting, Security and

Personalization, SOA Layer, Enhanced Analytics� Offers a “thick” client with DataQuant for Workstation, or a pure HTML, browser

based client with DataQuant for WebSphere

38



But wait…

There’s more…

39



40



FIREWALL/ENCRYPTION

FIREWALL/ENCRYPTION

Cognos 8 BI

Portal

Ad Hoc Query

Deep Comparative Analysis

Event Lifecycle Management

Dashboarding & Scorecarding

All Administration

CognosOffice Connection

ROUTER/FIREWALL

WEB SERVERS:

Web Gateways, Web Services API

Industry Standard Portals

Cognos Application Firewall

HTTP Server

WebSphere Portal Workplace

QUERY

SERVICE

JOB SCHEDULING

SERVICE

PRESENT-ATION

SERVICE

CONTENTMANAGER

SERVICE

AUDITSERVICE

METRICSSERVICE

EVENTSERVICE

MONITORINGSERVICE

App. Server

Studio

eMailDistribution

DevelopmentTools

System MgmtJob Scheduling

System x (Win/Linux)

System p (AIX/Linux)

Rose

WebSphere Data/MetaStage

DB2 CubeViews

All RDBMS & OLAP sources

WebSphere Information Integrator (Classic, Content, UIMA)

Reporting & Analysis

Data Warehouse

CONTENT & METRICS STORE

DB2 UDB

SECURITY PROVIDERSLDAP, Active Directory, NTLM, Netegrity, SAPBW, Cognos Namespace, Custom Providers

Directory ServerAccess Manager

AUDITRDBMS, UNIX, Sys, Logs,NT Event Log

System Monitoring

DB2 UDB

SQL MDX COMMON METADATA

Cognos 8 BI for Information Management

Business Analysis

Business Metrics

Business Performance Management

Design

Implementation

Optimization

Outsourcing – Insurance Industry

& IBM Software

41



42



IBM Cognos 8 Business Intelligence (BI) for System z

� This will be an entirely new offering for the System z platform with a

target availability of the second half of 2008.

� Cognos 8 BI for System z combines the targeted information delivery and analysis capabilities of Cognos 8 BI with the power and

reliability of System z to:

– Enable organizations to report and analyze hundred of millions of transactions directly on the mainframe.

– Ensure everyone across the organization can quickly identify and respond to critical business trends.

– Provide a competitive advantage through improved decision making from business data.

43



DB2 9 for z/OS Value Unit Edition (VUE)New Workload Offering available OTC

� Drive Net New Workload Growth in new and existing DB2 for z/OS customers by

– Offering customers DB2 9 as OTC for qualified net new workloads on z/OS

– Net new workloads limited to commercial (packaged) applications, data

warehouses, and WebSphere. Applications running on a measurable

standalone machine or LPAR.

– Customers can now add new applications and workload to System z without

affecting their MLC cost.

� Strengthens System z role as a cornerstone for new applications and projects such as SOA, data warehousing, operational BI, ERP, and more

Steve Mink

44



IBM Information Server for System zBenefits of this architecture

– Greater Efficiency Leveraging System z

– z/OS MIPs consumption dramatically reduced vs. USS or MVS approaches

– Minimizes impact on other z/OS software costs

– All Job Processing is on zLinux (except the z/OS data access)

– DB2 workload on z/OS can qualify for offload to ZIIP specialty engines

– High performance z data connectivity

– Batch Pipes for DB2 load, DRDA to DB2 over hipersockets

– SQL to Classic over hipersockets

– Integration with MQ and therefore with the Data Event Publishers

– Seamless integration with other IBM Information Server platforms

– Same operational architecture and metadata Repository

– Eliminates deployment issues

– Maintains value of DataStage for z/OS investments

45



What else is coming for DW on z?

� SHORT TERM– DB2 9

– Even more DB2 compatibility– Warehouse, reporting & optimizer enhancements

– BCU Methodology (Sizing Tool, Best Practices)

– InfoSphere Master Data Management Server for Linux on System z

� MEDIUM TERM – 1 to 1 1/2 YEARS– Shark

– Large table scans pushed down to Shark– Better price/performance

– Additional Support for DW on z

– MQT Advisor

– Cube Views support– Common Web-based Admin Console– Alphablox on Linux on z

� LONGER TERM– Additional capabilities supporting DW on z

– Data Mining– DWE Design Studio IDE

46



DW on System z Support Initiatives

� BI Sizing Tool for System z

– Target availability via Techline early 2H07

– Input: Txn rates for 5 different txn profiles & DW raw data size

– Output: CPU requirements (including zIIP estimate) plus guidelines for memory and storage

� DW on System z Best Practices Guidance

– Target V1 end of 2Q07 (will be track on IOD 2008)

– Recommended Architecture & Components

– DB2 Configuration & Tuning Guidance

47



48



49



And drive carefully out there...

Documents

Information Management System z - IBM Management System z 2008 ... DB2 9 for zOS and DB2 tools Seminar ... DB2 Optimization Expert, DB2 Optim solution, IBM Dataquant,