DB2 for z/OS: Version 9 and Beyond

© 2009 IBM Corporation

DB2 for z/OS: Version 9 and Beyond

Gareth JonesIBM DB2 for z/OS [email protected]

2


Important Disclaimer

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.

WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR

• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE.

3


Agenda

DB2 9 highlights and news

DB2 X

4


DB2 9 Highlights and News

5


• Index compression• Partition By Growth tables• Plan stability• Volume based backup / recovery• Automatic object creation

Simplification, Reduced TCO

• More online schema changes • Online REBUILD INDEX, Online REORG

improvements• Cloned tables• Trusted context and ROLEs• Parallel Sysplex clustering improvements• 64-bit virtual storage improvements

RAS, Performance,

Scalability, Security

• pureXML• Optimistic locking for WebSphere• LOB performance, usability• Native SQL procedure language• SQL improvements that simplify porting

Application Enablement

• Many SQL improvements• Dynamic index ANDing• Histogram statistics• New built-in OLAP expressions• Optimization Service Center

Dynamic Warehousing

DB2 9 for z/OS: Addressing Corporate Data Goals

DB2 9 for z/OS delivers on more than

225 requirements submitted by customers, business

partners, and worldwide user group

communities

6


64-bit Evolution and Buffer Pool Enhancements

7


64 bit Evolution (Virtual Storage Relief)

EDM Pool (EDMPOOL)

DBD Pool (EDMDBDC)

Global Stmt Pool (EDMSTMTC)

2GB

Skeleton Pool (EDM_SKELETON_POOL)

CT / PTOthers

CT / PTSKCT / SKPT

IFCID 217: detailed DBM1 virtual storage health

IFCID 225: consolidated DBM1 virtual storage health

STMT CACHE BLOCKS: 100% moves above 2GB

– Can be up to 100m or more.

SQL STATEMENT CACHE: approx. 60% moves above 2GB

– Statement above bar (split).

EDM SKCT/SKPT – 100% moves above 2GB

– This can be 100-200m for primarily static

Other areas move above 2GB, examples:– Parse tree– Control blocks for pagesets and RTS – DMTR

Virtual Storage Constraint is still an important issue for many DB2 customers. The following changes provide some relief:

8


64-bit DDF – Shared Private with DBM1DDF address space runs in 64-bit addressing mode– Shared 64-bit memory object avoids xmem moves between DBM1 and

DDF and improves performance– VSCR

DBM1 DIST

z/OS Shared Private Storage

2GBBar

Shared memory: new virtual storage type allowing multiple address spaces to share storage.Similar to ECSA – always addressable, avoids AR and XM moves.Different from ECSA – only available to those address spaces registering with z/OS to share this storage.Reduces data formatting and data movementReduces virtual storage– It exists once, instead of in

each address space

9


WLM Buffer Pool ManagementWLM-assisted buffer pool management– -ALTER BUFFERPOOL ()

AUTOSIZE(YES)– z/OS 1.8

– DB2 registers BP to WLM and reports synch read I/O delays to WLM

– DB2 periodically reports BP hit stats to WLM

– WLM projects effect of adjusting BP size on workload performance goals • Takes into account overall system

storage usage– WLM drives DB2 exit to adjust size

if appropriate• DB2 9 restricts to +/- 25%

BP1800MB

DB2 WLM

600MB

Min

1GBMax

800MB

Current

BP1Name

DSNB555I WLM RECOMMENDATION TO ADJUST SIZE FOR BUFFER POOL bpname HAS COMPLETEDOLD SIZE = csize BUFFERSNEW SIZE = nsize BUFFERS

PK75626 will be required to enable automatic WLM buffer pool management.!

10


SQL Enhancements

11


SQL: Productivity, DB2 family & porting

XML MERGE & TRUNCATESELECT FROM UPDATE, DELETE, MERGEINSTEAD OF TRIGGERBIGINT, VARBINARY, BINARY, DECIMAL FLOATNative SQL Procedure LanguageNested compoundOptimistic locking

LOB File reference variable & FETCH CONTINUEFETCH FIRST & ORDER BY in subselect and fullselectINTERSECT & EXCEPTROLE & trusted contextMany new built-in functions, caseless comparisonsIndex on expressionImproved DDL consistency CURRENT SCHEMA

12


INTERSECT/EXCEPT

INTERSECT UNION

R1 R2

EXCEPT

R1 R2 R1 R2

subselect

subselectUNIONEXCEPTINTERSECT

(fullselect) DISTINCT

ALL (fullselect)

13


INSTEAD OF TRIGGERSCREATE TABLE WEATHER (CITY VARCHAR(25), TEMPF DECIMAL(5,2))CREATE VIEW CELCIUS_WEATHER (CITY, TEMPC) AS

SELECT CITY, (TEMPF-32)*5.00/9.00 FROM WEATHER

CREATE TRIGGER CW_INSERT INSTEAD OF INSERT ON CELCIUS_WEATHER

REFERENCING NEW AS NEWCW DEFAULTS NULLFOR EACH ROW MODE DB2SQL

INSERT INTO WEATHER VALUES (NEWCW.CITY, 9.00/5.00*NEWCW.TEMPC+32)

CREATE TRIGGER CW_UPDATE INSTEAD OF UPDATE ON CELCIUS_WEATHER

REFERENCING NEW AS NEWCW OLD AS OLDCW DEFAULTS NULLFOR EACH ROW MODE DB2SQL

UPDATE WEATHER AS WSET W.CITY = NEWCW.CITY,

W.TEMPF = 9.00/5.00*NEWCW.TEMPC+32WHERE W.CITY = OLDCW.CITY

14


MERGE“Upsert”– A combination of insert and update, MERGE is a single SQL operation

– Single row or multi-row

If a row matches the ON condition it is updated,

Otherwise it is inserted.

MERGE INTO account AS TUSING VALUES (:hv_id, :hv_amt) FOR 5 ROWS AS S(id,amt)ON T.id = S.idWHEN MATCHED THEN

UPDATE SET balance = T.balance + S.amtWHEN NOT MATCHED THEN

INSERT (id, balance) VALUES (S.id, S.amt)NOT ATOMIC CONTINUE ON SQL EXCEPTION;

15


SELECT athlete, score FROM FINAL TABLE(UPDATE events SET SCORE=SCORE+10

WHERE EVENT=‘BOXING’);

SELECT MEMBER_ID, UPSERT_INDICATOR FROM FINAL TABLE(MERGE INTO MEMBER_PROFILE AS A

INCLUDE (UPSERT_INDICATOR CHAR(1))USING (VALUES (20, 'PAUL', 22) ) AS B

(MEMBER_ID, MEMBER_NAME, MEAL_PREFERENCE)ON (A.MEMBER_ID = B.MEMBER_ID)WHEN MATCHED THEN

UPDATE SET A.MEMBER_NAME = B.MEMBER_NAME,A.MEAL_PREFERENCE = B.MEAL_PREFERENCE,UPSERT_INDICATOR = 'U'

WHEN NOT MATCHED THEN INSERT (MEMBER_ID,MEMBER_NAME,MEAL_PREFERENCE,UPSERT_INDICATOR)VALUES (B.MEMBER_ID, B.MEMBER_NAME, B.MEAL_PREFERENCE, 'I')

NOT ATOMIC CONTINUE ON SQLEXCEPTION);

First example shows SELECT returning data from the nested UPDATESecond example shows SELECT returning data from the nested MERGEwith an INCLUDE column.– This include column returns the compare column from the ON clause and an

indicator of whether the row was Updated or Inserted.

SELECT FROM UPDATE/DELETE/MERGE

16


TRUNCATEAllows mass delete of all rows in base tables or declared global temporary tables – Simple, segmented, partitioned, universal table spaces.

– If table contains LOB or XML columns also truncates auxiliary table spaces.

– IMMEDIATE option – operation cannot be rolled back• Allows immediate reuse of allocated space for subsequent insert operations in the same UOW

without committing.

Deletes rows without firing DELETE triggers

Option to REUSE or DROP STORAGE

TRUNCATE table-nameDROP STORAGE

TABLE REUSE STORAGE

IGNORE DELETE TRIGGERS

RESTRICT WHEN DELETE TRIGGERS IMMEDIATE

17


New Data Types: DECFLOAT and BIGINTDECFLOAT– Well suited to typical customer financial calculations– Similar to “calculator” mathematics

• Eliminates rounding errors by using base 10 math• Has up to 34 digits of precision

– DECFLOAT(16)• 10+384 to 10-383 Positive & Negative

– DECFLOAT(32)• 10+6144 to 10-6143 Positive & Negative

– Floating point convenience with fixed point precision!

BIGINT– An exact numeric capable of representing 63-bit integers– -9223372036854775808 to 9223372036854775807

18


New Data Types: BINARY & VARBINARYBINARY fixed-length binary string– 1 to 255 bytes

VARBINARY variable-length binary string– 1 to 32704 bytes; maximum length determined by the

maximum record size associated with the table

Compatible with BLOBs

Not compatible with character string data types– Similar to FOR BIT DATA character strings– Can use CAST to change FOR BIT DATA character string into binary

string

Prior to DB2 9, FOR BIT DATA columns use character-based padding

19


Built-in Functions

OLAP– RANK– DENSE_RANK– ROW_NUMBER

Date and Time– TIMESTAMPADD– TIMESTAMP_ISO– TIMESTAMP_FORMAT– EXTRACT– MONTHS_BETWEEN

Sound representation– SOUNDEX– DIFFERENCE

String handling– ASCII_STR– EBCDIC_STR– UNICODE_STR– ASCII_CHR– EBCDIC_CHR– COLLATION_KEY– RPAD, LPAD

And More …

20


zIIP Enabled

for DRDA

Native SQL Procedural LanguageEliminates generated C code and compilation

Fully integrated into the DB2 engine

– An SQL procedure created without FENCED or EXTERNAL is a native SQL procedure

Appl pgm

CALL SP1

Appl pgm

CALL SP1

DB2DBM1

EDM pool

DDF

DB2 directory

SQL PL native logicSQLSQL

SP1

SQL PL native logicSQLSQL

SP1DRDA

21


DB2 SQL z z/OS V8commonluw Linux, Unix & Windows V8.2

Multi-row INSERT, FETCH & multi-row cursor UPDATE, Dynamic Scrollable Cursors, GET DIAGNOSTICS, Enhanced UNICODE for SQL, join across encoding schemes, IS NOT DISTINCT FROM, Session variables, range partitioning

Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scroll Cursors, UNION Everywhere, MIN/MAX Single Index Support, Self Referencing Updates with Subqueries, Sort Avoidance for ORDER BY, and Row Expressions, 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Tables, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, Call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, Client special registers, long SQL object names, SELECT from INSERT

Updateable UNION in Views, ORDER BY/FETCH FIRST in subselects & table expressions, GROUPING SETS, ROLLUP, CUBE, INSTEAD OF TRIGGER, EXCEPT, INTERSECT, 16 Built-in Functions, MERGE, Native SQL Procedure Language, SET CURRENT ISOLATION, BIGINT data type, file reference variables, SELECT FROM UPDATE or DELETE, multi-site join, MDC

z

luw

common

22


DB2 SQL z z/OS 9commonluw Linux, Unix & Windows 9

Multi-row INSERT, FETCH & multi-row cursor UPDATE, Dynamic Scrollable Cursors, GET DIAGNOSTICS, Enhanced UNICODE for SQL, join across encoding schemes, IS NOT DISTINCT FROM, Session variables, TRUNCATE, DECIMAL FLOAT, VARBINARY, optimistic locking, FETCH CONTINUE, ROLE, MERGE, SELECT from MERGE

Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scroll Cursors, UNION Everywhere, MIN/MAX Single Index Support, Self Referencing Updates with Subqueries, Sort Avoidance for ORDER BY, and Row Expressions, 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Tables, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, Call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, Client special registers, long SQL object names, SELECT from INSERT, UPDATE or DELETE, INSTEAD OF TRIGGER, Native SQL Procedure Language, BIGINT, file reference variables, XML, FETCH FIRST & ORDER BY in subselect and fullselect, caselesscomparisons, INTERSECT, EXCEPT, not logged tables, range partitioning, compression

Updateable UNION in Views, GROUPING SETS, ROLLUP, CUBE, 16 Built-in Functions, SET CURRENT ISOLATION, multi-site join, MERGE, MDC, XQuery

z

luw

common

23


DB2 SQL z z/OS 9commonluw Linux, Unix & Windows 9.5

Multi-row INSERT, FETCH & multi-row cursor UPDATE, Dynamic Scrollable Cursors, GET DIAGNOSTICS, Enhanced UNICODE for SQL, join across encoding schemes, IS NOT DISTINCT FROM, TRUNCATE, VARBINARY, FETCH CONTINUE, MERGE, SELECT from MERGE, index compression

Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scroll Cursors, UNION Everywhere, MIN/MAX Single Index, Self Referencing Updates with Subqueries, Sort Avoidance for ORDER BY, and Row Expressions, 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Tables, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, Call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, Client special registers, long SQL object names, SELECT from INSERT, UPDATE or DELETE, INSTEAD OF TRIGGER, Native SQL Procedure Language, BIGINT, file reference variables, XML, FETCH FIRST & ORDER BY in subselect & fullselect, caselesscomparisons, INTERSECT, EXCEPT, not logged tables, range partitions, data compression, session variables, DECIMAL FLOAT, optimistic locking, ROLE

Updateable UNION in Views, GROUPING SETS, ROLLUP, CUBE, more Built-in Functions, SET CURRENT ISOLATION, multi-site join, MERGE, MDC, XQuery, XML enhancements, array data type, global variables, Oracle syntax

z

luw

common

24


Performance, Performance, PerformanceSERVER

CLIENT

Data Management Client

Customer Client Application

SQL(X)

DB2 Server

XMLInterface

Interface

XMLStorage

RelationalStorage

Relational

XML Capabilities Inside the EnginepureXMLtm

Native storage Schema Index Functions Utilities

25


pureXML Support

SQL XML data type and native storage

Designed specifically for XML– Supports XML hierarchical structure storage

– Native operations and languages: XPath, SQL/XML, (XQuery in the future)

Not transforming into relational

Not using objects or nested tables

Not using LOBs

Integrated with relational engine, with all the utilities and tools support

26


What You Can Do with pureXML

Create tables with XML columns or alter table add XML columns

Insert XML data, optionally validated against schemas

Create indexes on XML data

Efficiently search XML data

Extract XML data

Decompose XML data into relational data

Construct XML documents from relational and XML data

All the utilities and tools support for XML

XMLDOC

XML Column

XMLIndex

XML

27


Optimization Evolution

28


Data Warehousing on z/OS – What is the driver?Customer commitment to the z platform

–Customers want to protect their significant investment in System z–TCO can be reduced through the utilization of existing processors, people, practices–TCO may also be achieved through a consolidation approach

New BI trends are changing the DBMS landscape–The distinction is blurring between warehouse and OLTP databases based on new

trends such as Dynamic Warehouse and Operational BI, driving: • The need for increased reliability, availability, security, and compliance in a DWH DBMS • The need for very current warehouse data, where proximity to the source provides an advantage

Many z customers already have a DWH on DB2 z/OS –This drives requirements into hardware and software, which in turn drives a trend–DB2 has responded with increased functionality and performance; hardware changes

are driving down costs

Specialty processors provide new ways to optimize TCO

–zIIPs and IFLs are driving down hardware and software costs; DWH/BI can make excellent use of these processors, ultimately driving TCO advantages

29


Dynamic Warehousing with System z Mission-critical analysis of operational dataRapid and secure user-access to data analysis

Interactive executive dashboards & information portalsOver 50 Warehousing features in V8 and DB2 9

V8 Materialized Query TablesV8 longer SQL statements, index keys, complex joinsV8 up to 4096 partitions in a single tableV8 and DB2 9 Improved SQL & optimizationDB2 9 Index compression added to data compressionDB2 9 Online Rebuild IndexDB2 9 Global Query OptimizationDB2 9 Dynamic index ANDing for improved star schema query supportDB2 9 Histogram Statistics

Cost optimization with parallel queries running on zIIP

30


Data Warehouse and BI Architecture for System z

Information Server for System z

DB2 for z/OS VUE

OLTPdata

Data Warehouse

Cognos 8 BI for System z

Enterprise BI

Complete ETL Solution

Core Offering for Enterprise Data Warehouse and BI:• Information Server for System z

A complete set of ETL tools for warehouse population and management • DB2 Value Unit Edition

A “new” value point for new DB2 z/OS workloads• Coming Soon! Cognos for System z (beta announced 2/26)

A comprehensive System z offering for Enterprise BI

The Enterprise Data Warehouse

31


Dynamic Index AND-ing for Star Join (Pair-Wise Join with Join Back)

Multi-index access steps are considered independent

Apply filtering to dimension tables before the fact table– Exploit single and/or multi-column fact table indexes

Runtime assessment of filtering– Pre-fact dimensions with poor filter factors can be discarded at runtime and

accessed post-fact

Independent join of each dimension table to fact table via index

The result of each pair-wise join is a set of fact table rids

Perform Rid Sort and Rid Merge (ANDing) to generate final fact table rid list

Final Rid list then used to retrieve data from Fact table

Join back to dimension table(s) as necessary.

32


Histogram Statistics - RUNSTATSV8 – DB2 has data skew awareness for single values

Histogram statistics addresses skews across ranges of data values

Summarizes data distribution on an interval scale

DB2 uses equal-depth histograms– Each quantile has about the same number of rows– Example - 1, 3, 3, 4, 4, 6, 7, 8, 9, 10, 12, 15 (sequenced),

cut into 3 quantiles

3/123151034/1249625/12 3411

FrequencyCardinalityHigh ValueLow ValueSeq No

33


Security Enhancements

34


Trusted Contexts – An Introduction

A TRUSTED CONTEXT establishes a trusted relationship between DB2 and an external entity such as a middleware server. For example:– WebSphere Application Server

– Lotus Domino

– SAP NetWeaver

– PeopleSoft V7

A set of trust attributes is evaluated to determine if a specific context is to be trusted.

A trusted context allows the external entity to use a database connection under a different user ID without the database serverauthenticating that ID.

It also allows an AUTHID to acquire database privileges associated with that trusted context, and not available outside it, via a ROLE.

35


Roles and Context-specific PrivilegesRoles provide the flexibility to grant privileges to an AUTHID only when the user is connected via a trusted context.

They greatly simplify management of authorization.

An individual role can be defined for any AUTHID using the trusted connection, in which case the user inherits the privileges granted to the individual role.

Where there is no individual role, any AUTHID using a trusted context inherits the privileges of the trusted context’s default role, if defined.

CREATE TRUSTED CONTEXT CTX1BASED UPON CONNECTION USING SYSTEM AUTHID WASADM1DEFAULT ROLE CTXROLEATTRIBUTES (ADDRESS '9.67.40.219')ENABLEWITH USE FOR JOE ROLE JROLE;

36


Application Design

37


Universal Table SpacesUniversal Table Space– Combination of segmented with partitioning options

• Better space management• Support of mass deletes / TRUNCATE

– If partitioned• Still must be one table per table space• Can choose Range Based partitioning (as before: PBR)• Can choose Partitioned By Growth (PBG)

– DROP / CREATE to migrate existing page sets

– Simple table spaces can not be created• Default table space is now Segmented (CM) or PGB (NFM)

38


CLONE TablesAllows fast replacing production data without renames and rebinds

– An alternative to online load replace

ALTER TABLE to create a Clone Table– All indexes, LOB and XML objects are also cloned– Structures ONLY – not data– Base and Clone tables share the same table space and index

names– Underlying data sets are differentiated by a data set instance

number

On single-table table spaces (partitioned or non-partitioned)

Use insert or load to populate clone tablesUtilities (except RUNSTATS) can operate on clone tables with new CLONE keywordEXCHANGE DATA switches logical names with underlying data

39


NOT LOGGED TablesIs actually NOT LOGGED tables, indexes, LOB, XML

ALTER / CREATE a TABLESPACE as NOT LOGGED– ALTER not allowed if in same UOW with an update to the table space

Indexes, LOB, and XML inherent the logging attribute of the base– This are considered “Linked” objects

Effects the UNDO / REDO records– Control information is still logged

LOB continue to log system pages & auxiliary indexes

A Unit of Recovery (UR) is still created

ALTER TABLESPACE … LOGGED/NOT LOGGED

LOG YES is a synonym for LOGGED

LOG NO is a synonym for NOT LOGGED

40


Other Design ChangesOptimistic Locking Support– Built-in timestamp for each row or page

• Automatically updated by DB2 as a GENERATED ALWAYS column• Allows timestamp predicate to validate that row has not changed since last access

Implicit Database Support– Automatic DB & TS– MAXPARTITIONS defaults to 256

– Support for Primary Key and Unique Keys, LOB and XML objects

– PK62178, sequence added to control number of implicit databases, default now 10000

STOGROUP support for SMS classes

ALTER TABLE RENAME Column

RENAME INDEX

New CATMAINT options– Switch schema name

– Change from owner to role (NFM)

– Change VCAT

RTS enhancement to identify unused indexes

41


LOB ImprovementsProgressive Streaming for LOB Locator Values– DB2 uses LOB size to determine whether to send LOB data to Java or DB2 CLI

clients in one go (<32KB), in chunks (<1MB) or as LOB locator (>=1MB)• Transparent to application using LOB locators

FETCH CONTINUE– Allows applications to retrieve LOB/XML data in pieces without the use of locators

File reference variables– A file reference variable allows direct transfer of LOB data between DB2 and the file

named in the variable

Utility Changes– LOAD / Cross load LOB column lengths > 32KB supported

– Logging for > 1GB LOBs

– REORG LOB reclaim space

– Online CHECK LOB and DATA

Elimination of LOB locks for improved availability and performance

42


OmniFind Text Search Support: Accessories Suite v1.2

• CONTAINS() built in function for text search• Search CHAR, VARCHAR, LOB, & XML columns• OmniFind provides a text index server• Efficient communication interactions with DB2 for z/OS• OmniFind text indexes are persisted into DB2 tables for

backup/recovery purposes

DB2

DB2

DB2DB2

ParallelSysplex

OmniFindServer

OmniFindServer

TCP/IP

Application

InvokeSQLAPI

43


DB2 9 Spatial SupportEnabling Open Geospatial Consortium (OGC)

compliant geospatial applications

– Spatial data types

– Spatial functions and predicates

– Spatial indexes

– Spatial search

– OGC-compliant spatial catalog

Phase II delivered in APARs PK51020, PK54451, …

Get updated IBM Spatial Support for DB2 for z/OS User’s Guide and Reference

MapGPSAddressStreetState…

44


Indexing Enhancements

45


Indexing EnhancementsLarger index pages allow for more efficient use of storage– Fewer page splits for long keys– More key values per page

Define RANDOM index keys to avoid hot spots with multiple processes inserting sequential keys

Rebuild Index SHRLEVEL CHANGE

Index compression provides page-level compression– Data is compressed to 4K pages on disk– 32K/16K/8K pages results in up to

8x/4x/2x disk savings– No compression dictionaries

• Compression on the fly• No LOAD or REORG required

4K

8K

16K

32K

46


Asymmetric Index Page SplitsMultiple Sequential Insert Patterns on an Index

Sequential inserts into the middle of an index resulted in some pages with 50% free space prior to DB2 9

New algorithm dynamically accommodates a varying pattern of inserts

47


Index on Expression

210,000200,000Rob2,00040,000Rex10,000400,000Matt5,00040,000Paul50020,000Gary

bonussalaryname

W2_TABLE

Query / Order on Total Compensation

Simple indexes can contain concatenated columnsCREATE INDEX TOTALCOMP ON

W2_TABLE(SALARY, BONUS)

Index on expression– Value of the index has been

transformed

– May not be the value of any of the columns that it is derived from

– Optimizer can use this indexCREATE INDEX TOTALCOMP ON

W2_TABLE(SALARY+BONUS)

48


Data Sharing

49


Data Sharing DB2 9 EnhancementsLog latch contention relief

Restart performance enhancements– Reduced impact of retained locks – released as rollbacks are completed

– Open data sets ahead of log apply

Command to remove GBP-dependency at object level– ACCESS DB MODE(NGBPDEP)– Typical usage would be before batch run– Command to “prime” open data set– ACCESS DB MODE(OPEN) [PART]

Auto-recover GRECP/LPL objects on group restart– Useful in Disaster Recovery or GDPS scenarios

DB2 overall health taken into account for WLM routing

Balance group attach connections across multiple members on same LPAR (V7 & V8 usermod)

Group wide outage no longer needed for new LOB locking protocol (apar)

50


Data sharing: LRSN Spin

LRSN updates every 16 microseconds– Unique for each log record on a single member– Log latch held while waiting for new LRSN

DB2 9 only waits for update when required– LRSN only needs to be unique per page– Improved logging performance by better concurrency

• Log latch does not need to be held while LRSN incremented• Log and transaction troughput can be increased

Reduced Latch class 19 waits– As result of increased logging activity, watch out for

• Log buffer contentions • More often checkpoints

– NFM required

51


Utilities

52


Utilities HighlightsMore online utilities

– Rebuild Index SHRLEVEL CHANGE

– Reorg LOB now supports SHRLEVEL REFERENCE (space reclamation)

– Check data, LOB and repair locate … SHRLEVEL CHANGE

– Check index SHRLEVEL REFERENCE supports parallel for > 1 index

– Clones for “online LOAD REPLACE”

Online REORG BUILD2 phase elimination

REORG parallelism for UNLOAD, RELOAD, LOG phases

Utility TEMPLATE switching

UNLOAD SKIP LOCKED DATA option

53


Utilities Highlights…MODIFY Recovery enhancements– “Retain” keyword added to improve management of copies

• LAST(n), LOGLIMIT, GDGLIMITVolume-based COPY/RECOVER (BACKUP SYSTEM/RESTORE SYSTEM)– RECOVER modified to enable object-level recovery from volume

FlashCopy– Full integration of tape into BACKUP/RESTORE SYSTEM utilities– Incremental FlashCopy, APAR PK41001Truncate log based on timestampRECOVER to any point-in-time with consistencyRECOVER RESTOREBEFORE to use an earlier image copyDisplay progress of RECOVER during log applyCOPY CHECKPAGE option always active– “Copy Pending” avoided if broken page encounteredCOPY SCOPE PENDING to copy only objects in “Copy Pending”

54


Utilities Performance ImprovementsCPU reductions in LOAD (with additional savings if data is PRESORTED), REORG, and REBUILD – Reductions mostly due to improved index processing (*

with exceptions)• 10 to 20% in Image Copy* (even with forced CHECKPAGE YES)• 5 to 30% in Load, Reorg, Reorg Partition, Rebuild Index

– Except REORG TABLESPACE SHR CHG PART with NPSIs

• 20 to 40% in Load• 20 to 60% in Check Index• 35% in Load Partition• 30 to 40% in Runstats Index• 40 to 50% in Reorg Index• Up to 70% in Load Replace Partition with dummy input

55


And Also …RLF enhancementsMany serviceability enhancements– Messages – Display commands– Refresh ERLY code without IPL– Many others …Converged TEMP SpaceZparm to prevent workfilemonopolizationSQL– FETCH FIRST/ ORDER BY in

SubselectRTS moved to catalog –automatic update

Global Query Optimization will allow DB2 to optimize a query as a whole rather than as independent parts– considers the effect of one

query block on another– considers reordering query

blocksServiceability– DISPLAY enhancements– Health Monitoring – internal

monitorDB2 9 can use more index lookaside for INSERT & DELETESSLImproved trace filteringCommand line processor

56


Post GA Service

57


DB2 9 Post GA Deliveries APAR PKxxxxx

Text search server Accessories Suite 1.2

New XMLTABLE and XMLCAST functions: PK51573

Incremental FlashCopy PK41001, z/OS 1.8 APAR OA17314

Trusted context enhancements PK44617, PK47579

New storage class Zparm for online CHECK utilities PK41711 Needed when PPRC is used.

ALTER TABLE ALTER COLUMN DROP DEFAULT PK56392Allow RESTORE SYSTEM recovery without requiring log truncation PK51979Spatial phase II PK51020, PK54451, …

BIND stability changes PK52522, PK52523

BIND changes convert plans with DBRMs to packages PK62876

58


DB2 9 Post GA Deliveries … continuedLOAD COPYDICTIONARY PK63324, PK63325

Support for DB2 utilities to read DFSORT parameters activated via ICEPRMxx members in PARMLIB (z/OS 1.10) PK59399

Restore V8 Group Attach behaviour: PK79228

Limit the number of implicitly-created databases: PK62178

Enable LOB locking enhancements without group restart: PK62027

Extend the APPEND option to LOB table spaces: PK65220

Trusted Context enhancements: PK44617

59


Incremental FlashCopy - PK41001

Incremental FlashCopy support added to BACKUP SYSTEM utilityDFSMShsm apar OA17314 in z/OS 1.8. – Reduced amount of data - only changed tracks are copied. – Complete copy of the source volume - with potentially less I/O– Incremental FlashCopy hardware relationship established for each

source volume in copy pool with corresponding target volumes– Multiple FlashCopy versions are not supported

New Keywords on BACKUP SYSTEM utility– ESTABLISH FCINCREMENTAL for incremental relationship – END FCINCREMENTAL last incremental FlashCopy taken followed

by withdraw of the incremental FlashCopy relationship

60


RESTORE SYSTEM change PK51979

Allow recovery without requiring log truncation

Original system point-in-time recovery design to truncate DB2 logs to prior log point

Allow DB2 to be restarted in System Point-in-Time Recovery mode without truncating logs to a prior log point.

The RESTORE SYSTEM Utility can then be used to recover DB2 to the current end of log.

In a data sharing environment, only non-quiesced members will be required to restart with the new DSNJU003 parm

– All members restarted in SYSPITR mode must be restarted with the same SYSPITR CRESTART value

61


Private protocol DRDA (new help in DSNTP2DP) Plans containing DBRMs, ACQUIRE(ALLOCATE)

packages, ACQUIRE(USE) XML Extender new XML typeDB2 MQ XML user-defined functions and stored procedures

use the new XML functionsmsys for Setup DB2 Customization Center removed

install panelsSimple table spaces segmented or partitioned by growthLoading DSNHDECP directly

Use the new interface

DB2 9 deprecated or removed function

62


BIND changes for plan stability and prepare for future release – PK52523

New REBIND PACKAGE option to control whether REBIND PACKAGE saves old package copies.. – PLANMGMT(EXTENDED) - A package has one active copy, and

two additional old copies (PREVIOUS and ORIGINAL) are preserved.

– New ZPARM added for default, tooNew REBIND PACKAGE option to allow users to revert a package to an older copy, effectively undoing a prior REBIND invocation.– SWITCH(PREVIOUS) - The PREVIOUS copy is activated – SWITCH(ORIGINAL) - The ORIGINAL copy is activated 4.

Option to disallow binding for private protocol as well as binding DBRMs into plans– DRDA to be required in future release– Binding packages to be required in future release

63


Business needs • Reduce CPU time• Improve business agility • Service Oriented Architecture

Application developers need• Powerful new SQL enhancements • Portability with SQL and data

definition compatibility • PureXML for a powerful SQL and

XML interface to XML data

Database Administrators need• Improve availability and

performance • More flexible security and

easier regulatory compliance • Better web application & data

warehouse function and performance

• LOB function, performance, usability

Why Migrate to DB2 9 for z/OS?

64


Migration process enhancements: ENFM speed, CM*Much less performance regression: – Earlier improvements– Plan stability & tools for avoiding access path issues

CCSIDs and old product issues resolved in V8Simpler virtual storage considerationsLess impact from incompatible changesEarlier deliveries from vendors

Why is migration easier to DB2 9 for z/OS?

65


Changed online REORGImproved RUNSTATSOptimization improvements, EDMPOOL VSCRREOPT(AUTO) dynamic SQLReordered row formatIndex: larger page sizes, compression, index on expressionLOB lock avoidance

Most consumable DB2 9 improvements Very little to no action:

Utility CPU reductionsLogging improvementsImproved index page splitLarger prefetch, write & preformat quantitiesLOB performanceDDF VSCROptimization Service Center & Opt. Expert

66


DB2 9 for z/OS RedBooks & RedPapersPowering SOA with IBM Data Servers SG24-7259

LOBs with DB2 for z/OS: SG24-7270

Securing DB2 & MLS z/OS SG24-6480-01

DB2 9 Technical Overview SG24-7330

Enhancing SAP - DB2 9 SG24-7239

Best practices SAP BI - DB2 9 SG24-6489-01

DB2 9 Performance Topics SG24-7473

DB2 9 Optimization Service Center SG24-7421

Index Compression with DB2 9 for z/OS paper

DB2 9 Stored Procedures SG24-7604

67


Futures: Possible Items in DB2 X

68


DB2 for z/OS Into the FutureDelivering Customer Value

V7V8

DB2 92001

2004

2007

20xx

DB2 X

OnGoing themes:Performance Scalability

Reliability Availability ServiceabilitySecurity Productivity

Application DevelopmentSQL XML SOA

64 bit data definition on demand

pureXMLtm

69


DB2 X for z/OS Status

The following slides represent DB2 Development’s current thinking on some of the items that are candidates for DB2 X

It is still early in the development process, so the details will change

The intention is to give you some information on DB2’s future technical directions

DB2 Development values customer feedback

70


• Full 64-bit SQL runtime• Auto stats• Data compression on the fly• Query stability enhancements• Reduced need for REORG• Utilities enhancements

Simplification, Reduced TCO

• Wide range of performance improvements• More online schema changes • Catalog restructure for improved concurrency• Fine grained access control• Hash access to data• New DBA privileges with finer granularity

RAS, Performance, Scalability, Security

• pureXML enhancements• Session variables, Generated columns• Temporal queries• Last Committed reads• SQL improvements that simplify porting

Application Enablement

• Moving sum, moving average• Many query optimization improvements• Query parallelism improvements• Advanced query acceleration

Dynamic Warehousing

DB2 X for z/OS At a Glance Addressing Corporate Data Goals

71


Application Enablement, Portability pureXML enhancementsSession variables to provide a flexible way to pass data from one program to the next via SQL – CREATE VARIABLEGenerated columns

• ALTER TABLE employeeADD COLUMN upper_lastnameGENERATED ALWAYS AS(UPPER(lastname));

Allow non-NULL default values for inline LOBsLoading and unloading tables with LOBs

• LOBs in input/output files with other non-LOB data

Full Decimal Floating Point support‘Last committed’ locking semanticsImplicit castingTimestamp with timezoneGreater timestamp precision

72


Application Enablement, Portability …SQL stored procedure enhancements– Autonomous transactions– SQLPL in Scalar UDFs and triggers

64-bit ODBC Support

Prepare/Describe enhancements for optimistic locking

Special null indicator to indicate value not supplied or default

DRDA support of Unicode for system code points

Instance based statement hints

Allow caching of dynamic SQL statements with literalsNumeric-based and data-dependent paging

• When only a specific part of the result set is needed• New syntax for paging through result sets• Efficient access to desired portions of result set, based on size or on data values

73


pureXML Enhancements

XML schema validation in the engine for improved usability, performance

Binary XML exchange format for improved performance

XML multi-versioning for more robust XML queries

Allow easy update of sub-parts of an XML document

Introduction of XQuery syntax

Stored proc, UDF, Trigger enhanced support for XML

Composite index support for XML data

74


Temporal Data – Query ‘AS OF’

Table-level specification to control the management of data based upon time– New syntax in FROM clause to specify a time criteria for selecting historical

data– New syntax to CREATE TABLE and ALTER TABLE to specify that a table has

data versioning. ‘KEEP’ specification for how much history is kept.

A history table is created to store the deleted/old values of a table defined with data versioning– Data is synchronously inserted into the history table when the base table is

updated or deleted– Timestamp value used is consistent for all DELETE/UPDATE statements

within the same logical unit of work

‘FOR AUDITING’ makes SECADM the owner of the history table with auditing columns in history table being populatedREORG of history table will remove data that has expired past the KEEP time

75


Availability

More online schema changes for tablespaces, tables and indexesOnline REORG instead of DROP/CREATE or REBUILD INDEX Alterations are manifested with REORG, unless noted otherwise– Page size for table spaces and indexes – DSSIZE for table spaces– SEGSIZE– MEMBER CLUSTER– Convert single table segmented into UTS PBG– Convert single table simple into UTS PBG– Convert classic partitioned tablespace into UTS (either PBG or PBR)– Convert UTS PBR to UTS PBG– Convert PBG to hash (immediate, but RBDP index)– Ability to drop pending changes

Online REORG for LOBs, other Online REORG / utility improvementsOnline add active log

76


Single-Table Simple

Table Space

Single-Table SegmentedTable Space

PartitionedTable Space

Range-PartitionedUTS

Partition-By-GrowthUTS

X: ALTER TABLE SPACESX: ALTER TABLE SPACES

77


DB2 X Performance, Scalability Objectives

Provide significant Scalability and Performance improvements– Will be an important “feature” for DB2 X– Synergistic operation with latest z10 Series hardware and follow-on

machine• High n-way scalability • Large real memory exploitation • Hardware level optimization

– Synergistic operation with z/OS 1.10 software (probable minimum supported level)

– Improve transaction times– Lower CPU usage for both large and small DB2 subsystems

Virtual storage is most common constraint for large customers – Can limit the number of concurrent threads for a single member/subsystem

Increasing the number of concurrent threads will expose the next tier of constraints

78


Performance Hash access pathParallel index update at insertInline LOBsLOB streaming between DDF and rest of DB2– Faster fetch and insert, lower virtual storage consumptionDEFINE NO for LOBs (and XML)Enabling MEMBER CLUSTER for UTSEfficient caching of dynamic SQL statements with literalsOption to avoid index entry creation for NULL valueIndex include columnsWorkfile spanned records, PBG support, and in-memory enhancementsBuffer pool enhancements– Utilize z10 1MB page size– “Fully in memory” option

Internal performance optimizations– Improved CPU cache performance, exploit new h/w instructions– Streamlined DDF, RDS, DM, Index Mgr. performance-critical paths

Exploitation of SSD (Solid State Disk)

79


Query Performance

Safe query optimization

More effective optimizer query rewrite

Statistical views

Minimize materialization and size of intermediate results

IN list performance

UNION ALL performance

RID pool usage enhancements

Query parallelism enhancements: lifting restrictions, improving efficiency

Dynamic Index ANDing Enhancements

Index include columns

Workfile spanned records, PBG support, and in-memory enhancements

Exploitation of SSD

80


DB2 X: 64 bit Evolution (Virtual Storage Relief)

EDMPOOL

DBD Pool

Global Stmt Pool

2GB

Skeleton Pool

Working memory

DB2 9 helped (~ 10% – 15%)

DB2 X expect to move 90%+– More concurrent work– Reduce need to monitor– Able to consolidate LPARs– Reduced cost– Easier to manage– Easier to grow

Scalability: Virtual storage constraint is still an important issue for many DB2 customers.

EDMPOOLDBD Pool

Global Stmt Pool

Working memory

2GB

Skeleton Pool

Working memory

EDMPOOL

81


Running a Massive Number of Threads

DB2A(500 thds)

Coupling Technology

Data sharing and sysplex allows for efficient scale-out of DB2 imagesSometimes multiple DB2s / LPAR

Today

LPAR1

DB2D(500 thds)

DB2B(500 thds)

LPAR2

DB2E(500 thds)

DB2C(500 thds)

LPAR3

DB2F(500 thds)

DB2A(5000 thds)

Coupling Technology

More threads per DB2 imageMore efficient use of large n-waysSSI constraints are relievedEasier growth, lower costs, easier managementData sharing and Parallel Sysplex still required for HA and XXL scale

DB2 X

LPAR1

DB2B(5000 thds)

LPAR2

DB2C(5000 thds)

LPAR3

82


Other System Scaling Improvements

Other bottlenecks can emerge in extremely heavy workloads– several improvements planned to reduce latching and other system serialization contention– new option to for readers to avoid waiting for updaters– eliminate UTSERIAL lock contention for utilities– Exploitation of 64-bit common storage to avoid ECSA constraints

Concurrent DDL/BIND/Prepare processes can hit contention with one another– restructure parts of the DB2 catalog to avoid the contention

SPT01 64GB limit can be a constraint, especially if “plan stability”support is enabled– relieve 64GB limit for SPT01

83


Protect sensitive data from privileged users

– SYSADM without data access

Separate authority to perform security related tasksAllow EXPLAIN without execute privilege or ability to access dataAudit privileged users

Security Administrator

Tasks

System Administrator

Tasks

AccessMonitor

DB2 X: Business Security & Compliance Needs

“As of” query, temporal or versioned dataFine grained access control

Allow masking of value Restrict user access to individual cells

Use disk encryption

84


DB2 X: Productivity – Doing More with Less!

Auto statistics collectionCompress ‘on the fly’– Avoid need to run

utilityReduce contention, more online processingAutomatic config of IBM supplied UDFsand SPsEnhancements for Data Studio

Manual invocation of•RUNSTATS•COPY/BACKUP SYSTEM•QUIESCE•MODIFY RECOVERY•REORG

85


Autonomics and DBA Productivity…

Access path stability– Rapid detection of performance regression and autonomic corrective action

Hints enhancements– Allow hints specification without source statement change

– Enable statement replacement without source statement change

Checkpoint intervals based on both time and # log records

Run ‘must complete’ backout under pre-emptable SRB

Reduce need for reorganizations– Automatic clean-up of pseudo-deleted keys in index, zIIP enabled

– Compression on the fly during INSERT

– New index prefetch technology to avoid need for REORG of indexes

86


Optimization Stability and Control

Provide an unprecedented level of stability of query performance achieved by stabilizing access paths:– Static SQL

• Relief from REBIND regressions• Remove the fear of REBINDing!• Enable REOPT support

– Dynamic SQL • Remove the unpredictability of PREPARE• Extend Static SQL benefits to Dynamic SQL

SupportAccess path repositoryVersioning“Fallback”“Lockdown”Manual overridesHints: easily influence access paths without changing appsPer-statement BIND options

87


Optimization Stability and Control…

Access path lock-in and fallback for dynamic SQL– Remove the unpredictability of PREPARE– Extend static SQL benefits to dynamic

Access path repository in DB2 catalog (both static and dynamic)

Policy (introduced in v9) can be used to identify candidates for stabilizing

At PREPARE, DB2 checks if an access path was previously captured for a query• YES -- it is used as a “hint” for the compilation• NO -- the query is compiled without a “hint.” The resulting access path captured if query is a candidate for

stabilization. • Once prepared, the statement could be cached

Rebinding dynamic SQL: new REBIND QUERY command– Allows DB2 to generate new access paths at next prepare– Filtering capabilities based on queryno, or user-specified tag– Also allows for fallback to prior access path (REBIND QUERY SWITCH)

Optimizer hints integrated into access path repository to enable hints for dynamic without requiring changes to the app

BIND options at statement level granularity

Statistical views

88


DB2 X Utilities Enhancements– REORG SHRLEVEL(CHANGE) for LOBs– Online REORG enhancements

• SHRLEVEL(CHANGE) support for all catalog/directory objects• Support shrinking of PBG partitions• Option to cancel blocking threads• Faster SWITCH phase• Allow disjoint partition ranges• Permit movement of rows between partitions when LOB columns exist

– Allows REBALANCE or shrinking of PBG even though LOB columns exist

– Allows DISCARD to delete associated LOB values• Messages to estimate length of REORG phases and time to completion

89


DB2 X more utilities enhancements

Support of spanned records for UNLOAD of LOB data– Currently unload of LOBs >32K must use FRVs

– This allows inlining of LOBs with base row in unload dataset

– Provides portability of data

Performance enhancement for FRV processing with PDS datasets– UNLOAD 33% elapsed time reduction

– LOAD 84% elapsed time reduction

Extend support for UTF-16– Date, time & timestamp fields currently unloaded in UTF-8

– Cannot specify a char value for a graphic column in WHEN clause

90


DB2 X: More Utility Improvements– Improved COPY CHANGELIMIT performance

• Use RTS instead of SM page scans– Dataset level FlashCopy option– FlashCopy backups with consistency and no application outage– FlashCopy backups as input to:

• RECOVER (fast restore phase)• UNLOAD• COPYTOCOPY, DSN1COPY

– RECOVER “back to” log point– REPORT RECOVERY support for system level backups– MODIFY RECOVERY improved performance– RUNSTATS enhancements to support auto stats

• Support generation, use & deletion of statistics profiles• Auto sample rates• Deletion of history information• RUNSTATS on views

91


Data Warehousing

Moving Sum, Moving Average

Enhanced query parallelism technology for improved performance– Remove query parallelism restrictions

Dynamic Index ANDing improvements

In-memory techniques for faster query performance

Advanced query acceleration techniques

92


Key details about DB2 XCM, ENFM, NFM is plannedProbable Prerequisites

– z/OS V1.10 – DB2 9 for z/OS in NFM– z890, z990, z9 and above (no z800, z900)

Eliminated: – Private protocol DRDA (new help in DSNTP2DP)– Old plans and packages V5 or before REBIND– Plans containing DBRMs packages– ACQUIRE(ALLOCATE) ACQUIRE(USE)– XML Extender XML type– DB2 MQ XML user-defined functions and stored procedures XML functions– DB2 Management Clients feature (DB2 Administration Server, Control Center, &

Development Center) IBM Data Studio application & administration services– msys for Setup DB2 Customization Center install panels– BookManager use for DB2 publications Info Center, pdf

Documents

DB2 for z/OS: Version 9 and Beyond