EDM and Dynamic Statement Caching - IBM - United … DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00 ... DBD REQUESTS 19197.4K 891.24 115.49 38.72 DBD NOT FOUND 52.00 0.00 0.00 0.00 DBD HIT

© 2010 IBM Corporation

EDM and Dynamic Statement Caching

John J. CampbellIBM Distinguished EngineerDB2 for z/OS [email protected]


2

The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. While IBM may have reviewed each item for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Anyone attempting to adapt these techniques to their own environments do so at their own risk.Any performance data contained in this document were determined in various controlled laboratory environments and are for reference purposes only. Customers should not adapt these performance numbers to their own environments as system performance standards. The results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.

Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Disclaimer/Trademarks


3

Objectives

Focus on key areas– System address space CPU, EDM pools, dataset activity, logging,

lock/latch contention, DBM1 virtual and real storage, buffer pool and GBP,…

Identify the key performance indicators to be monitoredProvide rules-of-thumb to be applied– Typically expressed in a range, e.g. < X-Y

– If <X, no problem - GREEN– If >Y, need further investigation and tuning - RED– Boundary condition if in between - AMBER

– Investigate with more detailed tracing and analysis when time available

Provide tuning advice for common problems


4

EDM Pool – V7 Picture

EDMPOOL

2GB BarGlobal Dynamic

Statement Cache (Dynamically

Prepared Statements)

DBM1 Address Space

Database Descriptors (DBD)

CT/PT SKCT/SKPT

CT

PT

PT SKCT

SKPT

SKCT

SKPT

SKDS SKDS

DBD DBD

EDMPOOL

2GB BarDBM1 Address Space

CT

PT

PT SKCT

SKPT

SKCT

SKPT

DBD DBD

EDMDSPAC

Data Space

SKDS SKDS


5

EDM Pools – V8 Picture

EDMPOOL

EDMSTMTC

EDMDBDC

2GB Bar

Global Dynamic Statement Cache

(Dynamically Prepared

Statements)

DBM1 Address Space

Database Descriptors Pool (DBD)

EDM Pool (CT/PT

SKCT/SKPT)

CT

PT

PT SKCT

SKPT

SKCT

SKPT

SKDSSKDS

DBDDBD


6

2GB Bar

DBM1 Address Space

EDM Pools – V9 Picture

RDS Pool Below (part of CT/PT

that must be below)

(No external ZPARM)RDS Pool Above

(part of CT/PT that can be above)

Skeleton Pool(SKCT/SKPT)EDM_SKELETON_POOL

EDMPOOL

EDMSTMTC

EDMDBDC


(Dynamically Prepared

Statements)

Database Descriptors Pool(DBD)

SKDSSKDS

DBDDBD

CT PT

SKCT

SKPT

SKCT

SKPT

CT

PT

PT


7

EDM Pool – Tuning V8Field Name Description

QISEPAGE PAGES IN EDM POOL

QISECT PAGES HELD BY CT

QISEKT PAGES HELD BY PT

QISEKTG PAGES HELD BY SKCT

QISESKPT PAGES HELD BY SKPT

QISEFREE FREE PAGES

QISEFAIL FAILS DUE TO POOL FULL

QISECTG CT REQUESTS

QISECTL CT NOT FOUND

QISEKTG PT REQUEST

QISEKTL PT NOT FOUND

% NON-STEAL. PAGES IN USE = (HELD BY CT + HELD BY PT) / PAGES IN EDM POOL * 100

CT/PT HIT RATIO = (CT/PT REQUESTS – CT/PT NOT FOUND) / CT/PT REQUESTS * 100

EDM POOL QUANTITY /SECOND /THREAD /COMMIT--------------------------- -------- ------- ------- -------PAGES IN EDM POOL (BELOW) 70000.00 N/A N/A N/AHELD BY CT 138.88 N/A N/A N/AHELD BY PT 2492.74 N/A N/A N/AHELD BY SKCT 559.90 N/A N/A N/AHELD BY SKPT 63811.66 N/A N/A N/AFREE PAGES 2996.82 N/A N/A N/A

% PAGES IN USE 95.72 N/A N/A N/A% NON STEAL. PAGES IN USE 3.76 N/A N/A N/AFAILS DUE TO POOL FULL 0.00 0.00 0.00 0.00

...

CT REQUESTS 505.0K 143.17 1.00 0.56CT NOT FOUND 153.00 0.04 0.00 0.00CT HIT RATIO (%) 99.97 N/A N/A N/APT REQUESTS 16302.4K 4621.62 32.18 18.00PT NOT FOUND 386.4K 109.54 0.76 0.43PT HIT RATIO (%) 97.63 N/A N/A N/A


8

EDM Pool – Tuning V8 …

If EDM Pool is too small–Fewer threads can run concurrently

–Race condition with CTHREAD/MAXDBAT– Queuing or resource unavailable “-904”

–Need a balanced configuration–Increased response time due to loading of SKCT, SKPT

–Increased I/O against SCT02 and SPT01

–Note: No need to tune for 50% FREE PAGES

Increase EDM Pool size as needed to keep• FAILS DUE TO POOL FULL = 0 ** Very serious condition **• % NON-STEALABLE PAGES IN USE < 50%• CT/PT HIT RATIO > 95 to 99%

ROTROT


9

RDS Pool and Skeleton Pool – Tuning V9Field Name Description

QISEPAGE PAGES IN RDS POOL BELOW

QISECT PAGES HELD BY CT BELOW

QISEKT PAGES HELD BY PT BELOW

QISEFREE FREE PAGES IN RDS POOL BELOW

QISEFAIL FAILS DUE TO POOL BELOW FULL

QISESPGE PAGES IN RDS POOL ABOVE

QISECTA PAGES HELD BY CT ABOVE

QISEKTA PAGES HELD BY PT ABOVE

QISESFRE FREE PAGES IN RDS POOL ABOVE

QISESFAL FAILS DUE TO POOL ABOVE FULL

QISEKPGE PAGES IN SKEL POOL

QISEKTG PAGES HELD BY SKCT

QISESKPT PAGES HELD BY SKPT

QISEKFRE FREE PAGES IN SKEL POOL

QISEKFAL FAILS DUE TO SKEL POOL FULL

QISECTG CT REQUESTS

QISECTL CT NOT FOUND

QISEKTG PT REQUEST

QISEKTL PT NOT FOUND

EDM POOL QUANTITY /SECOND /THREAD /COMMIT-------------------------- -------- ------- ------- -------PAGES IN RDS POOL (BELOW) 12576.00 N/A N/A N/AHELD BY CT 84.14 N/A N/A N/AHELD BY PT 309.74 N/A N/A N/AFREE PAGES 12182.12 N/A N/A N/A

FAILS DUE TO POOL FULL 0.00 0.00 0.00 0.00

PAGES IN RDS POOL (ABOVE) 524.3K N/A N/A N/AHELD BY CT 0.00 N/A N/A N/AHELD BY PT 137.15 N/A N/A N/AFREE PAGES 524.1K N/A N/A N/A

FAILS DUE TO RDS POOL FULL 0.00 0.00 0.00 0.00

PAGES IN SKEL POOL (ABOVE) 1280.00 N/A N/A N/A HELD BY SKCT 61.85 N/A N/A N/A HELD BY SKPT 1139.35 N/A N/A N/A FREE PAGES 78.81 N/A N/A N/A

FAILS DUE TO SKEL POOL FULL 0.00 0.00 0.00 0.00...CT REQUESTS 221.1K 10.26 1.33 0.45 CT NOT FOUND 24899.00 1.16 0.15 0.05 CT HIT RATIO (%) 88.74 N/A N/A N/A PT REQUESTS 4803.3K 222.99 28.90 9.69 PT NOT FOUND 540.1K 25.07 3.25 1.09 PT HIT RATIO (%) 88.76 N/A N/A N/A

% PAGES IN USE BY CT/PT BELOW = (HELD BY CT + HELD BY PT) / PAGES IN RDS POOL BELOW * 100

CT/PT HIT RATIO = (CT/PT REQUESTS – CT/PT NOT FOUND) / CT/PT REQUESTS * 100


10

RDS Pool and Skeleton Pool – Tuning V9 ...

If RDS Pool Below 2GB is too small– Fewer threads can run concurrently

– Race condition with CTHREAD/MAXDBAT– Queuing or resource unavailable “-904”

– Need a balanced configuration

If Skeleton Pool is too small– Increased response time due to loading of SKCT, SKPT

– Increased I/O against SCT02 and SPT01

Increase size of RDS Pool Below 2GB as needed to keep• FAILS DUE TO POOL FULL = 0 ** Very serious condition **• % PAGES IN USE BY CT/PT BELOW < 75%

ROTROT

Increase Skeleton Pool size as needed to keep• CT/PT HIT RATIO > 95 to 99%

ROTROT


11

Other Recommendations – V8 and V9

To keep the EDM Pool storage under control– Use BIND option RELEASE(COMMIT)

– RELEASE(DEALLOCATE) should only be used for the most frequently executed plans/packages

EDMBFIT=YES|NO? – Affects space search algorithm for all EDM Pools

– EDMBFIT NO (‘first fit’ algorithm) >> Best for performance– EDMBFIT YES (‘best fit’ algorithm) >> Best for space utilisation

– Use of EDMBFIT YES can be ‘worst fit’ algorithm

– May lead to very high EDM LRU (LC24) contention on large pools– Resulting in non-linear DB2 performance

– If LC24 contention rate is greater than 1000 per second – Switch to EDMBFIT=NO

– General recommendation: Always use EDMBFIT=NO (default)


12

V8 to V9 Migration Considerations

EDM Pool changes in DB2 9 – need REBIND – SKCT/SKPT moved above 2GB

– A portion (close to 30%) of CT/PT moved above 2GB

– Average estimated reduction of 60% but wide fluctuation from 20 to 90%

– ** BUT **

– There are no stealable EDM components left below 2GB

– Increase in package size after REBIND

– Increased likelihood of hitting EDM Pool full conditionRecommendation for RDS Pool below 2GB– On V8, identify the max number of pages held by CT/PT at peak

– Plan on keeping at least 2 * 70% * CT/PT (V8 peak)

– Intent of 2x factor is to provide some headroom


13

DBD Pool

If DBD Pool is too small– Increased response time due to loading of DBD

– Increased I/O against DBD01

To keep the DBD Pool storage under control– Try to minimise the number of objects in a database

– Especially in data sharing environment where multiple DBD copies can be stored in the DBD Pool due to DBD invalidation by other members

– Smaller DBDs also help with contention (see Locking) – Compact DBD by REORG and MODIFY if many DROP TABLE in segmented

tablespace

Field Name Description

QISEDPGE PAGES IN DBD POOL

QISEDBD PAGES HELD BY DBD

QISEDFRE FREE PAGES

QISEDFAL FAILS DUE TO DBD POOL FULL

QISEDBDG DBD REQUESTS

QISEDBDL DBD NOT FOUND

EDM POOL QUANTITY /SECOND /THREAD /COMMIT------------------------- -------- ------- ------- -------...PAGES IN DBD POOL (ABOVE) 25600.00 N/A N/A N/A HELD BY DBD 25458.84 N/A N/A N/A FREE PAGES 141.16 N/A N/A N/A

FAILS DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00 ...DBD REQUESTS 19197.4K 891.24 115.49 38.72DBD NOT FOUND 52.00 0.00 0.00 0.00DBD HIT RATIO (%) 100.00 N/A N/A N/A

Increase DBD Pool size as needed to keep DBD HIT RATIO > 95 to 99% ROTROT


14

NO CACHING

DB2 Statement Caching – Overview

PREPARE SEXECUTE SCOMMITEXECUTE SPREPARE SEXECUTE S

PREPARE SEXECUTE SEXECUTE S

ProgramA

ProgramB

full prepare

prepared statement S



-514/-518

delete

DB2 z/OS

full prepare

full prepare


15

DB2 Statement Caching – Overview …

Significant cost to fully prepare a dynamic SQL statement

Dynamic statement caching introduced in DB2 for OS/390 and z/OS V5

– Used by dynamic SQL applications to reuse and share prepared statements

Conditions for reuse of SQL statement from dynamic statement cache

– SQL is dynamically prepared SELECT, UPDATE, DELETE or INSERT

– The statement text is identical - character for character (literals problematic)

– The authorisation ID is the same

– REOPT(VARS) disables use of cache for that plan/package

Two levels of caching

– Global Dynamic Statement Cache

– Local Dynamic Statement Cache


16

CACHEDYN=YES


P R E P A R E SE X E C U T E SC O M M ITE X E C U T E SP R E P A R E SE X E C U T E S

P R E P A R E SE X E C U T E SE X E C U T E S

P ro g ra m A

P ro g ra m B

p re p a re d s ta te m e n t S



-5 1 4 /-5 1 8

d e le te

D B 2 z /O S

S K D SS

fu ll p re p a re

s h o rt p re p a re

s h o rt p re p a re


17

Global Dynamic Statement Cache …

NOTES:Enabled by ZPARM CACHEDYN = YES Statement text and executable of the prepared statement (SKDS) is cached for reuse across all threadsOnly first prepare is full prepare, otherwise short prepare, which is a copy from global cache into thread storageNo prepared statement is kept in thread storage across commit


18

CACHEDYN=YES

KEEPDYNAMIC(YES)

Local Dynamic Statement Cache

avoided prepare

PREPARE SEXECUTE SCOMMITEXECUTE S

PREPARE SEXECUTE SEXECUTE S

ProgramA

ProgramB

full prepare

short prepare



DB2 z/OS

SKDSS


19

Local Dynamic Statement Cache …

Enabled by BIND option KEEPDYNAMIC(YES)Prepared statements kept in thread storage across commit so thatprepares can be avoided– Application logic needs to reflect the bind option

Same prepared SQL statement can be stored in several threads – Least Recently Prepared Statements are thrown away from the LDSC

at commit based on MAXKEEPD

– MAXKEEPD limits the stored executable only, the statement text is always stored in thread storage

A portion of the Local Dynamic Statement Cache is moved above 2GB– Rough estimation of V9 = 50% of V8

– VSCR from local dynamic statement cache savings will be small ifthe V8 size is heavily constrained by low MAXKEEPD…


20

Local Dynamic Statement Cache …

APAR PK21861 introduced new zparm CACHEDYN_FREELOCALBased on internal thresholds (subject to change), statements areinvalidated from the LDSC and the associated storage is releasedStatements are purged at end of sectionCACHEDYN_FREELOCAL settings

0 Off (default V8)

1If (LDSC >=500MB & DBM1 Used >=75%) then free >= 100KB statementIf DBM1 Used >=85% then free any statement (default V9)

2If (LDSC >=500MB & DBM1 Used >=80%) then free >= 100KB statementIf DBM1 Used >=88% then free any statement

3If (LDSC >=350MB & DBM1 Used >=75%) then free >= 100KB statementIf DBM1 Used >=88% then free any statement


21

Dynamic Statement Cache Performance

GDSC hit ratio = [Short Prepares] / [Short + Full Prepares]

LDSC hit ratio = [Prepares Avoided]/[Prepares Avoided + ImplicitPrepares] – If the statement is not found in the LDSC >> implicit prepare

–Can result in either Short or Full Prepare

Field Name Description

QXPREP NUMBER OF SQL PREPARE STATEMENTS

QISEDSI FULL PREPARE REQUESTS

QXSTIPRP IMPLICIT PREPARES

QXSTNPRP PREPARES AVOIDED

QXSTDEXP PREP STMT DISCARDED - MAXKEEPD

QXSTDINV PREP STMT DISCARDED - INVALIDATION

DYNAMIC SQL STMT QUANTITY /SECOND /THREAD /COMMIT------------------------ -------- ------- ------- -------PREPARE REQUESTS 124.5K 5.78 0.75 0.25FULL PREPARES 17446.00 0.81 0.10 0.04SHORT PREPARES 108.1K 5.02 0.65 0.22

GLOBAL CACHE HIT RATIO (%) 86.10 N/A N/A N/A

IMPLICIT PREPARES 0.00 0.00 0.00 0.00PREPARES AVOIDED 5603.00 0.26 0.03 0.01CACHE LIMIT EXCEEDED 0.00 0.00 0.00 0.00PREP STMT PURGED 3.00 0.00 0.00 0.00LOCAL CACHE HIT RATIO (%) 100.00 N/A N/A N/A

GDSC hit ratio should be > 90-95%ROTROT

LDSC hit ratio should be >70%ROTROT


22

Recommendations

Global Dynamic Statement Cache–Should be turned on if dynamic SQL is executed in the DB2

system

–Best trade-off between storage and CPU consumption for applications executing dynamic SQL

Local Dynamic Statement Cache –Should only be used selectively for application with a limited

number of SQL statements that are executed very frequently

–Should NOT be used for DB2 systems that are constrained in DBM1 31-bit virtual storage


23

WebSphere preparedStatement Object Cache

WebSphere manages a cache of previously created preparedStatement objects on a connectionWhen a new prepared statement is requested on a connection, the cached preparedStatement object is returned if availableCreating a new preparedStatement object is costly in Java besides the cost to prepare the SQL to DB2


24

WebSphere/DB2 – How Caches Play TogetherPreparedStatement objects are cached per connection - multiple objects for the same statement

object construction

object construction

short prepare

c1=getConnection()p1=c1.prepareStatement()rs1=p1.execute()p1.close()commit()c1.close()

ProgramA

ProgramB

full prepare

short prepare




delete

DB2 z/OS

SKDSS

WebSphere AS

c1=getConnection()p1=c1.prepareStatement()rs1=p1.execute()p1.close()commit()p1=c1.prepareStatement()rs1=p1.execute()p1.close()commit()c1.close() local cache global cache

preparedStatement object cache

preparedStatementobject p1

conn1

deletethread1

prepare/execute

delete

avoided constructor

thread2preparedStatementobject p1


25

WebSphere/DB2 – Using KEEPDYNAMIC YES

c1=getConnection()p1=c1.prepareStatem ent()rs1=p1.execute()p1.close()com m it()c1.c lose()

ProgramA

ProgramB

full prepare


DB2 z/OS

SKDSS

W ebSphere AS

c1=getConnection()p1=c1.prepareStatem ent()rs1=p1.execute()p1.close()com m it()p1=c1.prepareStatem ent()rs1=p1.execute()p1.close()com m it()c1.c lose() local cache global cache

preparedStatement object cache

preparedStatementobject p1

conn1

thread1

execute

executeavoided constructor

avoided constructor

avoided prepare

avoided prepare

object construction


26

WebSphere/DB2 – Using KEEPDYNAMIC YES

Using KEEPDYNAMIC YES– Bind the JCC packages with KEEPDYNAMIC(YES) in a separate collection

– Define Data Source properties (WebSphere customProperty)– -keepDynamic=1 – -JdbcCollection=collname

Pros – Avoids prepare of statements after commit that were prepared earlier

– Saves CPU on application server (WebSphere) as well as DB2 Cons– Statements are saved at thread level

– Consumes a significant amount of storage– Not recommended for storage constrained systems

– Distributed threads cannot go inactive– DB2 resources for distributed threads are not freed at commit– No Sysplex workload balancing


27

WebSphere/DB2 – Recommendations

Always use PreparedStatement object caching in WebSphere AND global dynamic statement caching in DB2 for JDBC applicationConsider using local dynamic statement cache for applications that execute a limited amount of SQL very frequently and –DB2 is not storage constrained, and

–Sysplex Workload Balancing is not required


28

John Campbell IBM DB2 for z/OS [email protected]

SessionEDM and Dynamic Statement Caching

Documents

EDM and Dynamic Statement Caching - IBM - United … DUE TO DBD POOL FULL 0.00 0.00 0.00 0.00 ... DBD REQUESTS 19197.4K 891.24 115.49 38.72 DBD NOT FOUND 52.00 0.00 0.00 0.00 DBD HIT