DB2 Under the Hood:An OS/390 Performance Primer

Jim WankowskiQuest Software

WWW.Quest.Com/DB2

Agenda

Defining PerformanceIdentifying/Resolving Performance Issues

The SubsystemPhysical Design ConsiderationsSpace managementApplication Design

What is Performance?

Performance: noun“The fulfillment of a claim, promise, or

request”

How does your company define performance?– System availability– Transaction throughput– Minimum response times (SLA’s)

Subsystem Monitoring

All aspects of the subsystem need to be monitored.–Not just SQL–Take a look at the big picture

•Where are your bottlenecks?

Subsystem Performance

Components to monitor–EDM Pool–RID Pool–Buffer pool–Sort pool

Monitoring Subsystem Performance

EDM Pool (Environmental Descriptor Management Pool)

– DBD (Database Descriptor)– CT (Cursor Table)– PT (Package Table)– SKCT (Skeleton Cursor Table)– SKPT (Skeleton Package Table)– Plan/Package authorization Cache (CACHESIZE > 0)– Dynamic SQL skeletons (Dynamic SQL caching

active)

“System Bufferpool”– Minimizes I/O against catalog and directory

Problem: An EDM pool that is too small causes:

Increased I/O activity against DSNDB01• SCT02• SPT01• DBD01

Increased response times due to loading the SKCTs, SKPTs, and DBDs Re-preparation of Dynamic SQL Fewer threads used concurrently, due to a lack of storageResource unavailable “-904”

What to monitor

EDM Pool hit ratio should be at least 80%– PT/CT’s

• 80-90%– DBD requests

• 100%– Pages used for Package/Cursor tables <50% of pool

Sizing EDM Pools

Determining size of EDM Pool– Average DBD size– Average Plan/Package size– Average cache size for Plans

Don’t be afraid to make EDM too big!

Managing EDM Pool Size

Reduce size of DBD’sTry to minimize # of objects in database

Use 32K Pieces for large databasesRun MODIFY utility regularly to remove old recovery info.

Dropped objectsREORG tablespaces when tables dropped/recreated

• Dataspaces for Dynamic CachingSpecify a portion of EDM pool in a dataspace

EDMPOOL DATA SPACE SIZE > 0

Impact of Plans/Packages onEDM Pool SizeAvoid large Plans

• Use Packages (avoid DBRM’s)• Break applications into separate Plans

DEGREE(ANY) maintains 2 access pathsOne each for Parallelism YES/NO

Bind w/AQUIRE(USE) vs. AQUIRE(ALLOCATE) when possible.Use RELEASE(COMMIT) for infrequently used Plans/Pkg’sUse RELEASE(DEALLOCATE) cautiously

• Can cause EDM pool to grow enormously.• Only change a few programs at a time.

– Monitor EDM pool and IRLM memory usage before/after changing from COMMIT to DEALLLOCATE

RID Pool

“Row Identifier” Pool– Enforces unique keys during multi-row

updates– Used for Storing and Sorting RID’s for:

• List Prefetch• Multiple index access• Hybrid Joins

Default size = 4MB– 1000MB max– 0 deactivates Prefetch and multi-IX scans

Problem

If RID pool is too small– Previously mentioned access paths can

revert to TS scans

How Big?– #SQL statements requiring RID processing

X Avg number of RID’s X 2 X 4 (5 for LOBs)

What to Monitor

Insufficient pool size– RID pool too small

RDS Limit– RID list > 25% #rows in table

• Prefetch turned off and TS scan results

– Determined at Bind time

DM Limit– RIDs req’d to satisfy query >

16 million– TS scan results

RID Pool TuningInsufficient pool size– Recalculate RID pool size

– # of concurrent RIDS x Avg #RIDs x 2 x5 bytes/RID

RDS Limit– Have tables grown since

BIND?

–Make sure stats are accurate

• RUNSTATS/REBIND

DM Limit– Is TS Scan best access?– Re-evaluate indexes– Add additional filtering

BufferpoolsBuffer Pages

Hiperpool

Virtual storage for temporarily holding data and IX pages– Use DBM1 address space

• Virtual bufferpools• Can define up to 80 BP’s

– Limited to 1.6GB total

– DBM1 + Hiperspace• Hiperpools• Additional 8GB of extended storage• “Holding tank” for infrequently updated data

– MVS Dataspace• Support up to 8M buffers• Allows for direct I/O from extended storage

DASD

Buffer Pages

Virtual BP

BufferpoolsHiperpools

Advantages– Less space required for virtual pools

• Reduce central storage requirements– Serve as cache for infrequently accessed

data• Reduced response times vs. accessing DASD

– Use hiperpools when paging rates are high on virtual pools

Disadvantage– No direct I/O from Hiperpool

• Data is transferred back into virtual pool

Monitoring Bufferpools-Display BP

3rd Party monitor

What to Monitor Buffer Pool hit ratio

(GETPAGES – pages read)/GETPAGES

Hiperpool Hit Ratio– #pages read from

hiperpool/pages written

Page Externalization– Excessive writes to disk

• Deferred write threshold (DWQT)

• Vertical Deferred Write Threshold (VDQWT)

Effective Use of BufferpoolsThe single biggest performance mistake is to stick everything in BP0!– DB2 has 80 bufferpools available for a reason!

Separate bufferpools for:Catalog and Directory (BP0)DSNDB07Tablespaces Indexes

• Large VPSIZE• More than one

Small, Read-only tablesLarge tablespaces w/random processingSmall frequently updated tables and IX’sTest environment for isolating test cases

Sort PoolsSort pools should be made as large as possible– 240K – 64MB

• The larger the sort pool, the more efficient the sort

• Default to 1MB

Minimizes externalizing to physical sort files and BP’s

•DSNDB07

Optimizing SortDSNDB07– Do not use BP0!– 4 –5 equally sized datasets– Keep in primary space– Separate volumes– Also used for:

• View, temp table, and nested table expression materializations

• Non-correlated in-lists

Application Considerations– Avoid ORDER by, Group By, Distinct– Avoid sorting VARCHARs– Only select required columns

Physical Design

DB2 Catalog/DirectoryPhysical Objects– Tablespace– Index

Problem

A poorly designed/maintained catalog can result in:– Slow response time from catalog queries– Contention with other users

The DB2 CatalogDesign Considerations– BP0 is isolated to the catalog only– Keep catalog and directory on separate

volumes• Do not put other DB2 objects on catalog volume

– Keep EXTENTS to a minimum• Use REORG or RECOVER to re-allocate

– Do not migrate catalog datasets (DFSMS)

DB2 CatalogMaintenance–Eliminate redundant grants in AUTH

tables• Can be significant in older DB2 environments• Can cause significant performance impact

–Schedule regular maintenance• Treat Catalog like application

– RUNSTATS– REORG

•Reorg related directory tables at same time

– SYSDBASE + DBD01– SYSPLAN + SCT02– SYSPACKAGE + SPT01

DB2 Catalog

To avoid contention:Limit DDL and Binds to off hoursCreate “canned” queries for catalog reports using VIEWSKeep RUNSTATS current

Tablespace Design

What type of tablespace?Simple SegmentedPartitioned

SimpleIn General avoid, use Segmented

Advantage• If “READ ONLY” and data from 2 tables is being

retrieved in a hierarchical order

– Data from tables can be co-mingled onto same pages

Disadvantages• Data from multiple tables on same page• Page/Table locks can lock data from multiple tables• Tablespace scans can be much slower if multiple tables

in TS• Utilities lock entire TS• Dropped table space not immediately available

SegmentedBest choice for most applicationsAdvantage

• TS scans only scan segments assigned to table• Multiple tables data is not co-mingled

– Data pages organized into segments (4-64)– Table locks do not affect other tables

• Dropped table or mass delete space immediately available

– Only space map updated– No REORG required

• Less logging for mass delete

Disadvantage• LOAD REPLACE, RECOVER, and COPY must be for

entire tablespace

Segmented

Choosing SEGSIZE

For efficient sequential prefetch on small tables, choose a segsize = #pages in largest table

Pages Segsize<28 4-28

>28<128 32>128 64

PartitionedBest choice for very large tables or when Parallelism requiredAdvantages

• Spread data across multiple devices and bufferpools

• Partition independence– Utilities more manageable– Higher availability

• I/O Parallelism– Prefetch data from multiple partitions

concurrently• Query Parallelism

– Partitions can be spread across SYSPLEX members

Partitioned

Disadvantage• Fixed # of partitions

– Table must be dropped to add/remove partitions (changing in V8)

• TS Scans less efficient than segmented• More open datasets

Index considerations

Separate bufferpools from tablespacesAvoid updated columns– Especially for partitioning keys

Have thorough knowledge of SQL– Composite key IX’s can sometimes be more

efficient than single column keys– Use Asc/Desc to avoid sorting

Space Management

Efficient DASD UtilizationCompressionObject Maintenance

Efficient DASD Utilization

Allocate DB2 datasets to use only primary space– Excessive extents can cause inefficient I/O

Determine best page size for application– Data sharing

• 8K or 16K page size can reduce overhead in coupling facility

Use HSM to manage datasets– Move infrequently accessed datasets to slower devices

Compression

Use COMPRESS when applicable– More rows/page minimizes I/O

• Higher BP hit ratios• Fewer Getpage requests

– Always use hardware compression • Significantly less overhead than

EDITPROC/FIELPROC– Less DASD– Use DSN1COMP to check compression rate

CompressionConsiderations–For heavily updated data

• Increase PCTFREE to avoid increases in:–Synchronous reads–Lock requests–Getpage requests

–Decoding is much cheaper than encoding

–Overhead for TS scans can be higher–Cheaper for IX scans

Maintenance

Proper maintenance is critical for optimal performance– REORG– RUNSTATS– Real-Time Statistics

Reorg

What causes fragmentation?– Insert/Update

• Check PCTFREE and FREEPAGE– VARCHAR fields being updated

Monitoring for REORGsWhen to Reorg a tablespace:– CLUSTER RATIO < 95%

• SYSINDEXES– CLUSTERATIO < 95%

• SYSINDEXPART– FAROFFPOS > 10% of CARD

– Excessive row relocation• SYSTABLEPART

– NEARINDREF+FARINDREF > 10% of CARD

– Excessive extents (>50)– Excessive drop space

• Simple TS only– PERCDROP > 10%

– LOB tablespaces• SYSLOBSTATS

– ORGRATIO > 2

Monitoring for REORGs

When to REORG an Index–Excessive distance between LEAF

pages• SYSINDEXPART

–LEAFDIST > 200» Can cause Pre-fetch to be disabled» Should be monitored for growth

–LEAFFAR > 10% of NLEAF (SYSINDEXES)

–Excessive extents (>50)

RUNSTATS

Accurate statistics are a critical factor for performance monitoring and tuning

• RUNSTATS provides statistical information for:

1.Optimization of SQL2.Monitoring status of objects

RUNSTATSWhen to run RUNSTATS:

After LOAD, REORG, and REBUILD IXAfter creating new indexAfter heavy insert, update, delete activity

Performance Tip:Only run RUNSTATS against partitions which have changed.Consider using the SAMPLE option for column statistics to reduce processor costs• Be sure to check access paths when using

SAMPLE option

Monitoring RUNSTATSREBIND after RUNSTATS?

New index createdCLUSTERRATIOF changes <> 80%Changes more than 20%• Cardinality• NPAGES• NACTIVEF• NLEAF• Range of HIGH2KEY to LOW2KEY

Real Time Stats

New for V7• Facility for automating the collection

of DB2 statistics• Stored procedure DSNACCOR

1. Recommends when to REORG, Copy, or run RUNSTATS

2. Identifies TS and IX’s that have exceeded their dataset extents

3. Identifies objects in restricted state4. Stores data in new stats database DSNRTSDB5. Application must be built around DSNACCOR to

return information

Real Time StatsCreate stats database– DDL in SDSNSAMP(DSNTESS)

Set collection interval– Install panel DSNTIPO “Real TIME STATS”– DSNZPARM = STATSINT– Default interval = 30 Minutes– Data sharing environment has 1 interval per member

Start statistics database– START DATABASE(DSNRTSDB)

Application DesignApplication prepSQL Design considerationsOptimization HintsBind considerations

Efficient application design is the single most important aspect of an efficiently performing subsystem

Program PreparationCheck Object ValidityCheck authorityCompute filter factorsChoose access pathPrefetch Y/N

ApplicationCode

Precompile

ExecutableLoad

Module

DBRM Bind

Optimizer

DB2Catalog

Planor

Package

LinkEdit

Compile

DSNDB01SKCTSKPT

Optimization TipsOrder predicates from most to least restrictiveAvoid sorts– ASC/DESC indexes can help

avoid excessive sorting

Avoid UNION clause– CASE expression more

efficient

AND MOST IMPORTANTLY…..

Make sure statistics are accurateUse stage 1 vs. stage 2 predicatesOnly select required columns– Avoid SELECT *

Keep predicates as restrictive as possible– Minimize # rows

returned– Minimize program

filtering and let DB2 do the work EXPLAIN!!!!EXPLAIN!!!!

Optimization Hints

Allows for user specified access path– Useful for maintaining access paths:

• across releases• After Rebinds

– Should be used cautiously

Using HintsSave access path info for all critical queries before using HINTSPlan_Table must be in correct 49 column formatDSNZPARM, Optimization Hints = YesAdd QUERYNO clause to SQLUpdate Plan_Table – Specify name in OPTHINT for desired access path– Static -REBIND PLAN(XYZ) EXPLAIN YES

OPTHINT(‘MYACCESS’)– Dynamic – Set Current Optimization Hint = ‘MYACCESS’

Bind ConsiderationsACQUIRE (ALLOCATE)/ RELEASE (DEALLOCATE)

• Best for long running applications• Good for batch processes

Advantage• Helps prevent deadlocking

Disadvantage• Reduces concurrency• Can lock all partitions of TS if LOCKPART YES• N/A for BIND PACKAGE• Can cause EDM pool overflow

Bind ConsiderationsACQUIRE(USE)/RELEASE(DEALLOCATE)

• Best choice for most applications

Advantage• Locks are taken only as needed• Least restrictive locks used• Locks released when plan terminates

Disadvantage• Can increase frequency of deadlocks

Bind ConsiderationsACQUIRE(USE)/RELEASE(COMMIT)

• Best for maximum concurrency

Advantages• Table/TS locks taken only when needed• Least restrictive lock selected• All locks released at COMMIT

– Unless CURSOR WITH HOLD

Disadvantage• Excessive overhead for frequent COMMITS• Increased frequency of deadlocks

SummaryBuild a strategy for how you will monitor/tune your DB2 environment– Try to identify most critical applications and begin

focusing on those– Set realistic performance expectations– Tune, monitor, tune, monitor…

THANK YOU FOR LISTENING!

For more information:WWW.Quest.Com/DB2