Upload
tess98
View
115
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
© 2009 SOFTWARE ENGINEERING GMBH 1
Real-Time Statistics–
How to arm your DB2 for 24x7
Ulf HeinrichCentral Canadian DB2 User’s Group June 8 - 9, 2009
© 2009 SOFTWARE ENGINEERING GMBH 2
Overview
� Real-Time Statistics basics� Real-Time Statistics advanced� Real-Time Statistics vs. DB2 Catalog Statistics� Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 3
Real-Time Statistics (RTS) basics
What is Real-Time Statistics (RTS)?
� Collect real time statistics for table and index spaces� Existence of two DB2 tables allows easy interpretation� Statistics externalized based on certain events� Counters
The values reflect the amount of object level activity that has occurred since last COPY, REORG, RUNSTATS
� Very important!!!
© 2009 SOFTWARE ENGINEERING GMBH 4
What is Real-Time Statistics (RTS) ?
� V7 or later (integrated into DB2 Catalog within DB2 9)� Capture statistics in memory and periodically written to the RTS
tables� SYSIBM.(SYS)TABLESPACESTATS for tables
� One row per table space/partition� SYSIBM.(SYS)INDEXSPACESTATS for indexes
� One row per index space/partition� Tables reside in a segmented table space (� PBG)
(DSNRTSTS/SYSRTSTS)� Table spaces reside in the DSNRTSDB/DSNDB06 database
Real-Time Statistics (RTS) basics
© 2009 SOFTWARE ENGINEERING GMBH 5
Object name Description
DSNRTSDB/DSNDB06 Database for Real-Time Statistics objects
DSNRTSTS/SYSRTSTS Table space for Real-Time Statistics objects
SYSIBM.(SYS)TABLESPACESTATS Table for statistics on table spaces and table space partitions
SYSIBM.(SYS)INDEXSPACESTATS Table for statistics on index spaces and index space partitions
SYSIBM.TABLESPACESTATS_IX Unique index on SYSIBM.(SYS)TABLESPACESTATS(columns DBID, PSID, and PARTITION)
(SYSIBM.DSNRTX01 DBID, PSID, PARTITION and INSTANCE)
SYSIBM.INDEXSPACESTATS_IX Unique index on SYSIBM.INDEXSPACESTATS(columns DBID, ISOBID, and PARTITION)
(SYSIBM.DSNRTX02 DBID, ISOBID,PARTITION and INSTANCE)
Real-Time Statistics (RTS) basics
© 2009 SOFTWARE ENGINEERING GMBH 6
How are they activated?
RTS processing will always happen in DBM1 address space… , but externalization has to be activated
� “Enabling” RTS prior to DB2 9� Create objects via DSNTESS � Explicit START DATABASE(DSNRTSDB) enables Real-Time
Statistics externalization
� “Enabling” RTS with DB2 9� DB2 9 already provides them
Real-Time Statistics (RTS) basics
© 2009 SOFTWARE ENGINEERING GMBH 7
How are they accessed/processed?
� DB2 itself updates the RTS through non-SQL procedures
� RTS tables can be queried by standard SQL
� RTS tables can be processed by IBM supplied stored procedure� DSNACCOR/DSNACCOX
- Allows to narrow scope on utility and object (-type)- Checks RTS based on predefined formulas- Recommends objects for REORG, RUNSTATS, COPY- Exception table for general excludes
Real-Time Statistics (RTS) basics
© 2009 SOFTWARE ENGINEERING GMBH 8
� Working with RTS� DBM1 allocates RTS blocks for each updated object
� At first update since the pageset/partition is opened� RTS runs in DBM1 address space
� CPU time is included in DBM1's SRB time� The system task is created during START DB2
� Free RTS blocks when� Pagesets/Partitions are closed� After statistics are written to RTS tables
� In a data sharing system, statistics are collected by each member
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 9
� Affecting RTS� CREATE/DROP table space� Insert / Update / Delete� Massdelete only updates massdelete counters� Rollback� REORG - online REORG triggers changes during Log Apply� RUNSTATS� COPY� LOAD � REBUILD - sets REORG counters to zero� RECOVER - sets utility timestamps to null� Trigger may cause statistics updated for other tables
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 10
� Externalizing the RTS� RTS manager externalizes in-memory statistics to the RTS
Tables on a timer interval� STATSINT in ZPARM - default 30 minutes (DB2 9 15 min.)� REAL TIME STATS in DSNTIPO install panel
� STOP DATABASE SPACENAM command� Flush in-memory statistics for all target objects
� STOP DATABASE(DSNRTSDB)� Flush all in-memory statistics
� STOP DB2 MODE(QUIESCE)� A utility operation (e.g. LOAD, REORG, RUNSTATS, COPY,
REBUILD, RECOVER)
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 11
� No externalization of the RTS� STOP DB2 MODE(FORCE)� Utilities that have a RTS-object in the utility list� limited for TEMP and Work Files Database� Read only objects� DEFINE NO objects � at the Tracker Site
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 12
Columns of SYSIBM.TABLESPACESTATS (DB2 7 and 8)DBNAME CHAR( 8) NOT NULL, NAME CHAR( 8) NOT NULL, PARTITION SMALLINT NOT NULL, DBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULT,TOTALROWS FLOAT , NACTIVE INTEGER , SPACE INTEGER , EXTENTS SMALLINT , LOADRLASTTIME TIMESTAMP , REORGLASTTIME TIMESTAMP , REORGINSERTS INTEGER , REORGDELETES INTEGER , REORGUPDATES INTEGER , REORGUNCLUSTINS INTEGER , REORGDISORGLOB INTEGER , REORGMASSDELETE INTEGER , REORGNEARINDREF INTEGER , REORGFARINDREF INTEGER , STATSLASTTIME TIMESTAMP , STATSINSERTS INTEGER , STATSDELETES INTEGER , STATSUPDATES INTEGER , STATSMASSDELETE INTEGER , COPYLASTTIME TIMESTAMP , COPYUPDATEDPAGES INTEGER , COPYCHANGES INTEGER , COPYUPDATELRSN CHAR(6) FOR BIT DATA,
COPYUPDATETIME TIMESTAMP
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 13
Columns of SYSIBM.SYSTABLESPACESTATS (DB2 9)UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULT,NACTIVE INTEGER,NPAGES INTEGER,EXTENTS SMALLINT,LOADLASTTIME TIMESTAMP,REORGLASTTIME TIMESTAMP,REORGINSERTS INTEGER,REORGDELETES INTEGER,REORGUPDATES INTEGER,REORGDISORGLOB INTEGER,REORGUNCLUSTINS INTEGER,REORGMASSDELETE INTEGER,REORGNEARINDREF INTEGER,REORGFARINDREF INTEGER,STATSLASTTIME TIMESTAMP,STATSINSERTS INTEGER,STATSDELETES INTEGER,STATSUPDATES INTEGER,STATSMASSDELETE INTEGER,COPYLASTTIME TIMESTAMP,COPYUPDATEDPAGES INTEGER,COPYCHANGES INTEGER,COPYUPDATELRSN CHAR(6) FOR BIT DATA,COPYUPDATETIME TIMESTAMP,IBMREQD CHAR(1) NOT NULL,DBID SMALLINT NOT NULL,PSID SMALLINT NOT NULL,PARTITION SMALLINT NOT NULL,INSTANCE SMALLINT NOT NULL WITH DEFAULT 1,SPACE BIGINT,TOTALROWS BIGINT,DATASIZE BIGINT,UNCOMPRESSEDDATASIZE BIGINT,DBNAME VARCHAR(24) NOT NULL,NAME VARCHAR(24) NOT NULL
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 14
Columns of SYSIBM.INDEXSPACESTATS (DB2 7 and 8)DBNAME CHAR( 8) NOT NULL, INDEXSPACE CHAR( 8) NOT NULL, PARTITION SMALLINT NOT NULL, DBID SMALLINT NOT NULL, ISOBID SMALLINT NOT NULL, PSID SMALLINT NOT NULL, UPDATESTATSTIME TIMESTAMP NOT NULL WITH DEFAULTTOTALENTRIES FLOAT , NLEVELS SMALLINT , NACTIVE INTEGER , SPACE INTEGER , EXTENTS SMALLINT , LOADRLASTTIME TIMESTAMP , REBUILDLASTTIME TIMESTAMP , REORGLASTTIME TIMESTAMP , REORGINSERTS INTEGER , REORGDELETES INTEGER , REORGAPPENDINSERT INTEGER , REORGPSEUDODELETES INTEGER , REORGMASSDELETE INTEGER , REORGLEAFNEAR INTEGER , REORGLEAFFAR INTEGER , REORGNUMLEVELS INTEGER , STATSLASTTIME TIMESTAMP , STATSINSERTS INTEGER , STATSDELETES INTEGER , STATSMASSDELETE INTEGER , COPYLASTTIME TIMESTAMP , COPYUPDATEDPAGES INTEGER , COPYCHANGES INTEGER , COPYUPDATELRSN CHAR(6) FOR BIT DATA, COPYUPDATETME TIMESTAMP
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 15
Columns of SYSIBM.SYSINDEXSPACESTATS (DB2 9)UPDATESTATSTIMETIMESTAMP NOT NULL WITH DEFAULT,NLEVELS SMALLINT,NPAGES INTEGER,NLEAF INTEGER,NACTIVE INTEGER,SPACE INTEGER,EXTENTS SMALLINT,LOADRLASTTIME TIMESTAMP,REBUILDLASTTIMETIMESTAMP,REORGLASTTIME TIMESTAMP,REORGINSERTS INTEGER,REORGDELETES INTEGER,REORGAPPENDINSERT INTEGER,REORGPSEUDODELETES INTEGER,REORGMASSDELETEINTEGER,REORGLEAFNEAR INTEGER,REORGLEAFFAR INTEGER,REORGNUMLEVELS INTEGER,STATSLASTTIME TIMESTAMP,STATSINSERTS INTEGER,STATSDELETES INTEGER,STATSMASSDELETES INTEGER,COPYLASTTIME TIMESTAMP,COPYUPDATEDPAGES INTEGER,COPYCHANGES INTEGER,COPYUPDATELRSN CHAR(6) FOR BIT DATA,COPYUPDATETIME TIMESTAMP,IBMREQD CHAR(1) NOT NULL,DBID SMALLINT NOT NULL,ISOBID SMALLINT NOT NULL,PSID SMALLINT NOT NULL,PARTITION SMALLINT NOT NULL, INSTANCE SMALLINT NOT NULL WITH DEFAULT 1,TOTALENTRIES BIGINT, DBNAME VARCHAR(24) NOT NULL,NAME VARCHAR(128)NOT NULL,CREATOR VARCHAR(128)NOT NULL,INDEXSPACE VARCHAR(24) NOT NULL,LASTUSED DATE
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 16
� Real-Time Statistics are not used by the DB2 optimizer
� Real-Time Statistics are intended to provide youmost current object statistics on demand without anyoverhead
� Use RTS for threshold based database maintenance
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 17
RTS pertaining to backups:
� COPYLASTTIMEThe timestamp of the last full or incremental image copy on the table space or partition.
� COPYUPDATEDPAGESThe number of distinct pages that have been updated since the last COPY.
� COPYCHANGES The number of insert, delete, and update operations since the last COPY.
� COPYUPDATELRSN The LRSN or RBA of the first update after the last COPY.
� COPYUPDATETIME The timestamp of the first update after the last COPY.
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 18
DSNACCOR/DSNACCOX formulas to determine maintenance candidates
� Full COPY on table spaces� Full COPY on index space� Incremental COPY on table space� REORG on table space� REORG on index space� RUNSTATS on table space� RUNSTATS on index space� Extensive extents� Objects with exceptions (e.g. COPYPEND)
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 19
Execution within your job scheduler
Evaluation oftable spacesand indexes
Tailoringof utility JCL
Thresholds
RTS integration for the batch maintenance cycle
Start
QUIESCEREORGCOPY
MDB2ALTX MDB2SDB2
MDB2RUN
other
ALTER + REORGgenerates
Delete Copy Datasets
alters IX COPY YES/NO and generates
MODIFY MERGECOPY
MDB2RB MDB2MO MDB2MC
executes REBINDs
generates
MDB2LSTAMDB2MD
Delete Copy Datasets
generates checks the COPY statusEnd
generates
analyzes space usage
inserts
RTSRTS
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 20
Since object characteristics can be queried on demand, nowRTS even opens up
Real Time Maintenance on Demand
Goals:� Increased business agility� Runtime reduction for maintenance tasks� Real-time statistics also for volatile and clone objects� Less costs for gathering statistics
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 21
level 1cyclic
level 2re-activ
level 3automated
level 4on demand
level 5autonomic
staticmaintenance
jobs for allobjects
manualjob
building
totalmaintenance,
manualexecution
generatedmaintenance
jobs forspecificobjects
manualjob
building
selectivemaintenance,
manualexecution
generated,threshold
basedmaintenance
jobsautomated,thresholdbased jobgeneration
selectivemaintenance,
staticexecution
dynamicmaintenancejobs based onreal-time data
continuous,thresholdbased Jobgeneration
selectivemaintenance,
automatedexecution
dynamicmaintenance
integratedvia SLA‘s
continuous,DB2
basedgeneration
selectivemaintenanceautomatedexecution
pro
ced
ure
op
erat
ion
exec
uti
on
manual intervention
DB2 Database Maintenance – past, present, future
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 22
On Demand database maintenance benefits
� 24 x 7 data availability – even if database maintenance is required (RUNSTATS, COPY, REORG)
� Best availability and best performance� Immediately pinpointing urgent maintenance requirements� Direct execution of important Utility tasks
� faster Recovery� better performance (RUNSTATS, REORG on Indexes)� better load balancing making use of low workload periods� less “classic” batch window requirements
Real-Time Statistics (RTS) advanced
© 2009 SOFTWARE ENGINEERING GMBH 23
DB2 CatalogStatistics
(Optimizer-)RUNSTATS
BIND
SELECT …FROM …WHERE …
ExecutableCode
Optimizer
RTS vs. DB2 Catalog Statistics
Historically we have two typesof RUNSTATS:
1. Optimizer RUNSTATS to improve performance
© 2009 SOFTWARE ENGINEERING GMBH 24
DB2 CatalogStatistics
(Optimizer-)RUNSTATS
BIND
SELECT …FROM …WHERE …
ExecutableCode
REBIND
Optimizer Optimizer
RTS vs. DB2 Catalog Statistics
Historically we have two typesof RUNSTATS:
1. Optimizer RUNSTATS to improve performance
© 2009 SOFTWARE ENGINEERING GMBH 25
DB2 CatalogStatistics
(Analytical-)RUNSTATS
Utility Decision(REORG, COPY etc)
RTS vs. DB2 Catalog Statistics
Historically we have two typesof RUNSTATS:
1. Optimizer RUNSTATS to improve performance
2. Analytical RUNSTATS to analyze maintenance needs
© 2009 SOFTWARE ENGINEERING GMBH 26
DB2 CatalogStatistics
(Analytical-)RUNSTATS
Real-TimeStatistics
Utility Decision(REORG, COPY etc)
Utility Decision(REORG, COPY etc)
RTS vs. DB2 Catalog Statistics
With RTS we now have two repositoriesRUNSTATS � DB2 catalog RTS � RTS objects
© 2009 SOFTWARE ENGINEERING GMBH 27
Real-TimeStatistics
Utility Decision(REORG, COPY etc)
DB2 CatalogStatistics
(Analytical-)RUNSTATS
Utility Decision(REORG, COPY etc)
RTS vs. DB2 Catalog Statistics
� Stop using catalog statistics for analytical RUNSTATS
� Use Real-Time Statistics for accurate statistics
� Save CPU! Eliminate analytical RUNSTATS
� Eliminate the “lag” time
� Secure your optimizer base
© 2009 SOFTWARE ENGINEERING GMBH 28
DB2 CatalogStatistics
(Optimizer-)RUNSTATS
BIND
SELECT …FROM …WHERE …
ExecutableCode
REBIND Real-TimeStatistics
Optimizer Optimizer
Utility Decision(REORG, COPY etc)
RTS vs. DB2 Catalog Statistics
© 2009 SOFTWARE ENGINEERING GMBH 29
DB2 CatalogStatistics
(Optimizer-)RUNSTATS
Mini-BIND
SELECT …FROM …WHERE …
DynamicStatement
Cache
Optimizer
RTS vs. DB2 Catalog Statistics
© 2009 SOFTWARE ENGINEERING GMBH 30
0
5
10
15
20
25
30
Runtime in h forgenerationbased onanalyticalRUNSTATSRuntime in h forgenerationbased on RTS
RTS vs. DB2 Catalog Statistics
Resulting savings from exploiting RTS
� RTS fully eliminates the need for analytical RUNSTATS
© 2009 SOFTWARE ENGINEERING GMBH 31
0
500
1000
1500
2000
2500
3000
3500
Number ofRUNSTATSbefore
No RUNSTATSneeded withRTS
0
5
10
15
20
Duration ofRUNSTATS in hbefore
Duration ofRUNSTATS withRTS
RTS vs. DB2 Catalog Statistics
Resulting savings from exploiting RTS
� RTS reduces the resource requirements to determineobjects that require database maintenance to zero
© 2009 SOFTWARE ENGINEERING GMBH 32
Resulting savings from exploiting RTS
If you haven’t exploited threshold based maintenance already,the savings will be even more significant
- e.g. Full/incremental copies based on update rates- 77% elapsed time reduction- 74% CPU reduction
- e.g. RUNSTATS based on update rates- 83% elapsed and CPU reduction
RTS vs. DB2 Catalog Statistics
© 2009 SOFTWARE ENGINEERING GMBH 33
RTS good practice� Avoid Timeouts or Deadlocks with RTS manager� Use Uncommitted Read lock isolation when accessing RTS
tables� For disaster recovery
• Recover RTS objects after DB2 Catalog and Directory objects are recovered
• Explicitly issue START DATABASE(DSNRTSDB) after RTS objects are recovered
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 34
Omit weak points when enabling RTS� Validate table space, table and index definitions for orphans,
widows and null stats and inconsistencies� Data may not be accurate until a REORG/RUNSTATS/COPY
is done
V7: all values are null*V8/V9: all counters are zero (only updated if timestamp not null)
all timestamps are null*except REORGLASTTIME (creation timestamp)
* REORG, RUNSTATS, COPY required
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 35
RTS initialisation
e.g. UPDATE SYSIBM.TABLESPACESTATS
SET REORGLASTTIME = ‘0001-01-01-00.00.00.000000’,REORGINSERTS = 0,REORGINSERTS = 0,REORGINSERTS = 0,REORGMASSDELETE = 0
WHERE REORGLASTTIME IS NULL;UPDATE SYSIBM.TABLESPACESTATS A
SET TOTALROWS(SELECT B.CARDF FROM SYSIBM.SYSTABLEPART B
WHERE A.DBNAME = B.DBNAMEAND A.NAME = B.TSNAMEAND A.PARTITION = B.PARTITION)
WHERE A.TOTALROWS is NULL;
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 36
Other things to consider
� All in-memory statistics are lost in case of a DB2 abend� Unable to capture statistics when DSNRTSDB is stopped or
statistics tables are unavailable� Statistics could be inaccurate if running 3rd party vendor utilities
without flushing the in-memory statistics� Some stats may not be valid in a utility restart scenario� Use SHRLEVEL CHANGE when running utilities against RTS� Don't mix RTS objects with other user objects in a utility list operation
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 37
RTS Benefits:
� Current statistics without the need for expensive RUNSTATS� Current object characteristics that can be used to trigger utilities
threshold based � New opportunities to do database maintenance in dynamic 24x7
environments� Statistics for volatile tables without compromising access paths� Provides individual data for clone and base table� RUNSTATS only required for the DB2 optimizer (BIND/REBIND)� Easy access via SQL or DSNACCOR/DSNACCOX
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 38
Will LEO be back once upon a time?
DB2 Vnext goals
1. Save CPU time
2. Reduce disk space
3. Reduce software costs
4. Improve business agility
5. Improve productivity
Tips and Tricks
© 2009 SOFTWARE ENGINEERING GMBH 39
Questions?