PBR RPN & Other Availability Improvements in Db2 12 RPN and other availability... · PBR or PBR RPN •PI73421 •Allow externalization of RTS even if tablespace is in UTRW state

Session code:

PBR RPN & Other Availability Improvements in Db2 12

Haakon RobertsIBM

A11

07.11.2018 11:00-12:00 Platform: Db2 for z/OS

1

Disclaimer

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole

discretion. Information regarding potential future products is intended to outline our general product direction and it should not

be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a

commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products

may not be incorporated into any contract. The development, release, and timing of any future features or functionality described

for our products remains at our sole discretion.

Agenda

• Revisiting 10 byte log RBAs

• V12 Scalability

• PBR RPN

• Online schema

• Recent Developments

• Future Ideas

• Summary

10 byte log RBA revisited - background

• Prior to V11, RBAs and LRSNs were 6 bytes

• V11 supports 10 byte RBAs and LRSNs• LRSNs due to run out in 2042 if no delta

• RBA: • D34C4322BAEC converts to 00000000D34C4322BAEC

• LRSN• D34C4322BAEC converts to 00D34C4322BAEC000000

• 1 yottabyte

• Approx 30,000 more years for LRSN


• BSDSs converted using DSNJCNVT• Must be converted prior to migration to V12

• Pagesets may be converted to extended format• CREATE• REORG• LOAD REPLACE• REBUILD INDEX


• ZPARMs• OBJECT_CREATE_FORMAT• BASIC/EXTENDED• Default is BASIC

• UTILITY_OBJECT_CONVERSION• BASIC/EXTENDED/NONE/NOBASIC• Default is NONE (no conversion)

10 byte log RBA revisited - thresholds

• F00000000000• DSNJ032I issued every log switch or restart• Until BSDSs are converted

• FF8000000000• Soft update limit• Updates to basic format user pagesets fail -904 00C2026D

• FFFF00000000• DSNJ033E• Db2 shut down and only restartable in maint mode until BSDSs converted

• Or RBA is reset

• FFFFFFFFFFFF• Hard update limit• Updates to all basic format pagesets fail -904 00C2026E

Beware

• Customers convert the BSDSs but not any pagesets• No longer get DSNJ032I messages• No other messages issued until hit the FF8000000000 limit causing updates to fail and application outages• Then convert user pagesets, since no errors against catalog/directory• This accelerates logging until hit FFFFFFFFFFFF limit at which point no more conversions are possible because

conversions need to update cat/dir but cat/dir is in basic format

• Customers hitting limits convert to datasharing thinking that that will avoid the need to convert pagesets to extended format• If BSDS not converted, and RBA is FF8000000000 then LRSN will have delta applied to make LRSN

FF8000000000• This will give a max of 4 years 5 months before 6 byte LRSN runs out• If LRSN is higher, e.g. FFFF00000000, then this will give only 19 hours before 6 byte LRSN runs out• Converting BSDS at this stage will not help

• FFFF00000000 converts to 00FFFF00000000000000• Once we move to 01000000000000000000 then the pageset must be extended format to support this value

Recommendations

• Convert BSDSs and pagesets once in 11 NFM• No benefit to basic format• Converted BSDSs can save CPU• Converted pagesets can avoid LRSN spin in data sharing• Do not assume everything is fine just because you converted the BSDSs and no longer get DSNJ032I

• If you hit the FF0000000000 (or 00000000FF0000000000) limit, do not ignore the catalog and directory• Cat/dir pagesets MUST be converted before FFFFFFFFFFFF limit is hit

• What is Db2 doing to help?• All documentation updated to highlight the importance of conversion and in particular cat/dir conversion• New APAR will remove zparms so new objects will be extended format and pagesets will be converted to

extended format by utilities• New APAR will issue message to console once per life of a Db2 system when a basic format pageset is opened

• Note: In datasharing there is no need to convert all pagesets before going beyond RBA 00000000FFFFFFFFFFFF

Scalability

• Improved storage management for EDM pools• No longer fail if needed storage exceeds limit• EDMSTMTC – max EDM statement cache pool• EDMDBDC – Max DBD pool• EDM_SKELETON_POOL – Max skeleton pool

• Automatic management of LOB and XML pools• Removed LOBVALA/LOBVALS/XMLVALA/XMLVALS• Settings are 2Gb per user and 50Gb per subsystem

• Larger active log datasets• Max size increased from 4Gb to 768Gb• No change in max number of active log datasets

10

Partition Management – prior to V12

• DB2 page numbers are absolute

• 5 byte RIDs describe partition, page and row within page

• DSSIZE limits are dependent upon number of partitions, creating complex scalability constraints

• Alter of DSSIZE requires REORG of entire table space

• Running out of space in a partition causes an application outage and resizing may not be possible without dropping and recreating the table space

Partition Management - Relative Page Numbering

• Simplicity, usability, scalability, availability

• Support relative page numbers for range partitioning

• CREATE or ALTER TABLESPACE PAGENUM(RELATIVE)• Alter from either classic partitioned or PBR• No support for tablespaces with limitkeys truncated at 40 bytes• Must be fixed before converting to PBR RPN

• Deferred alter followed by materializing REORG

• Table space - level REORG required for conversion• No PIT recovery to prior to materializing REORG

• Separate page numbers from partition numbers


• Support partition-level DSSIZE up to 1Tb per part• Regardless of number of partitions or page size• Max table space size increase from 16Tb to 4Pb

• Online alter to increase DSSIZE - immediate

• Online alter to decrease DSSIZE - pending alter requiring table space -level REORG

• RID increase from 5 bytes to 7 bytes – requiring mapping table change• New mapping table format optional until new function enabled

• REORG requires part-level inline image copies

Sample CREATE TABLESPACE

CREATE TABLESPACE TSPBR IN DB1

PAGENUM RELATIVE

DSSIZE 8G

NUMPARTS 3

(PARTITION 1 DSSIZE 3G,

PARTITION 3 DSSIZE 9G);

14

PARTITION PAGENUM DSSIZE

1 R 3,145,728K

2 R 8,388,608K

3 R 9,437,184K

SYSTABLEPART

Sample ALTER TABLESPACE

ALTER TABLESPACE TSPBR IN DB1

ALTER PARTITION 1 DSSIZE 5G

ALTER PARTITION 3 DSSIZE 12G;

15

PARTITION DSSIZE

1 5,242,880K

2 8,388,608K

3 12,582,912K

SYSINDEXPART

Sample CREATE INDEX

CREATE INDEX IXPI ON TB1(COL1)

PARTITIONED

DSSIZE 1G;

16

PARTITION DSSIZE

1 1,048,576K

2 1,048,576K

3 1,048,576K

SYSINDEXPART

Sample ALTER INDEX

ALTER INDEX IXPI

ALTER PARTITION 1 DSSIZE 3G

ALTER PARTITION 3 DSSIZE 8G;

17

PARTITION DSSIZE

1 3,145,728K

2 1,048,576K

3 8,388,608K

SYSINDEXPART


• New format log records support relative page numbering for all page sets• Not just range partitioned• From before activation of new function onwards, so V11 products need

toleration code• PI34223 delivers DB2 11 support for DB2 12 log records

• Improved DSN1LOGP serviceability• New PART option, filtering & enhanced log record formatting

Insert Partition

• ALTER TABLE ADD PARTITION ENDING AT xxx

• Alternative syntax• ALTER TABLE ADD PARTITION ENDING AT xxx ALTER PARTITION n ENDING AT yyy

• Physical partition number is at the end

• Logical partition number based on position

• Attributes inherited from SYSTABLESPACE values• Or previous logical partition if SYSTABLESPACE values are null

• Pending change, affected partition placed in AREOR

• Only affected partition(s) need to be reorged

• PBR only

• No PIT recovery prior to materializing REORG

Online Schema

• ALTER INDEX COMPRESS YES places index in AREOR instead of RBDP• UTS table spaces only• Materialized through REORG INDEX or REORG TABLESPACE

• Support pending column-level ALTERs• Avoid availability constraints & conflict with other deferred alters• ALTER TABLE ALTER COLUMN SET DATA TYPE• Materialize changes through online table space – level REORG• Governed by DDL_MATERIALIZATION zparm

• ALWAYS_IMMEDIATE• ALWAYS_PENDING

• UTS only

• TRANSFER OWNERSHIP• Database objects

• DATABASE, INDEX, STOGROUP, TABLESPACE, TABLE, VIEW

REORG

• Fail REORG on Flashcopy failure if wholly reliant on inline FlashCopy• Previously left object in copy-pending, now leave object in RW• No need for –TERM UTIL• Enable full exploitation of Flashcopy in REORG

• Prevent copy-pending on LOB tablespace during REORG of PBG• Growth of new PBG partition during REORG LOG phase would leave new LOB

tablespace in copy-pending

Recovery

• Support recovery to PIT prior to materializing REORG for PBG table spaces• Equivalent to PBR support delivered in V11

• RECOVER now defaults to PARALLEL• Up to 35% ET improvement

• New RECOVER SCOPE UPDATED option for PIT recovery• Skip recovery for non-updated pagesets

• Retry auto-GRECP and auto-LPL recovery• Periodically up to 3 times based on internal timer• For indoubt or PA URs, LPL recovery attempts commence once indoubt or postponed

abort is resolved

General

• Better utilities concurrency• COPYTOCOPY, MODIFY RECOVERY, MERGECOPY now compatible with other utilities

• IFI306 records returned in correct version• Avoid need to REORG prior to enabling subscriptions after adding or altering columns

• Auto-bind releases resources prior to application commit• Retrofit to V11 in PI46379

• Single phase migration• No ENFM phase• New function activated through –ACTIVATE FUNCTION LEVEL command• APPLCOMPAT rules, fallback rules continue to apply

Recent Availability Enhancements

• Prevent RTS externalization from interfering with CATMAINT during migration• PI49628 (V11)

• Mark utility I/O lock as critical• Prevent DB2 failures when short on IRLM storage• PI81019 (V11)

• Allow update of partitioning key columns• For tables created prior to V5 APAR PQ16946• These columns are flagged as non-updatable• Convert them to be updatable during conversion to table-controlled partitioning or conversion to

PBR or PBR RPN• PI73421

• Allow externalization of RTS even if tablespace is in UTRW state• PI87403

24


• Improved support for ALTER ADD PARTITION in V11• Prior to PI42999

• If limitkey on last partition is MAXVALUE, then it must be lowered before adding new partition.• In 11, alter of limitkey is a deferred alter, placing partition in AREOR state• Attempt to ALTER TABLE ADD PARTITION fails since it is an immediate alter which is incompatible with

deferred alter• This forces a REORG of the last partition which can discard data records that need to be preserved then

loaded back into table• A later REORG is required after the new partition is added

• With PI42999• ALTER TABLE ALTER PARTITION limitkey is changed to be an immediate change if altering the last partition

and the high key is MAXVALUE (or MINVALUE)• The partition is left in REORP, allowing for ALTER TABLE ADD PARTITION and a single REORG to resolve

• Storage reduction for UNLOAD of XML data• >4Gb per utility down to 10s of Kb• PI84464

25

Shadow Target

Input data


• LOAD REPLACE SHRLEVEL REFERENCE

• LOAD utility pre-validation• Including CHECK constraints

• Support usual drain options, including SWITCHTIME

• Works for partition load also• Logical NPSI partitions left in RBDP• Like INDEXDEFER

• PI67793 & PI87474

• V12 APAR: PI69095


• Allow ROTATE PARTITION on MQTs or tables with MQTs defined• PH00194• Undo a restriction that was added by PI77487• Need to invalidate static packages for both ROTATE and alter of limitkeys on

MQTs

• Avoid package invalidation on PBR conversion• PH01819 (V12 only)

27


• Improved DDL break in on dynamic SQL• Serial nature of invalidation process in dynamic statement cache makes it

difficult for DDL to break in• PH00637 (V12 only)

28

Beforestmts /

uowddl

totalddlfail

sqltotal

sqlfail

time (min)

2 23 19 2522 407 106 17 14 2360 461 10

Afterstmts /

uowddl

totalddlfail

sql total

sql fail

time (min)

2 30 0 2440 600 106 30 0 1720 600 10

Future ideas

• Improved online schema processing

• More online schema changes

• Improved support for REORG part-level inline image copies to tape

Summary

• Continuing focus on improved availability & elimination of application impact

• More deferred ALTERs and materializing REORGs

• More granular processing and increased usability and flexibility in schema

• Improved and less disruptive autobind processing

• Less disruptive migrations and continuous delivery

Session code:

Please fill out your session

evaluation before leaving!

Haakon [email protected]

A11

31

Please fill out your session

evaluation before leaving!

Documents

PBR RPN & Other Availability Improvements in Db2 12 RPN and other availability... · PBR or PBR RPN •PI73421 •Allow externalization of RTS even if tablespace is in UTRW state