1

DB2z LOG RBA Reset

Jay Seiler(Systems Engineer) Jseiler@LLBean.comLaurie Pratt(DBA) Lpratt@LLBean.com

L.L. Bean, Inc

2

Agenda

• LLBean overview• LLBean DB2 history• LLBean’s DB2z current environment• The problem• Planning• Testing• Implementation• Questions L.L. Bean, Inc

3

LLBean Overview

• Founded in 1912 by Leon Leonwood Bean• Our 100th anniversary this year !• Headquartered in Freeport, Maine• L.L.Bean began as a one-room operation selling a

single product, the Maine Hunting Shoe • We have grown to a global, multi-channel

organization

L.L. Bean, Inc

4

LLBean Overview

• L.L.Bean products are rigorously tested, guaranteed to last and always shipped free; trusted source for quality apparel, reliable outdoor equipment and expert advice for 100 years

• Our 220,000 sq. ft. Flagship campus of stores in Freeport is open 24 hours a day, 365 days a year and welcomes more than three million visitors each year

L.L. Bean, Inc

5

LLBean Overview• Winner of National Retail Federation’s “Top 10

Customers' Choice Award”; rated in the top 3 for past 4 years; nominated once again this year

• L.L.Bean products are recognized around the world for their practical design and high quality; many items have been in the product line for several decades

L.L. Bean, Inc

6

LLBean DB2 History

• 1991 Brought in DB2z v2.3 • 1996 Launched LLBean website (DB2 LUW) • 1996 Implemented Data Sharing• 2003 Implemented a distributed front end gui

client accessing DB2z data via stored procedures for our customer service reps to take orders on the phones and handle telephone inquiries

• 2006 Eliminated Data Sharing • 2010 migrated to v9.1 L.L. Bean, Inc

7

LLBean’s DB2z Current Environment

• Two subsystems in our Production environment–DBQ1: originally intended as decision support and it is

small. –DBP1: large, 3TB

• V9.1 nfm/non data sharing• Virtual tape for image copies• Had a z9 (2674 mips) at start of planning,

but a z10 (same mips) by the time we did the RBA Reset L.L. Bean, Inc

8

The Problem

• On March 8, 2011, from the console:*DSNJ032I -DBP1 DSNJW307 WARNING - APPROACHING END OF 761THE LOG RBA RANGE OF FFFFFFFFFFFF. CURRENT LOG RBA IS F00035BCE000. DSNJ033E -DBP1 DSNJW307 THE LOG RBA MUST BE RESET 762BEFORE THE END OF THE LOG RBA RANGE IS REACHED

• This was the beginning of our journey

L.L. Bean, Inc

9

How long do we have before the end of the log RBA is reached?

• Looks like more than a year based on our processing rates in March: FFFF00000000 (high log RBA)

minus F065441C3F42 (current RBA) equals F99BBE3C0BE

• Divided by the average RBA increase per a day in March equals 397 DAYS

• Divided by the average RBA increase per a day in our peak (busiest) season equals 78 DAYS

• The decision was made to start planning the RBA Reset immediately

L.L. Bean, Inc

10

Planning

L.L. Bean, Inc

LOG RBA Reset process for non data sharing environment

IBM link to their documentation on the process:http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db29.doc.admin/db2z_subsystemendrba.htm

In a nutshell:• Use IDCAMS to delete and redefine the table spaces SYSUTILX, SYSCOPY,

and SYSLGRNX and their corresponding indexes. Then, re-initialize these page sets.

• APAR PK28576 – new DSNZPARM, SPRMRRBA, to enable special processing to reset the RBAs in the DB2 data for non data sharing DB2 subsystems using DB2’s COPY utility.

• Cold Start the subsystem back to an RBA value of 0• Image copy all tablespaces and indexes with zparm SPRMRRBA on• Stop DB2, Turn off zparm SPRMRRBA, Start DB2 for normal access

11

L.L. Bean, Inc

12

RBA vs. LRSN

• RBA–Relative Byte Address–Used by non data sharing subsystem

• LRSN– Log Record Sequence Number–Used by data sharing subsystems

L.L. Bean, Inc

13

Planning Continued

• IBM was contacted and provided great support:• information from other companies that had done an RBA

Reset• Bank of Italy’s plan

• Using DIAGNOSE TYPE(576) in the image copy for indexes which reduces what is written for an image copy dataset

• Formalized as a project with a project manager, systems programmer, DBA and a DBA who knew DPROP• Attended March NEDB2UG. John Iczkovits talked about LBI

(large block interface) for image copy datasets. Never benchmarked, but did use this for the image copies. L.L. Bean, Inc

14

Planning Continued

• Looked for obsolete tables to drop• Looked for tables where purges could be more aggressive•Used real time stats to determine how to break up image

copy jobs. • Turned COPY YES on for all indexes• Created validation queries to determine if any tablespaces

or indexes had been missed during the image copy process• Planned backout for RBA implementation; this would use

SRDF (mirrored data)L.L. Bean, Inc

15

DataPropagator(DPROP)aka: Infosphere Replication Server

• DataPropagator captures inserts/updates/deletes from the log to be able to propagate the data changes to other systems. DataPropagator keeps track of the last RBA propagated to know where to start in the log when DPROP is stopped and started.

• This added time to our outage to insure that DPROP was caught up with the log when we shut down DB2 and making sure that DPROP worked appropriately when DB2 was restarted for normal access after the RBA Reset.

L.L. Bean, Inc

16

Testing

• Tested on 2 non prod subsystems• Only one non prod subsystem had DataPropagator; needed

to test with DPROP twice• Did a dry run just image copying all page sets in production to

help determine how many jobs could run concurrently

L.L. Bean, Inc

17

Implementation• Sat • 11pm Brought down prod systems• Once everything was down, broke the SRDF links• 11:45pm DBA made sure DPROP was completely caught up• midnight Started image copies

It took 1.5 hours to do the image copies • Sun • 1:30am Restarted DB2 for normal access,

Validation, cold start of DPROP and DPROP steps• 4:30am Developers ran batch jobs to insert/delete/update• 4:45am Go decision was made, Reconnect SRDF,

Bring up prod servers & apps• 6:30am Call centers online• 8am Batch started

L.L. Bean, Inc

18

Questions?

Thank you.

L.L. Bean, Inc