Migrating to a new db2 LUW version the hard way

By reading just the title I'm sure that lots of you are thinking why people just cannot do things in an easy and well ordered fashion ... the way it issupposed to be. Well, sometimes there is just no way around.

Maybe explaining why will set things more in perspective: living in acountry where three languages are spoken, makes sorting chars achallenge. You would want to have names - whether they are Dutch,French or German - to be sorted the way your customer wants to seethem on his reports and this without losing any performance.

You should know that once you have created your database, the choice inwhich things are sorted is made for good and that flipping on anyconfiguration switch is not going to help you changing all that. Sure, youcan change your sorting by adding clauses to each and every query, butbe assured that sorting will happen by using algorithms which is alwaysmuch slower than every other option. It is just fine for testing purposesand that's the end of that discussion.

So the hard way ... which means – in short and by far the complete list:

-  generate ddl-  arrange new disk space-  prepare the new database with the new codepage to get the sorting

right-  arrange downtime-  export data with a script invoking HighPerformanceUnload or when

the downtime has to be kept to the minimum, set up a replication ofthe data by using Change Data Capture

-  import data-  check integrity-  point all software to the new database-  put the old database out of commission and free up the discs

… and repeat this cycle for every of your databases – in our case we had800 databases spread over the multiple environments – and you're done.We had to come up with a strategy that would avoid failure and since wehad a team of thirteen all migrations had to be done in one and exactlyone standardized way. Besides that we to bypass all of the flaws we foundin the tools we used:

a)  db2look didn't deliver ddl on which we could blindly count on as theorder of creating elements isn't always correct. To give one example– think of the order of creation of tables and views or storedprocedures. Remove database objects and grants that weren’tneeded anymore as a second example. For the databases that we

 

were obligated to use CDC generated columns should be differentlydefined and foreign keys shouldn’t be enforced at the targets’ side.Once all data replicated the current definitions would get restored.

b) HighPerformanceUnload wouldn't always export data in the form oflobs or xml. Most of the times the starting point of a migration wasan isolated online database and in other cases an offline backup ofthe database on disc and HPU couldn’t export the data in a correctmanner at first.

c)  there is nothing out there that can generate a script to load any kindof data into any kind of table (think of generated always columns,lobs, xml, ...) ... let alone load it in parallel

d) once all data loaded how are you going to get your tables out of'check integrity' state? In which order are you going to checkintegrity for your tables?

We came up with some Korn shell scripts that helped us to stand up andface all of the previously described problems. By declaring a few Unixenvironment settings on both the source and target server the scriptswhere given directions on how to proceed.A first script generated a ddl on which we could count on for 95% of ourcases. We had to interact anyway on those databases where e.g.federation was activated as passwords get blanked out (which is ok!). Asecond script executed the export and generated a load script suited foreach type of table or column for a database. A third script, now on thetarget system, broke the monolithic load script into pieces so that we wereable to load the data in parallel. A fourth script to check the numbers ofexported rows per table against the number of loaded rows and to searchfor any other kind of error. Finally a fifth script figuring out how to get thetables checked on their referential integrity.Now we still needed one Word document to rule them all. This documentdescribed the procedure that the other team members had to follow andgave the team directions on how to migrate every database in the sameway.

The effort that it took to get such a smooth migration? One year of shellprogramming, a number of pmr's and testing the sequence of actions tocome up with one procedure for all cases. One month and a fewadjustments later the majority of the databases was migrated from a v9.1with collating sequence UCA400 to a v9.5 UCA500R1.

Eddy Coppens 

SuadaSoft