The Tao of DB2

The Tao of DB2Achieving Balance & Understanding with DB2

Craig S. MullinsMullins Consulting, Inc.http://www.CraigSMullins.com

Session: B8Wed, May 25, 2005 @ 8:30 AM - 9:40 AM

Platform: DB2 for z/OS

http://www.craigsmullins.com/

2

© 2005, Craig S. Mullins

The Tao of DB2Achieving Balance & Understanding with DB2

Listen, well, my intern and youtoo can achieve balance andunderstanding with DB2.

Will this take long?I already told ourmanager I know this stuff.

3


Tao Te Ching

• Ancient Chinese philosophy of life.• Written by Lao Tzu over 2000 years ago.• The Tao, or the Way, is a universal power through which all

life flows.• The Tao Te Ching proposes a model by which the individual

and society can embody a philosophy to achieve a harmonious and balanced existence.

• This presentation will attempt to apply some of the concepts of the Tao to DB2 and database administration.

4


Understanding the Tao

• What is Tao?• “The Way”

The way that can be spoken ofIs not the constant way;The name that can be named Is not the constant name.

5


Tao Terms

• Tao – a nonsense syllable; it represents the mystery we can never understand

• Li – the patterns of nature• Tzu-Jan – “by itself so”• Wu-Wei – not forcing• Te – virtue; skill at living

Understanding the Tao of DB2 and armed with database Te, the DBA uncovers the Li of the application programs and removes the SQL tweaks that were working against Wu-Wei and the system experiences a performance gain that is Tzu-Jan.

6


The DB2 that can be mastered, is notthe DB2 that runs on z/OS. One who excels at tending to his lack of understanding can begin to embrace the glory of DB2, and call himself DBA.

I do notunderstand…

The Taoist DBAThe Intern

7


Indeed it is long sincethe people have been perplexed.

-Lao Tzu

• “One could study for ten thousand years and travel the four corners of the globe and yet still be ignorant,” said the Taoist DBA.

• Lesson: DB2 is a large, complicated piece of systems software. No one can fully understand everything it is and can do.• When you think you understand DB2 a new version will come out

with new features and changes to the old ways.

• Always be open to learning; never assume you know everything.

The term Tao refers to the vast and great…

-Chu Hsi

8


Were it (Tao) definable, it could then beused to obviate conflict…

-Lao Tzu

• The users were howling, the system was not responding. “What should I do?” screamed the intern. The Taoist DBA pulled up his PDS member and pressed <ENTER>. “Oh, has this happened before?” asked the intern. “How many times?” said the DBA.

• Lesson: Minimize conflicts through written standards and goals; prepare for recurring problems with predefined scripts.• SLAs

• DBA standards

• Development and SQL standards

• Canned jobs

9


The more laws & restrictions there arethe poorer people become.

-Lao Tzu

• “I followed the corporate standardsto the letter, but it doesn’t work,” spatthe frustrated intern. “The best thingabout standards,” said the Taoist DBA,“is that there are so many of them fromwhich to choose.”

• Lesson: Although it is advisable to create DB2 standards, for every standard there is a reason to make an exception.

If nothing within you stays rigid,outward things will disclose themselves.

-Kuan-Yin

10


Straightforward words seem paradoxical… -Lao Tzu

Lesson: Sometimes the manuals are noteasy to understanduntil you get used tothe terminology &the writing style.• Do not panic.• Augment manuals with other material.• Learn by doing.

The term li includes the innumerable vein-like patterns in the Tao.

-Chu Shi

11


Having examined the appearance of the thing,one seizes upon its essence.

-Lao Tzu

• “The mouse in the trap squeals with pain,” explained the Taoist DBA, “not first examining the cheese for a spring.”

• Lesson: Know your data; monitor your systems.• Distribution

• Static vs. dynamic data

• Indexes

• Cardinality

• CPU usage• Buffer Pool efficiency; memory

usage• SQL: static & dynamic• System parameters

The sage…avoids extremes and excesses,and does not become complacent.

-Lao Tzu

12


So I must document, butremain flexible whilealways learning this thingI will never fully master?

The Taoist DBA

The Intern

You begin to grasp yet you are far away frombeing enlightened. I

cannot yet retire with a good conscience,

young intern. We must continue the lesson…

13


And so they begin to work on a requestfor a new database…

is rented under

reports to

completes

rents /is in

employed by

receives

makes

rents under

is made on

is rented as

CUSTOMERCustomerNumNameAddressPhone

CreditCardNumCreditCardExDateStatusCode

MOVIEMovieNumNameDirectorDescriptionStarRatingGenreRental Rate

STOREStoreNumManagerAddressPhone

EMPLOYEEEmployeeNumNameAddressPhoneSocialSecNumHireDateSalarySupervisor (FK)

PAYMENT

Payment Transaction NumTypeAmountPaymentDateStatus

MOVIE RENTAL RECORDRental Record DateRental DateDue DateRental StatusRental RateOverdue Charge Amt

MOVIE COPYMovie Copy NumGeneral Condition

JoinDate

We begin with the data

model…

14


What is springs from what is not. -Lao Tzu

• “Where do I begin,” asked the intern. The Taoist DBA replied, “Begin at the beginning… and continue working until you reach the end.”

• Lesson: Every database must begin its life as a well-designed data model. It must be physically implemented with an understanding of the capabilities of DB2.• An improperly designed DB2 database can cause horrible

performance problems.• From logical data model to physical implementation, all steps must be

planned and well thought-out for success to follow.• Errors in planning at the initial stages can result in disaster at later

stages, requiring re-design and/or re-writing code.

15


Lay plans for the accomplishment of the difficult before it becomes difficult; make something big by starting with something small.

-Lao Tzu

• “That programmer from accounting is asking for more changes; should I stop what I am doing to work on those?” grumbled the intern. “When it is too late to do it right, we must inevitably do it over,” said the Taoist DBA as he nodded noncommittally.

• Lesson: Take the time to plan the database implementation including both database design and program design. • If you proceed without knowing the requirements you will be

continually changing everything forever.

16


Thus Something and Nothing produce each other…

-Lao Tzu

• “Does this DDL look right,” asked the intern. “Can one know what one does not know? The vessel is not empty, nor is it full. We may even know not where the vessel is…” the Taoist DBA rather cryptically muttered.

• Lesson: Define each column carefully.• Must a value be known?

• NULL or NOT NULL

• Is there a valid default value for the column?

• What is the data? • Numeric, alphanumeric, date/time

17


Use what is naturally useful. -Chuang Tzu

• “I do not know which of these options to choose. Should I just use the defaults,” moaned the intern. “I’m for whatever is right,” exclaimed the Taoist DBA. “Choose wisely and we will not have this conversation again.”

• Lesson: Understand every option at your disposal before choosing any parameter. Use the one that best matches the needs of the database, the application, or the situation.• Avoid DB2’s defaults.

All things can do something, andcan equally succeed.

-Kuo Hsiang

18


Let everything be allowed to dowhat it naturally does,so that its nature will be satisfied.

-Ching Hao

• When the intern asked “Why will DB2 not allow SPACES in this DATE column,” the Taoist DBA smiled and tried to answer the intern in writing. But he wrote his answer with a Fig Newton.

• Lesson: No kludging; use the features of DB2 as they were designed and meant to be used. Examples:• Use DATE, TIME, and TIMSTAMP for chronological data• Use numeric data types for numbers, character data types for alphanumeric• Use referential constraints (declarative RI) and CHECK constraints instead of

using programs to enforce constraints• Use whatever number of the 60 buffer pools your implementation

may need (instead of only BP0)

20


Though adjoining states are within sight of one another…the people of one state will grow old and die without having had any dealings with those of another.

-Lao Tzu

• Confused, the intern asked “But when should I partition the data?” The Taoist DBA slapped the intern and asked if it would have been beneficial for a partition to have existed between them.

• Lesson: When the amount of data is so large as to have a reasonable expectation that you will need to administer it in “pieces” consider partitioning.• There is no “mythical” cutoff of pages/rows after which

you must partition.• Largest simple or segmented TS is 64 GB

• Largest partitioned TS is 128 TB.

21


When the consistency of the way is known, the mind is receptive to its states of change.

-Lao Tzu

• The intern asked “Why is my data corrupt?” and the Taoist DBA answers “It is always the drunkard who knows not why he is slumped on the floor.” Learning, the intern slowly realizes he has only himself to blame.

• Lesson: Database constraints improve data integrity. When RI and CHECK constraints are defined to the database then DB2 is receptive to changes in the state of the data and manages them appropriately.

Yield, and maintain integrity;Be whole and all things come to you.

-Lao Tzu

22


Therefore he discards the one and takes the other

-Lao Tzu

• “With all of these constraints, my database will never get out of sync, will it?” queried the intern. The Taoist DBA continued to balance his checkbook, looking for that missing penny.

• Lesson: Even with constraints sometimes our data will lack integrity (e.g.., Loading without enforcing constraints.)

• Be prepared to get things re-synchronized:• Used the CHECK utility with the DISCARD option to remove data

that does not match the constraint.

23


It is easy to maintain a situationwhile it is still secure.

-Lao Tzu

• The Taoist DBA stole the intern’s lunch money to teach him the need for security. “I am hungry,” said the intern. “Yet I am not,” said the Taoist DBA.

• Lesson: Understand the security and authorization needs of your DB2 databases and applications before you develop them, far in advance of moving to production.• Build programs with knowledge of DB2’s abilities.• Multi-level security (DB2 V8) can provide row-level security.• Views possibly can help, too.• Using secondary authorization Ids can greatly simplify your DB2

security implementation.• Cooperation with RACF/ACF2 security staff is essential.

24


To accept the irrevocableis to let go of desire.

-Lao Tzu

• The test system was hosed. The intern had revoked the authority of an application DBA who had quit last month. “Why can no one access their tables?” asked the intern. “Can you touch what is not there?” asked the Taoist DBA.

• Lesson: Be careful with DB2 security. When revoking an authorization GRANTed by a user having WITH GRANT OPTION, the REVOKE will cascade and also remove all the authority the user granted.• Consider disallowing WITH GRANT OPTION, especially in

production systems.

25


The database is built, tested,and ready to deploy… is our work done?

The Taoist DBA

The Intern

As the Winter approachesthe squirrel busies him-self gathering nuts for

the barren season. He is ready should misfortunebefall him… you are not.

26


Lay plans for the accomplishment of the difficult before it becomes difficult; make something big by starting with something small.

-Lao Tzu

• “Be ready,” said the Taoist DBA “for everything, because eventually everything will happen.” The intern wept.

• Lesson: Be sure to schedule appropriate backups for each and every table space; understand the volatility of the data and the type of recovery that may been needed.• Incremental versus full backups

• More than once-a-day, Daily, Weekly, Monthly, etc.

• Test the recoverability of your databases

27


When the people lack a sense of awe, then some awful visitation will descend upon them.

-Lao Tzu

• “This backup stuff is pretty easy,” said the intern. “I’m outta here,” was the sole reply of the Taoist DBA, and all that remained was his spinning chair and a cooling mug of coffee.

• Lesson: Keep your disaster recovery plan up-to-date and be sure to test it at least annually. Take nothing for granted.• Remember always that contingency planning and recovery from a

disaster is a complex and time-consuming endeavor.

…he is fit to lead who takes the country’s disasters upon himself.

-Lao Tzu

28


Achieve success, but without force. -Lao Tzu

• “I’ve heard people say ‘If it don’t fit, force it’,” said the intern. And his Taoist mentor told him to force his mouth shut.

• Lesson: Force fitting a resolution to a problem is not a wise approach for the DB2 professional.

• Example: Resolve the problem instead of forcing

-START DATABASE (DSN8D81A) SPACENAM (DSN8S81E) ACCESS (FORCE)

RECP run the RECOVER utilityCHKP run the CHECK utility

29


If you would have a thing shrinkyou first must stretch it.

-Lao Tzu

• “How much storage space is needed for this database,” asked the intern. “What amount of space is required to store sand?” replied the Taoist DBA. “How much sand – a grain or a whole beach?” asked the intern. “Exactly,” replied his mentor.

• Lesson: Plan for database growth. Understand how the business use of the database will cause data to expand or contract. Use DB2 features to your advantage.

• Free space – allow sufficient free space to grow• Compression – reduce the size of larger objects

30


Reduce the size and populationof the state.

-Lao Tzu

• Lesson: Prepare a data archival plan for each and every DB2 table in your system.• Less data in a table space can result in more efficient access; as old

data gets archived…• …there may be more relevant rows per page, improving read efficiency;• …table space scans will run faster;• …backup/recovery and REORG will run faster;• …less disk space may be needed.

• Archiving stale, or outdated, information can enhance the accuracy of database reporting and queries.

• Archived data can be kept on cheaper storage media.

• The plan may be No Archiving.

31


OK, proper backups arescheduled and we have a DR plan; it is even tested.Surely we are ready to move on?

The Taoist DBA

The Intern

Is the work of the bumble-bee ever completed? There is always more

nectar to gather.

Performance

32


Life is the blended harmonyof yin and yang.

-Chuang Tzu

• When the intern’s tuning attempt does not solve the problem, he consults the Taoist DBA for assistance. “You glue wings to your arms, yet you cannot fly.” Confused, the intern moped away, and the problem persisted.

• Lesson: Most performance problems are not solved on the first try; many performance problems require multiple things to be changed for results to be achieved.

• Example: Adding an index to optimize a query. If it is static, you have to REBIND. If not, you might also have to modify the query, too.

• DB2 Performance = System + Database + Application

33


It is supplied with all its needsand is constantly at easebecause it follows its own plan…

-Lao Tzu

• Lesson: The DB2 optimizer formulates access paths which it packages into plans and follows for optimum efficiency.• Build effective SQL that encompasses as many of the “needs” of the

application as possible; this eases performance tuning.

• As data patterns change and DB2 evolves new features, rebinding a new “plan” can quickly adapt the program to the data and updates.

• The novice, with his data access built into his program is ill at ease as it takes long hours to adapt to changes.

34


The unseen design of things is more harmonious than the seen.

-Heraclitus

• The intern requested of his teacher, “Teach me all the mysteries of the Optimizer.” The Taoist DBA replied “A mystery is named such because it cannot be totally understood.”

• Lesson: sometimes that which we cannot understand 100% is still well-designed and required.• No one (outside of IBM) fully understands every nuance of the

optimizer

• We can learn general rules, one of which is to entrust the optimizer to create more efficient access than we can write ourselves

• Guideline: Do as much work as possible within the SQL

• Guideline: DB2 code is more efficient than my/your code

35


• “The programmers continue to blame DB2 for their errors,” lamented the intern. “A monkey chatters until you give it a banana. Then the monkey chatters again tomorrow.”

• Lesson: Let DB2 be DB2… understand and use DB2 as it is meant to be used; educationis required.

• Every day is an opportunity to learn.• The parable of the overconfident programmer…

• To not learn from your experiences is torelegate one self to never understanding thenature of DB2

The more it works, the more comes out. -Lao Tzu

36


Without a door, the room cannot be entered,and without windows it is dark.Such is the utility of non-existence.

-Lao Tzu

• Lesson: Use the EXISTS construct for efficient existence checking. Example:• As long as Jones exists, the query succeeds; DB2 need not find all of

the Jones, just one (any one).

SELECT 1 FROM SYSIBM.SYSDUMMY A WHERE EXISTS (SELECT 1 FROM DSN8810.EMP B WHERE LASTNAME = ‘JONES’ AND A.IBMREQD = A.IBMREQD);

It is only by means of beingthat non-being may be found.

-Lao Tzu

37


Keep a thing in order before disorder sets in. -Lao Tzu

• “When shall we REORG the table space?” asked the intern.“When we must, and only then,” replied the Taoist DBA.

• Lesson: Create a plan for each DB2 object basedon an understanding of the activity that will be performed on it. Monitor DB2 Catalog statistics to determine when to reorganize.• Is the object static or dynamic?

• Static (no new rows) = no free space• Changes? Rare; Variable? No = infrequent REORG• Variable? Yes = more frequent REORG (compression?)

• How frequently does the object change?• Schedule REORG frequency appropriately

• Is downtime possible? • Online vs. offline REORG

• Modify the REORG schedule as access patterns change.

38


A small country may have many machines,but the people will have no use for them…

-Lao Tzu

• “Doesn’t a DBMS like DB2 mean that we have to buy more hardware to support it?” questioned the intern. “Only if you plan not to work,” replied the Taoist DBA curtly.

• Lesson: The well-tuned DB2 environment will not necessitate additional CPU power. • Use DB2 as it was meant to be used and it will be efficient.• Do not use DB2 like a flat file system – that means index the right

data, code the proper predicates, and join data as necessary. • Do not read every row into the program and check the data like you

would a master file. (VSAM vs. DB2)

• Do not needlessly limit DB2 (e.g. only 5 indexes per table).• Educate every DB2 user on proper SQL coding techniques.

39


In walking, just walk.In sitting, just sit.Above all, don’t wobble.

-Yun-Men

• The order entry system was down. The main table space needed to be recovered. As the Taoist DBA focused on his task his intern fidgeted, fretting about the managers milling around his office. “Is the system up and running?” The Taoist DBA presses <ENTER> and winks.

• Lesson: Focus and concentration save the day in stressful situations; do not panic and avoid worrying about non-essential things (e.g. other tasks, managers, etc.)• Repeat past successes.

• Automate as much as possible.

40


Do that which consists in taking no actionand order will prevail.

-Lao Tzu

• Continuing to teach his intern, the Taoist DBA whispers softly, “When all around you is in flux, sip your coffee and contemplate the situation. Not every problem requires a solution.”

• Lesson: Sometimes the proper course of action is to do nothing. This can be difficult to endure.• Some “problems” solve themselves.

(or perhaps the solution is worse than the problem)

Pure in heart, like uncut jade,he cleared the muddy waterby leaving it alone.

-Lao Tzu

41


Learn when to stop and youwill meet with no danger.

-Lao Tzu

• “The users request that we cancel one of their long-running jobs,” stated the intern. “And who will pay double to undo that damage?” was the lone question posed by the Taoist DBA.

• Lesson: Sometimes it is better to let a task run to completion than to stop it and incur the cost of backing out.• Always issue COMMITs in your batch jobs.

42


Change Management

A lot of things can gowrong to cause DB2 performance problems? I am sure, though, that there is more to learn.

The Taoist DBA

The Intern

You have learned much.Consider though, the

caterpillar and the butterfly;How can we plan for

such changes?

43


Through his experience,the sage becomes aware that all things change

-Lao Tzu

• “The users want to change the data type of several columns,” stated the intern. “The only constant is change,” chanted the Taoist DBA.

• Lesson: Be prepared to support many different types of change requests. Some will be simple, some complex. All require careful planning to implement.• Change management is not simple even with DB2 V8 and Online

Schema Change:• What can change using ALTER? What cannot?• Versions and performance

The difficult and the easycomplement each other.

-Lao Tzu

44


When considering any thing,do not lose its opposite.

-Lao Tzu

• “So, all I have to do is drop the table and re-create it with the new column in the middle?” asked the intern. “Yes, that is all,” replied the Taoist DBA, shaking his head sarcastically.

• Lesson #1: Dropping an object drops all related objects and authority. Understand the connectedness of all DB2 objects.

• Lesson #2: Changes may need to be undone; be ready. A DBA must be ready to undo anything he does – and then redo it again.• Seriously consider a change management tool that will keep track of

changes for you and automate the process.• Without such a tool you will need to manually track and

store your DROP and CREATE statements somehow.

45


This interaction of yin and yang is called the Way, and the resulting creative processis called change.

-I Ching

• Realization dawning upon him, the intern mutters “Dealing with change seems like it will be a big part of my job.” With a grin, the Taoist DBA replies, “You have no idea how true that is.”

• Lesson: A DBA must be prepared to implement changes to database structures, but also to react to the changes made around the DBMS.• Most performance problems are caused because something changed –

but finding out what changed can be a daunting task.

46


DBA Guidelines

Is there yet more wisdom you can share with me?

The Taoist DBA

The Intern

That you are wise enough to ask speaks volumesabout your progress…

47


Do not attempt to be a particular type of person, but be many types simultaneously.

-Chuang Tzu

• When the intern balked at helping with a WebSphere problem his Taoist mentor told him “When you say it is not your job, soon it will not be your job.”

• Lesson: A good DBA must be a jack-of-all-trades. Expanding your knowledge and sphere of influence is always a good thing.

48


To know, yet to think that one does not know is best; not to know, yet to think that one knows will lead to difficulty.

-Lao Tzu

• “Oh, I’ve seen this problem before. All I have to do is to add an index like before and…” said the intern excitedly. “To a hammer all problems look like nails,” replied the Taoist DBA.

• Lesson: Do not rush to a solution. Take in all there is to learn and though you should rely on your past experience, do not force fit that experience to every problem.• Every day is an opportunity to learn something new.

• Humility is a quality that DBAs would do well to exhibit.

49


One who excels in employing others humbles himself before them…this is known as making use of the efforts of others.

-Lao Tzu

• “I feel like I have to know everything,” complained the intern. “Someone has to,” laughed the Taoist DBA.

• Lesson: The job of a DBA is complex and you cannot know everything. Learn from others and use the knowledge of others to succeed. • Learn who knows what within your company.

• Other DBAs, network folks, system programmers, application developers, transaction folks, web masters, etc.

• Network with other DBAs in your city/area.

• Network with others using the web, e-mail, etc.

50


Administrators who are wisedo not seek honors for themselves.

-Lao Tzu

• “Surely it must be annoying to answer the same question multiple times a day?” said the intern, quite smugly. “Hmmm,” said the Taoist DBA, “how many times have I answered that?”

• Lesson: A DBA must be ready to assist without wanting to bask in glory. Solving problems is more important than who caused them, and even who solved them.

With wise administrators,all can exist in unity

-Lao Tzu

51


My words are very easy to understand andvery easy to put into practice, yet no one can understand them or put them into practice.

-Lao Tzu

Though the words of the sage are simple,and his actions easily performed,they are few among many,who can speak or act as a sage.

-Lao Tzu

In this way he accomplishesthat which is required of him;without dwelling upon it in any way,he gives of his wisdom without display.

-Lao Tzu

52


Governing a large stateis like boiling a small fish.

-Lao Tzu

• Squinting confusedly at his mentor the intern mumbled “…but a fish has nothing to do with my job.” The Taoist DBA corrected his intern saying “There are lessons to be learned everywhere and from everything.”

• Lesson: Don’t fix what isn’t broken… • A small fish can be spoiled simply by handling it.

So can a well-tuned DB2 application.

53


People often fail on the verge of success.By giving as much care to the end as to the beginning, there will be few failures.

-Lao Tzu

• “I feel my confidence rising,” said the intern, “but I am frustrated that I am not yet a master.” The Taoist DBA winked and said “Then you know how it feels.”

• Lesson: Persistence is a quality that all successful DBAs possess. Perplexed by a problem, it is always that next “thing” that will solve it… and that is a good quality.

54


Go with the flow of things, and you will find yourself at one with the mysterious ways of the universe.

-Chuang Tzu

• “As a final lesson, let me tell you that you will learn to be ‘one with DB2’ as you apply your knowledge and these lessons to the job at hand. Don’t panic, use your training, and the Tao of DB2 will flow through you…” were the next to last words the Taoist DBA had to say to the intern.

55


To retire when the task is accomplishedis the way of heaven.

-Lao Tzu

You are now on the path toDB2 enlightenment… now

I can retire in peace.

Before you go,what is your home

phone number?

Ahhh, truly you are now ready… alas, I am unlisted.The Retired DBA

The Taoist Intern

58


The Tao of DB2Session: B8

Craig S. MullinsMullins Consulting, Inc.

15 Coventry Court

Sugar Land, TX 77479

http://www.CraigSMullins.com

[email protected]

Technology

The Tao of DB2