48
Waits Defined Kyle Hailey http://perfvision.com

Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Embed Size (px)

Citation preview

Page 1: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Waits Defined

Kyle Hailey

http://perfvision.com

Page 2: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.2Top 36 Foreground Waits19. write complete waits 20. library cache lock21. SQL*Net more data from dblink22. log file switch (checkpoint incomplete)23. library cache load lock24. row cache lock25. local write wait26. sort segment request27. process startup28. unread message29. file identify30. pipe put31. switch logfile command32. SQL*Net break/reset to dblink33. log file switch (archiving needed)34. Wait for a undo record35. direct path write (lob)36. undo segment extension

1. db file sequential read2. log file sync3. db file scattered read4. latch free5. enqueue6. SQL*Net more data from client7. direct path read8. direct path write9. buffer busy waits10. SQL*Net more data to client11. log buffer space12. log file switch completion13. library cache pin14. SQL*Net break/reset to client15. io done16. file open17. free buffer waits18. db file parallel read

minus OPS, RAC, PQ, Resource Mgr http://www.oraperf.com

Page 3: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.3

Tuning Methodology

MachineRun queue (CPU)

reduce CPU usage or add CPUs

Paging Reduce memory usage or add memory

Oracle Waits >> CPU

Tune waits We are going to We are going to concentrate here on concentrate here on WAITSWAITS

Page 4: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.4

For Analysis CPU Time

For comparision with Wait Time

Wait Time Top Waits

If Wait Time >> CPU Time, need top waits

Wait Arguments p1, p2, p3For detailed analysis of some waits

SQL that caused waitsFor detailed analysis of some waits

Page 5: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.5

Wait and CPU Data Sources

I StatspackTop 5 Timed Events

II 10g ASHOEM

ASH Report : ashrpt.sql

Custom queries on v$active_session_history

III V$session & v$session_wait Custom queries

Page 6: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.6

I Statspack

Look at Top 5 Timed Events

Top 5 Timed EventsTop 5 Timed Events~~~~~~~~~~~~~~~~~~ % Total~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Call TimeEvent Waits Time (s) Call Time-------------------------------------------- ------------ ----------- ----------------------------------------------------- ------------ ----------- ---------buffer busy waits 2,748 250 78.72buffer busy waits 2,748 250 78.72CPU time 32 10.16CPU time 32 10.16free buffer waits 1,588 15 4.63free buffer waits 1,588 15 4.63write complete waits 10 8 2.51write complete waits 10 8 2.51log buffer space 306 5 1.51log buffer space 306 5 1.51

Top 5 Timed EventsTop 5 Timed Events~~~~~~~~~~~~~~~~~~ % Total~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Call TimeEvent Waits Time (s) Call Time-------------------------------------------- ------------ ----------- ----------------------------------------------------- ------------ ----------- ---------buffer busy waits 2,748 250 78.72buffer busy waits 2,748 250 78.72CPU time 32 10.16CPU time 32 10.16free buffer waits 1,588 15 4.63free buffer waits 1,588 15 4.63write complete waits 10 8 2.51write complete waits 10 8 2.51log buffer space 306 5 1.51log buffer space 306 5 1.51

Page 7: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.7

II OEM 10g Performance Tab

Page 8: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.8

II OEM 10g Zoom-In

Page 9: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.9

II Top Activity Page

Page 10: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.10

II 10g v$active_session_history

The best source Only in 10g Can simulate in any version since v7

Page 11: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.11

II Querying ASH

select select event as "TOP 5 Timed Events", event as "TOP 5 Timed Events", cnt as "Count" from (cnt as "Count" from ( select decode(ash.session_state,select decode(ash.session_state, 'ON CPU','CPU',en.name) event ,'ON CPU','CPU',en.name) event , count(decode(ash.session_state,count(decode(ash.session_state, 'ON CPU','CPU',en.name)) cnt'ON CPU','CPU',en.name)) cntfrom from v$active_session_history ash,v$active_session_history ash, v$event_name env$event_name enwhere en.event# (+) = ash.event# where en.event# (+) = ash.event# and SAMPLE_TIME > sysdate - (5/(24*60)) and SAMPLE_TIME > sysdate - (5/(24*60)) group by decode(ash.session_state,'ON CPU','CPU',en.name)group by decode(ash.session_state,'ON CPU','CPU',en.name)order by count(decode(ash.session_state,order by count(decode(ash.session_state, 'ON CPU','CPU',en.name)) desc'ON CPU','CPU',en.name)) desc) where rownum < 6;) where rownum < 6;

select select event as "TOP 5 Timed Events", event as "TOP 5 Timed Events", cnt as "Count" from (cnt as "Count" from ( select decode(ash.session_state,select decode(ash.session_state, 'ON CPU','CPU',en.name) event ,'ON CPU','CPU',en.name) event , count(decode(ash.session_state,count(decode(ash.session_state, 'ON CPU','CPU',en.name)) cnt'ON CPU','CPU',en.name)) cntfrom from v$active_session_history ash,v$active_session_history ash, v$event_name env$event_name enwhere en.event# (+) = ash.event# where en.event# (+) = ash.event# and SAMPLE_TIME > sysdate - (5/(24*60)) and SAMPLE_TIME > sysdate - (5/(24*60)) group by decode(ash.session_state,'ON CPU','CPU',en.name)group by decode(ash.session_state,'ON CPU','CPU',en.name)order by count(decode(ash.session_state,order by count(decode(ash.session_state, 'ON CPU','CPU',en.name)) desc'ON CPU','CPU',en.name)) desc) where rownum < 6;) where rownum < 6;

Top 5 Timed Events CountTop 5 Timed Events Count

------------------------------------ --------------------------------------------- ---------

log file parallel write 8log file parallel write 8CPU 7CPU 7control file parallel write 3control file parallel write 3null event 3null event 3log file sync 2log file sync 2

Top 5 Timed Events CountTop 5 Timed Events Count

------------------------------------ --------------------------------------------- ---------

log file parallel write 8log file parallel write 8CPU 7CPU 7control file parallel write 3control file parallel write 3null event 3null event 3log file sync 2log file sync 2

Page 12: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.12

II Querying ASH for P1,P2,P3

SID EVENT P1 P2 P3SID EVENT P1 P2 P3

--- ----------------------- --- --- ------ ----------------------- --- --- ---

236 db file scattered read 3 35291 16236 db file scattered read 3 35291 16

245 db file scattered read 1 13646 2245 db file scattered read 1 13646 2

248 read by other session 1 13646 1248 read by other session 1 13646 1

245 db file scattered read 1 8457 7245 db file scattered read 1 8457 7

248 db file scattered read 1 42226 8248 db file scattered read 1 42226 8

245 db file sequential read 1 13591 1245 db file sequential read 1 13591 1

SID EVENT P1 P2 P3SID EVENT P1 P2 P3

--- ----------------------- --- --- ------ ----------------------- --- --- ---

236 db file scattered read 3 35291 16236 db file scattered read 3 35291 16

245 db file scattered read 1 13646 2245 db file scattered read 1 13646 2

248 read by other session 1 13646 1248 read by other session 1 13646 1

245 db file scattered read 1 8457 7245 db file scattered read 1 8457 7

248 db file scattered read 1 42226 8248 db file scattered read 1 42226 8

245 db file sequential read 1 13591 1245 db file sequential read 1 13591 1

select select

session_id, session_id,

event,event,

p1, p2, p3p1, p2, p3

from v$ashfrom v$ash

select select

session_id, session_id,

event,event,

p1, p2, p3p1, p2, p3

from v$ashfrom v$ash

Page 13: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.13

II ASH Report ASH report @?/rdbms/admin/ashrpt.sql Pick interval over last 7 days !

1) General info

2) Top User Events ***

3) Top Background Events

4) Top Event P1/P2/P3 Values

5) Top Service/Module

6) Top Client IDs

7) Top SQL Command Types

8) Top SQL Statements ***

1) General info

2) Top User Events ***

3) Top Background Events

4) Top Event P1/P2/P3 Values

5) Top Service/Module

6) Top Client IDs

7) Top SQL Command Types

8) Top SQL Statements ***

9) Top SQL using literals

10) Top Sessions ***

11) Top Blocking Sessions

12) Top Sessions running PQs 

13) Top DB Objects

14) Top DB Files

15) Top Latches

16) Activity Over Time ***

9) Top SQL using literals

10) Top Sessions ***

11) Top Blocking Sessions

12) Top Sessions running PQs 

13) Top DB Objects

14) Top DB Files

15) Top Latches

16) Activity Over Time ***

Page 14: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.14

III V$session_wait

select

decode(w.wait_time, 0, w.event , 'CPU') as "TOP 5 Timed Events”,

count(*) from v$session s,

v$session_wait w

where w.sid=s.sid

and s.status='ACTIVE'

and s.type='USER'

and w.event not in ('jobq slave wait',

'rdbms ipc reply')

group by

decode(w.wait_time, 0, w.event , 'CPU')

order by count(*) desc;

select

decode(w.wait_time, 0, w.event , 'CPU') as "TOP 5 Timed Events”,

count(*) from v$session s,

v$session_wait w

where w.sid=s.sid

and s.status='ACTIVE'

and s.type='USER'

and w.event not in ('jobq slave wait',

'rdbms ipc reply')

group by

decode(w.wait_time, 0, w.event , 'CPU')

order by count(*) desc;

Moment in Time data

Page 15: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.15

III V$session_wait

Top 5 Timed Events Top 5 Timed Events COUNT

----------------------------------- ----------

enq: US – contention 4

CPU 3

buffer busy waits 1

latch: row cache objects 1

Top 5 Timed Events Top 5 Timed Events COUNT

----------------------------------- ----------

enq: US – contention 4

CPU 3

buffer busy waits 1

latch: row cache objects 1

Page 16: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.16

III V$session_wait

col status for a35 select s.sid, s.sql_hash_value, decode(w.wait_time, 0, w.event , 'CPU') as status, w.p1, w.p2, w.p3from v$session s, v$session_wait w where w.sid=s.sid and s.status='ACTIVE' and s.type='USER' and w.event not in ('jobq slave wait', 'rdbms ipc reply');

col status for a35 select s.sid, s.sql_hash_value, decode(w.wait_time, 0, w.event , 'CPU') as status, w.p1, w.p2, w.p3from v$session s, v$session_wait w where w.sid=s.sid and s.status='ACTIVE' and s.type='USER' and w.event not in ('jobq slave wait', 'rdbms ipc reply');

Page 17: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.17

III V$session_wait

SID SQL_HASH STATUS P1 P2 P3

---------- -------------- -------------------------------- ---------- ---------- ----------

234 82347421 CPU 1431502854 39 0

235 3336613934 enq: US - contention 1431502854 44 0

236 1772152815 enq: US - contention 1431502854 42 0

238 2750335498 enq: US - contention 1431502854 44 0

240 343101472 enq: US - contention 1431502854 44 0

246 1782401401 enq: US - contention 1431502854 44 0

248 3333220954 CPU 1650815232 1 0

252 323960517 enq: US - contention 1431502854 44 0

260 1272059733 CPU 1431502854 44 0

SID SQL_HASH STATUS P1 P2 P3

---------- -------------- -------------------------------- ---------- ---------- ----------

234 82347421 CPU 1431502854 39 0

235 3336613934 enq: US - contention 1431502854 44 0

236 1772152815 enq: US - contention 1431502854 42 0

238 2750335498 enq: US - contention 1431502854 44 0

240 343101472 enq: US - contention 1431502854 44 0

246 1782401401 enq: US - contention 1431502854 44 0

248 3333220954 CPU 1650815232 1 0

252 323960517 enq: US - contention 1431502854 44 0

260 1272059733 CPU 1431502854 44 0

Moment in Time data

Page 18: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.18

Waits we will Ignore

Background Idle RAC Resource Manager

Page 19: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.19

Background Waits

Filter Out Background Waits

Statspack ASH : SESSION_TYPE='FOREGROUND‘ V$session_wait : type='USER'

Page 20: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.20

Background Waits

ASH Avoid Background waits in ASH with

V$session_wait joined to v$session

Select …from v$active_session_history where SESSION_TYPE='FOREGROUND'

Select …from v$active_session_history where SESSION_TYPE='FOREGROUND'

select …from v$session s, v$session_wait w where w.sid=s.sid and s.type='USER'

select …from v$session s, v$session_wait w where w.sid=s.sid and s.type='USER'

Page 21: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.21

Idle Waits

Filter Out 10g

where wait_class != ‘Idle’Create a list

9iCreate a list with

Documentation List created from 10g

Select name from v$event_name where wait_class=‘Idle’;

Select name from v$event_name where wait_class=‘Idle’;

Page 22: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.22

Parallel Query Waits

Filter Out Parallel Query Wait events are unusable

Save waits are both idle and waits

Parallel Query Waits start with ‘PX’ or ‘KX’PX Deq: Par Recov ReplyPX Deq: Parse Reply

Page 23: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.23

RAC Waits

You are on your own Check documentation If you are not using RAC then no worries 10g

9i RAC and OPS waits usually contain the word “global”

Select event from v$event_name where wait_class=‘Cluster’;

Select event from v$event_name where wait_class=‘Cluster’;

Page 24: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.24

Resource Manager Waits

Resource manager throttles userCreates waitObfuscates problems

10g

select name from v$event_name where wait_class='Scheduler';

select name from v$event_name where wait_class='Scheduler';

Page 25: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.25

Wait Arguments: P1,P2,P3

Each Wait has a 3 parameters P1,P2,P3 Give detailed information Meaning different for each wait Meaning definitions in V$event_name

Select

name,

parameter1,

parameter2,

parameter3

from v$event_name;

Page 26: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.26

Wait Arguments ExampleNAME PARAMETER1 PARAMETER2 PARAMETER3----------------------------------- --------------- --------------- ---------------

latch: cache buffers chains address number triesfree buffer waits file# block# set-id#buffer busy waits file# block# class#latch: redo copy address number trieslog buffer spaceswitch logfile commandlog file sync buffer#db file sequential read file# block# blocksenq: TM - contention name|mode object # table/partitionundo segment extension segment#enq: TX - row lock contention name|mode usn<<16 | slot sequencerow cache lock cache id mode requestlibrary cache pin handle address pin address 100*mode+namesplibrary cache load lock object address lock address 100*mask+namesppipe put handle address record length timeout

Page 27: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.27

Waits

I/O

Library Cache

Locks

Undo

Redo

Buffer Cache

SQL*Net

Wait Tree

Page 28: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.29Top 12 Waits NAME Count % Total

1. db file sequential read 23,850.00 11.67%2. log file sync 20,594.00 10.08%3. db file scattered read 15,505.00 7.59%4. latch free 11,078.00 5.42%5. enqueue 7,732.00 3.78%6. SQL*Net more data from client 7,510.00 3.67%7. direct path read 5,840.00 2.86%8. direct path write 4,868.00 2.38%9. buffer busy waits 4,589.00 2.25%10. SQL*Net more data to client 3,805.00 1.86% 11. log buffer space 2,990.00 1.46% 12. log file switch completion 2,878.00 1.41%

Above is over 80% of wait times reported

Anjo Kolk www.oraperf.com

Page 29: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.30

Top 36 Foreground Waits19. write complete waits 20. library cache lock21. SQL*Net more data from dblink22. log file switch (checkpoint incomplete)23. library cache load lock24. row cache lock25. local write wait26. sort segment request27. process startup28. file identify29. pipe put30. switch logfile command31. SQL*Net break/reset to dblink32. log file switch (archiving needed)33. Wait for a undo record34. direct path write (lob)35. undo segment extension36. undo segment tx slot

1. db file sequential read2. log file sync3. db file scattered read4. latch free5. enqueue6. SQL*Net more data from client7. direct path read8. direct path write9. buffer busy waits10. SQL*Net more data to client11. log buffer space12. log file switch completion13. library cache pin14. SQL*Net break/reset to client15. io done16. file open17. free buffer waits18. db file parallel read

minus OPS, RAC, PQ, Resource Mgr

Page 30: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.31

Waits 1-9

1. db file sequential readTune SQL, speed up disks (5-15ms), increase buffer cache

2. log file syncCommit less, put redo logs on faster disks

3. db file scattered read FTS - Tune SQL, add indexes, speed up disks (5-15ms)

4. latch free – need p1,p2,p35. enqueue – need p1,p2,p3 and SQL6. SQL*Net more data from client

Usually OK, reduce data transferred, possible Network problems

7. direct path readsorts or PQO - tune IO, sort less

8. direct path writedirect path load or temp io, improve disk speed

9. buffer busy waits – need p1,p2,p3 and SQL

Page 31: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.32

Waits 10-18

10. SQL*Net more data to clientUsually OK, reduce amount of data transferred, possible Network tuning needed

11. log buffer spaceIncrease log buffer

12. log file switch completionIncrease log file sizes

13. library cache pinReduce parsing the same cursor concurently

14. SQL*Net break/reset to clientCheck for errors in sql statement

15. io doneIo done by IO slaves, oracleIxxx could also be an idle wait – sort of worthless

16. file openReduce logons/logoffs or keep log files open

17. free buffer waitsIncrease buffer cache size

18. db file parallel readTune sql, tune io subsystem, increase buffer cache

Page 32: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.33

Waits 19-2719. write complete waits

Increase buffer cache (or tune DBWR)20. library cache lock

Avoid hard parsing same cursor at same time21. SQL*Net more data from dblink

Reduce data transfer, check net response22. log file switch (checkpoint incomplete)

Add log files (or increase log file size)23. library cache load lock

Avoid hard parsing same cursor at same time24. row cache lock

need p125. local write wait

truncates / reduce cache size26. sort segment request

SMON busy, process rollback problem27. process startup

If MTS, increase min servers and dispatchers in init.ora

Page 33: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.34

Waits 28-36

28. file identifyKeep log files open, reduce checkpoints

29. pipe putSpeed up pipe readers

30. switch logfile commandAvoid switching log files

31. SQL*Net break/reset to dblinkCheck for errors in sql statement sent

32. log file switch (archiving needed)Archive log running out of space

33. Wait for a undo record ??

34. direct path write (lob)Improve IO, reduce lob write size

35. undo segment extensionUse UNDO or with RBS, increase RBS size, avoid OPTIMAL

36. undo segment tx slotUse UNDO, increae # of RBS segs

Page 34: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.35

Difficult Waits

Latches Locks Buffer Busy Row Cache Lock

Page 35: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.36

Concepts: Latches

Protect areas of memory from concurrent use Light weight locks

Usually a held in a bit in memoryOften supported by atomic processor callFast and cheapGone if memory is lost

Often used in cache coherency managementChanges to a datablock

Exclusive Generally Sharing reading has been introduced for some latches

Page 36: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.37

Finding Latches

“latch free” Covers many latches, find the problem latch by

1. select name from v$latchname where latch# = p1; OR

2. Find highest sleeps in Statspack latch section

In 10g, important latches have a wait event latch: cache buffers chains latch: shared pool

latch: library cache

Page 37: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.38

Latches : Find Highest Sleeps

Use Statspack (or AWR report)

Latch Name Get Requests Misses SleepsSleeps Spin Gets Sleep1

------------------- -------------- ----------- ---------------------- ---------- -----

shared pool 3,044,462 184,517 19,09819,098 169,226 0

library cache 2,242,805 79,554 15,19815,198 66,497 0

kks stats 263,464 2,842 1,7741,774 1,152 0

row cache object 434,844 955 147147 816 0

library cache lock 666,668 734 5454 680 0

library cache pin 741,963 635 4242 593 0

enqueues 133,577 373 40 333 0

enqueue hash chains 133,751 89 19 70 0

Latch Name Get Requests Misses SleepsSleeps Spin Gets Sleep1

------------------- -------------- ----------- ---------------------- ---------- -----

shared pool 3,044,462 184,517 19,09819,098 169,226 0

library cache 2,242,805 79,554 15,19815,198 66,497 0

kks stats 263,464 2,842 1,7741,774 1,152 0

row cache object 434,844 955 147147 816 0

library cache lock 666,668 734 5454 680 0

library cache pin 741,963 635 4242 593 0

enqueues 133,577 373 40 333 0

enqueue hash chains 133,751 89 19 70 0

Page 38: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.39

Important Latches Cache Buffers Chains

Hot blocks, distribute activity, reduce update time, avoid “select for update”

Cache Buffers LRU Chain Too much activity , uses multiple buffer caches and/or increse LRU latches

Shared Pool Too much hard parsing, too small a shared pool

Library Cache Latch Hard Parsing

Use bind variables Uses cursor_sharing=force Increase shared pool size if reloads are high Avoid invalidations

Soft Parsing Hold cursors open Use session_cached_cursors Use cursor_space_for_time

Page 39: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.40

Concepts: Locks

“Enqueue” wait – covers all locks pre 10 Protect data against concurrent changes Lock info written into data structures

Block headersData blocksWritten in cache structures

Shareable in compatible modes

Page 40: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.41

Finding Locks Statspack not much help V$session_wait needs lots of decoding

P1 tells Lock Type and Mode P2,P3 give more data

Usually Need SQL to solve

select sid,

event,

chr(bitand(P1,-16777216)/16777215)||

chr(bitand(P1,16711680)/65535) as "Type",

mod(p1,16) as "mode"

from v$session_wait

where event = 'enqueue‘;

select sid,

event,

chr(bitand(P1,-16777216)/16777215)||

chr(bitand(P1,16711680)/65535) as "Type",

mod(p1,16) as "mode"

from v$session_wait

where event = 'enqueue‘;

SID EVENT Ty mode P2 P3

--- ------- -- ---- ------- ----

240 enqueue TX 6 2686995 433

SID EVENT Ty mode P2 P3

--- ------- -- ---- ------- ----

240 enqueue TX 6 2686995 433

Page 41: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.42

Locks 10g

10g breaks Enqueues out enq: HW - contention Configuration enq: TM - contention Application enq: TX - allocate ITL entry Configuration enq: TX - index contention Concurrency enq: TX - row lock contention Application enq: UL - contention Application

Page 42: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.43

Locks : TM & TX

select * from v$lock where type in ('TX', 'TM');

SID TY ID1 ID2 LMODE REQUEST CTIME BLOCK--- -- ---------- ---------- ----- ------- ----- -----151 TX 589855 1592 0 6 4049 0135 TM 53737 0 3 0 4058 0151 TM 53737 0 3 0 4049 0135 TX 589855 1592 6 0 4058 1

select * from v$lock where type in ('TX', 'TM');

SID TY ID1 ID2 LMODE REQUEST CTIME BLOCK--- -- ---------- ---------- ----- ------- ----- -----151 TX 589855 1592 0 6 4049 0135 TM 53737 0 3 0 4058 0151 TM 53737 0 3 0 4049 0135 TX 589855 1592 6 0 4058 1

Page 43: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.44

Buffer Busy Cases

Undo Header - Not enough rollback segments

Undo Block – hot spot in RBS

Data Block Table Block w/ DML : Table lacks Free lists Table I/O : Multiple sessions reading same block into data cache

Note: in 10g, becomes “read by other session” Index : has hot block, partition or change SQL

Segment Header – add free Lists (with Datablock) Free List – add free lists groups

Page 44: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.45

Buffer Busy Waits

Statspack

Buffer wait Statistics for DB: ORA9 Instance: ora9 Snaps: 2 -3

-> ordered by wait time desc, waits desc

Tot Wait Avg

Class Waits Time (s) Time (ms)

------------------ ----------- ---------- ---------

data block 1,490 2 1

undo header 2 0 0

Buffer wait Statistics for DB: ORA9 Instance: ora9 Snaps: 2 -3

-> ordered by wait time desc, waits desc

Tot Wait Avg

Class Waits Time (s) Time (ms)

------------------ ----------- ---------- ---------

data block 1,490 2 1

undo header 2 0 0

Page 45: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.46

Buffer Busy Solutions

IF Buffer Busy Waits highFirst look at v$waitstat

Class Time/Count

SQL> select * from v$waitstat;

CLASS COUNT TIME------------------ ---------- ----------data block 1 1segment header 0 0free list 0 0undo header 0 0undo block 0 0

SQL> select * from v$waitstat;

CLASS COUNT TIME------------------ ---------- ----------data block 1 1segment header 0 0free list 0 0undo header 0 0undo block 0 0

Page 46: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.47

Row Cache Lock

Need p1 to see the cache type

SQL> select cache#, parameter from v$rowcache; CACHE# PARAMETER---------- -------------------------------- 1 dc_free_extents 4 dc_used_extents 2 dc_segments 0 dc_tablespaces 5 dc_tablespace_quotas 6 dc_files 7 dc_users 3 dc_rollback_segments 8 dc_objects 17 dc_global_oids 12 dc_constraints

SQL> select cache#, parameter from v$rowcache; CACHE# PARAMETER---------- -------------------------------- 1 dc_free_extents 4 dc_used_extents 2 dc_segments 0 dc_tablespaces 5 dc_tablespace_quotas 6 dc_files 7 dc_users 3 dc_rollback_segments 8 dc_objects 17 dc_global_oids 12 dc_constraints

Page 47: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.48

Row Cache Lock Statspack

^LDictionary Cache Stats for DB: ORA9 Instance: ora9 Snaps: 1 -2->"Pct Misses" should be very low (< 2% in most cases)->"Cache Usage" is the number of cache entries being used->"Pct SGA" is the ratio of usage to allocated size for that cache

Get Pct Scan Pct Mod FinalCache Requests Miss Reqs Miss Reqs Usage----------------- --------- ------ ------- ----- -------- ----------dc_object_ids 45 0.0 0 0 958dc_objects 89 0.0 0 0 1,129dc_segments 69 0.0 0 0 807dc_tablespaces 12 0.0 0 0 13dc_usernames 22 0.0 0 0 19dc_sequences 120,003 0.0 0 120,003 5

^LDictionary Cache Stats for DB: ORA9 Instance: ora9 Snaps: 1 -2->"Pct Misses" should be very low (< 2% in most cases)->"Cache Usage" is the number of cache entries being used->"Pct SGA" is the ratio of usage to allocated size for that cache

Get Pct Scan Pct Mod FinalCache Requests Miss Reqs Miss Reqs Usage----------------- --------- ------ ------- ----- -------- ----------dc_object_ids 45 0.0 0 0 958dc_objects 89 0.0 0 0 1,129dc_segments 69 0.0 0 0 807dc_tablespaces 12 0.0 0 0 13dc_usernames 22 0.0 0 0 19dc_sequences 120,003 0.0 0 120,003 5

Page 48: Waits Defined Kyle Hailey . #.2 Copyright 2006 Kyle Hailey Top 36 Foreground Waits 19. 19.write complete waits 20. 20.library cache

Copyright 2006 Kyle Hailey

#.49

Areas of Contention

Buffer Cache Disk I/O Library Cache Redo SQL*Net Undo

We will go through these areas and the wait eventsthat are relevant to them.