ASH – Active Session History Feel the Power Kyle Hailey

ASH – Active Session History

Feel the Power

Kyle Haileyhttp://perfvision.com

#.2

Copyright 2006 Kyle Hailey

ASH Whole New Paradigm in

Technology

The Power of ASH lies in

Simplifying Performance Tuning

Totally new and exciting methodology Cheaper, Quicker, Richer and better tasting too

#.3


Why should you care?

Because ASH can Change your life …

10g immediately Accessible Geeks: Via scripts in SQL Managers : Graphical OEM

Stuck on Oracle 7,8,9 ?… my apologies because 10g rocks, but …no worries, the data is accessible for you too via scriptshttp://perfvision.com/sash.php

#.4


Here’s a piece of Personal history

#.5


This is the story of how ASH

changed my Life

Once upon a time I was stuck in dull tech support …

Finally I was promoted to head DBA, thinking my life was getting better…

#.6


But … It was then that I learned I needed to know what the

databases were doing ALL the time

Because lives and more importantly my job depended on it

#.7


What could I do?!

Clearly a job for Stats Pack

Because stats pack showed me every hour

All the Stats! All the top Bottlenecks Almost all the top SQL

#.8


So why wasn’t I feeling better?

Hmmm … but … wait

What if my database hits a bottleneck 15 minutes after my last stats pack?!

What about that rouge coder who writes bad SQL … can I find his module gone mad?

Ok I found the top SQL but … now what? What do I do with the top WAIT?

#.9


If lives (and my job) depended on my databases

Was once an hour really enough ? Could Stats Pack find that rogue coder? Could Stats Pack tell me why an SQL

bottlenecked? Could Stats Pack tell me the root of a wait

bottleneck?

#.10


Don’t tell anyone but …

I couldn’t even decipher Stats Pack half the timeOften, I didn’t understand statspack’s 1000+ linesOften, I didn’t understand statspack’s 1000+ lines

#.11


Oh no … it’s 3am my manager calls

“why is the database hanging?!” Who did it? Where did the SQL get blocked? What if the bottlenecked started 5 minutes

after my last Stats Pack?

#.12


In that moment of crises Stats Pack couldn’t tell

me…

but ASH can

#.13


The crises of my day have left the “ASHes” of

today.

I didn’t have ASH butYou do

#.14


It’s a Revolution and it’s an Evolution

Oracle 6 … ie the dark ages … there was once the Cache Buffer Hit Ratio

Oracle 7 … turned the lights on … Wait Events … hallelujah I can see the light

Oracle 10g … ASH has landed

(like the wheels on the suitcases)

#.15


ASH uses Sampling New Paradigm

Clear Vision of the future Use new sampling technology Lets go of the need to know 100% Statistical approximation

Cheaper Richer Multidimensional Data Identifies

SQL Sessions Objects Machine User

#.16


To identify Players before ASH

Sessions# sessions x (# wait events + statistics)Example (150 x (800+200) = 150,000 )

SQLv$sql

Could be 10000s Takes out latches that compete with other sql executions

ObjectsV$segstat 9i+

Could be 1000s of objects

Expensive !

#.17


ASH – Intelligence for the new Millennium

Intelligently Collects Data It self adjusts for your needs More activity, more data collected Less activity, less data collected

Those old methods collected everything Obfuscated the problem, too many statistics too late Costly Too Granular – once an hour ?! Give me a break

ASH Samples Session State

TIME?? ?? ?? ?? ??

Sessions change a lot quicker but can get the main picture via sampling by sampling faster

Every second Every second

10:00:00 10:00:01 10:00:02 10:00:03 10:00:04 10:00:05

#.19


Sampling is like taking Pictures

#.20


If happens a lot or for long … we’ll catch it, guaranteed

#.21


Session States

IO CPU IdleWait

Work Work LatencyLatencyContention Contention

#.22


IDLE

Example : SQL*Net Message from Client 10g

9i or below – just have to keep trackStatspack STATS$IDLE_EVENT

select name

from v$event_name

where wait_class='Idle';

58 Rows

#.23


CPU

ASH: SESSION_STATE = “ON CPU” ASH: wait_time > 0

10g: v$session

9i: v$session_wait

w.wait_time != 0 /* on CPU */ and

s.status='ACTIVE' /* ACTIVE */

Identifying CPU via v$session_wait is Identifying CPU via v$session_wait is a huge break througha huge break through

#.24


IO ASH: SESSION_STATE=‘WAITING’

&

10g: WAIT_CLASS=‘User I/O’ 9i:

Look at 10g for ideas db file sequential read db file scattered read db file parallel read direct path read

#.25


WAITING ASH: SESSION_STATE=‘WAITING’ ASH: WAIT_TIME <= 0 WAIT_CLASS

1. Administrative2. Application3. Cluster4. Commit5. Concurrency6. Configuration7. Network8. Other9. Scheduler10. System I/O

800+ WAIT

Session 1

Session 2

Session 3

Session 4

TIME

Samples for all users

10:15:00 10:15:01 10:15:02 10:15:03 10:15:04 10:15:05 10:15:06 10:15:07

#.27


ASH FieldsSQL> v$active_session_history Name Null? Type

----------------------------------------- -------- ----------------------------

SAMPLE_ID NUMBER SAMPLE_TIME TIMESTAMP(3) SESSION_ID NUMBER SESSION_SERIAL# NUMBER USER_ID NUMBER SQL_ID VARCHAR2(13) SQL_CHILD_NUMBER NUMBER SQL_PLAN_HASH_VALUE NUMBER SQL_OPCODE NUMBER SERVICE_HASH NUMBER SESSION_TYPE VARCHAR2(10) SESSION_STATE VARCHAR2(7) QC_SESSION_ID NUMBER QC_INSTANCE_ID NUMBER EVENT VARCHAR2(64) EVENT_ID NUMBER EVENT# NUMBER SEQ# NUMBER P1 NUMBER P2 NUMBER P3 NUMBER WAIT_TIME NUMBER TIME_WAITED NUMBER CURRENT_OBJ# NUMBER CURRENT_FILE# NUMBER CURRENT_BLOCK# NUMBER PROGRAM VARCHAR2(48) MODULE VARCHAR2(48) ACTION VARCHAR2(32) CLIENT_ID VARCHAR2(64

#.28


v$active_session_history

SESSION_ID NUMBER SESSION_SERIAL# NUMBER USER_ID NUMBER SERVICE_HASH NUMBER SESSION_TYPE VARCHAR2(10) PROGRAM VARCHAR2(64) MODULE VARCHAR2(48) ACTION VARCHAR2(32) CLIENT_ID VARCHAR2(64)

EVENT VARCHAR2(64) EVENT_ID NUMBER EVENT# NUMBER SEQ# NUMBER P1 NUMBER P2 NUMBER P3 NUMBER WAIT_TIME NUMBER TIME_WAITED NUMBER CURRENT_OBJ# NUMBER CURRENT_FILE# NUMBER CURRENT_BLOCK# NUMBER0

SQL_ID VARCHAR2(13) SQL_CHILD_NUMBER NUMBER SQL_PLAN_HASH_VALUE NUMBER SQL_OPCODE NUMBER QC_SESSION_ID NUMBER QC_INSTANCE_ID NUMBER

SAMPLE_ID NUMBER SAMPLE_TIME TIMESTAMP(3) When

Session

SQL

Wait

SESSION_STATE VARCHAR2(7) WAIT_TIME NUMBER State

TIME_WAITED NUMBER

Duration

#.29


Primary Fields of ASH

SESSION_ID

EVENT

SQL_ID

SAMPLE_TIME When

Session

SQL

Wait

SESSION_STATE State

TIME_WAITED Duration

#.30


Amazing things YOU can do with ASH

Consumers Top Session Top User Top SQL Top Object Top Module.Action Top Program Top Service Top Client Top Wait

No machine

X

Resources• CPU• Waits

• Event (800*)• I/O

• File• Block

• Time

(32 columns in ASH)

#.31


Groupings – Top ConsumerSESSION_IDSESSION_SERIAL# (signal SID reuse)SESSION_TYPE (FOREGROUND,BACKGROUND)

CURRENT_OBJ#CURRENT_FILE#CURRENT_BLOCK#

USER_ID (SYS, SYSTEM, SCOTT etc)

SERVICE_HASH (OE,GL,HR)

PROGRAM (SQL, JDBC, Forms etc)

MODULE.ACTION(PLSQL tagging)

CLIENT_ID (identifying users in a session pool)

SQL_IDQC_SESSION_ID - Query CoordinatorQC_INSTANCE_ID – RAC

EVENT + P1, P2, P3

#.32


Select session_id, count(*)

from v$active_session_history

where session_state= ‘ON CPU‘ and SAMPLE_TIME > sysdate – (5/(24*60))

group by session_id

order by count(*) desc;



where session_state= ‘ON CPU‘ and SAMPLE_TIME > sysdate – (5/(24*60))

group by session_id


Top CPU Session Who is the rogue session ?

#.33


Results Top CPU Session

SESSION_ID COUNT(*) ---------- ---------- 265 256 264 15 257 12 271 12 276 1

SESSION_ID COUNT(*) ---------- ---------- 265 256 264 15 257 12 271 12 276 1

SESSION_ID COUNT(*) ---------- ---------- 265 256 264 15 257 12 271 12 276 1

#.34


CPU with Bars

SESSION_ID COUNT(*) % Bar---------- ---------- ---------- ------------ 257 75 25 |*** | 263 62 21 |** | 256 32 11 |* | 264 9 3 | | 277 3 1 | | 258 1 0 | | 280 1 0 | |

SESSION_ID COUNT(*) % Bar---------- ---------- ---------- ------------ 257 75 25 |*** | 263 62 21 |** | 256 32 11 |* | 264 9 3 | | 277 3 1 | | 258 1 0 | | 280 1 0 | |

Bar shows 10% incrementsBar shows 10% increments

#.35




where session_state=‘WAITING’ and SAMPLE_TIME > SYSDATE - (5/(24*60))

group by session_id




where session_state=‘WAITING’ and SAMPLE_TIME > SYSDATE - (5/(24*60))

group by session_id


Top Waiting Session Top Waiting Session in last 5 minutes

#.36


SESSION_ID COUNT(*)---------- ---------- 272 224 254 8 249 5 276 5 277 4 270 1

SESSION_ID COUNT(*)---------- ---------- 272 224 254 8 249 5 276 5 277 4 270 1

Top Waiting Session Results

#.37


Top SQL from ASH

select

ash.SQL_ID ,

sum(decode(ash.session_state,'ON CPU',1,0)) "CPU",

sum(decode(ash.session_state,'WAITING',1,0)) -

sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" ,

sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" ,

sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL"

from v$active_session_history ash,

v$event_name en

where SQL_ID is not NULL and en.event#=ash.event#

group by sql_id

order by sum(decode(session_state,'ON CPU',1,1)) desc

select

ash.SQL_ID ,

sum(decode(ash.session_state,'ON CPU',1,0)) "CPU",

sum(decode(ash.session_state,'WAITING',1,0)) -

sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" ,

sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" ,

sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL"

from v$active_session_history ash,

v$event_name en

where SQL_ID is not NULL and en.event#=ash.event#

group by sql_id

order by sum(decode(session_state,'ON CPU',1,1)) desc

#.38


Top SQL from ASH Results

SQL_ID CPU WAITING IO TOTAL------------- ---------- ---------- ---------- ----------4c1xvq9ufwcjc 23386 0 0 233866wjw6rz5uvbp3 99 0 23 122968dm8hr9qd03 97 0 22 119938jp5gasmrah 90 0 25 115cv8xnv81kf582 42 0 9 516p9bzu19v965k 21 0 0 215zu8pxnun66bu 15 0 0 15db2jr13nup72v 9 0 0 97ks5gnj38hghv 8 0 0 8

SQL_ID CPU WAITING IO TOTAL------------- ---------- ---------- ---------- ----------4c1xvq9ufwcjc 23386 0 0 233866wjw6rz5uvbp3 99 0 23 122968dm8hr9qd03 97 0 22 119938jp5gasmrah 90 0 25 115cv8xnv81kf582 42 0 9 516p9bzu19v965k 21 0 0 215zu8pxnun66bu 15 0 0 15db2jr13nup72v 9 0 0 97ks5gnj38hghv 8 0 0 8

#.39


Top Sessionselect ash.session_id, ash.session_serial#, ash.user_id, ash.program, sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "WAITING" , sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "IO" , sum(decode(session_state,'ON CPU',1,1)) "TOTAL"from v$active_session_history ash, v$event_name enwhere en.event# = ash.event#group by session_id,user_id,session_serial#,programorder by sum(decode(session_state,'ON CPU',1,1))

select ash.session_id, ash.session_serial#, ash.user_id, ash.program, sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "WAITING" , sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "IO" , sum(decode(session_state,'ON CPU',1,1)) "TOTAL"from v$active_session_history ash, v$event_name enwhere en.event# = ash.event#group by session_id,user_id,session_serial#,programorder by sum(decode(session_state,'ON CPU',1,1))

select ash.session_id, ash.session_serial#, ash.user_id, ash.program, sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "WAITING" , sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "IO" , sum(decode(session_state,'ON CPU',1,1)) "TOTAL"from v$active_session_history ash, v$event_name enwhere en.event# = ash.event#group by session_id,user_id,session_serial#,programorder by sum(decode(session_state,'ON CPU',1,1))

#.40


Top Session Results

SESSION_ID SERIAL# USER_ID PROGRAM CPU WAITING IO

---------- ------- ---------- ------------------------- ------- ---------- ----------

247 61970 1 sqlplus 11698 0 0

277 1 0 oracle@labsfrh903 (LGWR) 14 21 0

276 1 0 oracle@labsfrh903 (CKPT) 19 10 0

278 1 0 oracle@labsfrh903 (DBW0) 29 0 0

280 1 0 oracle@labsfrh903 (PMON) 19 0 0

254 22617 5 Executor.exe 13 0 3

255 12877 5 Executor.exe 11 0 5

257 33729 5 Executor.exe 15 0 1

255 13417 5 Executor.exe 14 0 2

SESSION_ID SERIAL# USER_ID PROGRAM CPU WAITING IO

---------- ------- ---------- ------------------------- ------- ---------- ----------

247 61970 1 sqlplus 11698 0 0

277 1 0 oracle@labsfrh903 (LGWR) 14 21 0

276 1 0 oracle@labsfrh903 (CKPT) 19 10 0

278 1 0 oracle@labsfrh903 (DBW0) 29 0 0

280 1 0 oracle@labsfrh903 (PMON) 19 0 0

254 22617 5 Executor.exe 13 0 3

255 12877 5 Executor.exe 11 0 5

257 33729 5 Executor.exe 15 0 1

255 13417 5 Executor.exe 14 0 2

#.41


Top Session w/ Usernameselect

decode(nvl(to_char(s.sid),-1),-1,'DISCONNECTED','CONNECTED')

"STATUS",

topsession.session_id "SESSION_ID",

u.name "NAME",

topsession.program "PROGRAM",

max(topsession.CPU) "CPU",

max(topsession.WAITING) "WAITING",

max(topsession.IO) "IO",

max(topsession.TOTAL) "TOTAL"

from ( {previous query} ) topsession,

v$session s,

user$ u

where

u.user# =topsession.user_id and

/* outer join to v$session because the session might be disconnected */

topsession.session_id = s.sid (+) and

topsession.session_serial# = s.serial# (+)

group by topsession.session_id, topsession.session_serial#, topsession.user_id,

topsession.program, s.username,s.sid,s.paddr,u.name

order by max(topsession.TOTAL) desc

select

decode(nvl(to_char(s.sid),-1),-1,'DISCONNECTED','CONNECTED')

"STATUS",

topsession.session_id "SESSION_ID",

u.name "NAME",

topsession.program "PROGRAM",

max(topsession.CPU) "CPU",

max(topsession.WAITING) "WAITING",

max(topsession.IO) "IO",

max(topsession.TOTAL) "TOTAL"

from ( {previous query} ) topsession,

v$session s,

user$ u

where

u.user# =topsession.user_id and

/* outer join to v$session because the session might be disconnected */

topsession.session_id = s.sid (+) and

topsession.session_serial# = s.serial# (+)

group by topsession.session_id, topsession.session_serial#, topsession.user_id,

topsession.program, s.username,s.sid,s.paddr,u.name

order by max(topsession.TOTAL) desc

#.42


Top Session Finding a Rogue User

STATUS SESSION_ID NAME PROGRAM CPU WAITING IO--------------- ---------- ---------- ------------------------- ----- ---------- ----CONNECTED 247 CPU_Monger ChMgr304.exe 11704 0 0CONNECTED 277 SYS oracle@labsfrh903 (LGWR) 14 19 0CONNECTED 278 SYS oracle@labsfrh903 (DBW0) 29 0 0CONNECTED 276 SYS oracle@labsfrh903 (CKPT) 18 9 0CONNECTED 280 SYS oracle@labsfrh903 (PMON) 20 0 0DISCONNECTED 255 SYSTEM Executor.exe 11 4 5DISCONNECTED 257 SYSTEM Executor.exe 13 0 3DISCONNECTED 255 SYSTEM Executor.exe 14 0 2DISCONNECTED 257 SYSTEM Executor.exe 13 0 3

STATUS SESSION_ID NAME PROGRAM CPU WAITING IO--------------- ---------- ---------- ------------------------- ----- ---------- ----CONNECTED 247 CPU_Monger ChMgr304.exe 11704 0 0CONNECTED 277 SYS oracle@labsfrh903 (LGWR) 14 19 0CONNECTED 278 SYS oracle@labsfrh903 (DBW0) 29 0 0CONNECTED 276 SYS oracle@labsfrh903 (CKPT) 18 9 0CONNECTED 280 SYS oracle@labsfrh903 (PMON) 20 0 0DISCONNECTED 255 SYSTEM Executor.exe 11 4 5DISCONNECTED 257 SYSTEM Executor.exe 13 0 3DISCONNECTED 255 SYSTEM Executor.exe 14 0 2DISCONNECTED 257 SYSTEM Executor.exe 13 0 3

#.43


SASH – DB Detailed ASL@actAnalysis Begin Time : 2007-07-24 11:04:48Analysis End Time : 2007-07-24 11:19:45Start time, mins ago: 15Request Duration : 15Collections : 528Data Values : 3327Elapsed Time: 15 mins

WAIT_EVENT CNT % Active Ave_Act_Sess-------------------------------------- ---------- ------------latch free 10 .3 .02log buffer space 13 .39 .02buffer busy waits 14 .42 .03db file scattered read 15 .45 .03library cache pin 78 2.34 .15log file sync 213 6.4 .4ON CPU 726 21.82 1.38enqueue 855 25.7 1.62db file sequential read 1399 42.05 2.65 ------------sum 6.3

@actAnalysis Begin Time : 2007-07-24 11:04:48Analysis End Time : 2007-07-24 11:19:45Start time, mins ago: 15Request Duration : 15Collections : 528Data Values : 3327Elapsed Time: 15 mins

WAIT_EVENT CNT % Active Ave_Act_Sess-------------------------------------- ---------- ------------latch free 10 .3 .02log buffer space 13 .39 .02buffer busy waits 14 .42 .03db file scattered read 15 .45 .03library cache pin 78 2.34 .15log file sync 213 6.4 .4ON CPU 726 21.82 1.38enqueue 855 25.7 1.62db file sequential read 1399 42.05 2.65 ------------sum 6.3

#.44


SASH – ASL over time

@aveactTM NPTS AVEACT GRAPH CPU WAITS---------------- ------ ------- ---------------------- ---- -----06-AUG 13:00:00 270 .33 +- 2 29 5906-AUG 14:00:00 1040 2.24 ++--------2--- 341 198406-AUG 15:00:00 623 6.67 ++++------2---------- 438 371806-AUG 16:00:00 1088 2.59 ++--------2---- 335 248606-AUG 17:00:00 1104 1.26 ++----- 2 349 104306-AUG 18:00:00 1093 1.38 +++---- 2 663 84206-AUG 19:00:00 1012 1.74 ++------- 2 373 138806-AUG 20:00:00 1131 .99 +---- 2 304 82006-AUG 21:00:00 1111 1.22 ++----- 2 344 101206-AUG 22:00:00 1010 1.66 ++------ 2 414 125906-AUG 23:00:00 1120 1.08 +---- 2 298 91307-AUG 00:00:00 1024 .83 +--- 2 273 57607-AUG 01:00:00 1006 1.74 ++------- 2 319 142807-AUG 02:00:00 1090 2.47 ++--------2---- 347 234507-AUG 03:00:00 687 6.59 +++-------2---------- 382 414207-AUG 04:00:00 1004 1.95 ++++++--- 2 1299 65907-AUG 05:00:00 1104 3.08 +++++-----2------ 1170 222607-AUG 06:00:00 1122 1.91 +++++++-- 2 1582 55807-AUG 07:00:00 1115 1.06 +++--- 2 559 61807-AUG 08:00:00 1140 .81 ++-- 2 403 51907-AUG 09:00:00 1128 .88 ++--- 2 386 601

@aveactTM NPTS AVEACT GRAPH CPU WAITS---------------- ------ ------- ---------------------- ---- -----06-AUG 13:00:00 270 .33 +- 2 29 5906-AUG 14:00:00 1040 2.24 ++--------2--- 341 198406-AUG 15:00:00 623 6.67 ++++------2---------- 438 371806-AUG 16:00:00 1088 2.59 ++--------2---- 335 248606-AUG 17:00:00 1104 1.26 ++----- 2 349 104306-AUG 18:00:00 1093 1.38 +++---- 2 663 84206-AUG 19:00:00 1012 1.74 ++------- 2 373 138806-AUG 20:00:00 1131 .99 +---- 2 304 82006-AUG 21:00:00 1111 1.22 ++----- 2 344 101206-AUG 22:00:00 1010 1.66 ++------ 2 414 125906-AUG 23:00:00 1120 1.08 +---- 2 298 91307-AUG 00:00:00 1024 .83 +--- 2 273 57607-AUG 01:00:00 1006 1.74 ++------- 2 319 142807-AUG 02:00:00 1090 2.47 ++--------2---- 347 234507-AUG 03:00:00 687 6.59 +++-------2---------- 382 414207-AUG 04:00:00 1004 1.95 ++++++--- 2 1299 65907-AUG 05:00:00 1104 3.08 +++++-----2------ 1170 222607-AUG 06:00:00 1122 1.91 +++++++-- 2 1582 55807-AUG 07:00:00 1115 1.06 +++--- 2 559 61807-AUG 08:00:00 1140 .81 ++-- 2 403 51907-AUG 09:00:00 1128 .88 ++--- 2 386 601

#.45


SASH – ASL over time with waits@aveactnTO_CHAR(STA AAS AAS1 FIRST AAS2 SECOND GRAPH

----------- ------- ----- --------------- ----- --------------- ------------------------06 13:00:00 .30 .11 db file sequent .10 CPU - 206 14:00:00 2.24 .87 direct path wri .49 log file sync ++--------2---06 15:00:00 6.67 2.07 direct path wri 1.93 log file sync ++++------2----------06 16:00:00 2.59 1.01 direct path wri .57 log file sync ++--------2----06 17:00:00 1.26 .63 log file sync .32 CPU ++----- 206 18:00:00 1.38 .61 CPU .45 log file sync +++---- 206 19:00:00 1.86 .97 log file sync .39 CPU ++------- 206 20:00:00 .99 .38 log file sync .27 CPU +---- 206 21:00:00 1.22 .57 log file sync .31 CPU ++----- 206 22:00:00 1.66 .81 log file sync .41 CPU ++------ 206 23:00:00 1.08 .49 log file sync .27 CPU +---- 207 00:00:00 .83 .37 log file sync .27 CPU +--- 207 01:00:00 1.74 .96 log file sync .31 CPU ++------- 207 02:00:00 2.48 1.21 direct path wri .52 log file sync ++--------2----07 03:00:00 6.59 2.63 direct path wri 1.58 log file sync +++-------2----------07 04:00:00 1.95 1.29 CPU .27 log file sync ++++++--- 207 05:00:00 3.08 1.05 CPU .83 enqueue +++++-----2------

@aveactnTO_CHAR(STA AAS AAS1 FIRST AAS2 SECOND GRAPH

----------- ------- ----- --------------- ----- --------------- ------------------------06 13:00:00 .30 .11 db file sequent .10 CPU - 206 14:00:00 2.24 .87 direct path wri .49 log file sync ++--------2---06 15:00:00 6.67 2.07 direct path wri 1.93 log file sync ++++------2----------06 16:00:00 2.59 1.01 direct path wri .57 log file sync ++--------2----06 17:00:00 1.26 .63 log file sync .32 CPU ++----- 206 18:00:00 1.38 .61 CPU .45 log file sync +++---- 206 19:00:00 1.86 .97 log file sync .39 CPU ++------- 206 20:00:00 .99 .38 log file sync .27 CPU +---- 206 21:00:00 1.22 .57 log file sync .31 CPU ++----- 206 22:00:00 1.66 .81 log file sync .41 CPU ++------ 206 23:00:00 1.08 .49 log file sync .27 CPU +---- 207 00:00:00 .83 .37 log file sync .27 CPU +--- 207 01:00:00 1.74 .96 log file sync .31 CPU ++------- 207 02:00:00 2.48 1.21 direct path wri .52 log file sync ++--------2----07 03:00:00 6.59 2.63 direct path wri 1.58 log file sync +++-------2----------07 04:00:00 1.95 1.29 CPU .27 log file sync ++++++--- 207 05:00:00 3.08 1.05 CPU .83 enqueue +++++-----2------

#.46


Many Ways to Attack Problems

Confusing

How to Attack the problem? Top SQL?

Top wait for that SQL? Top Waiting Session ?

Top Waits for that Session Top Waits for Database?

Top Session waiting for that wait Top SQL for that wait

Solution - Graphics

#.47


Mining Data is Non Trivial

Many Dimensions to consider Constantly Varying time frames Luckily 10g automates all of this functionality.

collects all of the dataanalyses all the wait events reports on bottlenecks supplies solutions

Graphics add tremendous ease and power

Session 1

Session 2

Session 3

Session 4

TIME

Graphical ASH

#.49


Graph of User States

#.50


One Second Graph

#.51


15 Second Averages

#.52


Maximum CPU Line

#.53


Idle Users

#.54


OEM Perf Page

#.55


OEM Perf Page

#.56


ASHvs

Statistics

Statistics are more expensivehave lag time lack clear identification of culprits

#.57


Statistic Lag Time

CountersCounters

SamplesSamples

Slight LagsSlight Lags

#.58


CPU Lag Problem

ASH is the only way to see CPU usage realtime

V$sysstat reports CPU but is only updated at the end of the call. Long calls look deceiving like no CPU is being

used

#.59


CPU in ASH vs Stats

#.60


ASH Sampling Cheap

Three Methods

1. With PL/SQL it’s less that 1 % CPU with 10 average active sessions

2. DMA - direct memory attach to SGA Using C code 200x Cheaper

3. ASH in 10g should be even cheaper than DMA because it’s done in the kernel

#.61


ASH – In Memory

Collects active session data only History v$session_wait + v$session + extras Circular Buffer - 1M to 128M (~2% of SGA)

Flushed every hour to disk or when buffer 2/3 full (it protects itself so you can relax)

#.62


ASH Sizing …

Avg row around 150bytes 3600 secs in an hour ~ ½ Meg per Active Session per hour That’s generally over an hour of ASH

#.63


How ASH works

V$session_wait+ v$session + extras

MMON/MMNL


wrh$_active_session_history

Only writes out 1 in 10 rows via direct path inserts

#.64


Select reads backwards

-No latching

-No read consistency

-Index on time Insert point

ASH bufferInsert one direction

- Touch up wait times

#.65


Family of ASH Tables

v$session_wait


wrh$active_session_history

V$session_wait_history

DBA_HIST_ACTIVE_SESS_HISTORY

#.66


ASH Tables

v$session_wait SID

SEQ#

EVENT

P1TEXT

P1

P1RAW

P2TEXT

P2

P2RAW

P3TEXT

P3

P3RAW

WAIT_CLASS_ID

WAIT_CLASS#

WAIT_CLASS

WAIT_TIME

SECONDS_IN_WAIT

STATE

v$session_wait_history SID

SEQ#

EVENT#

EVENT

P1TEXT

P1

P2TEXT

P2

P3TEXT

P3

WAIT_TIME

v$event_nameEVENT_ID

EVENT#

EVENT_ID

NAME

PARAMETER1

PARAMETER2

PARAMETER3

WAIT_CLASS_ID

WAIT_CLASS#

WAIT_CLASS

v$active_session_historySAMPLE_ID

SAMPLE_TIME

SESSION_ID

SESSION_SERIAL#

USER_ID

SQL_ID

SQL_CHILD_NUMBER

SQL_PLAN_HASH_VALUE

SQL_OPCODE

SERVICE_HASH

SESSION_TYPE

SESSION_STATE

QC_SESSION_ID

QC_INSTANCE_ID

SEQ#

EVENT#

P1

P2

P3

WAIT_TIME

TIME_WAITED

CURRENT_OBJ#

CURRENT_FILE#

CURRENT_BLOCK#

PROGRAM

MODULE

ACTION

CLIENT_ID

current 10 samples hour

DBA_HIST_ACTIVE_SESS_HISTORY wrh$active_session_historySNAP_D

DBIDINSTANCE_NUMBER

SAMPLE_ID

SAMPLE_TIME

SESSION_ID

SESSION_SERIAL#

USER_ID

SQL_ID

SQL_CHILD_NUMBER

SQL_PLAN_HASH_VALUE

SQL_OPCODE

SERVICE_HASH

SESSION_TYPE

SESSION_STATE

QC_SESSION_ID

QC_INSTANCE_ID

SEQ#

EVENT_ID

P1

P2

P3

WAIT_TIME

TIME_WAITED

CURRENT_OBJ#

CURRENT_FILE#

CURRENT_BLOCK#

PROGRAM

MODULE

ACTION

CLIENT_ID

7 days (disk)1 in 10

#.67


Wait Time vs Time Waited

SESSION_STATEWaiting, on CPUBased on WAIT_TIME

WAIT_TIME (v$session, v$session_wait, v$ash)

0 => waiting>0 => CPU (value is time of last wait)

TIME_WAITED Actual time waited for event0 until wait finishes Fix up values (no one else can do this)

#.68


Oradebug

Dump to trace file

SQL> oradebug dump ash 5

SQL> Alter session set events ‘immediate tracename ashdump level 5’;

level 5 = # of minutesloader file rdbms/demo/ashldr.ctl

SQL> oradebug ashdump 5

SQL> Alter session set events ‘immediate

tracename ashdump level 5’;

#.69


INIT.ORAstatistics_level = Typical (default)

PARAMETER SESSION_VALUE INSTANCE_VAL

----------------------- ---------- ------------

_ash_sampling_interval 1000 1000

milliseconds

_ash_size 1048618 1048618

ASH buffer size

_ash_enable TRUE TRUE

Turn on/off ASH sampling, flushing and the V$ views on ASH

_ash_disk_write_enable TRUE TRUE

Flush to disk

_ash_disk_filter_ratio 10 10

write 1 in 10 points

_ash_sample_all FALSE FALSE

Sample including idle waits

#.70

10.2 ASH Extras BLOCKING_SESSION BLOCKING_SESSION_STATUS BLOCKING_SESSION_SERIAL# P1TEXT P2TEXT P3TEXT WAIT_CLASS WAIT_CLASS_ID XID

7004/21/23

#.71


10.2 Extras 10gR2 add fields to ASH

Blocking Session Id, serial# and state *** XID, transaction ID RAC event Fixup Plan Hash Fix up

10gR1 - during parsing no plan, good way to find parsing problems

10gR2 – get plan hash fixup – good but lose some ability to find parsing problems

Wait Class, needed this for grouping Force_matching_signature – sql profile id

#.72

10.2.0.3 ASH Extras

FORCE_MATCHING_SIGNATURE PLSQL_ENTRY_OBJECT_ID PLSQL_ENTRY_SUBPROGRAM_ID PLSQL_OBJECT_ID PLSQL_SUBPROGRAM_ID

7204/21/23

ALL_PROCEDURESALL_PROCEDURES where object_id = plsql_object_id where object_id = plsql_object_id and subprogram_id = plsql_subprogram_id and subprogram_id = plsql_subprogram_id

#.73

7304/21/23

PLSQL Tracking

Package/Procedure/SQL tracking 10.2.0.3calling_code SQL_ID COUNT(*)----------------------------------------------- ----------ORDERENTRY.NEWORDER 0uuqgjq7k12nf 258ORDERENTRY.NEWCUSTOMER 0bzhqhhj9mpaa 262ORDERENTRY.BROWSEANDUPDATEORDERS 41zu158rqf4kf 301ORDERENTRY.NEWORDER 0yas01u2p9ch4 569PARSE_SAME 3vjxpmhhzngu4 874ORDERENTRY.BROWSEANDUPDATEORDERS 05s4vdwsf5802 1669

select object_name package||.|| procedure_name, sql_id, count(*)from v$active_session_history ash, all_procedures procs where ash.PLSQL_ENTRY_OBJECT_ID = procs.object_id and ash.PLSQL_ENTRY_SUBPROGRAM_ID = procs.SUBPROGRAM_ID

#.74


11g ASH extrasSQL ElapsedSQL Elapsed SQL_EXEC_ID SQL_EXEC_START

IN_CONNECTION_MGMT IN_PARSE IN_HARD_PARSE IN_SQL_EXECUTION IN_PLSQL_EXECUTION IN_PLSQL_RPC IN_PLSQL_COMPILATION IN_JAVA_EXECUTION IN_BIND IN_CURSOR_CLOSE

SQL_PLAN_LINE_ID SQL_PLAN_OPERATION SQL_PLAN_OPTIONS

CURRENT_ROW# EVENT# QC_SESSION_SERIAL# CONSUMER_GROUP_ID FLAGS

ETCETC

CPU Analysis – non TimedCPU Analysis – non Timed

SQL Row Source SQL Row Source

REMOTE_INSTANCE#RAC – remote transfersRAC – remote transfers

TOP_LEVEL_SQL_ID

TOP_LEVEL_SQL_OPCODE

Recursive SQLRecursive SQL

#.75


11g ASH extrasRun-time SQL row source information

Identifies current row source within plan

SQL execution ID Is this same execution as last sample? can see how long sql has been running

Operation bit vector Capture non-timed operations Examples: fetch, binding, close

Remote instance id for Cache transfers Which instance sourced requested block?

#.76


How Many Active Sessions?

How much data does ASH Collect ? 1 CPU means max 1 Avg Active Session unless

there is a bottleneck

Big site examples: Oracle 4 way RAC internal apps

10,000 connected, 200 active

One Site 3000 connected, 30 Active

Site 12,000 connected, 100 active

#.77


Simulating ASH

ASH is new in 10g ASH data exist since V7 Need Diagnostic Pack License Collect it yourself

SASH – Simulated ASHhttp://perfvision.com/sash.php

Graphical Monitorhttp://perfvision.com/ashmon.php

Consumes < 1% CPU for 10 active sessions (a lot)

#.78


Sampling

Sampling is the future Took clean vision to create

Bit of a leap of faith Less Accurate but more powerful than past

Inaccuracies unimportant for the most part Feasible since version 7

You can implement it yourself Graphics are the only way to full harness the

power

#.79


Summary

To solve hard problems you need Sampling If you have 10g and a license you can use

V$active_session_historyOEM 10g (can access some 9 sampling)?/rdbms/admim/ashrpt.sql

If you are on 7,8,9 or 10g without the licenseNeed custom sampling

SASH – Simulated ASH http://perfvision.com/ash.php ASHMON – graphical http://perfvision.com/ashmon.php

#.80


ASH SummaryFaster, Cheaper, instant (no lag) …

but most importantly

Rich & Multidimensional Overview of system load Drilldowns into problems

Sql Session Wait event

Details information on problem resolution

OEM simplifies mining of ASH data

Documents

ASH – Active Session History Feel the Power Kyle Hailey