View
324
Download
3
Category
Preview:
Citation preview
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
1/47
Best Practices for InterpretingPL/SQL Hierarchical Profiles for Effective Tun
Avaloq | www.avaloq.com
Switzerland, UK, Luxembourg, Germany, France, Singapore, Hong Kong, Philippines, Austra
CON2082
26 October 2015
Martin Buechi, Lead Software Architect
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
2/47
Presenter Sharing Passion for Efficient Development of High-Quality DB A
Avaloq: Swiss core banking system vendor
Development started 1993 (Oracle 7), 200 PL/SQL developers
Business logic in 20 M lines of PL/SQL in database
Approach also relevant for small shops with little DB code
Martin Büchi: Architect responsible for Oracle foundation
Oracle PL/SQL Developer of the Year 2009
Other Oracle interests: modularization, ILM, tools
Goals
Share passion. Help you improve your development.
No selling of consulting services or products.
Performance Is Key
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
3/47
Performance is a key feature of any application
Goals of Performance Profiling
“The universal experience of program
who have been using measurement tobeen that their intuitive guesses fa
Donal
Performance is what users experience:The response time from the click
to the display of the result on the screen.
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
4/47
Design for performance, then measure performance & tune where ap
Sample tuning process 1. Focus on most valuable business process and biggest resource consu
2. Explore alternatives to achieve same business outcome (requirements
3. Measure response time
4. Tune what takes most time. List improvement options. Implement best
End-to-end performance profile
BrowserNetworkMiddle tier
Database
Overall End-to-End Process and Profile
0 ms 100 ms 2
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
5/47
Often significant potential
Where to look for performance
Answer questions – get actionable information on important business
How long did PL/SQL take?
Why did it take so long?
Why Profile PL/SQL?
Level Potential # Presentation
Algorithm (e.g., PL/SQL) Very high Very few
Physical data modelling Usually lower Few
SQL Usually lower Very many
Infrastructure, instance Usually lower Many, past very
How can the time be reduced?
Are we done (economically optim
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
6/47
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
7/47
Start with dbms_hprof, then use dbms_profiler if reporting on line-level requnclear where time spent inside subprogram, code coverage reporting)
PL/SQL Profilers
Property dbms_profiler dbms_h
Since (and mostly unchanged since) 8.1.5 11gR1
Recording by line + -
Call sequence recording - +
Memory allocation recording - +
Low performance overhead - +
Recording table file
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
8/47
Package
Installed on all DBs: @$ORACLE_HOME/rdbms/admin/catproc.sql
grant execute on SYS.DBMS_HPROF to ;
Directory
create or replace directory PLSHPROF_DIR as '/tmp';
grant read, write on directory PLSHPROF_DIR to ;
Optional tables (SYS.DBMSHP_%) for reporting
@?/rdbms/admin/dbmshptab.sql
Installation of PL/SQL Hierarchical Profiler
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
9/47
PL/SQL API
dbms_hprof.start_profiling('PLSHPROF_DIR', 'myprofile.hpf');
/* activity to be profiled */
dbms_hprof.stop_profiling; /* optional, or just terminate se
Built into SQL Developer, TOAD, PL/SQL Developer, etc
SQL Developprofile into tadeletes raw fiusing built-in
Gathering of Profiles
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
10/47
Scoping
Avoid unnecessary actions (PL/SQL executions) when profiling is activa
Wait events in profile if and only if PL/SQL on stack profile time ≤ elap
– In profile: time in PL/SQL, wait in SQL executed by PL/SQL, Blocking wait in P(dbms_aq.dequeue, dbms_lock.request, dbms_alert.waitany, etc)
– Not in profile: “SQL*Net message from client”, wait in SQL execute directly by
Ideally, turn start / stop with top-level call or on same call stack level
Measure realistically
First vs. following executions
– On DB: PL/SQL code loading, SQL parsing, buffer cache
– In session: package initialization
Taking Useful Profiles
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
11/47
No out of the box solutions for
Gathering for other session and based on module like dbms_monitor for
No third-party tools for scoping of profiles / no always running recorder l
Build your own
Instrumentation wrapper that calls dbms_application_info.set_* and
Starts / stop PL/SQL hierarchical profiler (and other tools). Unique namesince re-start with same name overwrites. OK to concatenate.
Receives commands from other sessions, e.g., through global context
Raise an SR to support Bug 12883592 : ADDITIONAL FUNCTIONALITYINSPECT THE CURRENT STATE OF OTHER SESSION
Logon trigger as last resort
Build Your Own Advanced Gathering
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
12/47
Make Gathering & Reporting Accessible to Developers
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
13/47
create or replace procedure h isprocedure sleep(seconds number) is l_cnt number;begin
if seconds is not null then dbms_lock.sleep(seconds); enselect count(*) into l_cnt from dba_objects, dba_tablesp
end sleep;--procedure call_no_sleep
is begin sleep(null); end call_no_sleep;--procedure call_sleep(seconds number)is begin sleep(seconds); end call_sleep;
begincall_sleep(1); call_sleep(10);call_no_sleep;
end h;
Example with Hierarchy and Skew
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
14/47
Header P#V PLSHPROF Internal Version 1.0Start P#! PL/SQL Timer Started
...Call P#C PLSQL."SYS"."DBMS_LOCK"::11."SLEEP"#9689ba46Execute μs P#X 1000064 Return P#R
...
Content: Call, Elapsed Time with PL/SQL on Stack, Re
Namespace Object Type Subprogram S
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
15/47
May not need the most detailed format every time
Format options
Raw or processed (HTML, table, graphical) with or without navigation (d
Individual calls or aggregated (average, median, histogram)
Different perspectives
Look at profiles from multiple angles before taking actions.
A fool with a tool is still a fool
Study & understand output on toy example first
engineer: “efficiently get relevant, actionable information” scientist: “understand everything”
presenter: “nice graphics to explain”
Presentation & Reading Skills to Find Relevant Proble
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
16/47
Exampleoracle@srv:db > ${ORACLE_BIN}/plshprof -output myprofile mypPLSHPROF: Oracle Database 11g Enterprise Edition Release 11.64bit Production[7 symbols processed][Report written to 'myprofile.html']
oracle@srv:db > ls myprofile*.html
myprofile_2c.html myprofile_2n.html myprofile_fn.htmlmyprofile_md.html myprofile_ms.html myprofile_nsf.htmlmyprofile_pc.html myprofile_td.html myprofile_ts.htmlmyprofile_2f.html myprofile_aggr.html myprofile.htmlmyprofile_mf.html myprofile_nsc.html myprofile_nsp.htmlmyprofile_tc.html myprofile_tf.html
Script to concatenate to single HTML file in appendix.
HTML Output with plshprof
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
17/47
High SQL percentage?
Problem in SQL
Test system with bad I/O
Bad algorithm in PL/SQL with too many SQL calls
PL/SQL or SQL?
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
18/47
SQL trace shows
Bad SQL, slow I/O, concurrency waits Number of rows returned processed in SQL
Must start SQL trace first
1 dbms_monitor.session_trace_enable;2 dbms_hprof.start_profiling('PLSHPROF_DIR', 'myprof.hpf');
Bug 22085980 : SETTING A DATABASE EVENT IN A SESSION STOPS THIERARCHICAL PROFILER
PL/SQL Hierarchical Profile and SQL Trace
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
19/47
Duration = #Execution * #AverageDurationPerExecution fewer calls
Focus on high duration and many executions
Expensive and Called Multiple Times
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
20/47
May not directly yield biggest benefit, but why so many calls?
Counts: Do We Need to Call a Subprogram n Times?
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
21/47
Subtree with Much Higher Ind% than Next Line
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
22/47
Optionally with tool showing source code
Look at breakdown of time in report and examine source code.
Percentage of function as part of subtree not shown.
Breakdown of Children / Parent
Percentage of Descendants (not Subtree)
Percentage of total run
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
23/47
Very useful if time in some packages split across very many subprogram
Possible option to estimate tuning potential.
Time by Module
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
24/47
Skew in Subtree Between Callers (Parents)
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
25/47
Own call analysis tool for individual call and histogram reporting
select * from table(single_call('PLSHPROF_DIR', 'myprofile.h'K.H.H.SLEEP'));
Focused report
${ORACLE_BIN}/plshprof -trace '"K"."H"."H.SLEEP"' -skip 1 -coutput second myprofile.hpf
Skew in Subtree Calls from Same Parent
OCC PARENT SUBTREE FUNCTION DE
1 K.H.H.CALL_SLEEP 2,800,569 42
2 K.H.H.CALL_SLEEP 11,812,431 17
3 K.H.H.CALL_NO_SLEEP 1,762,206 12
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
26/47
When gathering cannot be properly scoped / focus without distorted
Subtree focus & number of invocations
plshprof -trace '"K"."H"."H.SLEEP"' -skip m -collect n ..
Always three dot-separated arguments between "". See raw trace for ex
– Standalone subprogram H in schema K "K"."H"."H"
– PL/SQL VM "".""."__plsql_vm"
– Anonymous PL/SQL "".""."__anonymous_block"
– Specific SQL "K"."P"."__dyn_sql_exec_line
Limit Reporting to Contiguous Set of Invocations of Su
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
27/47
Export to Microsoft Excel for Annotation and Processin
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
28/47
Example
Load into Tables SYS.DBMSHP_%
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
29/47
Careful: Hierarchy and Skew Lost in Import into Tables
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
30/47
Sample Domain-Specific Custom Reports for Generat
id type classif cond …
1 5 9090 [doc.bp.class(9089)=101619]…
2 5 5075 [rm$spread_deriv.rel_eam(doc) = '0'] …
create or replace package body k.deriv# isfunction cond$1(
i_doc doc_mgr#.t_doc
) return varchar2 is
1 . G en
er a t e
module function subtree_elapsed_time
deriv# cond$1 973
id type
1 5
2. Execute with HProf, load i
3. Join
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
31/47
Interactive flame graph (http://www.brendangregg.com/flamegraphs.html)
exec flatten('PLSHPROF_DIR', 'myprofile.hpf', 'myprofile.flaflamegraph.pl myprofile.flat > myprofile.svg
Graphical Presentations and Profiler for Other Langua
http://www.brendangregg.com/flamegraphs.htmlhttp://www.brendangregg.com/flamegraphs.htmlhttp://www.brendangregg.com/flamegraphs.html
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
32/47
Compact semicolon-separated time-aggregated call stack with durati
Execution count in brackets is our addition
H.H [1] 7
H.H [1];H.H.CALL_SLEEP [2] 4
H.H [1];H.H.CALL_SLEEP [2];H.H.SLEEP [2] 59
H.H [1];H.H.CALL_SLEEP [2];H.H.SLEEP [2];H.__static_sql_exec_line7 [2] 3609411
H.H [1];H.H.CALL_SLEEP [2];H.H.SLEEP [2];DBMS_LOCK.SLEEP [2] 11003530H.H [1];H.H.CALL_NO_SLEEP [1] 1
H.H [1];H.H.CALL_NO_SLEEP [1];H.H.SLEEP [1] 12
H.H [1];H.H.CALL_NO_SLEEP [1];H.H.SLEEP [1];H.__static_sql_exec_line7 [1] 17621
flamegraph.pl Input from Sample Program
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
33/47
Semantics
y-axis: Call stack, x-axis: width proportional to duration, siblings sorted a
Interactive: Click to zoom / reset zoom, Search
flamegraph.pl Output of Sample Program
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
34/47
Generate cpuprofile from PL/SQL HProf trace
exec chrome_cpuprofile('PLSHPROF_DIR', 'myprofile.hpf','myprofile.cp
Files too big? Use (sparse / adaptive) sampling
Google Chrome Developer Tools cpuprofile
https://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-execution
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
35/47
{"head": {"functionName": "(root)", "id": 1,
"children": [{"functionName": "H.H", "hitCount": 7, "id": 2,"children": [{"functionName": "H.H.CALL_NO_SLEEP", "id": 3, ..."functionName": "H.H.SLEEP", "id": 4, ..."functionName": "H.__static_sql", "id": 5, ...
}, ...]
}]
},"startTime": 0, "endTime": 16375218,"samples": [2, 4, 5, 4, ... ],
"timestamps": [0, 1, 5, 1762199, ... ]
Chrome Input from Sample Program
For aggregated times
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
36/47
Opening .cpuprofile files
https://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-e
Google Chrome Developer Tools cpuprofile
https://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-executionhttps://developers.google.com/web/tools/chrome-devtools/profile/rendering-tools/js-execution
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
37/47
Semantics
y-axis: Call stack (top down)
x-axis: time
Flame Chart of Sample Program
Many more visualization tools tha
use, e.g., Callgrind / KCachegrind
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
38/47
Manual
Stare at two reports and compare manually SQL to compare runs loaded into SYS.DBMSHP_% tables
plshprof HTML with two input files
${ORACLE_BIN}/plshprof -output comp h1.hpf h2.hpf
Difference Between Runs (Tuning, Input Variation, etc
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
39/47
Sample Comparison Overview Report
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
40/47
Smaller files, but relevant information may be lost requiring re-gather
Syntax
dbms_hprof.start_profiling(…, max_depth => n)
Raw trace file content: Time summed up at last reported levelmax_depth => null max_depth => 2
P#C PLSQL."K"."H"::7."H.CALL_SLEEP" P#C PLSQL."K"."H"::7."H.CALLP#X 1 P#X 2800544P#C PLSQL."K"."H"::7."H.SLEEP" P#R
P#X 23P#C PLSQL."SYS"."DBMS_LOCK"::11."SLEEP"P#X 1000064P#R... /* statis SQL call */ P#RP#X 0P#R
Limiting Call Depth in Gathering
o m i t t e d d e t a i l s
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
41/47
Compilation options (plsql_%)
PL/SQL hierarchical profiler also works for wrapped and natively compile
With inlining (plsql_optimize_level=3 or pragma inline), calls to inlined suare not visible. Likewise when call to deterministic function is optimized a
Usage notes
Performance verhead low enough to run on production. Ensure enough free disk space and not on a critical file system if full. Ca
maximum file size.
No need for a fast server. Just compare relative times in PL/SQL (not SQPL/SQL).
Bits & Pieces 1/2
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
42/47
Other usages
Call stack until hang since no stop needed – alternative to oradebug dumerrorstack 1
Actual call tree (of test input only) – vs. complete possible static calls of
References
https://docs.oracle.com/database/121/ADFNS/adfns_profiler.htm#ADFN
Bits & Pieces 2/2
https://docs.oracle.com/database/121/ADFNS/adfns_profiler.htm#ADFNS023https://docs.oracle.com/database/121/ADFNS/adfns_profiler.htm#ADFNS023
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
43/47
Gathering
dbms_hprof.start_profiling(..., profile_uga | profile_pga =>
Often in combination with dbms_session.get_package_memory_utilizati
What’s in the trace
P#Z 1169723168
P#Z -2146643687
Reporting
plshprof -uga or -pga
dbms_hprof.analyze(..., profile_uga | profile_pga => true)
Memory Allocation Profiling: UGA or PGA
Oracle internal. Not documen
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
44/47
Conclusions
Performance is a key feature of any application
Oracle-provided tools are sufficient for analysis of PL/SQL hierarchical pexcept for skew analysis and graphical people—for whom this presentat
Understand output on toy example before analyzing real code.
Action items
Profile PL/SQL as part of end-to-end performance analysis.
Integrate gathering and reporting into your developer andkey business user UI.
Conclusions & Action Items
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
45/47
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
46/47
Thank you for your attention.
Martin Büchimartin.buechi@avaloq.com
8/20/2019 PLSQL Hierarchical Profiles for Effective Tuning
47/47
The KSH, PL/SQL, and Java code for analyzing and transforming PL/SQL Profiler traces as shown in the presentation can be found at
https://drive.google.com/open?id=0B40PcvWHeRrrZW1oRjg1WHFfOEE .
This is sample code. Not production quality code. No warranties.
Sample Code Only
https://drive.google.com/open?id=0B40PcvWHeRrrZW1oRjg1WHFfOEEhttps://drive.google.com/open?id=0B40PcvWHeRrrZW1oRjg1WHFfOEERecommended