Delivering Oracle Success
Top Down Performance Management with OEM Grid Control
Or … How I learned to stop worrying and love OEM Grid Control
John Darrah, DBAK
RMOUG QEWNovember 19, 2010
© DBAK 2010 2
About DBAK
• Oracle solution provider• Co-founded in 2005• Based in Englewood, CO• 2008 “Emerging Business of the Year” – South Metro
Denver Chamber of Commerce• More than 130 implementations, upgrades, conversions,
and support projects for 80+ clients• Average 15 years of Oracle expertise• Oracle Gold Partner
© DBAK 2010 3
The DBA Challenge
• Manage complex environments– N-Tier environments with several entry points to the
database or databases• Manage many database environments
– Fewer DBAs to manage more environments• Need to respond to performance problems
– DBAs must be able to track down root cause across several environments
– DBAs must be able to demonstrate the problem is not database related
© DBAK 2010 4
The Past
© DBAK 2010 5
The Present
© DBAK 2010 6
How do you monitor environments?
• SQL Trace• SQL*Plus Scripts• StatsPack / AWR• Third Party Monitoring Applications• Users• Prayer• OEM (dbconsole)• OEM Grid Control
© DBAK 2010 7
Which Monitoring tools are best?
• All of them have their place– All of the above methods have advantages and
disadvantages– Use the right tool for the right situation– Don’t become entirely dependent on GUIs
• Take a Top-Down Approach– OEM is an ideal solution for a top down tuning approach– Most other approaches are more suited for detailed
analysis of individual problems
© DBAK 2010 8
The need for a Top Down approach
• In complex environments it is difficult to find the problem much less address it<sarcasm>
• Database is always the performance bottleneck</sarcasm>
• The good old days of client server apps (i.e easy to trace) are long gone.
© DBAK 2010 9
“The WMS database is down!”
© DBAK 2010 10
Top Down Approach
• Use Groups and dashboards to quickly identify problems in the areas you care about– Group dashboards give a high level view of targets
© DBAK 2010 11
Top Down Approach cont.
• Start at a high level and drill down– Quickly identify problems and drill into root cause
© DBAK 2010 12
“The ERP database is down!”
© DBAK 2010 13
Drilling into RAC environments
© DBAK 2010 14
Performance Tab Summary
• OEM’s performance page provides – High level performance metrics on database
performance– A graphical representation of AWR data (10/11g)– Top activity analysis and drilldown– SQL Tuning Advisor (10/11g)
© DBAK 2010 15
OEM Performance Tab (10g)
© DBAK 2010 16
Top Activity
• Shows a 1-hour timeline of the top activity running on the database
• Displayed in 5-minute intervals • Timeline graph gives the ability to look at past
statements• ASH and SQL Tuning available from this page
© DBAK 2010 17
Top Activity
© DBAK 2010 18
Top Consumers
© DBAK 2010 19
SQL Tuning Advisor
• Runs a series of what-if scenarios and data analysis to better determine plan efficiency
• Provides a list of suggestions weighted by % improvement
• Only available with 10g or 11g + Diagnostics and Tuning option
© DBAK 2010 20
SQL Tuning Advisor
© DBAK 2010 21
“Our ETL ran long, what happened?”
© DBAK 2010 22
Grid Control can show historical as well as real time data
© DBAK 2010 23
ASH Reporting
• Shows active session history– V$ACTIVE_SESSION_HISTORY– DBA_HIST_ACTIVE_SESS_HIST
• The same report can be run from SQL*Plus– OEM Takes care of formatting and parameters
© DBAK 2010 24
ASH Reporting
© DBAK 2010 25
A note about 8i and 9i databases
• Requires additional setup– See section 10.3 of advanced configuration guide
• Lacks ADDM, AWR, ASH• Still provides valuable information to a DBA
© DBAK 2010 26
8i and 9i cont.
• Snapshots of an SQL statement
© DBAK 2010 27
“The web servers are crashing! What’s wrong with the database?!”
© DBAK 2010 28
Interdependent targets are grouped into systems• Different targets that fulfill a business need
– Host – Database– Listener – Web servers– Load Balancer
• All of the targets must be available to service the system
© DBAK 2010 29
Services can monitor systems in OEM
• Create tests that run synthetic transactions• Determine the availability of a service
– Critical system component availability– Service availability
• Beacons can test service availability and performance from many locations– Internal– External
• Beacons give visibility to end user experience
© DBAK 2010 30
Service test example
© DBAK 2010 31
AD4J expands Grid Control’s abilities
• AD4J stands for advanced diagnostics for Java• Provides the ability to inspect JVM heaps
– Memory leak detection• Provides the ability to profile individual threads• Provides the ability to tie a thread to a database session• Fully integrated into Grid Control as of OEM 11• Service Tests and Beacons provide the what - AD4J
provides the why
© DBAK 2010 32
AD4J Dashboard
© DBAK 2010 33
AD4J Active Thread Trace
© DBAK 2010 34
Beyond Performance Monitoring
• Performance monitoring it crucial but does not provide a complete solution– Look at present and past activities, not a look ahead– It is reactive– Tactical
• The complete IT shop needs to look forward as well– Capacity planning– Operational budgeting
© DBAK 2010 35
Beyond Performance Monitoring (cont.)
• OEM Grid Control repository is essentially an ODS – Performance metrics gathered from all targets– Configuration information about targets– Utilization regarding targets
• Data from the OEM repository can be mined– What is the average utilization of the EBS servers?
• How is that utilization trending?– What is my breakout of different models of servers?
• Sun Op Center takes this concept further
© DBAK 2010 36
The Future
• OEM Grid Control will continue to evolve– Many recent acquisitions will be woven into the OEM
brand/frameworkSun Op Center
• Oracle will continue to improve on the end to end top down vision monitoring solution– RUEI– Op Center