Performance Diagnosis Using DMV's & DMF's in SQL 2005 & 2008

Performance diagnosis using DMV’s and DMF’s inSQL Server 2005 & 2008 and 2008

Performance diagnosis using DMV’s and DMF’s

Prepared for

Administrators, Support Engineers, Administrators, Support Engineers, Customers,Field Engineer andConsultantsConsultants

Monday, 21 January 2008

Version 1

Prepared by

Nickson Dicson

Support Engineer

[email protected]

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication and is subject to change at any time without notice to you. This document and its contents are provided AS IS without warranty of any kind, and should not be interpreted as an offer or commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.

The descriptions of other companies’ products in this document, if any, are provided only as a convenience to you. Any such references should not be considered an endorsement or support by Microsoft. Microsoft cannot guarantee their accuracy, and the products may change over time. Also, the descriptions are intended as brief highlights to aid understanding, rather than as thorough coverage. For authoritative descriptions of these products, please consult their respective manufacturers.

This deliverable is provided AS IS without warranty of any kind and MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, OR OTHERWISE.

All trademarks are the property of their respective companies.

©2005 Microsoft Corporation. All rights reserved.

Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Page iiPerformance diagnosis using DMV’s and DMF’s in SQL Server 2005 & 2008, Administrators, Support Engineers, Customers,Field Engineer and Consultants, Version .1 Prepared by Nickson Dicson"document.doc" last modified on 21 Jan. 08, Rev 41

Revision and Signoff Sheet

Change Record

Date Author Version Change reference

16-Dec-2007 Nickson Dicson 1.0 Initial version

Page iiiPerformance diagnosis using DMV’s and DMF’s in SQL Server 2005 & 2008, Prepared by Nickson Dicson last modified on 21st Dec 2008

Table of Contents

Introduction.................................................................................................................................................. 1

Purpose................................................................................................................................................... 1

Audience.................................................................................................................................................. 1

Knowing and understanding DMV’s and DMF’s ....................................................................................2

Retrieving DMV’s Definition...................................................................................................................... 3

Categorization of DMV’s and DMF’s........................................................................................................4

1) SQLOS / Engine 4

2) SQL Execution related DMV’s 6

3) IO Related. 6

4) Transaction Related. 7

5) Index Related. 7

6) Service Broker. 8

7) Replication Related. 8

8) Full Text Search Service. 9

9) CLR 9

Causes of SQL Server slowness............................................................................................................10

1) System resource. 10

2) Slow query / plan. 11

3) Tempdb contention. 11

4) Blocking / Deadlocks. 12

Querying DMV’s & DMF’s for performance diagnosis..........................................................................14

1) System Resource................................................................................................................................ 14

a) CPU 14

b) Memory 17

c) I/O Disk Subsystem 20

2) Blocking............................................................................................................................................... 23

3) Monitoring Worker threads and Schedulers....................................................................................24.

Page ivPerformance diagnosis using DMV’s and DMF’s in SQL Server 2005 & 2008, Prepared by Nickson Dicson last modified on 21st Dec 2008

DMV’s and DMF’s introduced in SQL Server 2008................................................................................25

Summary................................................................................................................................................... 26

Page vPerformance diagnosis using DMV’s and DMF’s in SQL Server 2005 & 2008, Prepared by Nickson Dicson last modified on 21st Dec 2008

INTRODUCTION

Purpose

This document provides information on diagnosing and identifying the system bottleneck using DMV’s and DMF’s which can be used to analysing and troubleshooting SQL Server 2005 performance. It provides brief information and conceptual knowledge on cause of system bottleneck. I have further added SQL Server 2008 DMV’s however complete and detailed queries for the additional SQL 2008 DMV’s will be compiled in later versions.

Audience

This document is intended for SQL Server Support Engineer, Database Administrators, Technology solution specialists, consultants and Field Engineers involved in maintaining and troubleshooting performance in SQL Server 2005 and SQL Server 2008.

Page 1

"document.doc" Created by Nickson Dicson

KNOWING AND UNDERSTANDING DMV’S AND DMF’S

One among the common and the most serious problem faced in SQL Server deployed environment which gets the professionals on hot seat is the big ‘P’ Performance. Troubleshooting performance is necessary however to troubleshoot one need to first know the cause and to diagnose the performance problem.

SQL Server 2005 introduced lots of new development one among them was dynamic management view (DMV) and dynamic management function (DMF) which changed the entire approach of collecting system information and troubleshooting SQL Server performance.

Dynamic management views and functions return server state information that can be used to monitor the health of a server instance, diagnose problems, and tune performance.

There are two types of dynamic management views and functions:

1) Server-scoped dynamic management views and functions. These require VIEW SERVER STATE permission on the server.

2) Database-scoped dynamic management views and functions. These require VIEW DATABASE STATE permission on the database.

In earlier version of SQL Server we used to use system tables (virtual tables) like sysprocesses and syslockinfo to view system state and locking information respectively. These tables were called "virtual" because the rows are never physically stored on disk but are materialized from underlying server data structures when a user queries the table. There were couple of other tables that were used however this could not give SQL Server or system information in much detail. With introduction of dynamic management view (DMV) or dynamic management function (DMF) in SQL Server the approach towards troubleshooting and collecting system information have changed. DMV's and DMF's exposed more information and accurate.

The dynamic management views do not require any parameters whereas the dynamic management functions generally accept parameters to qualify what data to return but still return their data as a rowset.

SQL Server virtual tables discussed above were based on a class named UTRowset. However SQL Server 2005 uses streaming table valued function (STVF) for developing DMV's which gave more flexibility and gather information from system table structure. In a nutshell In SQL 2005 some of the DMV’s are written using the STVF framework while others are implemented using the older UTRowset framework.

Page 2


RETRIEVING DMV’S DEFINITION

As we can retrieve the definition for user stored procedure we can also see the definition of DMV's, querying the global data structure. We can use the sp_helptext procedure to get the body of DMV's. Eg sp_helptext 'DMV'. You can also use the OBJECT_DEFINITION function for the same.

SELECT OBJECT_DEFINITION(OBJECT_ID('sys.dm_os_memory_objects')) AS [Object Definition];

Views that contain format "OPENROWSET (TABLE name)" are STVF-based views, while ones that contain "OPENROWSET(name)" are UTRowset-based views.

Below are the example of "OPENROWSET(name)" and UTRowset-based views.

CREATE VIEW sys.dm_clr_tasks AS

SELECT *

FROM OpenRowset(TABLE DM_CLR_TASKS)

CREATE VIEW sys.dm_os_memory_objects AS

SELECT *

FROM OpenRowSet(sysmemobjs)

Page 3


CATEGORIZATION OF DMV’S

SQL Server 2005 introduced some what around 80 plus DMV’s and these are very well categorized on various SQL Server components associated with system and SQL Server engine etc.

For instance SQLOS related DMV’s are more concentrated on collecting system / engine related information which are alternatives to collecting dumps or process level information for in depth troubleshooting.

Below is brief description on some of the DMV’s associated to SQL Server and system.

1) SQLOS / Engine.

Sr no DMV / DMF's Description 1 sys.dm_os_tasks A task represents a single batch of request. It maintains the context

that is used to run the batch.

2 sys.dm_os_workers This table contains a row for every worker in the system. Worker could be mapped to a thread or can be fiber. A task, representing a batch, is assigned a worker (one-to-one mapping). This association is kept until the task ends. After this, the worker is available to be assigned to a new task.

3 sys.dm_os_threads A worker can either be a fiber or mapped to a thread that is used to run a request on a scheduler. In the case where lightweight pooling is turned off (default), there is a one-to-one mapping between worker and thread. This mapping is maintained until the worker is deallocated either due to memory pressure or if it has been idle for a long time. When lightweight pooling is turned on, there is a many-to-one mapping between workers to thread (Essentially many fibers (i.e. workers) could be run on an individual thread). Workers and threads are pooled so as to support a lot of connections (like a thread pool).

4 sys.dm_os_wait_stats This table contains information on waits encountered by executing threads. It replaces DBCC SQLPerf (WaitStats). There are three general types of waits: Resource Waits, Queue Waits and External Waits.

5 sys.dm_os_waiting_tasks This DMV describes the wait queue of tasks, which are waiting on some resource. It is a simultaneous representation of all wait-queues in the system.

6 sys.dm_os_sysinfo It shows miscellaneous set of useful information about the machine, resources available and consumed by SQL Server.

7 sys.dm_os_stacks This table is used internally to keep track of debug data like outstanding allocations.

8 sys.dm_os_schedulers This table contains one row per scheduler in the system. This is mapped to an individual processor in the system. If an affinity was specified, then the scheduler will always run on the particular processor. This table is primarily used to monitor the health of a scheduler or to identify run-away tasks.

Page 4


9 sys.dm_os_ring_buffers This table lists all ring buffers in SQL Server. Ring Buffer is a memory buffer that keeps history of a number of last records. Ring Buffer implementation utilizes circular memory buffer as a main storage for records.

10 sys.dm_os_performance_counters This view returns a row per performance counter maintained by the Server.

11 sys.dm_os_memory_clerks Memory Clerks implement access to memory nodes interface to allocate memory. It also tracks the memory allocated using the clerk for diagnostics. Every component allocating substantial amount of memory needs to create its own memory clerk and allocate all its memory using the clerk interfaces. During startup components create their corresponding clerks.

12 sys.dm_os_loaded_modules This dynamic management view has a row for each module loaded into the server address space.

13 sys.dm_os_latch_stats A latch is a light weight synchronization object used by various components within SQL Server. A latch wait occurs when a latch request can not be granted immediately because the latch is held by some other thread in a conflicting mode. Note that this DMV only tracks latch waits, it does not track latch requests that were granted immediately or failed without waiting. Specifically the num_of_requests only tracks the number of waits. This DMV can be used to narrow down or pinpoint the source of latch contention by examining the relative wait numbers and wait times for the different latch classes.

14 sys.dm_os_hosts SQL Server can have external components run inside its process. An example of an external component is an OLEDB provider used to run Distributed Queries. In order to keep track of the resources it uses, SQL exposes the concept of a Host that allows this component to allocate memory, run tasks, etc. inside of SQL Server. This enables SQL Server to keep track of resource consumption by these components and react accordingly. This DMV exposes all the currently registered hosts on a SQL Server and the amount of resources they have used.

15 sys.dm_os_cluster_nodes When clustering is enabled, the SQL Server instance can run on any of the nodes of the cluster that are designated as part of the SQL Server virtual server. Each of the rows in this view represents the name of a node in the virtual server.

16 sys.dm_os_buffer_descriptors The DMV exposes all buffer pool buffer descriptors that are in use by a database on the server. Therefore, free or stolen pages will not be exposed. Also, pages that had errors when read will not be exposed. Pages in use by the resource database will be exposed.

Page 5


2) SQL Execution related DMV’s.

Sr no DMV / DMF's Description1 sys.dm_exec_connections This DMV provides information about the connections established to

this sql server on various databases by different users local/remote and the details of each connection.

2 sys.dm_exec_sessions Contains one row per authenticated session on the SQL Server.3 sys.dm_exec_requests Provides information about each request executing within SQL Server.4 sys.dm_exec_sql_text This Dynamic Management Function (DMF) provides the text of the

SQL statement given the sql_handle for that statement. SQL handles are direct hash maps of the sql texts and are rarely same for two different sql queries

5 sys.dm_exec_query_plan Returns the cached plan for a given plan_handle.6 sys.dm_exec_query_stats Provides aggregate performance statistics for cached query plans.

The view contains one row per query plan and the lifetime of the row is tied to the plan itself. When the plan is removed from the cache for whatever reason, the row is eliminated from dm_exec_query_stats.

7 sys.dm_exec_query_optimizer_info This view provides detailed statistics about the operation of the SQL Server query optimizer. It also provides optimization cost incurred on a query plan.

8 sys.dm_exec_cached_plans This DMV provides information about the query execution plans that are cached by SQL server for faster query execution.

9 sys.dm_exec_cursors This dynamic management function provides information about cursors opened on various databases and their details.

3) IO Related.

Sr no DMV / DMF's Description1 sys.dm_io_pending_io_requests This table contains a row for each pending I/O in the system. An I/O

that has been pending for a while (io_pending == TRUE) could indicate a problem with either the OS or the I/O hardware.

2 sys.dm_io_virtual_file_stats Returns I/O statistics for database files, including log files.3 sys.dm_io_cluster_shared_drives When clustering is enabled, the nodes of a cluster require a shared

disk array for storing data that may be accessed by a second node during failover. Each of the rows in this view represents a single disk of that shared disk array.

4 sys.dm_io_backup_tapes This dynamic management view identifies the list of backup devices and the status of requests to mount them up for backing up.

Page 6


4) Transaction Related.

Sr no DMV / DMF's Description1 sys.dm_tran_locks This view contains information about currently active lock manager

resources. This includes regular locks and lock related notification requests. Each row represents a currently active request to the lock manager that has either been granted or is waiting to be granted, i.e. the request is blocked by an already granted request.

2 sys.dm_tran_database_transactions Each row in this table represents a per transaction per database object, known as XDES. Each XDES holds transaction information relevant to the given database.

3 sys.dm_tran_session_transactions This gives transaction information for every session in SQL server.4 sys.dm_tran_version_store It returns a virtual table that displays all version records in the version

store. This is an expensive DMV to run as it has to query the entire version store that can potentially be very large. Each versioned record is stored as binary data along with some tracking/status information.

5 sys.dm_tran_current_transaction It returns a single row that displays state information of the transaction in the current session.

6 sys.dm_tran_current_snapshot It displays the tranaction sequence number of all active transaction at the time when the current snapshot transaction starts. If the current transaction is not running under snapshot isolation, it returns no rows. This is similar to dm_tran_transactions_snapshot except that it only shows the active transactions for the current snapshot.

7 sys.dm_tran_active_transactions Each row in this table represents an active transaction in this instance.

5) Index Related.

Sr no DMV / DMF's Description1 sys.dm_db_index_usage_stats This view allows you to display usage counts for individual indexes,

and the time of the last operation performed on individual indexes, for various kinds of operations. An index use is an individual seek, scan, lookup, or update on that index by one query execution. Information is reported both for operations caused by user-submitted queries, and operations caused by internally-generated queries, such as scans for gathering statistics.

2 sys.dm_db_partition_stats Returns page and row count information for every partition in the current database.

3 sys.dm_db_index_physical_stats Displays size and fragmentation information for the data and indexes of the specified table. For an index, one row of statistics can be returned for each level of the b-tree in each partition. For heaps, one row of statistics is returned for the allocation unit of each partition. For LOB data one row of statistics is returned for the allocation unit of each partition. For row-overflow data, one row of statistics is returned for each partition if any row-overflow data has been created.

Page 7


4 sys.dm_index_operational_stats Returns various performance statistics for indexes and tables within a given database

6) Service Broker.

Sr no DMV / DMF's Description1 sys.dm_broker_activated_tasks This Dynamic Management View (DMV) contains a row for each

stored procedure activated by Service Broker.

2 sys.dm_broker_connections This dynamic management view contains one row for each Service Broker network connection.

3 sys.dm_broker_forwarded_messages This dynamic management view contains a row for each Service Broker message that the SQL Server instance is in the process of forwarding.

4 sys.dm_broker_queue_monitors This dynamic management view contains a row for each queue monitor in the instance. A queue monitor manages activation for a queue.

7) Replication Related.

Sr no DMV / DMF's Description1 sys.dm_repl_articles This Dynamic Management Function (DMF) provides information

about different articles tables/SP’s published / subscribed as a part of SQL server replication at sql command level.

2 sys.dm_repl_schemas This Dynamic Management Function (DMF) provides information about different articles tables/SP’s published / subscribed as a part of SQL server replication at object level.

8) Full Text Search Service.

Sr no DMV / DMF's Description

Page 8


1 sys.dm_fts_active_catalogs This dynamic management view returns information on the full-text catalogs that have some population activity in progress on the server.

2 sys.dm_fts_crawls This dynamic management view returns information about the full-text index populations currently in progress.

3 sys.dm_fts_crawl_ranges This dynamic management view returns information about the specific ranges (sub-tasks) related to a full-text index population currently in progress.

4 sys.dm_fts_memory_buffers This dynamic management view returns information about memory buffers that belong to a specific memory pool that are used as a part of a full-text crawl or a full-text crawl range.

5 sys.dm_fts_memory_pools This dynamic management view returns information about the memory pools used during a full-text crawl, or a full-text crawl range

9) CLR

Sr no DMV / DMF's Description1 sys.dm_clr_tasks Every SQL OS task starts a CLR Host task whenever it requires

execution of managed code. This DMV shows information on all CLR tasks currently running.

2 sys.dm_clr_appdomains Application domain is a construct in the CLR that is the unit of isolation for an application. This dynamic management view has a row for each application domain in the server.

3 sys.dm_clr_properties This DMV exposes all the properties related to CLR integration. This DMV requires CLR to be enabled and initialized. CLR is initialized by execution of any managed routines, types or triggers.

4 sys.dm_clr_loaded_assemblies This dynamic management view has a row for each managed (CLR) user assembly loaded into the server address space.

Page 9


CAUSES OF SQL SERVER SLOWNESS

SQL Server as an RDBMS the primary aim is to serve the request from client processes. SQL Server very intelligently makes uses of its own internal component and external system components in serving its request in the best optimal ways. However there are many factors which could impact the overall performance. Over a period of time the database size may grow, users may increase and the hardware performance starts degrading and the server may not be able to handle the increase in load.

SQL Server could also perform slowly if there is contention in tempdb which is used by various queries and slow running queries etc.

Below are some of the main causes of slow performance.

1) System resource.

2) Slow query / plan.

3) Tempdb contention.

4) Blocking / Deadlocks.

1) System Resource

Every process executed through SQL Server for its completion requires a system resource engagement. And these resources are CPU, IO, memory and network etc. These resource forms an integral part of every system. SQL making use of these resources can be used optimally or underused due to bad planning or application design. You might find that the problem is a resource that is running near capacity and that SQL Server cannot support the workload in its current configuration. To address this issue, you may need to add more processing power, memory, or increase the bandwidth of your I/O or network channel. But before going for high level machine or additional resource it is important to diagnose whether there is contention /Bad performance in resource or some configuration problem.

Below are some of the points causing high CPU conditions.

a) Statistics are not updated. (Or not updated with FullScan after Bulk Load). This can cause the queries to choose wrong execution plan. For example, Nested Loop join can be replaced by Hash join and will cause CPU spike.

b) Table Variables are used in queries which have large number of rows and used in joins with Regular Tables. Table variables do not maintain statistics and can lead to bad execution plan.

c) Some stored procedure which uses Parameters. Depending upon parameters, the count of retrieved rows can be very different. For example, the query which retrieves the data of 4 months or 2 months based on parameter. These queries need different execution plan and if they use execution plan which is already in memory, they can cause CPU to spike due to re-compilation.

d) When the large query is rolled back, it needs to rollback all the tasks which are done and can cause CPU to go high.

Page 10


2) Slow query / Plan.

A query executed against SQL Server goes through different phase (parsing, normalization compilation, and optimization) till it reaches its actual execution. During this process a query plan is generated and SQL Server tries to generate the most optimal plan for faster response time. Fastest response time doesn’t necessarily mean minimizing the amount of I/O that is used, nor does it necessarily mean using the least amount of CPU—it is a balance of the various resources.

These query plan generated could be resource intensive based on the type of joins finalized. The Hash operator and Sort operator scan through their respective input data. With read ahead being used during such a scan, the pages are almost always available in the buffer cache before the page is needed by the operator. Thus, waits for physical I/O are minimized or eliminated. When these types of operations are no longer constrained by physical I/O, they tend to manifest themselves by high CPU consumption. Nested loops can be Disk IO bound as per its working it needs to scan through the inner table for its match in outer table. Nested loop joins have many index lookups and can quickly become I/O bound if the index lookups are traversing to many different parts of the table so that the pages can’t fit into the buffer cache.

Query plan high depends on cardinality estimates for generating a better plan if statistics are not up to date this could result in inaccurate cardinality estimate causing query optimizer to get wrong input finally a non optimal plan causing slow query execution. As a part of diagnosis we can check DBCC show plan for EstimateRows and EstimateExecutions attributes.

3) Tempdb contention.

Tempdb plays an important role in SQL Server. It allows creation of temporary tables, user objects and internal SQL tables too, it also allows sorting operation in it. Application can run huge DDL and DML operation against tempdb database. Tempdb normally faces contention due to high SQL Server and application DML/DDL activities and disk space issues.

Normally user activities cause tempdb space issues resulting in SQL Server slowness and application task failure. Below table gives detailed information on space utilization in tempdb database.

User objects These are explicitly created by user sessions and are tracked in system catalog. They include the following:

Table and index.

Global temporary table (##t1) and index.

Local temporary table (#t1) and index.

Session scoped.

Stored procedure scoped in which it was created.

Table variable (@t1).

Session scoped.

Stored procedure scoped in which it was created.

Internal objects These are statement scoped objects that are created and

Page 11


destroyed by SQL Server to process queries. These are not tracked in the system catalog. They include the following:

Work file (hash join)

Sort run

Work table (cursor, spool and temporary large object data type (LOB) storage)

As an optimization, when a work table is dropped, one IAM page and an extent is saved to be used with a new work table.

There are two exceptions; the temporary LOB storage is batch scoped and cursor worktable is session scoped.

Version Store This is used for storing row versions. MARS, online index, triggers and snapshot-based isolation levels are based on row versioning. This is new in SQL Server 2005.

Free Space This represents the disk space that is available in tempdb.

It is important to constantly monitor tempdb activities and maintain sufficient space on disk having tempdb database files. Preferably for better performance and I/O its recommend to place tempdb on a dedicated disk drive. Having tempdb on disk having other database files could also result in used database failure.

4) Blocking/Deadlocks.

Blocking is a common cause of SQL Server slow response. Blocking normally happens due to improper application design / database design, ad-hoc queries running against SQL Server or long running queries.

Blocking happens when one connection from an application holds a lock and a second connection requires a conflicting lock type. This forces the second connection to wait, blocked on the first.

Most blocking problems happen because a single process holds locks for an extended period of time, causing a chain of blocked processes, all waiting on other processes for locks.

Common blocking scenarios include:

Submitting queries with long execution times.

A long-running query can block other queries. For example, a DELETE or UPDATE operation that affects many rows can acquire many locks that, whether or not they escalate to a table lock, block other queries. For this reason, you generally do not want to intermix long-running decision support queries and online transaction processing (OLTP) queries on the same database. The solution is to look for ways to optimize the query, by changing indexes, breaking a large, complex query into simpler queries, or running the query during off hours or on a separate computer.

One reason queries can be long-running, and hence cause blocking, is if they inappropriately use cursors. Cursors can be a convenient method for navigating through a result set, but using them

Page 12


may be slower than set-oriented queries.

Cancelling queries that were not committed or rolled back.

This can happen if the application cancels a query; for example, using the Open Database Connectivity (ODBC) sqlcancel function without also issuing the required number of ROLLBACK and COMMIT statements. Cancelling the query does not automatically roll back or commit the transaction. All locks acquired within the transaction are retained after the query is cancelled. Applications must properly manage transaction nesting levels by committing or rolling back cancelled transactions.

Applications that are not processing all results to completion.

After sending a query to the server, all applications must immediately fetch all result rows to completion. If an application does not fetch all result rows, locks may be left on the tables, blocking other users. If you are using an application that transparently submits Transact-SQL statements to the server, the application must fetch all result rows. If it does not (and if it cannot be configured to do so), you may be unable to resolve the blocking problem. To avoid the problem, you can restrict these applications to a reporting or decision-support database.

SQL Server serves the application base don the request received. The client application has almost total control over (and responsibility for) the locks acquired on the server. Although the SQL Server lock manager automatically uses locks to protect transactions, this is directly instigated by the query type sent from the client application and the way the results are processed. Therefore, resolution of most blocking problems involves inspecting the client application.

A blocking problem frequently requires both the inspection of the exact SQL statements submitted by the application and the exact behaviour of the application regarding connection management, processing of all result rows, and so on.

Page 13


QUERYING DMV’S AND DMF’S FOR PERFORMANCE DIAGNOSIS

As discussed above in the previous topic regarding the causes of SQL Server slowness. SQL Server 2005 and 2008 provides with lots of DMV’s and DMV’s for diagnosing the performance problem and identifying the resource utilization and contention which can prove to be an important tool for troubleshooting the performance problem. DMV’s help in collecting the system state and in-depth information eliminating the need of collecting dumps to certain extent.

I have categorized the queries collecting specific components related information based on the causes mentioned above for clarity.

1) System resource

a) CPU

This section has queries for collecting diagnostic information for CPU bottleneck.

1) One of the causes of high CPU is cost of compilation / Optimization. Below is the DMV and query which can be used to find the cost of optimization.

sys.dm_exec_query_optimizer_info

Using sys.dm_exec_query_optimizer_info, you can get a good idea of time spent optimizing. If we take 2 snapshots of this DMV, you can get a good feel of the time spent in optimizing for a given time period.

Select * from sys.dm_exec_query_optimizer_info

Or

Select * from sys.dm_exec_query_optimizer_info where counter = 'elapsed time'

In the above query we would look at the Elapsed time which is the time period elapsed due to optimizations. Since the optimization process is very CPU bound. We can get a good measure of how much of compile and Recompile time is contributing to the high CPU problem.

2) The query below can be used to retrieve query performing Sort and has operation. Sorts and hash Joins consumes high CPU.

Note: - The query will itself appear in the output.

select *from sys.dm_exec_cached_planscross apply sys.dm_exec_query_plan(plan_handle)where cast(query_plan as nvarchar(max)) like '%Sort%'

or cast(query_plan as nvarchar(max)) like '%Hash Match%'

Page 14


3) You can also monitor the SQL Server schedulers using the sys.dm_os_schedulers view to see if the number of runnable tasks is typically nonzero. A nonzero value indicates that tasks have to wait for their time slice to run, and thus high values for this counter may be indicative of a CPU bottleneck.

select scheduler_id,current_tasks_count,runnable_tasks_countfrom sys.dm_os_schedulers

where scheduler_id < 255

4) The following query will give you a high level view of which currently cached batches or procedures are using the most CPU. The query aggregates the CPU consumed by all statements with the same plan_handle (meaning that they are part of the same batch or procedure). If a given plan_handle has more than one statement you may have to drill in further to find the specific query that is the largest contributor to the overall CPU usage.

select top 50 sum(qs.total_worker_time) as total_cpu_time, sum(qs.execution_count) as total_execution_count,count(*) as number_of_statements, qs.plan_handle from sys.dm_exec_query_stats qs

group by qs.plan_handle

order by sum(qs.total_worker_time) desc

5) You can also use the below query for more detailed information on query taking high CPU and associated database and object.

Select highest_cpu_queries.plan_handle, highest_cpu_queries.total_worker_time, q.dbid, q.objectid, q.number, q.encrypted, q.[text]

from (select top 50 qs.plan_handle, qs.total_worker_time from sys.dm_exec_query_stats qs order by qs.total_worker_time desc) as highest_cpu_queries

cross apply sys.dm_exec_sql_text(plan_handle) as q

order by highest_cpu_queries.total_worker_time desc

Page 15


6) The below query gives top 15 stored procedures that were recompiled (plan_generation_num value>1)

select top 15sql_text.text,sql_handle,plan_generation_num,execution_count,dbid,objectid from sys.dm_exec_query_stats aCross apply sys.dm_exec_sql_text(sql_handle) as sql_text

where plan_generation_num >1

order by plan_generation_num desc

Note:

The output data for the DMV (sys.dm_exec_query_stats) is tied to the compiled plan and so its lifetime is based on the time the plan remains in cache. The table only contains plans for cached DML statements (SELECT, INSERT, UPDATE, DELETE). Non-DML operations (e.g., CREATE INDEX, BACKUP, TRUNCATE, etc) are never found in this table, nor are direct calls to CLR procedures/triggers. Thus sys.dm_exec_query_stats may not account for all the resources used and you may need to use detailed tracing to determine the resources consumed by a given operation.

Page 16


b) Memory

This section has queries for collecting diagnostic information for Memory bottleneck.

1) Consumption of memory by internal SQL Server components can be assessed using sys.dm_os_memory_clerks DMV.

A quick way to check the amount of memory consumed through the multi-page allocators is:

select sum(multi_pages_kb) from sys.dm_os_memory_clerksORselect type, sum(multi_pages_kb) from sys.dm_os_memory_clerks where multi_pages_kb != 0 group by type

A quick way to check the amount of memory consumed through the single-page allocators is:

select sum(single_pages_kb) from sys.dm_os_memory_clerksOR

select type, sum(single_pages_kb) from sys.dm_os_memory_clerks where single_pages_kb != 0 group by type

2) AWE allocated memory in SQL Server.

select sum(awe_allocated_kb) / 1024 as [AWE allocated, Mb]

from sys.dm_os_memory_clerks

3) Below query gives the system memory information, SQL Server memory and CPU details.

select physical_memory_in_bytes / 1024 / 1024 as physical_memory_mb,virtual_memory_in_bytes / 1024 / 1024 as virtual_memory_mb,bpool_committed * 8 / 1024 as bpool_committed_mb,bpool_commit_target * 8 / 1024 as bpool_target_mb,bpool_visible * 8 / 1024 as bpool_visible_mb,cpu_count,hyperthread_ratio,scheduler_countfrom sys.dm_os_sys_info

Page 17


4) Query retrieving information on memory consumption by extended stored procedures.

select type, sum (single_pages_kb + multi_pages_kb + virtual_memory_committed_kb + awe_allocated_kb + shared_memory_committed_kb) KBfrom sys.dm_os_memory_clerks where type like '%SQLXP%' group by type

5) If SQL Server witnesses memory pressure on box. SQL Server tries to decommit its own memory and also tries to shrink cache. This explains that there is memory pressure on box. There could be Memory Cache pressure due to internal or external factor. Cache pressure could result in faster flushing out of cached data and this can be identified by speed of clock algorithm.Information about clock hands movements is exposed through dm_os_memory_cache_clock_hands DMV. Each cache entry has a separate row for internal and external clock hand. If you see increasing rounds_count and removed_all_rounds_count, then the server is under internal/external memory pressure depending on which hand is moving.

select *from sys.dm_os_memory_cache_clock_handswhere

rounds_count > 0 and removed_all_rounds_count > 0

6) In the above queries we can further make modification for more detailed information by including sys.dm_os_memory_cache_counters DMV.

select distinct cc.cache_address, cc.name, cc.type, cc.single_pages_kb + cc.multi_pages_kb as total_kb, cc.single_pages_in_use_kb + cc.multi_pages_in_use_kb as total_in_use_kb,

cc.entries_count, cc.entries_in_use_count, ch.removed_all_rounds_count, ch.removed_last_round_count

from sys.dm_os_memory_cache_counters cc join sys.dm_os_memory_cache_clock_hands ch on (cc.cache_address = ch.cache_address)

where ch.rounds_count > 0 and ch.removed_all_rounds_count > 0

order by total_kb desc

Page 18


7) Using Ring Buffers to diagnose the memory pressure. Significant amount of diagnostic information can be obtained from ring buffers DMV - sys.dm_os_ring_buffers.

Each ring buffer keeps record of last number of notifications of a certain kind.

For example some of the ring buffer type are below:

a) You can use information from resource monitor notification to identify memory state changes. Internally SQL Server has a framework that monitors different memory pressures. When memory state changes, resource monitor task generates a notification. This notification is used internally by the components to adjust their memory usage according to the memory state and it is exposed to the user through sys.dm_os_ring_buffers DMV

select record, * from sys.dm_os_ring_buffers where ring_buffer_type = 'RING_BUFFER_RESOURCE_MONITOR'

b) This ring buffer will contain records indicating server out of memory conditions. This record will tell which operation has failed (commit, reserve or page allocation) and the amount of memory requested.

select record, * from sys.dm_os_ring_buffers where ring_buffer_type = 'RING_BUFFER_OOM'

c) This ring buffer will contain records indicating severe buffer pool failures, including buffer pool out of memory conditions. This record will tell what failure (FAIL_OOM, FAIL_MAP, FAIL_RESERVE_ADJUST, FAIL_LAZYWRITER_NO_BUFFERS) and the buffer pool status at the time.

select record from sys.dm_os_ring_buffers where ring_buffer_type = 'RING_BUFFER_BUFFER_POOL'

Page 19


C) I/O (Disk Subsystem).

For Diagnosing IO bottleneck Perfmon counters are very commonly used however these are DMV's which can be used for better information on wait stats, latch waits.

These latch waits account for the physical I/O waits when a page is accessed for reading or writing and it is not available in the buffer pool. When the page is not found in the buffer pool, an async I/O is posted and then the status of the IO is checked.

If IO has already completed, the worker proceeds normally otherwise it waits on PAGEIOLATCH_EX or PAGEIOLATCH_SH depending upon the type of request.

1) The following DMV query can be used to find IO latch wait stats:

select * from sys.dm_os_wait_stats

select wait_type, waiting_tasks_count, wait_time_msfrom sys.dm_os_wait_stats where wait_type like 'PAGEIOLATCH%' order by wait_type

Sample output as below.

wait_type waiting_tasks_count wait_time_ms

------------------------ -------------------- --------------------

PAGEIOLATCH_DT 0 0

PAGEIOLATCH_EX 19 750

PAGEIOLATCH_KP 0 0

PAGEIOLATCH_NL 0 0

PAGEIOLATCH_SH 230 10281

PAGEIOLATCH_UP 24 1046

You can identify an IO problem if your waiting_task_counts and wait_time_ms deviate significantly from what you see normally. For this, it is important to get a baseline of performance counters and key DMV query outputs when your SQL Server is running smoothly. These wait_types can provide you with a good indication if your IO subsystem is experiencing a bottleneck but it does not provide you with any visibility on the physical disk(s) that are experiencing the problem.

Page 20


2) You can use the following DMV query to find currently pending IO requests. You can execute this query periodically to check the health of IO subsystem and to isolate physical disk(s) that are involved in the IO bottlenecks.

select database_id, file_id, io_stall,io_pending_ms_ticksfrom sys.dm_io_virtual_file_stats(NULL, NULL)t1,

sys.dm_io_pending_io_requests as t2where t1.file_handle = t2.io_handle

Database_id File_Id io_stall io_pending_ms_ticks

----------- ------- -------------------- --------------------

9 2 10322 51

9 2 10322 51

7 1 104511 48

A sample output above. It shows that on a given database, there are three pending IOs at this moment. You can use the database_id and file_id to find the physical disk the files are mapped to. The io_pending_ms_ticks represent the total time individual IOs are waiting in the pending queue.

3) The following DMV query can be used to find which batches/queries are generating most IOs

select * from sys.dm_exec_query_stats

The query below retrieves logical reads and writes which can be used to find the logical reads done mainly for query plan and the records accessed from cache.

select top 5 (total_logical_reads/execution_count) as avg_logical_reads, (total_logical_writes/execution_count) as avg_logical_writes, Execution_countfrom sys.dm_exec_query_stats order by (total_logical_reads + total_logical_writes)/execution_count Desc

4) The query bellow retrieves physical reads which is actual IO done from the Disk subsystem and the records accessed from disk and not from memory cache.

select top 5 (total_physical_reads/execution_count) as avg_logical_reads, Execution_countfrom sys.dm_exec_query_stats order by (total_physical_reads/execution_count) Desc

Page 21


5) We can also include plan_handle column to retrieve the query plan for which there is high read/write values for further query tuning. Below is the query for the same.

select dbid, objectid, query_plan from sys.dm_exec_query_plan (0x05000400A3B73E12B8C10705000000000000000000000000)

Output as below:

=============

dbid objectid query_plan

------ ----------- ---------------------------------------------------------------------------------------------------------------

4 306100131 <ShowPlanXML xmlns=http://schemas.microsoft.com/sqlserver/2004/07/showplan Version="1.0" Build="9.00.3042.00"><BatchSequence><Batch><Statements><StmtSimple StatementText="CREATE PROCEDURE sp_get_sqlagent_properties
AS
BEGIN
D

Page 22


http://schemas.microsoft.com/sqlserver/2004/07/showplan

2) Blocking.

Blocking can be broadly categorized into waits for logical locks such as wait to acquire an X lock on a resource or waits that results from lower level synchronization primitives such as latches. Blocking could result in slow execution of query and system pressure. Finding out waittype of blocked process can be helpful in analysing the cause and troubleshooting blocking. SQL Server 2005 provides more detailed and consistent wait information, reporting approximately 125 wait types compared with SQL Server 2000’s 76 wait types.

1) The DMV’s that provide this information range from sys.dm_os_wait_stats for overall and cumulative wait for SQL Server to the session specific sys.dm_os_wait_tasks that breakdown waits by session. This DMV provides details on the wait queue of tasks, which are waiting on some resource.

select * from sys.dm_os_wait_stats

select * from sys.dm_os_waiting_tasks

2) We can identify the blocked session and blocking session ID from the above query and can provide the session ID in the query below for getting information for blocked process.

select session_id, wait_type,wait_duration_ms,blocking_session_id, resource_description from sys.dm_os_waiting_tasks where session_id=53

The above query will output the session id (53) being blocked by a blocking session id (blocking_session_id) and the lock wait time in milliseconds.

3) Once we get the above output we can further diagnose the blocking for more information about granted locks or waiting for locks, using the sys.dm_tran_locks DMV. This dmv in conjunction with sys.partitions will give us information like the lock mode and active session status.

select request_session_id, resource_type, resource_database_id, (case resource_type

WHEN 'OBJECT' then object_name(resource_associated_entity_id)

WHEN 'DATABASE' then ' ' ELSE (select object_name(object_id) from sys.partitions

where hobt_id=resource_associated_entity_id)END) as object_name, resource_description, request_mode, request_status

from sys.dm_tran_locks

The above query will give us the output as sysprocesses in SQL 2000 however more detailed information can be retrieved using this DMV's.

Page 23


4) This query lists the top 10 waits in SQL Server. These waits are cumulative but you can reset them using dbcc sqlperf ([sys.dm_os_wait_stats], clear)

select top 10 * from sys.dm_os_wait_stats order by wait_time_ms desc

3) Monitoring Worker threads and Schedulers

1) To view kernel and user mode time for thread with the dynamic management view (DMV) sys.dm_os_threads. When combined with sys.dm_os_workers and sys.dm_os_schedulers it is possible to see details that pertain to the system and scheduler utilization on an active server.

SELECT kernel_time + usermode_time as TotalTime, * FROM sys.dm_os_threadsORDER BY kernel_time + usermode_time desc

2) The following query can be run against a live SQL Server 2005 installation to see the milliseconds that have elapsed since the scheduler last checked the timer list.

SELECT yield_count, last_timer_activity, (SELECT ms_ticks from sys.dm_os_sys_info) - last_timer_activity AS MSSinceYield, * FROM sys.dm_os_schedulers WHERE is_online = 1 and is_idle <> 1 and scheduler_id < 255

The above query normally can be used during scheduler hung or 17883/ 17884 errors.

Page 24


Page 25


DMV’S AND DMF’S INTRODUCED IN SQL SERVER 2008

Below are some of the DMV’s introduced in SQL Server 2008.

Srno DMV / DMF's1 sys.dm_cdc_errors 2 sys.dm_cdc_log_scan_sessions 3 sys.dm_cryptographic_provider_properties 4 sys.dm_database_encryption_keys 5 sys.dm_db_mirroring_auto_page_repair 6 sys.dm_db_mirroring_past_actions 7 sys.dm_exec_query_memory_grants 8 sys.dm_exec_query_resource_semaphors 9 sys.dm_exec_query_transformation_stats 10 sys.dm_filesream_oob_handles 11 sys.dm_filesream_oob_requests 12 sys.dm_os_memory_brokers 13 sys.dm_os_memory_nodes 14 sys.dm_os_nodes 15 sys.dm_os_process_memory 16 sys.dm_os_spinlock_stats 17 sys.dm_os_sys_memory 18 sys.dm_os_resource_governor_configuration 19 sys.dm_os_resource_governor_resource_pools 20 sys.dm_os_resource_governor_workload_groups 21 sys.dm_tran_commit_table

Page 26


22 sys.dm_xe_map_values 23 sys.dm_xe_object_columns 24 sys.dm_xe_objects 25 sys.dm_xe_packages 26 sys.dm_xe_session_event_actions 27 sys.dm_xe_session_events 28 sys.dm_xe_session_object_columns 29 sys.dm_xe_session_targets 30 sys.dm_xe_sessions

Summary:-

========

From the above information and the queries DMV’s definitely proves to be an important tool for diagnosing and troubleshooting system and performance problem. These DMV queries can be scheduled as a job in SQL Server to be executed in fixed interval or executing during performance problem and getting the output in text format. This document is more emphasised on SQL 2005 however I may include more of SQL 2008 in later versions.

Page 27


Page 28


Documents

Performance Diagnosis Using DMV's & DMF's in SQL 2005 & 2008