Gems to help you troubleshoot query performance

© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1

2

3



4


5

And what are the main reasons a query displays poor performance?

We might find queries are using what may be deemed as excessive resources, such as CPU, memory or I/O.

That may very well be a consequence of an inadequate indexing strategy for our workloads.

Lack of useful or updated statistics might also lead to inefficient execution plans, or slowness might be a symptom of extended blocking.

And last but not least, several server-wide configurations can and will affect overall performance, from the well-known MaxDOP to CPU Affinity, from Lock threshold to max worker threads. Misconfiguring these can have a severe impact.

These are some of the aspects we will cover next.

Checklist for Analyzing Slow-Running Queries (http://msdn.microsoft.com/en-us/library/ms177500.aspx)

How to troubleshoot slow-running queries on SQL Server 7.0 or on later versions (http://support.microsoft.com/kb/243589/en-us)

6


http://msdn.microsoft.com/en-us/library/ms177500.aspx

http://support.microsoft.com/kb/243589/en-us

When a query or update takes longer than expected, one should ask the following questions, which address the reasons for slow-running queries:

1. Is the performance problem related to a component other than queries? For example, is the problem slow network performance? Are there any other components that might be causing or contributing to performance degradation?

Performance Monitor can be used to monitor the performance of SQL Server and non-SQL Server related components. Several DMVs are also available that allows us to explore how SQL Server resources are being used.

2. If the performance issue is related to queries, which query or set of queries is involved?

Use SQL Server Profiler to help identify the slow query or queries.

Use the sys.dm_exec_query_stats and sys.dm_exec_requests dynamic management views to find similar queries that collectively consume a large number of resources.

3. How do I analyze the performance of a slow-running query?

After you have identified the slow-running query or queries, which may even be the source of blocking in the system, you can further analyze query performance by producing a Showplan. The showplan can be a text, XML, or graphical representation of the query execution plan that the query optimizer generates. You can produce a Showplan using Transact-SQL SET options, SQL Server Management Studio, or SQL Server Profiler.

The information gathered by these tools allows you to determine how a query is executed by the SQL Server query optimizer and which indexes are being used. Using this information, you can determine if performance improvements can be made by rewriting the query, changing the indexes on the tables, or perhaps modifying the database design.

4. Was the query optimized with useful statistics?

The query optimizer uses statistics to create query plans that improve query performance. For most queries, the query optimizer already generates the necessary statistics for a high-quality query plan; in a few cases, you need to create additional statistics or modify the query design for best results.

5. Are suitable indexes available? Would adding one or more indexes improve query performance?

Analyze the execution plan to ascertain the possibility of creating missing indexes. Database Engine Tuning

7


http://msdn.microsoft.com/en-us/library/ms189741(v=sql.105).aspx

http://msdn.microsoft.com/en-us/library/ms177648(v=sql.105).aspx

Advisor can also recommend the creation of necessary statistics.

6. Are there any data or index hot spots? If you have a large volume of data, do you need to partition it?

Data manageability is the main benefit of partitioning, but if your tables and indexes on them are partitioned similarly, partitioning can also improve query performance.

Consider using disk striping. Disk striping can be implemented by using RAID (redundant array of independent disks) level 0, where data is distributed across multiple disk drives.

7. Is the query optimizer provided with the best opportunity to optimize a complex query?

Simplify queries and allow the QO to do its job more efficiently. Perhaps storing intermediate results in temporary tables that are later on joined in a statement that returns the expected results might produce simpler and more efficient plans.

9/15/2016 3:00 PM

7

8


9


10


The query plan consists of physical operators which implement the operation described by logical operators.

The order of these determine how data is accessed. A query plan also determines what support objects must be created, such as worktables or workfiles in tempdb. Remember that these decisions rely heavily on statistics, which are somewhat shown in the execution plan in the form of estimated rows.

Some physical operators may perform other types of operations e.g. the Aggregate operator calculates an expression containing MIN, MAX, SUM, COUNT or AVG, for example.

And we might have execution warnings.

More information can be found here:

Showplan Logical and Physical Operators Reference (https://msdn.microsoft.com/en-us/library/ms191158.aspx)

11


https://msdn.microsoft.com/en-us/library/ms191158.aspx

Flow of data is read from right to left

The width of the arrows gives an indication of the number of rows in each dataset, like the one coming from the Customer table CL IX scan. This can often be a clue to high resource usage

For joins: the outer table appears on top

Resultsets are created from each join pair which is then passed to the next join.

Two crucial facts affect join performance:

• Order in which more than two tables are joined

• Selection of outer/inner table

Joins producing smaller result sets are performed first

Local predicates are applied before the join

Joins that reduce the number of rows are performed first

Aggregation may be performed before the join

This is all dependent on how accurate statistics the SQL Server Query processor has available at compile time.

Query:

select p.Title + ' ' + p.FirstName + ' ' + p.LastName as FullName, c.AccountNumber, s.Name

from Person.Person p

join Sales.Customer c

on c.PersonID = p.BusinessEntityID

join Sales.Store s

on s.BusinessEntityID = c.StoreID

where p.LastName = 'Koski';

12


There are 2 importante concepts to grasp when dealing with execution plans in SQL Server. Those are the:

Query hash, which is a binary hash value calculated on the query and used to identify queries with similar logic.

In turn, a SQL handle is a token for the SQL text that relates to a batch.

…And we have the query_plan_hash, which is a binary hash value calculated on the query execution plan and used to identify similar query execution plans.

In turn, the plan handle is a sort of token for an execution plan. This handle is a transient identifier and remains constant only while the plan remains in the cache.

We will further explore the use of these concepts in this session.

13


14


15



16


17


18


19

When you are troubleshooting query performance, metrics are never too much. With that in mind, based on customer feedback, but also our own CSS support, we have added runtime information to Showplan, and exposed a new Extended Event in SQL Server 2016. We are adding the new Showplan info and Extended Event in SQL Server 2014 Service Pack 2, and as usual deliver added value to in-market versions.

Regarding Showplan, take a test query on which we are doing a Clustered index Scan, and look the information available until SQL Server 2014:

<RunTimeInformation> <RunTimeCountersPerThread Thread="0" ActualRows="8001" ActualEndOfScans="1" ActualExecutions="1" /> </RunTimeInformation>

We can see the scan returned 8001 rows, but that doesn’t give you much insight regarding performance of the scan.

But below we can observe what kind of detail we can get from runtime stats in SQL Server 2016. These include the actual rows read, I/O Reads, CPU time, and elapsed time, all exposed per-thread:

<RunTimeInformation> <RunTimeCountersPerThread Thread="0" ActualRows="8001" ActualRowsRead="10000000" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="965" ActualCPUms="965" ActualScans="1" ActualLogicalReads="26073" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" /> </RunTimeInformation>

We have also introduced a new Extended Event (query_thread_profile), allowing more insight on the performance of each node and thread:

Note: Showplan time scale is shown in milliseconds, while the xEvent shows microseconds for CPU and total time.


20

We have also introduced a new Extended Event (query_thread_profile), allowing more insight on the performance of each node and thread:

Note: Showplan time scale is shown in milliseconds, while the xEvent shows microseconds for CPU and total time.


21


22

If a row is filtered out by an non-sargable predicate, nowhere in the query plan will show that. Non-sargable pushing happens very frequently. This is very confusing from troubleshooting perspective where you can see high CPU or large logical reads (from tracing), but the query plan doesn't reflect that.

Some query execution plans in Microsoft SQL Server include pattern of evaluating a filter on top of a table or index scan/range operation. Some parts of the filter predicate may match an index key and may therefore be used to run an index seek or range scan. The remaining parts of the predicate are known as "residual" and must be evaluated for each row output by the scan or range operation. This would correspond to a filter operator. However, to improve performance, SQL Server can push such a filter down to the table access operator itself.


23

Although this approach improves performance overall, under some conditions (for example, in the case of an inaccurate cardinality estimation that is related to parameter sensitivity), the scan-below filter may be processing a larger number of rows than expected. This fact may be hidden during query performance troubleshooting when you are using an actual execution plan, because the actual number of rows that is returned will correspond to the number of rows after the residual predicate is applied and not the actual number of rows that are scanned from a table or index.

Improved diagnostics for query execution plans that involve residual predicate pushdown in SQL Server 2012 and 2014 - https://support.microsoft.com/en-us/kb/3107397


24

https://support.microsoft.com/en-us/kb/3107397


25


26

Estimated query plans do not contain any run-time information (e.g. actual row counts, memory grant usage, execution time warnings, etc.).

Query completion is a prerequisite for the availability of an actual query plan.

Actual query plans unsuitable for troubleshooting complex performance issues:

• Long running queries

• Queries that run indefinitely and never finish execution.

Statistics profile infrastructure must be enabled.

• Specifying Include Live Query Statistics in SSMS enables the statistics infrastructure for the current query session.

Data is stored in DMV (sys.dm_exec_query_profiles) which is not highly scalable for widespread use.

Three other ways to enable the stats profile infra to view the LQS from other sessions (such as from Activity Monitor)*

• SET STATISTICS XML ON;

• SET STATISTICS PROFILE ON;

• Enable the query_post_execution_showplan extended event.

Enable in the target session.

query_post_execution_showplan extended event is a server wide setting that enable live query statistics on all sessions. To enable extended events, see Monitor System Activity Using Extended Events.

LQS requires the database level SHOWPLAN permission to populate the Live Query Statistics results page, and the server level VIEW SERVER STATE permission.

27


https://msdn.microsoft.com/en-us/library/bb630319.aspx


28

Provide side-by-side comparison of two different showplans, for easier identification of similarities and changes, that explain different behaviors.

Used with Query Store for the same purpose.

Revamp the entire Showplan analysis experience:

• Improved plan navigation

• Improved real estate

• Improved properties window

• Improved access to query text

• Offline comparison

• Compare plans between SQL Server versions

• Find differences and similarities

• Analyze plan shapes

• Analyze properties between plans

• Rich set of user options

Scenarios:

• Find why a query or batch suddenly slowed down as compared to a previous state.

Understand the impact of a query refactor by comparing to previous state.

Analyze how a specific performance-enhancing change introduced to the design (like an index) has effectively changed the plan.


29


32


33

In SQL Server 2014 SP2 and SQL 2016 post-RTM


34

This extended event is intended to help detect inaccurate or insufficient memory grant, when grant is larger than 5MB as minimum.


35


37


39

While spills are something that denotes some type of inefficiency, starting with IO usage where only memory usage was expected, but also estimation skews, they are introduced so that a query execution doesn’t exhaust the server’s memory and can effectively continue execution in spite its allocated resources are running out (memory spills).

Much like the enhancements in showplan, the xEvent enhancements introduced here are aimed at giving more contextual information over these unwarranted events and allow you to take action to optimize from a database and query design standpoints, to influence the optimizer to produce different choices.

The sort_warning and hash_warning extended events have been available for several versions of SQL Server, and they give you insight on the fact those events have happened and what type. It is actually the same information as SQL Profiler sort_warning and hash_warning events have available.

With SQL Server 2016, the information (event fields) you can retrieve from these 2 warnings have been greatly enhanced. Now can see the DOP level, how many pages were involved in the spill, memory statistics for the operator, the row count for the spill context (uniquely hashed for hash warning), besides the recursion level for hashing operations and the warning type, which were already present.


40



While spills are something that denotes some type of inefficiency, starting with IO usage where only memory usage was expected, but also estimation skews, they are introduced so that a query execution doesn’t exhaust the server’s memory and can effectively continue execution in spite its allocated resources are running out (memory spills).

Much like the enhancements in showplan, the xEvent enhancements introduced here are aimed at giving more contextual information over these unwarranted events and allow you to take action to optimize from a database and query design standpoints, to influence the optimizer to produce different choices.

The sort_warning and hash_warning extended events have been available for several versions of SQL Server, and they give you insight on the fact those events have happened and what type. It is actually the same information as SQL Profiler sort_warning and hash_warning events have available.

With SQL Server 2016, the information (event fields) you can retrieve from these 2 warnings have been greatly enhanced. Now can see the DOP level, how many pages were involved in the spill, memory statistics for the operator, the row count for the spill context (total sorted for sort warnings), besides the warning type, which was already present.


41



For hashing related spills, we also added a new Extended Event hash_spill_details. These will be triggered at the end of hash processing, if there is insufficient memory to process the build input of a hash join. Use this event together with any of the query_pre_execution_showplan or query_post_execution_showplan events to determine which operation in the generated plan is causing the hash spill.

Note that the event fields on this one are also present in the existing warning events (see below), in the proper context of either a sort or warning.


42


43


45


46


47

48

49


50


Technology

Gems to help you troubleshoot query performance