Upload
dageop-ltd
View
201
Download
3
Embed Size (px)
Citation preview
www.dageop.com
Optimising Queries
®
OQ-01 Query Optimiser Architecture
DR. SUBRAMANI PARAMASIVAM
(MANI)
About me
Dr. SubraMANI ParamasivamPhD., MCT, MCSE, MCITP, MCP, MCTS, MCSACEO, Principal Consultant & Trainer @ DAGEOP (UK) Email: [email protected] Blog: http://dataap.org/blog
Follow Us
https://www.facebook.com/pages/YOUR-SQL-MAN-LTD/
http://www.youtube.com/user/YourSQLMAN
https://twitter.com/dageop
https://uk.linkedin.com/in/dageop
Proud Sponsor
• SQLBits• SQL Saturdays• MCT Summit• SQL Server Geeks
Summit• Data Awareness
Programme• Dageop’s Data Day
®
www.DataAP.org
SPEAKER
Contents
• OQ-01 Query Optimiser Architecture• Phases
• Strategies
• Data access plans
• Auto-parameterisation
• Avoiding recompilation of queries
www.dageop.comOptimizing Queries
Optimising Queries
• Optimising Query is very important task to maintain the server resources.
• Most of the time server is hit by performance issues.
• Clean the SQL Server Cache.
• Well written T-SQL Queries will help to optimise.
www.dageop.comOptimizing Queries
Introduction to Query Optimiser Architecture
2 Major Components in SQL Server• SQL Server Database Engine
• Reading data between disk and memory
•Relational Engine (Query Processor)• Accepts queries, analyse & executes the plan.
www.dageop.comOptimizing Queries
Introduction to Query Optimiser Architecture
• Analyse Execution Plans (Cannot consider every plan)
• Estimate the cost of each plan
• Select the plan with low cost
www.dageop.comOptimizing Queries
Plan1
Plan2
Plan3
Plan4
Plan5
Plan6
Plan7
Plan1 – 80%
Plan2 – 60%
Plan3 – 10%
Plan5 – 90%
Plan6 – 70%
Plan3 – 10%
Analyse Estimate Select
COST X 2Plan & itself
BETTER UNDERSTANDING of Query Optimiser is a must
for both DBA & DEVELOPERS.
Main Challenges are
Cardinality & Cost Estimations
www.dageop.comOptimizing Queries
Query Optimiser
• 1st Part - Searching or enumerating candidate plans
• 2nd Part - Estimates the cost of each physical operator in that plan (I/O, CPU, and memory)
• This cost estimation depends mostly on the algorithm used by the physical operator, as well as the estimated number of records that will need to be processed (Cardinality Estimation).
PRIMARY WAY to interact with Query Optimizer is through EXECUTION PLANS
www.dageop.comOptimizing Queries
Query Optimiser
• Execution Plan• Trees consisting of a number of physical operators
• Physical operators• Index Scan
• Hash Aggregate
• result operator (root element in the plan)
• Nested Loops Join
• Merge Join
• Hash Join
• Algorithms
• Can be saved as .sqlplan (Can be viewed in SSMS)
www.dageop.comOptimizing Queries
Query Optimiser• Operators
• Logical• Relational algebraic operation• Conceptually describes what operation needs to be performed
• Physical• Implement the operation described by logical operators• Access columns, rows, index, table, views, calculations, aggregations, data integrity
checks • 3 methods
• Init() – Initializes a connection• GetNext() – 0 to many calls to get data• Close() – Some cleanup and close the connection.
• Query Optimizer chooses efficient physical operator based on logical.
www.dageop.comOptimizing Queries
Query Optimiser
www.dageop.comOptimizing Queries
Logical Physical Logical & PhysicalBitmap Create Assert AggregateBranch Repartition Bitmap Clustered Index ScanCache Clustered Index Delete Clustered Index ScanDistinct Clustered Index Insert Clustered Index UpdateDistinct Sort Clustered Index Merge CollapseDistribute Streams Hash Match Compute ScalarEager Spool Merge Join ConcatenationFlow Distinct Nested Loops Cursor
Full Outer Join Nonclustered Index Delete Inserted ScanGather Streams Index Insert Log Row ScanInner Join Index Spool Merge IntervalInsert Nonclustered Index Update Index ScanLazy Spool Online Index Insert Index SeekLeft Anti Semi Join Parallelism Parameter Table ScanLeft Outer Join RID Lookup Remote DeleteLeft Semi Join Stream Aggregate Remote Index ScanPartial Aggregate Table Delete Remote Index SeekRepartition Streams Table Insert Remote InsertRight Anti Semi Join Table Merge Remote QueryRight Outer Join Table Spool Remote ScanRight Semi Join Table Update Remote UpdateRow Count Spool SegmentSegment Repartition SequenceUnion Sequence ProjectUpdate Sort
SplitSwitchTable ScanTable-valued FunctionTopWindow Spool
Query Optimiser
• Invalid Plans• Removal of an index
• Removal of a constraint,
• Significant changes made to the contents of the database
• SQL Server under memory pressure
• Changing configuration options - max degree of parallelism
• Discard the plan cache and new optimization generated
www.dageop.comOptimizing Queries
Query Optimiser
• Highly complex pieces of software
• Even 30 years of research, still has technical challenges.
HINTS
• override the operations of the Query Optimizer
• caution and only as a last option
• Influence Query Optimizer to use certain plans
www.dageop.comOptimizing Queries
Query Optimiser - Interaction
www.dageop.comOptimizing Queries
Query Optimizer
Execution Plans
Trees, Algorithm, Physical operator
ACTUAL ESTIMATED
Graphics, Text, XML format
Phases
• Once we execute the SQL Statement, it will follow certain procedures from SQL Statement to Query Result
www.dageop.comOptimizing Queries
Parsing Query Compilation
Query Optimization
Query Execution
Phases
• If the Query is already in the plan cache then it will not generate any new execution plan for that query.
• Parsing:• The Query’s syntax will be validated and query is transformed in a tree.
Checks the objects for its existence which are used in the query.
• After the query validated, the final tree is formed.
• Query Compilation:• Query Tree will be compiled here.
www.dageop.comOptimizing Queries
Phases
• Query Optimization• The query optimizer takes as input the compiled query tree generated in the
previous step and investigates several access strategies before it decides how to process the given query.
• It will analyse the query to find most efficient execution plan.
• Query Execution• After the execution plan is generated, it is permanently stored and executed• Note: For Some statements, parsing and optimization can be avoided if the
database engines know that there is only one viable plan. This is called trivial plan optimization.
www.dageop.comOptimizing Queries
Strategies
• As a DBA, we need to choose right strategy for each SQL statement like scripts should be well written before executing the SQL statement.
• Statistics will be used to understand the performance of each query.
• Enable Set Statistics IO before query run. It will display the following information• How many scans were performed
• How many logical reads (reads in cache) were performed
• How many physical reads (reads on disk) were performed
• How many pages were placed in the cache in anticipation of future reads (read-ahead reads)
www.dageop.comOptimizing Queries
Strategies
• We can understand the query performance from statistics IO, If the query is good then the logical reads of the query result should be lower and few physical reads and scans.
• Enable Set Statistics Time, it will display the execution time of the query. It purely depends on the total activity of the server.
• Enable show execution plan, to see how the query performed.
• These are things will helpful for choosing the right strategy.
www.dageop.comOptimizing Queries
www.dageop.comOptimizing Queries
StrategiesChoose the right strategies based on below information from SQL Server
Data Access Plans
• If we need to optimize the data access plans then we need to start from creating files for database, Index on the tables and T SQL Statements.
• Steps:• Organize the file groups and files
• Apply partitioning in big tables
• Create the appropriate indexes/covering indexes
• Defragment the indexes
• Identify inefficient TSQL
• Diagnose performance problems
www.dageop.comOptimizing Queries
Data Access Plans
• Organize the file groups and files• Initially two files will be created while created a database (.mdf & .ldf). • .mdf file : Primary data file for each database. All system objects will be stored in this
file, including the user defined objects if .ndf file is not there.• .ndf file: These are secondary data files, these are optional. These files will have user
created objects.• .ldf file: These are the transaction log files. This could be one or more files for single
database.
• File Group:• Database files are logically grouped for better performance and improved
administration on large databases. When a new SQL Server database is created, the primary file group is created and the primary data file is included in the primary file group. Also, the primary group is marked as the default group.
www.dageop.comOptimizing Queries
Data Access Plans
• To obtain the performance of the data access, Primary file group must be separate and it should be only for system objects.
• Need to create one more file called secondary data file for user defined objects.
• Separating the system objects will improve the performance enhance the ability to access tables in cases of serious data failures.
• For frequently accessed tables containing indexes, put the tables and the indexes in separate file groups. This would enable reading the index and table data faster
www.dageop.comOptimizing Queries
Data Access Plans
• Apply partitioning in big tables• Table partitioning is nothing but splitting the large table into multiple small
tables so that queries have to scan less amount for data retrieving.
• Consider partitioning big fat tables into different file groups where each file inside the file group is spread into separate physical disks (so that the table spans across different files in different physical disks). This would enable the database engine to read/write data operations faster.
• Partitioning is very much needed for history tables.
• 2 types• Physical
• Logical
www.dageop.comOptimizing Queries
Data Access Plans
• Create the appropriate indexes/covering indexes
• Create Non-Clustered index on frequently used columns
• Create index on column which is used for joining the tables
• Index for foreign key columns
• Create covering index for particular columns which are using frequently.
• Defragment the indexes
• Once index has created , it should maintain properly to avoid defragmentation.
• Maintaining the index will lead to performance gain.
www.dageop.comOptimizing Queries
Data Access Plans
• Identify inefficient TSQL• Don’t use * in the select statements. Mention column names while retrieving
the data. It will improve the performance of the query
• Avoid Deadlocks between two objects.
• Diagnose the performance issue• SQL Server has many tools to Monitor and diagnose the issues.
• Accessing the data will be more easier.
www.dageop.comOptimizing Queries
Auto-Parameterisation
• SQL Server Query Optimizer might decide to parameterize some of the queries.
• In this case the specific parameter will not make any impact on the execution plan. It will return the same execution plan.
• In SQL Server 2005 forced parameterization has been introduced and is disabled by default and can be enabled in database level.
• To differentiate from forced parameterization, auto-parameterization is also referred as simple parameterization.
www.dageop.comOptimizing Queries
Avoiding Recompilation of Queries
• In SQL Server 2005 recompiles the stored procedures, only the statement that causes recompilation is compiled, rather than the entire procedure.
• Recompilation will occur in following ways,• On Schema change of objects
• On Change of the SET options
• On statistics change of tables.
www.dageop.comOptimizing Queries
Avoiding Recompilation of Queries
• On Schema Change of Objects• Adding and dropping column, constraints, index, indexed view and trigger.
• On change of the SET options• When executing the stored procedure, the compiled plan is created and it will store
the environment setting of a connection (SET OPTION) .• Recompilation will occur, if the stored procedure run on different environment and
with different SET option then it will not use the existing plan which it is created first time.
• On statistics change of tables• SQL server maintains a modification counter for each table and index.• If the counter values exceed the defined threshold, the previously create compiled
plans is considered stale plan and new plan will be created.
www.dageop.comOptimizing Queries
Avoiding Recompilation of Queries
• Temporary table modification counter threshold is 6. Stored procedure will be recompiled when stored procedure create a temp table insert 6 or more rows into this table.
• For Permanent table the counter threshold is 500.
• We can increase the temp table counter threshold to 500 as same as permanent table.
• Use table variable instead of Temporary table.
www.dageop.comOptimizing Queries
Review
Query Optimiser ArchitecturePhases
Strategies
Data access plans
Auto-parameterisation
Avoiding recompilation of queries
www.dageop.comOptimizing Queries
®
www.dageop.com