70
C O N F I D E N T I A L Jun 28, 2022 1 MarketView Database Tuning Best Practice StarCite Engineering May 13, 2009

MarketView Database Tuning Best Practice

  • Upload
    totie

  • View
    31

  • Download
    0

Embed Size (px)

DESCRIPTION

MarketView Database Tuning Best Practice. StarCite Engineering May 13, 2009. Agenda. Identify the hot-spot Long Running DML Inefficient Indexes More dig into (tempdb, disk usage, etc) Improve the Long Running DML Improve the Inefficient indexes TempDB improvement Production Data - PowerPoint PPT Presentation

Citation preview

Page 1: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 1

MarketView Database Tuning Best Practice

StarCite Engineering

May 13, 2009

Page 2: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 2

Agenda

• Identify the hot-spot– Long Running DML– Inefficient Indexes– More dig into (tempdb, disk usage, etc)

• Improve the Long Running DML

• Improve the Inefficient indexes

• TempDB improvement

• Production Data

• Some hot topics

• Further topics

Page 3: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 3

Identify the hot-spot - Long Running DML

• Powerful DMV & DMF

Dynamic Management View & Dynamic Management Function

Page 4: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 4

Long Running DML Identify top 10 query by CPU cost

Page 5: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 5

Long Running DML Identify top 10 query by elapsed time

Page 6: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 6

Long Running DML MKV tools can identify to specific table

SELECT TOP 50qs.total_worker_time/qs.execution_count as [Avg CPU Time],qt.text as query_text, qt.dbid,qt.objectid FROM sys.dm_exec_query_stats qscross apply sys.dm_exec_sql_text(qs.sql_handle) as qtWhere qt.text like '%table_name%'ORDER BY [Avg CPU Time] DESC

Page 7: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 7

Inefficient indexes Identify the missing Index

Page 8: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 8

Inefficient indexes Identify the most costly unuage index

Page 9: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 9

Dig into

• TempDB

• Data file fragmentation

• Log file fragmentation

• Disk usage

Page 10: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 10

TempDB

• TempDB

Temp Table Temp Store Procedure

Table variables

Internal Table

Row version

Cursor

Provide it a best environment!Put it on faster disk(raid5).

Provide separated disk for tempdb, and avoid the working files sit with os files.

When you have multiple CPUs may increase tempdb file numbers can help.

Increase tempdb initial size it depends?

Page 11: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 11

TempDB (Space Used By)User objects These are explicitly created by user sessions and are tracked in system catalog. They

include the following: Table and index. Global temporary table (##t1) and index. Local temporary table (#t1) and index. Session scoped. Stored procedure scoped in which it was created. Table variable (@t1). Session scoped. Stored procedure scoped in which it was created.

Internal objects These are statement scoped objects that are created and destroyed by SQL Server to process queries. These are not tracked in the system catalog. They include the following:

Work file (hash join) Sort run Work table (cursor, spool and temporary large object data type (LOB) storage) As an optimization, when a work table is dropped, one IAM page and an extent is

saved to be used with a new work table.There are two exceptions; the temporary LOB storage is batch scoped and cursor

worktable is session scoped.

Version Store This is used for storing row versions. MARS, online index, triggers and snapshot-based isolation levels are based on row versioning. This is new in SQL Server 2005.

Free Space This represents the disk space that is available in tempdb.

Page 12: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 12

TempDB (Space Used By)User objects

+

Internal objects

+

Version Store

+Free Space

Total Space Usage

Select SUM (user_object_reserved_page_count)*8 as user_objects_kb, SUM (internal_object_reserved_page_count)*8 as internal_objects_kb, SUM (version_store_reserved_page_count)*8 as version_store_kb, SUM (unallocated_extent_page_count)*8 as freespace_kbFrom sys.dm_db_file_space_usageWhere database_id = 2

Here is one sample output (with space in KBs).user_objets_kb internal_objects_kb version_store_kb freespace_kb---------------- -------------------- ------------------ ------------8736 128 64 448

Page 13: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 13

TempDB

Some of the common issues with tempdb are as follows:

• Running out of storage space in tempdb.

• Queries that run slowly due to the I/O bottleneck in tempdb.

• Excessive DDL operations leading to a bottleneck in the system tables.

• Allocation contention.

Page 14: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 14

TempDBlook for multiple counters to cross check the validity of your findings.• PhysicalDisk Object: Avg. Disk Queue Length represents the average number of physical read and write

requests that were queued on the selected physical disk during the sampling period. If your I/O system is overloaded, more read/write operations will be waiting. If your disk queue length frequently exceeds a value of 2 during peak usage of SQL Server, then you might have an I/O bottleneck.

• Avg. Disk Sec/Read is the average time, in seconds, of a read of data from the disk. Any numberLess than 10 ms - very goodBetween 10 - 20 ms - okayBetween 20 - 50 ms - slow, needs attentionGreater than 50 ms – Serious I/O bottleneck

• Avg. Disk Sec/Write is the average time, in seconds, of a write of data to the disk. Please refer to the guideline in the previous bullet.

• Physical Disk: %Disk Time is the percentage of elapsed time that the selected disk drive was busy servicing read or write requests. A general guideline is that if this value is greater than 50 percent, it represents an I/O bottleneck.

• Avg. Disk Reads/Sec is the rate of read operations on the disk. You need to make sure that this number is less than 85 percent of the disk capacity. The disk access time increases exponentially beyond 85 percent capacity.

• Avg. Disk Writes/Sec is the rate of write operations on the disk. Make sure that this number is less than 85 percent of the disk capacity. The disk access time increases exponentially beyond 85 percent capacity.

Page 15: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 15

TempDB

When you have identified an I/O bottleneck, you can address it by doing one or more of the following:

• Check the memory configuration of SQL Server. If SQL Server has been configured with insufficient memory, it will incur more I/O overhead.

• Increase I/O bandwidth.

* Add more physical drives to the current disk arrays and/or replace your current disks with faster drives. This helps to boost both read and write access times. But don't add more drives to the array than your I/O controller can support.

* Add faster or additional I/O controllers. Consider adding more cache (if possible) to your current controllers.

• Examine execution plans and see which plans lead to more I/O being consume. It is possible that a better plan (for example, index) can minimize I/O. If there are missing indexes, you may want to run Database Engine Tuning Advisor to find missing indexes

Page 16: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 16

TempDB

1. The following DMV query can be used to find which batches/requests are generating the most I/O.

select top 5 (total_logical_reads/execution_count) as avg_logical_reads, (total_logical_writes/execution_count) as avg_logical_writes, (total_physical_reads/execution_count) as avg_phys_reads, Execution_count, statement_start_offset as stmt_start_offset, sql_handle, plan_handle from sys.dm_exec_query_stats order by (total_logical_reads + total_logical_writes) Desc

2. You can also identify I/O bottlenecks by examining the latch waits. These latch waits account for the physical I/O waits when a page is accessed for reading or writing and the page is not available in the buffer pool.

Select wait_type, waiting_tasks_count, wait_time_ms from sys.dm_os_wait_stats where wait_type like 'PAGEIOLATCH%' order by wait_type

Page 17: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 17

TempDB

1. Put it on faster disk(raid5).

2. Provide seperated disk for tempdb

3. Increase tempdb file number. A suggestion number is 1 processer 1 file

4. Increase tempdb initial size

Page 18: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 18

Disk Term

• Hard Drive: HDD or disk, non-volatile storage device which stores digitally encoded data on rapidly rotating platters with magnetic surfaces.

• Platter: Two or more per HDD.

• Cylinder: All tracks which can be accessed by the heads while the access arms are stationary.

• Track: Concentric rings on a platter.

• Sector – 1. Wedge-shaped sections of a platter, classically 64 sectors per track.– 2. Bits which lie at the intersection of a track & a sector, usually 512 bytes.

(This is a standard definition & one we will use most.)

• File Allocation Unit: Some integral number of 512 byte sectors which are treated as a unit by the OS.

Page 19: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 19

Disk Issue Implicit• Output of diskpar (Windows 2000 Resource Kit) (Above Windows 2000 should use

diskpart.exe)C:\>diskpar -i 0---- Drive 0 Geometry Infomation ----Cylinders = 12161TracksPerCylinder = 255SectorsPerTrack = 63BytesPerSector = 512DiskSize = 100027630080 (Bytes) = 95393 (MB)---- Drive Partition 0 Infomation ----StatringOffset = 32256PartitionLength = 49319424HiddenSectors = 63PartitionNumber = 1PartitionType = de

– By default, for years Windows instantiated 63 hidden sectors in all new partitions. – These hidden sectors contain the master boot record (MBR).– Note the typos:

• “StatringOffset” instead of “StartingOffset”.• “Infomation” instead of “Information”

Page 20: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 20

Disk Issue Implicit

Page 21: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 21

Disk Issue Implicit (File Allocation Unit)

• fsutil fsinfo ntfsinfo c:• Output for default NTFS format

C:\>fsutil fsinfo ntfsinfo c:NTFS Volume Serial Number : 0x3a16ff9d16ff5879Version : 3.1Number Sectors : 0x000000000a2397ffTotal Clusters : 0x00000000014472ffFree Clusters : 0x000000000025b76aTotal Reserved : 0x00000000000051f0Bytes Per Sector : 512Bytes Per Cluster : 4096Bytes Per FileRecord Segment : 1024Clusters Per FileRecord Segment : 0Mft Valid Data Length : 0x0000000007c90000...

Page 22: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 22

Disk Issue Implicit (File Allocation Unit)

……8KBData Page

Extent 64KB

So we need change the file allocation unit to 64KB that could reduce I/O significantly.

Page 23: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 23

Disk file fragmentation

• Two types of disk Reading :1. sequential reading (Index seek)

2. Random reading(Index scan)

• Tuning: 1. Increase initial data file size

2. Set correct data file increasing size

3. Schedulable clean up file fragmentation

File A in DataBase A

File A in DataBase A

File B in DataBase B

File B in DataBase B

Unused space

Page 24: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 24

Log file fragmentation

• More log file cause data fragmentation

• Small size log file.

• Less log file (VLF). A suggestion number is 5

• Schedule backup log file.

Page 25: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 25

Improve the Long Running DML

Page 26: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 26

Improve the Long Running DMLSelect dbuser1_.id as id190_,

From

DB_SITE_USERS dbsiteuser0_,

DB_USERS dbuser1_,

SITE_MODULES sitemodule2_,

DB_WORKSHOPS dbworkgrou3_,

DB_USER_ROLES dbuserrole4_,

DB_ROLE_MODULES dbrolemodu5_,

DB_ROLES dbrole6_,

WORKGROUP_RESOURCES workgroupr7_,

RESOURCES resource8_,

RESOURCE_ITEMS resourceit9_

Where

dbsiteuser0_.SITE_ID=35

AND dbsiteuser0_.DELETED_FLAG=0

AND dbuser1_.USER_ID=dbsiteuser0_.USER_ID

AND dbuser1_.DELETED_FLAG=0

AND sitemodule2_.SITE_ID=35

AND sitemodule2_.MODULE_ID=17

AND sitemodule2_.DELETED_FLAG=0 AND dbworkgrou3_.SITE_ID=35 AND dbworkgrou3_.DELETED_FLAG=0 AND dbrole6_.SITE_ID=35 AND dbrole6_.ROLE_ID=dbrolemodu5_.ROLE_ID AND ( dbrole6_.NAME in ( 'RFP EscalatiON','BrAND Manager','Executive Management' ) ) AND dbrole6_.DELETED_FLAG=0 AND dbuserrole4_.WORKGROUP_ID=dbworkgrou3_.WORKGROUP_ID AND dbuserrole4_.SITE_USER_ID=dbsiteuser0_.SITE_USER_ID AND dbuserrole4_.DELETED_FLAG=0 AND dbrolemodu5_.ROLE_ID=dbuserrole4_.ROLE_ID AND dbrolemodu5_.ACTIONS % 8-4>=0 AND dbrolemodu5_.DELETED_FLAG=0 AND workgroupr7_.WORKGROUP_ID=dbuserrole4_.WORKGROUP_ID AND workgroupr7_.DELETED_FLAG=0 AND resourceit9_.RESOURCE_ID=workgroupr7_.RESOURCE_ID AND resourceit9_.DELETED_FLAG=0 AND resource8_.id=resourceit9_.RESOURCE_ID AND resource8_.DELETED_FLAG=0 AND ((resourceit9_.RESOURCE_ITEM_XREF_ID=1381

or resourceit9_.ALL_RESOURCE_ITEM_XREFS=1 ) AND resourceit9_.RESOURCE_ITEM_TYPE_ID=1 or ( resourceit9_.RESOURCE_ITEM_XREF_ID in (90) or resourceit9_.ALL_RESOURCE_ITEM_XREFS=1) AND resourceit9_.RESOURCE_ITEM_TYPE_ID=2 )

Page 27: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 27

Improve the Long Running DML• SELECT u.user_name as userName, u.email as email • FROM RESOURCE_ITEMS as ri with(nolock) • JOIN WORKGROUP_RESOURCES as wr with(nolock) • ON ri.deleted_flag=0 AND wr.deleted_flag = 0 • AND ri.resource_item_type_id IN( 1,2) • AND ri.resource_item_xref_id IN(1381, 90) • AND ri.site_id=35 • AND ri.resource_id = wr.resource_id • JOIN DB_USER_ROLES ur with (nolock) • ON ur.workgroup_id = wr.workgroup_id AND ur.deleted_flag = 0 • JOIN DB_ROLES r with (nolock) • ON r.role_id = ur.role_id • JOIN DB_ROLE_MODULES rm with (nolock) • ON r.deleted_flag = 0 AND rm.deleted_flag =0 AND r.role_id = rm.role_id • AND r.role_id in ('35\18','35\81','35\82') AND r.site_id = 35 • AND (rm.MODULE_ID=17 or all_modules = 1) AND (rm.ACTIONS % 8-4>=0) • JOIN DB_SITE_USERS su ON su.site_user_id = ur.site_user_id AND su.deleted_flag = 0 • JOIN DB_USERS u ON u.user_id = su.user_id AND u.deleted_flag = 0

Page 28: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 28

Improve the Long Running DML

How about the How about the index?index?

Page 29: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 29

Improve the Long Running DML

Page 30: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 30

Improve the Long Running DML• How can we improve the long running reading query?1. Restrict the size of input. The leftmost table selection is most important factor for the

performance of query.2. Reduce the size of output. Only read you need read data from the output tables.3. Try you best not involve the String, Date comparison or like. If you must, please make

sure correct use and avoid the index scan happen.4. Read the query plan generated by sql server. For small or medium size query try you best

use “INNER LOOP” or “HASH MATCH”. For large size query try you best use “MERGE JOIN”.

5. If customer of your system can accept some level data inconsistency, please put with(nolock) table hint that will reduce read lock contention. If customer of your system cannot accept data inconsistency and your sql server version is 2005, you also can try SNAPSHOT ISOLATION level (VERSION DATA) but that will come without free (each query will occupy more memories than normal other isolation level).

6. Join type need be carefully considered. Avoid “OUTTER JOIN” and “RIGHT JOIN”.7. Performance killer “Table Spooler”, “Stream Aggregation”. Count(*) will impose the “Table

Spooler” and “Stream Aggregation”, try you best use other way for implementation. Holy Grail Algorithm

8. MSDN will be your best assistance.

Page 31: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 31

Improve the Long Running DML

• How can we improve the long running create, update or delete query?

1. Restrict the size of data you want to change in the table. Try you best control data changes within the row level, otherwise sql server may escalate the lock level from row level to table level or schema level. With(rowlock) table hint is not 100 percent true.

2. Large data change need to be batch mode (within one transaction). Multiple transactions data changes will impose contention and even further dead lock.

3. Change only your need change data. Because that possibly will not reflect changes to some NONE-CLUSTER indexes.

4. Avoid concurrent long running changing transactions. If really need that, separate them into small transactions (NESTED TRANSACTION).

Page 32: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 32

Improve the Long Running DML

MKV performance bottleneck due to the long running transactions.

Application transaction

Sql server 2005

Db transaction

Db transaction

Db transaction

enlist

enlist

enlist

Update table A

Update table B

Update table C

Table A

Table B

Table C

1

*

1

*

FINE-GRAINED TRANSACTION or COARSE-GARINED TRANSACTION ?

Page 33: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 33

Improve the Inefficient indexes

• Identify the slowness reason from indexes perspective

• Identify the slowness reason from contention perspective

• Reduce the contention of update by unused indexes

• Apply the trace configuration for diagnostic purpose if deadlock found.

• Group review & Performance testing.

Page 34: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 34

Dynamic management views and functions return server state information that can be used to monitor the health of a server instance, diagnose problems, and tune performance.

• Index Related Dynamic Management Views and Functions – sys.dm_db_index_operational_stats– sys.dm_db_index_usage_stats– sys.dm_db_missing_index_details– sys.dm_db_missing_index_groups– sys.dm_db_index_physical_stats– sys.dm_db_missing_index_columns– sys.dm_db_missing_index_group_stats

Dynamic Management Views and Functions

Page 35: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 35

The missing indexes feature uses dynamic management objects and Showplan to provide information about missing indexes that could enhance SQL Server query performance.

• Dynamic Management Objects– sys.dm_db_missing_index_group_stats– sys.dm_db_missing_index_groups – sys.dm_db_missing_index_details – sys.dm_db_missing_index_columns

About the Missing Indexes Feature - Indexes perspective

Returns summary information about missing index groups, for example, the performance improvements that could be gained by implementing a specific group of missing indexes.

Returns information about a specific group of missing indexes, such as the group identifier and the identifiers of all missing indexes that are contained in that group.

Returns detailed information about a missing index; for example, it returns the name and identifier of the table where the index is missing, and the columns and column types that should make up the missing index.

Returns information about the database table columns that are missing an index.

Page 36: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 36

• Find the 10 missing indexes with the highest anticipated improvement for user queries

select top 10 d.*, s.avg_total_user_cost, s.avg_user_impact, s.last_user_seek,s.unique_compiles from sys.dm_db_missing_index_group_stats s,sys.dm_db_missing_index_groups g,sys.dm_db_missing_index_details dwhere s.group_handle = g.index_group_handleand d.index_handle = g.index_handleorder by s.avg_total_user_cost * s.avg_user_impact * (s.user_seeks + s.user_scans) desc

Examples - Indexes perspective

Page 37: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 37

• Suggested index columns & usages SELECT mig.*, statement AS table_name,

column_id, column_name, column_usageFROM sys.dm_db_missing_index_details AS midCROSS APPLY sys.dm_db_missing_index_columns (mid.index_handle)INNER JOIN sys.dm_db_missing_index_groups AS mig ON mig.index_handle =

mid.index_handleORDER BY mig.index_group_handle, mig.index_handle, column_id;

Examples - Indexes perspective

Page 38: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 38

• The result set of sys.dm_db_missing_index_details returns this information in the equality_columns, inequality_columns, and included_columns columns.

• The result set returned by sys.dm_db_missing_index_columns returns this information in its column_usage column.

• Create Index by Using Missing Index Information– List the equality columns first (leftmost in the column list). – List the inequality columns after the equality columns (to the right of

equality columns listed). – List the include columns in the INCLUDE clause of the CREATE INDEX

statement. – To determine an effective order for the equality columns, order them

based on their selectivity; that is, list the most selective columns first.

Indexes perspective - Improve

Page 39: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 39

• It is not intended to fine tune an indexing configuration.

• It cannot gather statistics for more than 500 missing index groups.

• It does not specify an order for columns to be used in an index.

• For queries involving only inequality predicates, it returns less accurate cost information.

• It reports only include columns for some queries, so index key columns must be manually selected.

• It returns only raw information about columns on which indexes might be missing.

• It does not suggest filtered indexes.

• It can return different costs for the same missing index group that appears multiple times in XML Showplans.

• It does not consider trivial query plans.

Limitations of the Missing Indexes Feature

Page 40: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 40

declare @dbid int

select @dbid = db_id()

select 'object' = object_name(object_id),index_id

,'user reads' = user_seeks + user_scans + user_lookups

,'system reads' = system_seeks + system_scans + system_lookups

,'user writes' = user_updates

,'system writes' = system_updates

from sys.dm_db_index_usage_stats

where objectproperty(object_id,'IsUserTable') = 1

and database_id = @dbid order by 'user reads' desc

Index Perspective – Balance (1)

Page 41: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 41

declare @dbid int

select 'object'=object_name(o.object_id), o.index_id

, 'usage_reads'=user_seeks + user_scans + user_lookups

, 'operational_reads'=range_scan_count + singleton_lookup_count

, range_scan_count, singleton_lookup_count, 'usage writes' = user_updates

, 'operational_leaf_writes'=leaf_insert_count+leaf_update_count+ leaf_delete_count

, leaf_insert_count,leaf_update_count,leaf_delete_count

, 'operational_leaf_page_splits' = leaf_allocation_count

, 'operational_nonleaf_writes'=nonleaf_insert_count + nonleaf_update_count + nonleaf_delete_count

, 'operational_nonleaf_page_splits' = nonleaf_allocation_count

from sys.dm_db_index_operational_stats (@dbid,NULL,NULL,NULL) o ,sys.dm_db_index_usage_stats u

where objectproperty(o.object_id,'IsUserTable') = 1

and u.object_id = o.object_id and u.index_id = o.index_id

order by operational_reads desc, operational_leaf_writes, operational_nonleaf_writes

Index Perspective – Balance (2)

Page 42: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 42

• Slow down the performance: INSERTs, UPDATEs, and DELETEs

• Whether or not the indexes will be used, decided by Query Optimizer.– Selectivity: the percentage of rows in a table that are returned by a query

• Index statistics – Used by Query Optimizer for selectivity info– Density (Conceptually reverse to selectivity)

– DBCC SHOW_STATISTICS (table_name, index_name)

Index Perspective – Statistics

Page 43: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 43

• sys.dm_db_index_operational_stats

• sys.indexes

USE G3ProdMKV_MarketViewdeclare @dbid intselect top 10 dbid=database_id,objectname=object_name(s.object_id),indexname=i.name, i.index_id,partition_number,row_lock_count,row_lock_wait_count,row_lock_wait_in_ms,[block %]=cast (100.0 * row_lock_wait_count / (1 + row_lock_count) as numeric(15,2)),[avg row lock waits in ms]=cast (1.0 * row_lock_wait_in_ms / (1 + row_lock_wait_count) as

numeric(15,2))from sys.dm_db_index_operational_stats (@dbid, NULL, NULL, NULL) s, sys.indexes iwhere objectproperty(s.object_id,'IsUserTable') = 1 and i.object_id = s.object_idand i.index_id = s.index_id order by row_lock_wait_count desc

Contention perspective - List Indexes with the Most Contention

Page 44: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 44

The following sample scripts lists the top 10 statements by average CPU time.

– sys.dm_exec_query_stats– sys.dm_exec_sql_text(sql_handle)SELECT TOP 10qs.total_worker_time/qs.execution_count as [Avg CPU Time],qs.execution_count,SUBSTRING(qt.text,qs.statement_start_offset/2,(case when qs.statement_end_offset = -1then len(convert(nvarchar(max), qt.text)) * 2 else qs.statement_end_offset end - qs.statement_start_offset)/2) as query_text, qt.dbid, dbname=db_name(qt.dbid), qt.objectidFROM sys.dm_exec_query_stats qscross apply sys.dm_exec_sql_text(qs.sql_handle) as qtORDER BY [Avg CPU Time] DESC

Contention perspective - Highest Average CPU Time and Highest Execution Counts

Page 45: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 45

• sys.indexes

• sys.objects

• sys.dm_db_index_usage_stats

select top 10 object_name(i.object_id), i.namefrom sys.indexes i, sys.objects owhere i.index_id NOT IN (select s.index_id

from sys.dm_db_index_usage_stats s where s.object_id=i.object_id and i.index_id=s.index_id and database_id = db_id())

and o.type = 'U' and o.object_id = i.object_idorder by object_name(i.object_id) asc

Contention perspective - Contention of update by unused indexes

Page 46: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 46

• Enables the specified trace flags.

• Syntax – DBCC TRACEON ( trace# [ ,...n ][ , -1 ] ) [ WITH NO_INFOMSGS ]

trace# - Is the number of the trace flag to turn on. n - Is a placeholder that indicates multiple trace flags can be specified. -1 - Switches on the specified trace flags globally. WITH NO_INFOMSGS - Suppresses all informational messages.

• Examples– DBCC TRACEON (1222); – DBCC TRACEON (1222, -1);– DBCC TRACEON (1222, 3206, -1);

Contention perspective – Deadlock diagnostic

Page 47: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 47

• Whenever you add one index into the table. Should view the impact to the INSERT, UPDATE and DELETE by the index you added .

Set statistics time on

Set statistics io on

Updated sql before/after

Set statistics time off

Set statistics io off

If updated sql slow than before, you should know the table read/write ratio. If read most, it should be acceptable. But if write most, it definitely impose bad impact overall.

Impact by indexes

Page 48: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 48

TempDB improvement

• Tempdb is often overlooked when finding out problems for performance issues with SQLServer

• LocationSeparate tempdb from the user databases and SQL Server binaries and place the tempdb on faster disk array.

alter database tempdb modify file(name=tempdev, filename='D:\sqldata\tempdb.mdf')alter database tempdb modify file(name=templog, filename='D:\sqldata\templog.ldf')

Page 49: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 49

TempDB improvement

• DB Size– Default: 8 MB– Query the current sizeSELECT name AS FileName, size*1.0/128 AS FileSizeinMB, CASE max_size WHEN 0 THEN 'Autogrowth is off.' WHEN -1 THEN 'Autogrowth is on.' ELSE 'Log file will grow to a maximum size of 2 TB.' END, growth AS 'GrowthValue', 'GrowthIncrement' = CASE WHEN growth = 0 THEN 'Size is fixed and will not grow.' WHEN growth > 0 AND is_percent_growth = 0 THEN 'Growth value is in 8-KB pages.' ELSE 'Growth value is a percentage.' ENDFROM tempdb.sys.database_files;

Page 50: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 50

TempDB improvement

• Update the sizealter database tempdb modify file(name=tempdev, filename='D:\sqldata\tempdb.mdf', size=XXXMB)

Page 51: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 51

TempDB improvement

• Number of data files– More data files for tempdb will reduce contention– Recommend: number of processors in system.alter database tempdb add file(name=tempdev2, filename='d:\sqldata\tempdb2.ndf', size=200MB, filegrowth=10%)

• Separate data file and log file– Tempdb does not need backups and it can well be in simple recovery

model. This is the recommendation and also the default.

Page 52: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 52

TempDB improvement• Practice - Location

– Move C partition -> D partition

• Result

Page Response time (improvement in percent)

  No Change

Move from C: -> D:

with default

size

Move from C: -> D:

with 800M size

Move from C: -> D:

with 320M size

Move from C: -> D:

with 64M size

Average 1.813051 1.434593 FAIL 1.701 1.64125

Maximum 5.983458 5.408847 FAIL 6.83278 6.060153

90 Percent 2.774966 2.236746 FAIL 2.48322 2.350949

Page 53: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 53

TempDB improvement (Resource Compare)

Page 54: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 54

Production Data

Page 55: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 55

Production Data

Date Release Number

Total Action Numbers

< 3s total action numbers

Average Percent Page <3s

5/15/2009

5/13/2009

4/20/2009

MKV 8.8

MKV 8.7.1

MKV 8.7

106,429

81,018

81,672

103,174

75,300

75,325

96.9%

93%

92.2%

Page 56: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 56

Production Data

Date Release Number

Total Action Numbers

< 3s total action numbers

Average Percent Page <3s

5/15/2009

5/13/2009

4/20/2009

MKV 8.8

MKV 8.7.1

MKV 8.7

106,429

81,018

81,672

103,174

75,300

75,325

96.9%

93%

92.2%

Page 57: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 57

Some Hot Topics

• NUMA

• CLUSTERED DATABASE CONNECTION

Page 58: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 58

Some Hot Topics (NUMA

Page 59: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 59

Some Hot Topics (NUMA)

Page 60: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 60

Some Hot Topics (NUMA)

Page 61: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 61

Some Hot Topics (NUMA)

Page 62: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 62

Some Hot Topics (NUMA)

Page 63: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 63

Some Hot Topics (NUMA)

Page 64: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 64

Some Hot Topics (CLUSTERED DATABASE CONNECTION)

Page 65: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 65

Some Hot Topics (CLUSTERED DATABASE CONNECTION)

Page 66: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 66

Some Hot Topics (CLUSTERED DATABASE CONNECTION)

Page 67: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 67

Some Hot Topics (CLUSTERED DATABASE CONNECTION)

Page 68: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 68

Some Hot Topics (CLUSTERED DATABASE CONNECTION)

• Microsoft JDBC support configure the failover partner information

dbs/mss/server = tstsapdb1;FailoverPartner=tstsapdb2;Database=TST

Page 69: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 69

Further topics

1. Disk tuning.

2. Sql server high availability and data replication

3. Sql server full text search

4. TempDB further tuning

Page 70: MarketView Database Tuning Best Practice

C O N F I D E N T I A L Apr 22, 2023 70