SQL Server 2005 Performance Enhancements for Large Queries
Joe [email protected]/joe_chang.asp
When & Why Migrate to 2005
Performance gains in large queriesIn memory
Disk performance
Data Warehouse ApplicationsEasiest to migrate
Usually internal users
Read-only queries
TPC-H benchmark
Decision SupportLarge volumes of data
Complex queries – 22 queries, 2 data refresh
Power & Throughput metricsPower – run 1 queries at a time
Throughput – run multiple concurrent streams
System Details
2003 Oct: SQL Server 2000 EE build 782
16 Itanium 2 1.5GHz/6M, 64GB, 215 disks
2004 Aug: Oracle 10g EE16 Itanium 2 1.5GHz/6M, 64GB, 166 disks
Unisys ES7000 Aries 420
2005 Jun: SQL 2005 HP Integrity rx8620
16 Itanium 2 1.6GHz/9M, 64GB, 342 disks
2005 Jul: Oracle 10g R2 - 236 disks
OS: Windows Server 2003 Datacenter, SP1 for last
TPC-H 1000GB Results
Power Throughput
Composite
SQL 2000 7,330.6 3,687.4 5,199.1
Oracle 10g 13,614.2 7,131.3 9,853.3
SQL 2005* 19,241.0 9,666.3 13,637.8
Oracle 10g R2*
20,385.3 11,140.4 15,069.9
S2K5 SP1* 23,346.3 13,458.6 17,725.9
*1.6GHz/9M processor, others: 1.5GHz/6M
10
100
1,000
10,000
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
TPC-H 1000GB - Power
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18Q19 Q20 Q21 Q22
SQL 2000 2530.8 60.2 873.2 807.4 1058.7 70.8 838.2 822.8 5085.8655.4 224.5 666.0 728.7 149.4 110.9 269.1 181.3 3214.4283.2 125.7 3489.6 156.3
Oracle 10g 1068 117.5 79.2 84 334.4 67 420.7 242.3 1268189.6 172.8 182.8 1251 48.5 122.9 227.9 372.7 2146460.1 156.3 1908 189.2
SQL 2005 987.3 26.2 59.2 76.4 235.7 49 236.9 163.9 1084104.7 112.4 292 405.7 74.1 42.1 210.4 90.5 1456193 76.4 1686 182.6
10X
TPC-H 1000GB - Power
10
100
1,000
10,000
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
10
100
1,000
10,000
Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
TPC-H 1000GB - Throughput
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18Q19 Q20 Q21 Q22
SQL 2000 7111 523.7 4892 3816 5009 185.3 5545 20526 243412241 1464 3677 5320 1108 884.6 3057 5198 152997042 4029 15943 643.4
Oracle 10g 4394 1294 501 366.5 1909 252.5 2594 2496 57481137 866.3 1315 3857 262.3 639.5 770.6 1733 215801140 1236 9459 791.5
SQL 2005 5460 197.1 1486 1640 2447 161.4 1861 2311 64641857 1006 1555 2256 580.4 253.6 821 1630 41261037 2376 5320 644.1
100
1,000
10,000
100,000
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2
Av
era
ge
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
TPC-H 1000GB - Throughput
100
1,000
10,000
100,000
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11
Av
era
ge
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
100
1,000
10,000
100,000
Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22
Av
era
ge
se
co
nd
s
SQL 2000 Oracle 10g
SQL 2005 Oracle 10g R2
TPC-H Query 1
SQL 2000 to 20052.56X Power1.30X Throughput
SELECT L_RETURNFLAG ,L_LINESTATUS ,SUM(L_QUANTITY) AS SUM_QTY, SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE , SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS SUM_DISC_PRICE, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)) AS SUM_CHARGE, AVG(L_QUANTITY) AS AVG_QTY ,AVG(L_EXTENDEDPRICE) AS AVG_PRICE, AVG(L_DISCOUNT) AS AVG_DISC,COUNT(*) AS COUNT_ORDERFROM LINEITEM WHERE L_SHIPDATE <= DATEADD(dd, -90, cast('1998/12/01' as smalldatetime))GROUP BY L_RETURNFLAG ,L_LINESTATUS ORDER BY L_RETURNFLAG ,L_LINESTATUS
TPC-H Query 3
SQL 2000 to 200514.7X Power3.3X Throughput
SELECT TOP 10 L_ORDERKEY, SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS REVENUE, O_ORDERDATE ,O_SHIPPRIORITYFROM CUSTOMER INNER JOIN ORDERS ON C_CUSTKEY = O_CUSTKEY INNER JOIN LINEITEM ON L_ORDERKEY = O_ORDERKEYWHERE C_MKTSEGMENT = 'BUILDING' AND O_ORDERDATE < '1995-03-15' AND L_SHIPDATE > '1995-03-15'GROUP BY L_ORDERKEY ,O_ORDERDATE ,O_SHIPPRIORITY ORDER BY REVENUE DESC ,O_ORDERDATE
Query 8
SQL 2000 to 20055.0X Power8.9X Throughput
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
10GB Line Item in memory test
SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism
Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
10GB Line Item in memory test 1
SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism
Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P
100
1,000
10,000
100,000
1,000,000
12 13 14 15 16 17 18 19 20 21 22
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
100
1,000
10,000
100,000
1,000,000
12 13 14 15 16 17 18 19 20 21 22
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
10GB Line Item in memory test 2
SQL Server 2000SP4Query time in milli-seconds versus Max Degree of Parallelism
Limited parallelism gains beyond 4-8PSQL Server 2005June CTPContinued parallelism gains beyond 4P
10GB in memory test
Max Degree of Parallelism 1
Max DOP 4
Max DOP 16
30% reduction in total time at 1P
50% reduction in total time at 16P
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
SQL 2000 SQL 2005
100
1,000
10,000
100,000
1,000,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
SQL 2000 SQL 2005
100
1,000
10,000
100,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
SQL 2000 SQL 2005
In-Memory Table Scan
Xeon systems have better table scan performance on SQL 2000SQL 2000 default table scan performance depends on build
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
5,000
Default RowLock PagLock TabLock NoLock
MB
/sec
SQL 2000 Clust. Index Scan SQL 2000 Heap Table Scan
SQL 2005 Clust. Index Scan SQL 2005 Heap Table
Table Scan to Disk
SQL 2005 Table Scan performance probably limited by disk system
0
200
400
600
800
1,000
1,200
1,400
1,600
Default RowLock PagLock TabLock NoLock
MB
/sec
SQL 2000 Clust. Index Scan SQL 2000 Heap Table Scan
SQL 2005 Clust. Index Scan SQL 2005 Heap Table
Loop Join – SQL 2000 & 2005
SQL 2000
SQL 2005
No scaling from 1-2P
Better overall performance, scaling from 1-8P
0
50
100
150
200
250
300
350
400
100 1,000 10,000 100,000000's rows per query
00
0's
ro
ws
/se
c
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
0
100
200
300
400
500
600
100 1,000 10,000 100,000000's rows per query
00
0's
ro
w/s
ec
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
Hash Join
SQL 2000
SQL 2005
Unpredictable behavior below 3M rowsLimited scaling
Consistent behavior < 1M rowsNot as much fall off in large joins
0
500
1,000
1,500
2,000
2,500
3,000
100 1,000 10,000 100,000000's rows per query
00
0's
ro
ws
/se
c
DOP 1
DOP 2
DOP 4
DOP 8
DOP 16
0
500
1,000
1,500
2,000
2,500
3,000
100 1,000 10,000 100,000000's rows per query
00
0's
ro
ws
/se
c
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
Merge Join – SQL 2000
SQL 2005 does not use parallel merge joins? Slight decrease in non-parallel performance.
SQL 2000 parallel merge joins has serious problems, runs slower, consumes more CPU. It should be possible to performance a parallel merge efficiently?
0
500
1,000
1,500
2,000
2,500
100 1,000 10,000 100,000000's rows per query
00
0's
ro
ws
/se
c
DOP 1 DOP 2 DOP 4 DOP 8 DOP 16
LiteSpeed Backup Performance
0
200
400
600
800
1000
1200
2.6:1 5:1 8:1Compression Ratio
MB
/sec
Threads 6 8 10 12
HP rx8620, 16 x 1.5GHz Itanium 2 processors2 EVA 5000 (4 HSV110 controllers, 8 enclosures x14 disks each8 x 2Gbit/sec FC ports
SQL 2005 Performance Summary
Significant improvements in large queries
Data Warehouse Applications
Improved disk performance
More Information
www.sql-server-performance.com/joe_chang.asp
SQL Server Quantitative Performance SQL Server Quantitative Performance AnalysisAnalysis
Server System ArchitectureServer System Architecture
Processor PerformanceProcessor Performance
Direct Connect Gigabit NetworkingDirect Connect Gigabit Networking
Parallel Execution PlansParallel Execution Plans
Large Data OperationsLarge Data Operations
Transferring StatisticsTransferring Statistics
SQL Server Backup Performance with SQL Server Backup Performance with LiteSpeedLiteSpeed
System Configuration
rx8620
16 Itanium 21.5GHz
HSV110 HSV110 HSV110 HSV110
8 2Gb/sFC ports 6 SCSI Disks
rx862016 Itanium 2
1.5GHz
HSV110 HSV110 HSV110 HSV110
8 2Gb/sFC ports 6 SCSI Disks
Bookmark Lookup Performance
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
1 2 4 8 16Degree of Parallelism
Ro
ws/
sec
2000 Clustered Index
2005 Clustered Index
2000 Heap
2005 Heap
SQL 2005 has better bookmark lookup performance to Clustered Indexes SQL 2000 has better bookmark lookup performance to Heap organized tables