View
221
Download
2
Category
Preview:
Citation preview
DAT309:SQL Server CE 2.0 Performance Tuning, Deployment, And Scalability
Kevin Collins
Microsoft
Senior Program Manager
Agenda
SQL Server CE with ADO.NET performance tips
SQL Server CE QP performance tips
Mobile application deployment techniques
Sync scaling strategies
Native Versus Managed
Absolute fastest data access is OLEDBCE via eVC
Much more code and harder to maintain
ADO.NET Verses ADOCE Perf
1. Seek to a random value1. Seek to a random value2. Read entire row2. Read entire row3. Repeat (n) times3. Repeat (n) times4. Close the cursor4. Close the cursor
1. SELECT * FROM tab WHERE id=n1. SELECT * FROM tab WHERE id=n2. Read entire row2. Read entire row3. Close the cursor 3. Close the cursor 4. Repeat (n) times4. Repeat (n) times
SeekSeek TestTest
Number of rowsNumber of rows
00
55
1010
1515
2020
2525
3030
3535
4040
11 1010 100100 10001000
Tim
e (
se
co
nd
s)
Tim
e (
se
co
nd
s)
ADO.NETADO.NET ADOCEADOCE
Single Fetch without parametersSingle Fetch without parameters
Number of rowsNumber of rows
00
5050
100100
150150
200200
250250
11 1010 100100 10001000
Tim
e (
se
co
nd
s)
Tim
e (
se
co
nd
s)
ADO.NETADO.NET ADOCEADOCE
Data Reader Versus Data Set
Whenever possible use Data Reader
Significantly faster than DataSet, especially when using larger result sets
NCS saw 5X performance improvements going to Data Reader
http://www.ncssuite.com/NewsArticles/NZ%20NETArticleR1.pdf
Direct access to SQL CE
No double buffering of data
DataSet updates ALL columns
DownsideNot bound to controls
Not scrollable or updateable
Data Reader – Optimal Usage
When reading entire rows, consider using GetValues() rather than calling GetXXX multiple times
Use strongly typed getters rather than GetValue()
Refer to the columns by ordinal rather than name
Using SqlTypes is more expensive than CLR types
**** List by ordinal versus by nameExample:Example:
object[] row = new object[rdr.FieldCount];object[] row = new object[rdr.FieldCount];while (rdr.Read())while (rdr.Read()){{ // Fewer function calls inside the data provider// Fewer function calls inside the data provider rdr.GetValues(row);rdr.GetValues(row); // Process the row here// Process the row here} }
GetValues() Versus GetValue(i) #12
demodemo
QP Optimizer
Heuristic
Rewrites query into semantically equivalent form that leads to a better execution plan
Based on syntax
Rewrite rules are applied if applicable
Adjacent inner joins are merged together so alternative join orders may be considered... FROM (Table_1 INNER JOIN Table_2 ON Table_1.Col = Table_2.Col) INNER JOIN
Table_3 ON Table_1.Col = Table_3.Col ...
... FROM Table_1, Table_2, Table_3 WHERE Table_1.Col = Table_2.Col AND Table_1.Col = Table_3.Col ...
Cost BasedDetermines
Base table scan type (File scan / Index scan)
What indexes, if any, are used
Join order, join algorithm
Sort / filter placement
How does it do itEnumerates a selected subset of all possible execution plans
Finds out the plan with lowest estimated cost
Generates executable data structures that implement the plan
Schema Tips
Variable length columnsKeeps db size down
Row might move with column growthSize increase
Defragmentation
Denormalize?
Joins of more than 5 tablesQP might not come up with optimal plan
Modifying SQL Statements
These have no impactOrder of tables in old-style FROM clause“… FROM Foo, Bar …” ==
“… FROM Bar, Foo …”
Order of predicates in the same clause“… WHERE Col1 = 2 AND Col2 > 10 …” ==
“… WHERE Col2 > 10 AND Col1 = 2 …”
Order of INNER JOIN tables“… FROM Foo INNER JOIN Bar …” ==
“… FROM Bar INNER JOIN Foo …”
Optimizer re-arranges the order in search for better execution plan
Subquery Versus Join
Evaluate both
Subquery may be written using JoinJoin can evaluate tables in different order from subquery
Subquery may not need to scan all rowsReturns after seeing first occurrence
IN subqueriesAlways re-writen to join
No need to re-write to explicit join
Subquery Versus Join #4Subquery Versus Join #4
demodemo
Subquery Versus Join
Orders that have at least one item with discount of 25% or moreOrders that have at least one item with discount of 25% or more
SELECT "Order ID" FROM Orders O WHERE EXISTS (SELECT "Order ID" FROM "Order Details" SELECT "Order ID" FROM Orders O WHERE EXISTS (SELECT "Order ID" FROM "Order Details" OD WHERE O."Order ID" = OD."Order ID" AND Discount >= 0.25) OD WHERE O."Order ID" = OD."Order ID" AND Discount >= 0.25)
SELECT DISTINCT O."Order ID" FROM Orders O INNER JOIN "Order Details" OD ON O."Order SELECT DISTINCT O."Order ID" FROM Orders O INNER JOIN "Order Details" OD ON O."Order ID" = OD."Order ID" WHERE Discount >= 0.25ID" = OD."Order ID" WHERE Discount >= 0.25
Filters In WHERE Versus HAVING
Non-aggregate filters in WHERE clause instead of HAVING clause
WHERE is evaluated before HAVING
Result is reduced early in query processing
Filters – WHERE Versus HAVING
Order count for customers where id start with ‘A’Order count for customers where id start with ‘A’
• SELECT "Customer ID", COUNT("Customer ID") FROM Orders GROUP BY SELECT "Customer ID", COUNT("Customer ID") FROM Orders GROUP BY "Customer ID" HAVING "Customer ID" >= 'A' AND "Customer ID" < 'B‘"Customer ID" HAVING "Customer ID" >= 'A' AND "Customer ID" < 'B‘• SELECT "Customer ID", COUNT("Customer ID") FROM Orders WHERE SELECT "Customer ID", COUNT("Customer ID") FROM Orders WHERE "Customer ID" >= 'A' AND "Customer ID" < 'B' GROUP BY "Customer ID""Customer ID" >= 'A' AND "Customer ID" < 'B' GROUP BY "Customer ID"
Filters – WHERE Versus Filters – WHERE Versus HAVING #5HAVING #5
demodemo
Create Useful Indexes
Create selective indexesSelectivity is ratio of qualifying rows to total rows
Index on Orders.OrderID is selective
Index on Orders.ShipVia is not selective
SQL CE only uses one index per tableIndexes will slow down DML
Increases database size
Evaluate use of applicationHeavy DML less indexes
Heavy querying more indexes
Useful Indexes #6Useful Indexes #6
demodemo
Multi-Column Indexes
CREATE INDEX Emp_name on Employees (“Last Name” ASC, “First Name” ASC)
It may optimizeWHERE “Last Name” = ‘Doe’WHERE “Last Name” = ‘Doe’ AND “First Name” = ‘John’ORDER BY / GROUP BY “Last Name”ORDER BY / GROUP BY / DISTINCT “Last Name”, “First Name”DISTINCT “Last Name”, “First Name”
It cannot optimizeWHERE “First Name” = ‘John’ORDER BY / GROUP BY / DISTINCT “First Name”ORDER BY / GROUP BY “First Name”, “Last Name”DISTINCT “Last Name”
Joins
Indexes used to speed up JOINsEngine automatically creates index for PK and FK
Not good for duplicate values
If no PK/FK index join columns
Join Columns Index Join Columns Index Versus Non-Indexed #2Versus Non-Indexed #2
demodemo
Optimizable Versus Non-Optimizable Clauses
Optimizable clausesColumn operator <constant or variable> or
<constant or variable> operator Column
>, <, >=, <=, BETWEEN, IN, and LIKE (prefix matching, such as LIKE 'John%')
Non-optimizable clausesNOT, <>, (NOT) EXISTS, NOT IN, NOT LIKE, and intrinsic functions
Turn clauses into optimizable onesUPPER(“Customer ID”) = ‘ANTON’ -> “Customer ID” = ‘ANTON’
ORDER BY / GROUP BY / DISTINCT
ORDER BY / GROUP BY Prefix set of index columns must match ORDER BY / GROUP BY columns with exact order
Index more columns, but not fewer ones
DISTINCTIndex columns must match projection list
ORDER BY – Index Versus Sort
The following queries are run on tables of different cardinality with and without indexesThe following queries are run on tables of different cardinality with and without indexes
SELECT "Customer ID" FROM Customers ORDER BY "Customer ID"SELECT "Customer ID" FROM Customers ORDER BY "Customer ID"SELECT "Order ID" FROM Orders ORDER BY "Order ID"SELECT "Order ID" FROM Orders ORDER BY "Order ID"SELECT "Order ID" FROM "Order Details" ORDER BY "Order ID"SELECT "Order ID" FROM "Order Details" ORDER BY "Order ID"
Outer Join Performance Issues
SQL CE doesn’t re-arrange outer joins
Potentially modify database design Products – Categories scenario
Instead of inserting NULL as “Category ID”, add “Misc” or “No Category” category value
Outer join is avoided when joining the two tables together
Parameterized Queries
Compile once and execute many timesGood for a set of queries that are only different in a few constant valuesHold on to the SqlCeCommand object that caches the compiled planCreate multiple SqlCeCommand objects if you run multiple parameterized queries at the same time
Example:Example:// Create a parameterized query// Create a parameterized querycmd.CommandText = “UPDATE myTable set col1=? where cmd.CommandText = “UPDATE myTable set col1=? where col2=?";col2=?";
// Avoid SqlCeParameter(string, object) constructor// Avoid SqlCeParameter(string, object) constructor cmd.Parameters.Add("p1", SqlDbType.Decimal);cmd.Parameters.Add("p1", SqlDbType.Decimal);cmd.Parameters.Add("p2", SqlDbType.SmallInt);cmd.Parameters.Add("p2", SqlDbType.SmallInt);cmd.Prepare();cmd.Prepare();
// Execute multiple times …// Execute multiple times … cmd.Parameters["p1"].Value = 123456.54;cmd.Parameters["p1"].Value = 123456.54;cmd.Parameters["p2"].Value = 1;cmd.Parameters["p2"].Value = 1; cmd.ExecuteNonQuery();cmd.ExecuteNonQuery();
Parameterized Queries
Base Table Cursor
Bypasses the Query ProcessorFetches all the columns in a rowThe fastest way of reading all the columns from a table
Example:Example:
// Create and execute SqlCeCommand// Create and execute SqlCeCommandSqlCeCommand cmd = new SqlCeCommand(“Authors",cnn);SqlCeCommand cmd = new SqlCeCommand(“Authors",cnn);cmd.CommandType = CommandType.TableDirect;cmd.CommandType = CommandType.TableDirect;IDataReader dr = cmd.ExecuteReader();IDataReader dr = cmd.ExecuteReader();
// Retrieve Results// Retrieve Resultswhile(dr.Read())while(dr.Read()){ { Console.WriteLine("Name = " + dr["au_lname"]);Console.WriteLine("Name = " + dr["au_lname"]);}}dr.Close();dr.Close();cnn.Close();cnn.Close();
Seek/SetRange – Basic Usage
Used to open base table indexThe fastest way of selecting range of values
Example:Example:cmd.CommandType = CommandType.TableDirect;cmd.CommandType = CommandType.TableDirect;cmd.CommandText = "Orders";cmd.CommandText = "Orders";
//Assume: Index contains one column [datetime]//Assume: Index contains one column [datetime]cmd.IndexName = "SomeIndex"; cmd.IndexName = "SomeIndex"; object[] start = new object[1];object[] start = new object[1];object[] end = new object[1];object[] end = new object[1];start[0] = new SqlDateTime(2001, 1, 1); start[0] = new SqlDateTime(2001, 1, 1); end[0] = new SqlDateTime(2002, 2, 3;end[0] = new SqlDateTime(2002, 2, 3;
cmd.SetRange(DbRangeOptions.Match, start, end); cmd.SetRange(DbRangeOptions.Match, start, end); SqlCeDataReader rdr = cmd.ExecuteReader(); SqlCeDataReader rdr = cmd.ExecuteReader(); rdr.Seek(DbSeekOptions.FirstEqual, rdr.Seek(DbSeekOptions.FirstEqual, new SqlDateTime(2001,3,4));new SqlDateTime(2001,3,4)); while(rdr.Read()) { } while(rdr.Read()) { } // Read data in the usual way// Read data in the usual way
Seek/SetRange Versus Scrollability
SqlCeDataReader has limited scrolling abilityDataAdapter
does not leverage this capabilitymay be used with base-table cursor (TableDirect) can use CommandBuilder with base-table cursor
Example:Example:// Set the index range// Set the index range cmd.SetRange(DbRangeOptions.InclusiveStart, start, end); cmd.SetRange(DbRangeOptions.InclusiveStart, start, end); SqlCeDataReader rdr = cmd.ExecuteReader(); SqlCeDataReader rdr = cmd.ExecuteReader();
// Seek to a value (customer name)// Seek to a value (customer name) rdr.Seek(DbSeekOptions.FirstEqual,”Collins”);rdr.Seek(DbSeekOptions.FirstEqual,”Collins”);rdr.Read();rdr.Read();// Process the row in the usual way// Process the row in the usual way // Seek to another value (customer name)// Seek to another value (customer name) rdr.Seek(DbSeekOptions.LastEqual,”Sabino”);rdr.Seek(DbSeekOptions.LastEqual,”Sabino”);rdr.Read();rdr.Read();// Process the row in the usual way// Process the row in the usual way
rdr.Close();rdr.Close();
Use Queries?
SELECT "Customer ID" FROM Orders WHERE "Order ID" = 10123SELECT "Customer ID" FROM Orders WHERE "Order ID" = 10123VSVS
Base TableBase Table
Encryption?
190 188
347 335
0
50
100
150
200
250
300
350
Tim
e in
Se
co
nd
s
SELECT UPDATE
Encryption vs No Encryption
EncyrptedNo Encryption
160K rows
UPDATE Table Set NonIndexCol = ‘KevinCol’
SELECT DateDataType FROM Table
Dell Axim XScale PXA250 (400mhz)
Storage Media?
Varying perf for different storage mediaRAM, SD, CF, IPSM (NOR), Toshiba NAND
SD is faster than MMChttp://www.dpreview.com/reviews/minoltadimagex/page7.asp
CF perf varies widelyhttp://www.dpreview.com/articles/mediacompare/default.asp?sort=fwread
142 155 144209
314348
0
100
200
300
400
Tim
e In
Sec
onds
SELECT UPDATE
Various storage medias
RAM
SD
IPSM
Delta’s from CPU’s and OS versions
255
188
148
246
11288
155
44 33
050
100150200250300
Tim
e i
n s
ec
on
ds
RDA Pull Update SELECTData-Reader
Intermec 700 64mb RAM, db in RAM
XScale 250PPC 02
XScale 255PPC 02
XScale 255PPC 03
Delta’s from CPU’s and OS versions
272
191
102
48
0
100
200
300
Tim
e in
se
cond
s
Update SELECTData-Reader
HTC Phone Edition 32mb RAM SA1100, db on SD
PPC 02
PPC 03
Misc
Database compactCorrect any potential corruption
Tables in contiguous orderIf PK/Unique index present, ordered by index
Refreshes statistics
QP StatisticsAllows for more accurate cost estimate
SupportsIndex cardinality (for selectivity of equality predicate)
Doesn’t supportMin / Max
Histogram
Change temp db location to durable media for large transactions
SQL CE QP doesn’t distinguish between RAM and durable media
Performance white papershttp://www.microsoft.com/sql/ce/
Nabisco Case Study And Demo
1500 Delivery 1500 Delivery TrucksTrucks
107 Distribution 107 Distribution CentersCenters
Web ServerWeb Server SQL Server 2000 SQL Server 2000 FailoverFailover
802.11b RF802.11b RF
IBM DB/2IBM DB/2
XMLXML
SAP Business ObjectsSAP Business Objects
Ruggedized Ruggedized PocketPC DevicePocketPC Device
with SQL Server CEwith SQL Server CE
Nabisco Deployment
Rolled custom CAB fileOriginally used eMVT wizard
Load CAB file and database on CF cardAllows for cold boot recovery in field
Used version flag in publicationIf version changes, then it is replicated down to the device
This flags download of new CAB file from Internet Information Server and updates latest version
Nabisco Scaling
One 4-Proc SQL Server 2K
One 4-Proc IIS Win2K
Start of day averages are ~15 seconds
End of day averages are ~15 seconds
Tech-Ed Europe 2002 Mobile ComNet
1500 PocketPC 2002 Wi-Fi devices
C# application with SSCE v2.0 Schedule maintenance
Attendees download all sessions (synched with ComNet devices) to see session changes and change personal itinerary
Used SQL Server SP3 replication fixesLocking reduced by changing to dynamic queries from SP’s
Used NTLB for Internet Information Server
SyncCombination of merge and RDA
One article in publication
Four tables for RDA Pulls
DataGridsPaging
RepeatersMultiple sessions
Timely links
Key FeaturesScheduling
VLAN14 ExboNet Net
VLAN13 UntrustVLAN12
VLAN11 COMMSNET
Infrastructureat Event Site
WEB11for Local
Downloads
InternetSite
Supplid Router
AD11AD / DNS/ WINS 1
AD12AD / DNS/ WINS 2
Delegates PCs
MSEVSGAT11DHCP, ISA andVPN (Router)
Speakerscomputer
Wireless LAN
PDA withWireless LAN
Candidates Laptopswith Wireless LAN
MSEVSGAT12DHCP, ISA andVPN (Router)
2 x IP Address T.B.A1 x Internet OutGoing
1 x For VPN Routed Connection
VLAN 21 VLAN 21
RedundentLinks
Session Room Back Office
Wireless for Officeand speaker network
Speaker orOffice network LAPTOPS
or PC
Internet
GAT13DHCP, NAT
Buddy ServerIIS
Buddy ServerSQL
Buddy System
InternetLINX
Heart beatLAN
SQL01SQL Cluster
SQL02SQL ClusterDisk
Heart beatLAN
Public Addresses
VLAN 3
VLAN 2
ISA05Secure VPN
Server 1(Online)
ISA03 forHTTP
Publishingfor Production
Sites
AD01AD / DNS / WINS 1
AD02AD / DNS/ WINS 2
WEB03service.aspx
MOM01MOM
Service
Infrastructureat London ASP
Site
PRODSQL01Production SQL
Cluster
PRODWEB02Production WEB
WSRV01.asmx WEB
Service
ISA01 forHTTP
Publishingfor Events Sites
Media ServerMedia Server
ISA02 forHTTP
Publishingfor Events Sites
ISA04 forHTTP
Publishingfor Production
Sites
ISA06Secure VPN
Server 7(Backup)
PRODWEB01Production WEB
MOM02MOM SQL Server
WEB04service.aspx
PRODSQL02Production SQL
Cluster
Disk
NLB
WEB01service.aspx
WEB02service.aspx
MAN01Management LAN
NLB
WSRV02.asmx WEB
Service
The Hardware
New HP supplied for COMMS NetServer Use Model Number of
ServersProcessors and Speed
Memory Disk
ISA DL380 6 2 x 1Ghz 1Gb RAM 4 x 18Gbs
AD DL380 4 2 x 1Ghz 1Gb RAM 4 x 18Gbs
WEB DL360 8 2 x 1Ghz 1Gb RAM 2 x 18Gbs
SQL Clusters DL580 4 4 x 700mhz 2Gb RAM 4 x 18Gbs
Fibre Channel disk
2 N/A N/A 32 x 18Gb disks
SQL DL380 1 2 x 1Ghz 1Gb RAM 4 x 18Gbs
AD DL380 2 2 x 1Ghz 1Gb RAM 4 x 18Gbs
WEB DL360 1 2 x 1Ghz 1Gb RAM 2 x 18Gbs
GATEWAY DL580 2 4 x 700mhz 2Gb RAM 4 x 18Gbs
Performance
Average Sync on device (RDA and merge) ~20 seconds
Average IIS CPU utilization ~5%
Merge Agent duration <1 sec min, 42 sec max, average <1 sec
Scaling Merge Replication
SET @run_much_faster = True
Upgrade to SQL Server 2K SP3 and associated QFE 798 (kb 810185)
Significant performance improvements on concurrent sync PLUS SQL Slammer fix
Defrag indexes on SQL Server
DBCC SHOWCONTIG (MSMerge_Contents) WITH TABLERESULTS, ALL_INDEXES
DBCC INDEXDEFRAG (Driver, MSMerge_Contents, 1-4)
Optimizes join between MSMerge_Contents and base tables
Avg. Bytes free per page should be low
Avg. Page density should be high
Scaling Merge Replication
Use of TempDB is high, don’t use autogrow
Put data files and log files on different disks
Separate out NTEXT/IMAGE data-types from base table
Prevents large unchanged columns from synch
Investigate using Upload_Only optionNabisco utilized this option
Investigate mixing RDA with Merge replication
Investigate lowering retention period from default of 14 days
Investigate using re-publishers
Investigate using IIS Web farm with NTLB
Scaling Merge ReplicationChange to row level tracking
Decreases # of rows in MSMerge_ContentsChange per article “@column_tracking = N‘false‘”
Increase generations per batchDecreases number of times looping through Setup_Belongs
Reduce retention period from default of 14 days@retention=nDownside is potential expired subscriptions
Split large batches of changes by calling sp_MSmakegenerationKeep_partition_changes=True
Avoids sending deletes to all subscribers on partition changesFiltering
Avoid complex join filter or subset filter statementsIndex columns involved in filtersSet the join unique key property = true when joining between PK-FKDe-Normalize table to have filtered column on each table to reduce complex join filters
Publicationshttp://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsql2k/html/sql_replmergepartitioned.asphttp://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/sql/maintain/Optimize/MERGPERF.asp
Questions?
Community Resources
Community Resourceshttp://www.microsoft.com/communities/default.mspx
Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/
NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx
User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx
evaluationsevaluations
© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
Recommended