Upload
harvey-montgomery
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Building the Fastest SQL Servers
Brent OzarMicrosoft Certified Solutions Master (MCSM)Brent Ozar PLF, LLC
DBI328
Photo Licensed with Creative Commons: http://www.flickr.com/photos/philly22/4279056817/
How I Built SQL Servers
How I Thought I Did It:Carefully baseline application to determine load patternsHand-select components perfectly matched to application needsGo live with a faster, more reliable environment
Photo Licensed with Creative Commons: http://www.flickr.com/photos/dangoodphoto/5202499523/
How I Built SQL Servers
How I Thought I Did It:Carefully baseline application to determine load patternsHand-select components perfectly matched to application needsGo live with a faster, more reliable environment
How I Actually Did It:Took a few rush measurements with Perfmon at random timesTried to figure out what parts were rightGo live without testingSomebody changes the app and my work was useless anyway
Photo Licensed with Creative Commons: http://www.flickr.com/photos/icathing/26603225/
How Microsoft Designs SQL Server Appliances
Systematically review thousands of SQL ServersDistill use cases down to a few common patternsChoose hardware components that are very likely to work great for those patternsPublish an incredibly detailed test checklist to make sure the hardware is working as designed
How Microsoft Designs SQL Server Appliances
Systematically review thousands of SQL ServersDistill use cases down to a few common patternsChoose hardware components that are very likely to work great for those patternsPublish an incredibly detailed test checklist to make sure the hardware is working as designed
Let’s steal their hard work.
Session Agenda
Define common SQL Server use patternsUnderstand the right hardware for a patternRecognize the server designs we can reuseLearn how to test our own hardware
Define
Common SQL Use Patterns
OLTP: Transactional Processing
How It’s Accessed:Inserts, Updates, Deletes throughout the day
How It’s Stored:Large numbers of tables with lots of complex relationships, but usually under a terabyte of data.
Examples:Web sites, ecommerce, payroll
Data Warehousing
How It’s Accessed:Loaded in short windows overnight,then read-only with big reads through the day for reports
How It’s Stored:Just a few tables, but many historical records in each table, and often over a terabyte of data.
Examples:10 years of sales history, stock prices, patient history
The Real Difference
An OLTP user says, “I need to fetch Mr. Jones’s order. This transaction takes 3 seconds, but I want it to finish instantly.”
A data warehouse user says,“I need all sales data for bicycles from last year. This report takes 15 minutes, but I want it to finish in 30 seconds.”
Understand
The Right Hardware for Pattern
SQL Server Hardware At Its Simplest
OLTP scenario: 50GB table.
CPU
Memory
(64GB)
Drives(100GB
)
End User
The Right Hardware for the Fastest OLTP
Instant queries = cache all data in memoryMaximize memory sizeMinimize data size (drop extra indexes, right-size fields)
Instant transactions = blazing fast log fileSSD RAID 10 for multiple databasesCan get away with dedicated RAID 10 magnetic for a single DB
Avoiding locking issues = blazing fast TempDB with RCSI
Read Committed Snapshot Isolation
I Didn’t Say Blazing Fast Data Drives.
Once the data’s in cache, data drive speed rarely matters.
SQL Server restarts will mean slow performanceAggressively monitor data size, memory sizeWhen you run out, things get ugly fast
Note: this session is about building really fast servers,not really easy servers. Easy = RAID 10 everything.
SQL Server Hardware At Its Simplest
Data warehouse scenario: 500GB table.
CPU
Memory
(256GB)
Drives(1TB)
End User
The Right Hardware for the Fastest DWs
Instant queries = cache all data in memory (we can’t)
Maximize memory sizeMinimize data size (drop extra indexes, right-size fields)Maximize storage throughput to retrieve what isn’t cached
Instant transactions = blazing fast log fileSSD RAID 10 for multiple databasesCan get away with dedicated RAID 10 magnetic for a single DB
Avoiding locking issues = blazing fast TempDB with RCSI
Read Committed Snapshot Isolation
Recognize
Server Designs We Can Reuse
Photo Licensed with Creative Commons: http://www.flickr.com/photos/leejordan/2484541616/
Download: BrentOzar.com/go/fastdw
Fast Track Data Warehouse Reference Arch.
Microsoft provides sample framework for vendorsVendors like Dell, HP, IBM provide specific server, storage, switching parts as an approved whole
Microsoft gives you the testing instructions for validationVendors provide the proof, and you re-validate it
You can use this same knowledge with your gear too
Sample Fast Track Reference Architecture
Dual Xeon processors, 6 cores each: MCR 3,168 MB/sec
Four 8Gbps Host Bus Adapters (HBAs):Delivers 3,123 MB/sec
48 15k drives in multiple RAID 5s:Delivers 3,146 MB/sec
Notice the balance
SQL Server Can Handle More Than You Think
Maximum Consumption Rate (MCR):How fast each core can consume data from storageStarting point: 200MB/sec of sequential reads per core
Dual socket, quad core: 1,600MB/secFour socket, quad core: 3,200MB/secFour socket, ten core: 8,000MB/sec
Your Hardware Does Less Than You Think
From BrentOzar.com/posters
Typical Storage Throughput
New SSDs Do More Than You Think
Learn
How to Test Our Own Hardware
Photo Licensed with Creative Commons: http://www.flickr.com/photos/boostinchick/4956027368/
Test Storage Quickly with CrystalDiskMark
Free tool with instructions:BrentOzar.com/go/cdmPick 5 tests, 4000MB testfile, drive letterSequential:roughly akin to backups, large table scans4K QD32:vaguely similar to activeOLTP server or TempDB
Test Storage Slowly with SQLIO
Free tool with instructions:BrentOzar.com/go/sqlioLots of possible options,collect the whole setUse a test file larger than yourSAN’s cache (say, 20GB)Don’t run this on a live server
Your Goals
Test with CrystalDiskMark to get a quick ideaTry two simultaneous CrystalDiskMark tests against two different drive letters to see if your multipathing worksWhen that works, amp up to SQLIO and really push itProve the hardware can achieve the Maximum Consumption Rate before you bother installing SQL Server
Remember
Storage Isn’t the Only Solution
How to Reduce Storage Throughput Needs
Keep memory free for SQL Server data cachingDesign and maintain your indexesCompress dataUse partitioned views to segment old archive data into tables with 100% fill factor, perfect statistics, read-only filegroupsConsider columnstore indexes for reporting queriesMerry-go-round scans with SQL Server Enterprise Edition
Merry-Go-Round Scans in Enterprise Edition
tblSales – 1,000GB
Brent’s Query
Merry-Go-Round Scans in Enterprise Edition
tblSales – 1,000GB
Brent’s Query
Merry-Go-Round Scans in Enterprise Edition
tblSales – 1,000GB
Brent’s Query
Dandy’s Query
Merry-Go-Round Scans in Enterprise Edition
tblSales – 1,000GB
Brent’s Query
Dandy’s Query
Merry-Go-Round Scans in Enterprise Edition
tblSales – 1,000GB
Brent’s Query
Dandy’s Query
Merry-Go-Round Scans in SQL Server Type R
tblSales – 1,000GB
Dandy’s QueryDandy’s Query
What We Covered
Defined common SQL Server use patternsOLTP: “I want the query to finish instantly.”DW: “I want the query to finish in 30 seconds.”
Understood the right hardware for a patternOLTP: Cache it all in memoryDW: Get blazing fast
Recognized the server designs we can reuseDW: MS Fast Track Data Warehouse Reference Architecture
Learned how to test our own hardware and compare itRemembered that storage isn’t the only solution
Photo Licensed with Creative Commons: http://www.flickr.com/photos/atxjen/4014857345/
Track Resources
@sqlserver@TechEd_NA#msTechEd
mvaMicrosoft Virtual Academy
SQL Server 2012 Eval Copy
Get Certified!
Hands-On Labs
Resources
Connect. Share. Discuss.
http://northamerica.msteched.com
Learning
Microsoft Certification & Training Resources
www.microsoft.com/learning
TechNet
Resources for IT Professionals
http://microsoft.com/technet
Resources for Developers
http://microsoft.com/msdn
Complete an evaluation on CommNet and enter to win!
MS Tag
Scan the Tagto evaluate thissession now onmyTechEd Mobile
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.