Spark the future.

May 4 – 8, 2015Chicago, IL

Realizing the Benefits of Upgrading Microsoft SQL Server Victor IsakovMCA, MCM, MCT, MVPvictor@sqlserversolutions.com.au

BRK2579

AbstractWith Microsoft planning to release a new version of SQL Server every couple of years your organisation should take a new approach to upgrading your SQL Server solution(s) so as to realise numerous benefits and maximize your return on investment. You should consider an upgrade project as an opportunity to make your SQL Server infrastructure more resilient, agile and optimized.In this session we will concentrate on the refactoring opportunities that you can take advantage of to improve performance, manageability or availability. With useful tips and examples along the way this should prove to be a very practical session that can help you build the case for upgrading SQL Server within your organisation.

Rationale / Motivation“Business As Usual” MentalityUpgrade like-for-likeNo innovationNo analysis of new capabilities to see if there is a fitNo re-investigation if current solution is still optimal / relevant

“Groundhog Day” DatabasesDatabase “stuck” doing the same thing day in, day out

Organizations not getting value for investmentLicensing Cost: Enterprise Agree / Software AssuranceStorage CostNo improvement in throughput which would translate to business benefits

Goal: Change the conversation with business

AgendaCurrent StateUpgrade ProcessDatabase Engine ConfigurationManageabilityHigh AvailabilityDatabaseRefactoring OpportunitiesFuture State

Current State

Industry ObservationsMicrosoft is planning on releasing a new version of SQL Server every 18-24 monthsSQL Azure updated continuously

Industry is still managing SQL Server like they were in the 1990’sIn a lot of cases even more risk averseNeed to be more agileLess processes for SQL Server infrastructure compared to other IT infrastructure

Change ManagementRelease ManagementIncident Management

SQL Server 2005 support ends April 12, 2016

Upgrade BlockersVendorSoftware not supported on later SQL Server editionsLearn from “mistakes of past”

RiskDatabase / code will break afterUnknown dependenciesA lot of risk is “perceived risk”

Lack of ResourcesIT Pro / DevelopersDomain level knowledgeThere are plenty of tools out there than can help analyze your current state[BRK4551] Upgrading and Migrating SQL Server (Bob Taylor)[BRK2577] Upgrading to Microsoft SQL Server 2014 and to Microsoft Azure (Mike Shelton; Ross LoForte)

CaveatsApplying Previous Knowledge / ExperienceExample: Failover Clustering

Automatically Using New FunctionalityExamples: Availability Groups

Failover Clustering: Fileshare Witness

“Business As Usual Forever” (BAUF) MentalityStart engaging the businessStart thinking tactically, strategically

ProcessesTechnology by itself rarely deliversNeed to change you processes

Upgrade Process

OpportunityAn upgrade project represents an opportunity to standardize, optimize, consolidate, etcLearn from the “mistakes of the past”

Develop SOE for WindowsStandardization

Disk layoutWindows Core

Develop SOE for SQL ServerStandardizationConfiguration

Define Tiers

TiersDefine Tiers for your SQL Server infrastructure For most organizations 3 tiers are sufficient

Consider future growthConsider Azure

Define Service Levels for TiersImportant to engage the business for Service Levels:

Importance to businessPerformanceAvailability

Map applications to appropriate TiersDecouple an application from a SQL Server instanceAn application becomes a service that gets delivered to the businessGoal is provide agility at the application level

Tiers (Concept)

DATABASE

TIER 1 TIER 2 TIER 3

A B C A B C A B

• Separate LUNs for Data and Log• Separate LUNs for TEMPDB’s

Data and Log

• Separate LUNs for Data and Log

• 8 vCPUs• 128 GB• AG

• 4 vCPUs

• 64 GB• Cluster

• 4 vCPUs

• 32 GB

• 4 vCPUs

• 32 GB• Cluster

• 4 vCPUs

• 16 GB

• 2 vCPUs

• 8 GB

• 2 vCPUs

• 8 GB

• 1 vCPU• 4 GB

LOB App X

SharePoint X

BizTalk X

VMware ESX

X

GIS X X

…

EnvironmentsDeveloper (DEV)Can use SQL Server Developer Edition to reduce licensing costs

User, Acceptance & Testing (UAT)Can use SQL Server Developer Edition to reduce licensing costs

Production (PROD)

Consider using cloud for UAT (and DEV) environments

Upgrade Process I still see organizations not doing some of the basics as part of the upgrade processIn some cases problems persist through subsequent upgradesRun these basics:Run DBCC CHECKDB with the DATA_PURITY optionRun DBCC UPDATEUSAGEChange page verification to CHECKSUM

Can use TDE to ensure checksum is calculated for all database pagesChange database compatibility flag as appropriate

SQL Server Features

SQL Server Time LineMicrosoft is more aggressively releasing new version of SQL Server going forward

SQL Server Version RTM Date Delta

SQL Server 2016 Summer 2016* Approx. 24 months

SQL Server 2014 April 2014 25 months

SQL Server 2012 March 2012 23 months

SQL Server 2008 R2 April 2010 20 months

SQL Server 2008 August 2008 33 months

SQL Server 2005 November 2005 120 months

SQL Server 2000 November 2000 24 months

SQL Server 7.0 November 1998 -

SQL Server 2008 (R2)CompressionPage compressionRow compression

Backup CompressionAdded to SQL Server Standard Edition in SQL Server 2008 R2

Extended EventsFILESTREAMNew DATE and TIME Data Types

SQL Server 2012Availability Groups“Evolution of Database Mirroring”Secondary replicas can be readableCan have multiple readable secondary replicasCan run DBCC / backup operations off secondary replicas

Columnstore IndexesRead-onlyNonclustered

SQL Server 2014Clustered Columnstore IndexesBuffer Pool ExtensionsAvailable in all SQL Server SKUs

Availability Group Enhancements8 replicasImproved availability with the readable secondaries when the primary replica being “offline”

In-Memory OLTP TablesMemory-optimized database engine integrated into the SQL Server engine

In-Memory OLTP is optimized for OLTP

Delayed Durability

Database Engine Configuration

Maximum Memory SupportSQL Server 2014 Standard Edition Supports 128GBEarlier versions of SQL Server Standard Edition were limited with the amount of RAM supported

SQL Server 2012: 64GBSQL Server 2008R2: 64GBSQL Server 2008: Operating system maximumSQL Server 2005: Operating system maximum

Take Advantage of Window Server CoreMuch smaller memory footprintAdditional benefits

Reduced “surface area”Less patching required*

Ability to achieve higher virtualization densitiesSupported since SQL Server 2012

Buffer Pool ExtensionExtend SQL Server’s Buffer

BenefitsIncreased random I/O throughputReduced I/O latencyIncreased transaction throughputImproved read performance with a larger hybrid buffer poolA caching architecture that can take advantage of present and future low-cost memory drives

Extend SQL Server’s Buffer Pool to Flash StorageDesigned for OLTP workloadsOnly clean pages are written to BPE fileNo need for RAIDOffload I/O path from SAN, SMB shares, etc

Database Engine Configuration Optionsoptimize for ad hoc workloadsImproves efficiency of the plan cache for workloads that contain many single use ad hoc batches

When a batch is compiled for the first time the Database Engine only stores a small compiled plan stub in the plan cache, instead of the full compiled planHelps to relieve memory pressure by not allowing the plan cache to become filled with compiled plans that are not reused

backup compression defaultDetermines whether the server instance creates compressed backups by default

blocked process thresholdSpecifies the threshold (in seconds) at which blocked process reports are generated

Uses the Deadlock Monitor background threadThe blocked process report is done on a best effort basisThere is no guarantee of any real-time or even close to real-time reporting

DeadlocksContention is often overlooked as root cause for performance problems1204 and 1222 Trace Flags are so “old school”Investigate how to use Extended Events insteadEmail developers through database engine whenever a deadlock occurs

Great way of getting deadlocks solved

Trace FlagsInvestigate trace flags and assess whether they are appropriate for your SQL Server SOEsTRACE FLAG

DESCRIPTION Source

1117 Ensures equal autogrowth of all files in file group Fast Track DW 2.0 Architecture

1118 [tempdb] system database optimization(Use uniform extents only)

BOL

1204 / 1222 Deadlock detection BOL

3023 CHECKSUM option is automatically enabled for the BACKUP command

KB2656988

3226 Prevents successful backup operations from being logged to ERRORLOG

BOL

4199 Query Optimizer fixes BOL

Manageability

Backup CompressionReleased in SQL Server 2008Supported in Standard Edition since SQL Server 2008 R2Controlled at Different LevelsServer LevelBACKUP statement

Trace Flag 3023 (Backup CHECKSUMs)Trace flag 3023 automatically enables the CHECKSUM option for the BACKUP commandValidation of page checksum if the database has the PAGE_VERIFY option set to CHECKSUM and the database page was last written by using checksum protectionGeneration of a backup checksum over the backup streams that are written to the backup fileDocumented: KB2656988

Use to make sure that all backups use the backup CHECKSUMYou do not have to rewrite all the existing backup scriptsYou use utilities such as log shipping or maintenance plans

These utilities and the associated stored procedures do not provide a CHECKSUM option

SQL Server 2014: ‘backup checksum default’

High Availability

High AvailabilityAvailability Groups seem to be the “high availability de jour”Make sure you understand the use case for Failover Clustering versus Availability GroupsIt very much depends on your applicationVery highly recommended to implement a Proof-of-Concept

Failover ClusteringBe careful with applying previous knowledge and experienceFailover Clustering has substantially changed in Windows Server 2012 / R2Quorum types

Fileshare WitnessDynamic QuorumNetwork configuration

NIC TeamingSMB 3 Support

Make sure you re-evaluate Windows Server 2012 / R2

Failover Clustering: Configuration“Classic” configuration2 separate networks

1 dedicated to cluster traffic1 dedicated to cluster and client traffic

IPv6 disabled(Fileshare Witness)

Availability GroupsMultiple replicas with their own storage2 replicas in failover pairAsynchronous / synchronousReplicas can be utilized:

Read-only workloadsBackupsDatabase consistency checks

CaveatsDatabases must use FULL recovery modelCan affect application behaviour / performanceJob management more complex

Real World Example: Batch ProcessCRM database solution2TB OLTP database6TB document store database

Existing solutionSQL Server 2008 R2Failover Cluster + disk-based SAN

New solutionSQL Server 2014Availability Groups with local PCIeSSDs

Key batch processesEnd-of-DayEnd-of-Week

Availability Group OverheadNightly Batch Process

No AG AG (FO) AG (FO + Sync) FCI0

20

40

60

80

100

120

140

160

29

70 68

80

34

84 81

138

40

7281

102

Daily Overnight Rebuild

Min Max Average

No AG AG (FO) AG (FO + Sync) FCI0

100

200

300

400

500

600

700

212

436475

500

248

468513

634

227

452494

522

Weekend Overnight Rebuild

Min Max Average

Delayed DurabilityAllows client applications to continue without waiting for acknowledgement of COMMIT from transaction logNot hardening logs effectively before commitPotentially very useful for Availability GroupsBE AWARE OF IMPLICATIONS!

Database

Database File LayoutDatabases are only getting largerImpacts recoverabilityImpacts disaster recoveryDifficult to manage

Take advantage of file groupsImproved backup / restore capabilitiesImproved storage / capacity / performance management

SELECT * FROM sys.dm_io_virtual_file_stats

Real World Use Case: File GroupsDatabase file layout:

Real World Use Case: File Groupssys.dm_io_virtual_file_statsFile Name Number of

ReadsNumber of Bytes Read

I/O Stall Read

I/O Stall Queued Read (ms)

Number of Writes

Number of Bytes Written

I/O Stall Write (ms)

I/O Stall Queued Write (ms)

I/O Stall Size on Disk (bytes)

System 21,167,0163,307,566,972,9

2813,926,11

9 036,838,93

6430,417,707,0

0826,817,93

0 040,744,04

990,855,440,38

4

Log 35,307 63,196,548,096 88,158 053,295,31

9597,217,190,4

0028,414,50

7 028,502,66

5104,857,600,0

00

Indexes 18,295,8351,224,097,431,5

5211,704,40

6 026,995,16

0252,918,013,9

5218,969,15

0 030,673,55

648,003,153,92

0Deprecated 4,674 9,367,928,832 30,768 0 269 6,307,840 158 0 30,926 403,701,760Audit 84 5,005,312 87 0 16 131,072 5 0 92 104,857,600Temp 84 5,005,312 75 0 16 131,072 5 0 80 104,857,600

AccountMedia 1,830,9632,371,033,858,0

4838,852,55

6 0 9,865 129,916,928 4,109 038,856,66

581,893,785,60

0

AccountRebuildLog 594,016 403,912,130,560 1,244,190 0 46,657 634,077,184 33,863 01,278,05313,258,981,37

6

Entry 9,375,1443,432,744,927,2

3214,745,79

0 0 67,365 2,995,478,528 54,579 014,800,36

9106,430,464,0

00

Entry_Indexes 2,473,552 562,829,582,336 3,615,644 0 4,645,61442,354,139,13

6 3,916,374 07,532,01824,851,251,20

0

TransactionAudit 259,983 543,635,234,816 1,276,844 0 35,225 558,620,672 12,388 01,289,23231,876,710,40

0TransactionAudit_Indexes 5,898,013 733,996,007,424 4,606,510 0 5,890,031

52,926,455,808 4,223,188 08,829,698

39,321,600,000

Data 137 17,883,136 49 0 24 524,288 21 0 70 104,857,600

Database CompressionAvailable since SQL Server 2008Effectively fit more data in the Buffer PoolQuery performance improvementReduced backup size and duration

Row CompressionPage Compression

SQL Server Compression Estimatorssce.codeplex.com

Real World Use Case: Database CompressionPage compression was implemented when SQL Server 2005 was upgraded to SQL Server 2008Selectively enabled on tables with appropriate benefitStill have a 40% storage saving over the last Query performance improvedDatabase backups improvedSizeDuration

Real World Use Case: Database CompressionLarge DSS / OLAP tablesReport queries reduced from 2+ hours to >30 minutesNo application change required

Columnstore IndexesAvailable since SQL Server 2012Nonclustered Columnstore indexes were read-onlyClustered Columnstore available in SQL Server 2014

In-Memory ColumnstoreStores data in columnar format for massive compressionBoth memory and diskBuilt-in to core RDBMS engine

10-100x fasterWork on existing hardwareEasy upgradeEasy deployment

C1 C3 C5C4C2 C6

Columnstoreindex representation

SQL Server 2014: Clustered Columnstore IndexesUpdatable, clustered Columnstore now supportedNo need for separate base tableNo need for other indexes

New COLUMNSTORE_ARCHIVE compressionBetter compression at higher CPU cost: 80-90% compression

Far fewer data type restrictions than SQL Server 2012More operators support batch mode processingNeed to refactor your Transact-SQL code back potentially

Use case: Real-time analytics involving large scans and aggregations

Clustered Columnstore Batch ProcessingBiggest advancement in query processing in years! Data moves in batch through query plan operatorsHighly efficient algorithms, better parallelismSQL Server 2014Support for all flavours of JOINs

OUTER JOINSemi-join: IN, NOT IN

UNION ALLScalar aggregatesMixed mode plansImprovements in bitmaps, spill support, …

Real World Use Case: DSS/OLAPNo application change requiredSubstantial improvement in query performanceNotice there is no index usage for columnstore indexes

No nonclustered indexes

Real World Use Case: AuditingBusiness RequirementsCompliance reasons“Necessary evil”

WorkloadsPredominantly INSERT activityHardly ever queried / Never queried

TechnologyClustered ColumnstorePartitioning

OpportunityMinimal riskMinimal refactoring

Real World Use Case: Auditing (Storage)Heap:Baseline

Page Compression:38.5%

Clustered Columnstore:CCI: 13.4%ACCI: 10.4%

Real World Use Case: Auditing (Querying)No such thing as “free lunch”As expected we pay a performance penalty with point queries

Refactoring Opportunities

Technical DebtStart paying your “technical debt”Data types deprecated in SQL ServerBetter data typesBetter programming patternsNeed to develop strategy to mitigate future riskRelease ManagementChange Management

“Cloud readiness”

Deprecated Data TypesTEXT / IMAGE data types are deprecatedVariety of optionsVARBINARY(MAX)FILESTREAMFILETABLE

ConsiderationsPerformanceStorage Cost

Windows DeduplicationBuffer Pool / WIN32 pathDon’t forget impact on backup strategy

20 minutes versus 3 hours

“Better” Data TypesConsider NOT using DATETIME data typeDATETIMEDATETIME2

“Better” Programming PatternsSafer ProgrammingTRY / CATCHTRY_PARSETRY_COVERT

Review TRANSACTIONsBEGIN TRAN / COMMIT TRANSET XACT_ABORT

Common Table ExpressionsSequential patterns

Contained DatabasesAzure SQL Database version V12

Future State

GoalsDatabase is a “service” that meets business SLAsEngage the business ASAP in the Upgrade Project

Decouple database(s) from SQL Server instancesAgility between Tiers at database levelAbility to upgrade at database levelDocumentationRelease ManagementChange Management

Cloud Readiness

Visit Myignite at http://myignite.microsoft.com or download and use the Ignite Mobile App with the QR code above.

Please evaluate this sessionYour feedback is important to us!

© 2015 Microsoft Corporation. All rights reserved.