Mission critical features in SQL 2016David LythPat MartinPremier Field Engineers, Microsoft New Zealand

Hyperscale cloud

Deeper insights across data

SQL Server 2016 highlights

Performance Security Availability Scalability

Operational analyticsInsights on operational data; Works with in-memory OLTP and disk-based OLTP

In-memory OLTP enhancementsGreater T-SQL surface area, terabytes of memory supported, and greater number of parallel CPUs

Query Store Monitor and optimise query plans

Native JSON Expanded support for JSON data

Temporal DatabaseQuery data as at points in time

Always encrypted

Sensitive data remains encrypted at all times with ability to query

Row Level SecurityApply fine-grained access control to table rows

Dynamic Data MaskingReal-time obfuscation of data to prevent unauthorised access

Other enhancementsAudit success/failure of database operations

TDE support for storage of in-memory OLTP tables

Enhanced auditing for OLTP with ability to track history of record changes

Enhanced AlwaysOn

Three synchronous replicas for auto failover across domains

Round robin load balancing of replicas

Automatic failover based on database health

DTC for transactional integrity across database instances with AlwaysOn

Support for SSIS with AlwaysOn

Enhanced caching Cache data with automatic, multiple TempDB files per instance in multi-core environments

Support for Windows Server 2016

12 TB of memory supported

Mission critical features

Query Store

Pat Martin

Have You Ever…?…had your system slow down and everyone wait for you to magically fix the problem …

…upgraded an application to the latest SQL Server version and had an issue with a plan change slowing your application down?

With Query Store…

I CAN retrieve a full history of query execution

I CAN quickly pinpoint the most expensive queries

I CAN identify all queries that regressed

I CAN easily force a better plan from plan history with a single

line of T-SQL

I CAN safely manage server restart or upgrade

Compile MSG

Execute MSG

Query Store

Durability latency controlled by DB option

DATA_FLUSH_INTERNAL_SECONDS

Async Write-Back

Compile

Execute

SQL

Plan Store

Runtime Stats

Query Store

Schema

Query Data StoreCollects query texts (+ all relevant properties)

Stores all plan choices and performance metrics

Works across restarts / upgrades / recompiles

Simplifies performance troubleshooting

New Views

Intuitive and easy plan forcing

Keeping stability while upgrading to SQL

Sever 2016

Upgrade to SQL vNextKeep 110/120

Compatibility LevelFreeze plans

(optional)

Run Query Store

(establish performance

baseline)

Move to 130 Compatibility

Level and unfreeze plans

Monitor performance

and fix regressions with plan forcing

Monitoring performance using Query

Store The Query Store feature provides DBAs with insight on query plan choice and performance

/* (1) Turn ON Query Store */

ALTER DATABASE MyDB SET QUERY_STORE = ON;

/* (2) Review current Query Store parameters */SELECT * FROM sys.database_query_store_options

/* (3) Set new parameter values */ALTER DATABASE MyDBSET QUERY_STORE ( OPERATION_MODE = READ_WRITE, CLEANUP_POLICY = ( STALE_QUERY_THRESHOLD_DAYS = 30 ), DATA_FLUSH_INTERVAL_SECONDS = 3000, MAX_SIZE_MB = 500, INTERVAL_LENGTH_MINUTES = 15);

/* (4) Clear all Query Store data */ALTER DATABASE MyDB SET QUERY_STORE CLEAR;

/* (5) Turn OFF Query Store */ALTER DATABASE MyDB SET QUERY_STORE = OFF;

/* (6) Performance analysis using Query Store views*/SELECT q.query_id, qt.query_text_id, qt.query_sql_text, SUM(rs.count_executions) AS total_execution_countFROMsys.query_store_query_text qt JOIN sys.query_store_query q ON qt.query_text_id = q.query_text_id JOINsys.query_store_plan p ON q.query_id = p.query_id JOINsys.query_store_runtime_stats rs ON p.plan_id = rs.plan_idGROUP BY q.query_id, qt.query_text_id, qt.query_sql_textORDER BY total_execution_count DESC

/* (7) Force plan for a given query */exec sp_query_store_force_plan 12 /*@query_id*/, 14 /*@plan_id*/

DB-level feature exposed through T-SQL extensions

ALTER DATABASE

Catalog views (settings, compile and runtime stats)

Stored Procs (plan forcing, query/plan/stats cleanup)

Live Query Statistics

Live Query StatisticsCollect actual metrics about query while running

View CPU/memory usage, execution time, query progress, etc.

Enables rapid identification of potential bottlenecks for troubleshooting query performance issues.

Allows drill down to live operator level statistics:

Number of generated rows

Elapsed time

Operator progress

Live warnings, etc.

Query Store / LQS in action…

Pat Martin

Temporal Database

David Lyth

Real data sources are dynamic Historical data may be critical to business success

Traditional databases fail to provide required insights

Workarounds are…Complex, expensive, limited, inflexible, inefficient

SQL Server 2016 makes life easyNo change in programming model

Provides new insights

Why Temporal

Time Travel Data Audit

Slowly Changing Dimensions

Repair record-level corruptions

No change in programming model

New insights

INSERT / BULK INSERT

UPDATE

DELETE

MERGE

DML SELECT * FROM temporal

Querying

How to start with Temporal

CREATE temporal TABLE PERIOD FOR SYSTEM_TIME…

ALTER regular_table TABLE ADD PERIOD…

DDL

FOR SYSTEM_TIMEAS OF FROM..TOBETWEEN..ANDCONTAINED IN

Temporal Querying

Temporal table (actual data)

Insert / Bulk Insert

* Old versions

Update */ Delete *

How system-time works?

History Table

Temporal table (actual data)

Temporal Queries * (Time travel,etc.)

How system-time works?

History Table

Regular queries (current data)

* Include Historical Version

DepNum DepName MngrID From To

A001 Marketing 5 2005 2008

A002 Sales 2 2005 2007

A003 Consulting 6 2005 2006

A003 Consulting 10 2009 2012

DepNum DepName MngrID

A001 Marketing 5

A001 Marketing 6

A002 Sales 2

A002 Sales 5

A003 Consulting 6

A003 Consulting 10

DepNum DepName MngrID From To

A001 Marketing 6 2008 ∞

A002 Sales 5 2007 ∞A001

A002

A003

period of validity current time

∞

∞

Department (current)

Department (history)

Department (current + history)

2005 2015

SELECT * FROM DepartmentSELECT * FROM Department FOR SYSTEM_TIMEBETWEEN '2006.01.01' AND '2007.01.01'

SELECT * FROM Department FOR SYSTEM_TIMECONTAINED IN ('2007.01.01', '2009.01.01')SELECT * FROM Department FOR SYSTEM_TIME AS OF '2006.01.01'

Getting insights from temporal

A001

A002

A003

“Get actual row versions”AS OFBETWEEN..ANDCONTAINED IN

SELECT * FROM Department FOR SYSTEM_TIME AS OF '2010.01.01'

Facts:

1. History is much bigger than actual data

2. Retained between 3 and 10 years

3. “Warm”: up to a few weeks/months

4. “Cold”: rarely queried

Solution:

History as a stretch table:

PeriodEnd < “Now - 6 months”

Azure SQL Database

Provides correct information about stored facts at any point in time, or between two points in time.

There are two orthogonal sets of scenarios with regards to temporal data:

System(transaction)-time

Application-time

SELECT * FROM Person.BusinessEntityContactFOR SYSTEM_TIME BETWEEN @Start AND @EndWHERE ContactTypeID = 17

Performance

Temporal database support - BETWEEN

Temporal Database in action…

David Lyth

Row Level Security

Pat Martin

Fine-grained access control over specific rows in a database table

Help prevent unauthorised access when multiple users share the same tables, or to implement connection filtering in multitenant applications

Administer via SQL Server Management Studio or SQL Server Data Tools

Enforcement logic inside the database and schema bound to the table.

Protect data privacy by ensuring the right access across rows

SQL Database

Customer 1

Customer 2

Customer 3

Row Level Security

Fine-grained access control

Keeping multi-tenant databases secure by limiting access by other users who share the same tables.

Application transparency

RLS works transparently at query time, no client application changes needed.

Compatible with RLS in other leading products.

Centralised security logic

Enforcement logic resides inside the database and is schema-bound to the table it protects, providing greater security, reduced application maintenance and complexity.

Store data intended for many consumers in a single database/table while at the same time restricting row level read and write access based on users’ execution context.

Benefits of Row Level Security

Sample RLS use cases

A hospital can create a security policy that allows nurses to view data rows for their own patients only.

A bank can create a policy to restrict access to rows of financial data based on the employee's business division, or based on the employee's role within the company.

A multi-tenant application can create a policy to enforce a logical separation of each tenant's data rows from every other tenant's rows. Efficiencies are achieved by the storage of data for many tenants in a single table. Of course, each tenant can view only their own data rows.

CREATE SECURITY POLICY mySecurityPolicyADD FILTER PREDICATE dbo.fn_securitypredicate(wing, startTime,

endTime) ON dbo.patients

Predicate functionUser-defined inline table-valued function (iTVF) implementing security logicCan be arbitrarily complicated, containing joins with other tables

Security predicateApplies a predicate function to a particular table Two types: filter predicates and blocking predicates – blocking predicate not in SQL16 CTP2

Security policyCollection of security predicates for managing security across multiple tables

RLS concepts

Security

CREATE FUNCTION dbo.fn_securitypredicate(@wing int)

RETURNS TABLE WITH SCHEMABINDING AS

return SELECT 1 as [fn_securitypredicate_result] FROM

StaffDuties d INNER JOIN Employees e

ON (d.EmpId = e.EmpId)

WHERE e.UserSID = SUSER_SID()

AND @wing = d.Wing;

CREATE SECURITY POLICY dbo.SecPol

ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing)

ON Patients

WITH (STATE = ON)

Fine-grained access control over rows in a table based on one or more predefined filtering criteria, e.g., user’s role or clearance level in an organisation

Concepts:

Predicate function

Filter Predicate

Security policy

RLS implementation example

Two

App user (e.g., nurse) selects from Patients tableThree

Security Policy transparently rewrites query to apply filter predicate

Database Policy Manager

CREATE FUNCTION dbo.fn_securitypredicate(@wing int) RETURNS TABLE WITH SCHEMABINDING AS return SELECT 1 as [fn_securitypredicate_result] FROM StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() AND @wing = d.Wing;

CREATE SECURITY POLICY dbo.SecPol ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients WITH (STATE = ON)

FilterPredicate:

INNER JOIN…

SecurityPolicy

Application

Patients

One

Policy manager creates filter predicate and security policy in T-SQL, binding the predicate to the Patients table

Nurse

SELECT * FROM Patients

SELECT * FROM Patients SEMIJOIN APPLY dbo.fn_securitypredicate(patients.Wing);

SELECT Patients.* FROM Patients, StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() AND Patients.wing = d.Wing;

RLS in three steps

Row Level Security in action…

Pat Martin

Dynamic Data Masking

David Lyth

Dynamic Data Masking

Security Officer

ALTER TABLE [Employee] ALTER COLUMN [SocialSecurityNumber]ADD MASKED WITH (FUNCTION = ‘SSN()’

ALTER TABLE [Employee] ALTER COLUMN [Email]ADD MASKED WITH (FUNCTION = ‘EMAIL()’)

GRANT UNMASK to admin1

1) Security officer defines dynamic data masking policy in T-SQL over sensitive data in Employee table

2) App user selects from Employee table3) Dynamic data masking policy obfuscates the sensitive data in the query results

SELECT [Name], [SocialSecurityNumber], [Email], [Salary]FROM [Employee]

admin1 loginother login

Dynamic Data MaskingALTER TABLE Membership ALTER COLUMN FirstName ADD MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)');ALTER TABLE Membership ALTER COLUMN Phone# ADD MASKED WITH (FUNCTION = 'default()');ALTER TABLE Membership ALTER COLUMN Email ADD MASKED WITH (FUNCTION = 'email()');

EXECUTE AS USER = 'TestUser';SELECT * FROM Membership;REVERT;

--Grant unmask to see all dataGRANT UNMASK TO TestUser;EXECUTE AS USER = 'TestUser';SELECT * FROM Membership;REVERT;

-- Removing the UNMASK permissionREVOKE UNMASK TO TestUser;

--Remove masking on a columnALTER TABLE Membership ALTER COLUMN FirstName DROP MASKED;

ALTER TABLE [Membership] ALTER COLUMN [FirstName]ADD MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)'

ALTER TABLE [Membership] ALTER COLUMN [Phone#]ADD MASKED WITH (FUNCTION = 'default()');

ALTER TABLE [Membership] ALTER COLUMN [Email] ADD MASKED WITH (FUNCTION = ‘email()’) GRANT UNMASK to admin1

Dynamic Data Masking in action…

David Lyth

Summary

Pat Martin

Performance Security Availability Scalability

Operational analyticsInsights on operational data; Works with in-memory OLTP and disk-based OLTP

In-memory OLTP enhancementsGreater T-SQL surface area, terabytes of memory supported, and greater number of parallel CPUs

Query Store Monitor and optimise query plans

Native JSON Expanded support for JSON data

Temporal DatabaseQuery data as at points in time

Always encrypted

Sensitive data remains encrypted at all times with ability to query

Row Level SecurityApply fine-grained access control to table rows

Dynamic Data MaskingReal-time obfuscation of data to prevent unauthorised access

Other enhancementsAudit success/failure of database operations

TDE support for storage of in-memory OLTP tables

Enhanced auditing for OLTP with ability to track history of record changes

Enhanced AlwaysOn

Three synchronous replicas for auto failover across domains

Round robin load balancing of replicas

Automatic failover based on database health

DTC for transactional integrity across database instances with AlwaysOn

Support for SSIS with AlwaysOn

Enhanced caching Cache data with automatic, multiple TempDB files per instance in multi-core environments

Support for Windows Server 2016

12 TB of memory supported

Mission critical features

Resources

TechNet & MSDN FlashSubscribe to our fortnightly newsletter

http://aka.ms/technetnz http://aka.ms/msdnnz

http://aka.ms/ch9nz

Microsoft Virtual AcademyFree Online Learning

http://aka.ms/mva

Sessions on Demand

Speak to us if you’re interested in hearing more about our Premier Support offerings

ServicesNZ@Microsoft.com

Microsoft | Services

Complete your session evaluation now and win!

© 2015 Microsoft Corporation. All rights reserved.Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or

other countries.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.