Upgrade to SQL Server 2016 is justified as per Hybrid Cloud aspectThe Evolution of Microsoft Data Platform

Among the new features Microsoft introduced in the of SQL Server 2016 are a new Query Store, support for JSON documents, row-level security for filter predicates (SELECT), dynamic data masking, AES encryption for Endpoints and Always-On support for three synchronous replicas. The following is a breakdown of the improvements to analytics/BI and the new Azure support, respectively:

Analytics and Business Intelligence

PolyBase: a real-time link between structured and unstructured data Enhanced speed with in-memory OLTP to support operational analytics Non-clustered Columnstore indexes on a rowstore with a unique key Temporal tables: built-in support for audit history tables and slowly changing

dimensions Native support for "R" analytic applications More than 50 new DAX commands Parallel processing of tabular tables with multiple partitions Master data services (MDS) improvements, including faster Excel interaction Easier setup for Analysis Services for SharePoint PowerPivot mode Complete overhaul of the Reporting Services report building process

Azure- Cloud Related enhancements

Polybase Stretch database: dynamic storage and query access of older data in Azure Always Encrypted technology: for increased confidence in cloud-based databases Disaster Recovery Backup to Cloud Data Files in the cloud

We will discuss the features of Cloud specific.

PolyBase

As a platform, Hadoop provides an excellent tool to collect and then make discoveries against unstructured and semi-structured data. But once the discoveries have been made, they still reside in Hadoop. Special connectors such as Apache Sqoop allow Extract, Transform and Load (ETL) processes to move the data to SQL Server, but they require some finesse to work. Doable, not seamless and it required the physical movement of data.

First introduced in the Microsoft Analytic Platform System (APS, formerly, Parallel Data Warehouse), PolyBase provides a direct, real-time bridge between the relational world and Hadoop. But, PolyBase is not simply a physical bridge. It's a sophisticated query engine designed to use T-SQL against Hadoop while bypassing the MapReduce protocol when appropriate, allowing analysts to focus on the data using SQL rather than the more complex MapReduce protocols.

Installing PolyBase is straightforward. It's just another feature selection that has two software requirements. Windows Server 2012 R2 already has the .NET Framework 4.0 installed, so that just leaves the Java SE Runtime Environment (JRE 7.51 or higher) requirement. My installation used the JRE file: jre-8u51-windows-x64.exe.

Setting up PolyBase is a more complicated series of steps. There are no wizards and the documentation is in its early stages. First you have to change the server configuration for the ‘hadoop connectivity' setting. Next, the PolyBase yarn-site.xml file needs to be updated with a value from your Hadoop system if your Hadoop system is YARN-based. With that done, you'll follow a three-step process to first create an external data source, then an external file format, and finally an external table that uses the external data source and the external file format you just created. And if all goes well, you'll have an external table setup, using your newly created external data

source and external file format.

PolyBase allows…:

• Ad-hoc queries on Hadoop data• Import data from Hadoop/Azure Storage to SQL Server for persistent storage• Export data from SQL Server to Hadoop/Azure Storage for cold but queryable

storage• Seamless BI integration• Push-down computation to Hadoop

Walkthrough: Stretch DB

With Stretch DB, SQL Server 2016 creates a natural integration point with Microsoft Azure to leverage the cloud and the promise of limitless storage to move cold and warm data on-demand to Azure. This data is readily accessible for queries and this works seamlessly without any application change.

Let us consider a scenario where we want to seamlessly stretch cold data in order history to the cloud. Once configured the colder data in the order history table will start migrating to Azure without any downtime and change required for the application. SQL Server 2016 will keep moving colder data to Azure while maintaining hotter data on premise.

When the application makes a query where the part of the table that is stretched to Azure holds the data, SQL Server uses Microsoft Azure to run the query and return the data to the application. Setup is a multi-step process. The database server must be enabled using the server option ‘remote data archive'. Next, enable the database for stretch and, finally, the identified table. So, with a few clicks and a sign-on into Azure, the archival process should begin. Admin procedures include pausing and resuming the archive process, backup and restore, and most important, a process to unwind the stretch process and migrate the data back to an on-premises server. Undoubtedly, data access to the Azure cloud will be slower, but the cost trade-off could be justified.

Disaster recovery:

We can offer support to our customers to have secondary failover support to Windows Azure.

Microsoft SQL Server has an important high availability/disaster recovery feature called Always On. It allows you to deploy a database with multiple replicas that are kept synchronized and should the primary replica fail, one of the secondary replicas will take over. There are a few different Azure Resource Manager templates out there for deploying this in the Azure cloud. Here are just a few of them:

• SQL Server 2014 Always On including Domain Controllers• SQL Server 2016 Always On for Existing VNET and Domain

Additionally, there is also a template available through the Azure Commercial cloud portal.

The components involved in those deployments are continuously updated. Specifically, SQL Server 2017 has been released and PowerShell DSC modules such as xSqlServerDsc have been updated many times, e.g. xSqlServerDsc has recently been renamed to SqlServerDsc. There are also some features in SQL Always On that could potentially help simplify the deployments. One such feature is the Cloud Witness, which uses Azure Blob storage instead of a file share on a VM for cluster quorum.

 

Always Encrypted:

Technically, the Always Encrypted technology is a security upgrade. And its usefulness spans both on-premise databases, as well as those hosted in the cloud. But with accelerating migration of databases and systems to the cloud, security is at the top of requirements. The Always Encrypted feature gives you column-level security control over sensitive data, controlled by the application, so sensitive data is unavailable to DBAs and systems administrators. So if it is in the cloud, the cloud administrators do not have access to the encryption key.

Backup to Cloud:

• Near “bottomless” storage

• Off-site, geo-redundant

• No provisioning

• No device management

• Media Safety (decay free)

• Remote accessibility

Data files in cloud:

• Migrate Database to another machine

• Basic Disaster Recovery

• Separation of Compute and Storage

• For Azure VM, expand drive and size of Xdrive

• Better together: IaaS VM per minute billing