KoprowskiT_SQLRelay2014#6_Leeds_WADBForBeginners

Windows Azure SQL Database

Tips and Tricks for beginners

Tobiasz Janusz Koprowski | Independent Consultant

Premier

Gold

Silver

Bronze

Our awesome sponsors

ABOUT ME3

SQL Relay 2014 | Leeds Oct 28th | KoprowskiT - WASD Tips and Tricks for Beginners

Member of the Board Polish Information Processing Society

EMEA Vice-Chair of Global IT Community Association

Microsoft Certified Trainer (SharePoint, SQL Server, Security, Azure)

Subject Matter Expert at CQURE

SQL Server MVP (five years)

MVP Mentor Program Member

consultant, security & licensing auditor and trainer

active blogger, influencer, technical writer

last 9 years living in Data Processing Center

last 15 years banking and ICT

NOW going into Independent Consultancy

frequently speaker at several conferences: SQL Server Community Launch, Time for

SharePoint, CodeCamps, InfoMEET, SharePoint Community Launch, CISSP Day, SQL in the

City, InfoTRAMS, SQL Bits, IT Camp, SQL Saturday, CareerCon, Sharepoint & SQL

Connection, Kariera IT, SQL Rally, SQL Relay, InfoTech…

Microsoft Terminology Community / translations for Wndows 7/8 and Visual Studio

2010/2012

AGENDA

PART ONE: Slide Talks

plan, compability, features, functionality

PART TWO: Show Time

azure portal, database portal, management studio, 3rd party tools

PART THREE: The Future of DBA

short discussion...

APPENDIX

additional links, q&a

4


PART ONE:

Slide Talks

6


PART ONE: Private > IaaS > PaaS

PART ONE: Azure Short Introduction

• Windows Azure

Compute:

Virtualized compute environment based

on Windows Server

Storage:

Durable, scalable, & available storage

Management:

Automated, model-driven management of the service

• SQL Azure (cloud-based database)

Database:

Relational processing for structured/unstructured data

• App Fabric (.NET services)

Service Bus:

General purpose application bus

Access Control:

Rules-driven, claims-based access control

7


Data Centers for Azure Platform8


Data Centers for Azure Platform

North America

US Central - Iowa

US East - Virginia

US East 2 - Virginia

US Gov Iowa - Iowa

US Gov Virginia - Virginia

US North Central - Illinois

US South Central - Texas

US West – California

Asia

Asia Pacific East - Hong Kong

Asia Pacific Southeast – Singapore

Europe

North Europe - Dublin, Ireland

West Europe - Amsterdam, Netherlands

Japan

Japan East - Saitama Prefecture

Japan West - Osaka Prefecture

Oceania (announced, coming soon)

Australia - Sydney, New South Wales

Australia - Melbourne, Victoria

South America (preview)

Brasil - Sao Paulo,

Some

datacenters have servers

grouped inside containers -

each containing

1800-2500 servers.

9


Security Requirements for Azure Platform

As a Service Provider Microsoft has an obligation to passing the several rules for security:

• ISO/IEC 27001:2005

• SAS 70 Type 1 and II

Also they passing (continuisly) several data securing audits:

• PCI DSS

• SOX compliance

• HIPAA compliance

And of course requirements for Data Centers:

• Physical security of the data centers (locks, cameras, biometric devices, card readers, alarms)

• Firewalls, application gateways and IDS to protect the network

• Access Control Lists (ACLs) applied to virtual local area networks (VLANs) and applications

• Authentication and authorization of persons or processes that request access to data

• Hardening of the servers and operating system instances

• Redundant internal and external DNS infrastructure with restricted write access

• Securing of virtual machine objects

• Securing of static and dynamic storage containers

10


Security Requirements for Azure Platform

• ISO/IEC 27001:2005 Audit and CertificationISO Scope: The following Azure features are in scope for the current ISO audit: Cloud Services (including Fabric and RDFE), Storage (Tables,

Blobs, Queues), Virtual Machines (including with SQL Server), Virtual Network, Traffic Manager, Web Sites, BizTalk Services, Media Services,

Mobile Services, Service Bus, Workflow, Multi-Factor Authentication, Active Directory, Right Management Service, SQL Database, and HDInsight.

This includes the Information Security Management System (ISMS) for Azure, encompassing infrastructure, development, operations, and

support for these features. Also included are Power BI for Office 365 and Power Query Service.

• SOC 1 and SOC 2 SSAE 16/ISAE 3402 AttestationsScope: The following Azure features are in scope for the current SOC 1 Type 2 and SOC 2 Type 2 attestations: Cloud Services (includes stateless

Web, and Worker roles), Storage (Tables, Blobs, Queues), Virtual Machines (includes persistent virtual machines for use with supported

operating systems) and Virtual Network (includes Traffic Manager).

• United Kingdom G-Cloud Impact Level 2 AccreditationScope: The following Azure features are in scope for the IL2 accreditation: Virtual Machines, Cloud Services, Storage (Tables, Blobs, Queues,

Drives), and Virtual Network.

• Cloud Security Alliance Cloud Controls Matrix

• Federal Risk and Authorization Management Program (FedRAMP)

• Payment Card Industry (PCI) Data Security Standards (DSS) Level 1

• HIPAA Business Associate Agreement (BAA)

• Family Educational Rights and Privacy Act (FERPA)

11


PART ONE: Virtual Machines

Virtual Machines

Launch Windows Server and Linux in minutes

• Scale from 1 to 1000s of VM Instances

• Built-in Virtual Networking, Load Balancing

• Leverage Hybrid Consistency with On-Premises

• Microsoft SQL Server and SharePoint Server

• Oracle, MySQL, Redis, MongoDB

• Save money with per-minute billing

12


PART ONE: HDInsight

HDInsight

Our 100% Apache Hadoop-based service in the

cloud

• Scale to petabytes on demand

• Process unstructured and semi-structured data

• Develop in Java, .NET, and more

• No hardware to buy or maintain

• Pay only for what you use

• Spin up a Hadoop cluster in minutes

• Visualize your Hadoop data in Excel

• Easily integrate on-premises Hadoop clusters

13


PART ONE: Machine Learning

Machine Learning PREVIEW

Powerful cloud-based predictive analytics

• Designed for new and experienced users

• Proven algorithms from MS Research, Xbox and Bing

• First class support for the open source language R

• Seamless connection to HDInsight for big data

solutions

• Deploy models to production in minutes

• Pay only for what you use. No hardware or software to

buy.

14


http://bit.ly/JenANDMachineLearning

Shared Environment

Hardware Boundary

Hardware Boundary

Hardware Boundary

Hardware Boundary

BC

D

AA

B

B

C

CD

D

A

15


PART ONE: SQL Azure few days ago

We want to work with SQL Azure (oops SQL Database)

Size does matter?

Personal: 1GB, 5GB

Business: 10GB, 20 GB, 30 GB, 40 GB, 50 GB, 100 GB, 150 GB (3x50)

“Private”: less than 100 MB ($4.995 / month)

Version does matter?

10.25 my first experience, but after July 2011 Service Release…

11.0.9214.51 now > close to SQL Server 2012

Row Versioning-Based Isolation Levels

Both the READ_COMMITTED_SNAPSHOT and ALLOW_SNAPSHOT_ISOLATION database options are set to ON in

Microsoft Azure SQL Database.

Because SET <snapshot_option> in the ALTER DATABASE Transact-SQL statement is not supported, these database

options cannot be changed.

error msg 40544:The database has reached its size quota. Partition or delete data, drop

indexes, or consult the documentation for possible resolutions.

16


http://blogs.msdn.com/b/windowsazure/archive/2011/07/13/announcing-sql-azure-july-2011-service-release.aspx

PART ONE: SQL Azure now Size does matter?

Personal: 1GB, 5GB

Business: 10GB, 20 GB, 30 GB, 40 GB, 50 GB, 100 GB, 150 GB (3x50)

“Private”: less than 100 MB ($4.995 / month)

Size does matter!

BASIC

size: 100MB, 500MB, 1GB, 2GB

performance: 5DTU

STANDARD

size: 100MB, 500MB, 1GB, 2GB, 5GB, 10GB, 20GB, 30GB, 40GB, 50GB, 100GB, 150GB, 200GB, 250GB

performance: S0 10DTU, S1 20DTU, S2 50DTU

PREMIUM

size: 100MB, 500MB, 1GB, 2GB, 5GB, 10GB, 20GB, 30GB, 40GB, 50GB, 100GB, 150GB, 200GB, 250GB, 500GB

performance: P10 100DTU, P2 200DTU, SP3 800DTU

What is DTU

DTUs provide a way to describe the relative capacity of a performance level of Basic, Standard,

and Premium databases. DTUs are based on a blended measure of CPU, memory, reads, and writes.

17


PART ONE: SQL Azure and DTU

Service Tier/Performance

LevelDTU MAX DB Size

Max Worker Threads

Max SessionsBenchmark

Transaction RatePredictability

Basic

5 2 GB 30 300

16,600

transactions per

hour

Good

Standard/S0

10 250 GB 60 600521 transactions

per minuteBetter

Standard/S1

20 250 GB 90 900934 transactions

per minuteBetter

Standard/S2

50 250 GB 120 1,200

2,570

transactions per

minute

Better

Premium/P1

100 500 GB 200 2,400105 transactions

per secondBest

Premium/P2

200 500 GB 400 4,800228 transactions

per secondBest

Premium/P3

800 500 GB 1,600 19,200735 transactions

per secondBest

18


PART ONE: SQL Azure Collation

Collation does it matter?

default database collation used by Microsoft Azure SQL Database is SQL_LATIN1_GENERAL_CP1_CI_AS,

where LATIN1_GENERAL is English (United States), CP1 is code page 1252, CI is case-insensitive, and AS

is accent-sensitive

SQL_Latin1_General_CP1_CI_AS

When using an on-premise SQL Server, you can set collations at server, database, column, and

expression levels.

Microsoft Azure SQL Database does not allow setting the collation at the server level.

To use the non-default collation with Microsoft Azure SQL Database, set the collation with the Create

Database Collate option, or at the column level or the expression level.

SQL Database does not support the Collate option with the Alter Database command.

By default, in SQL Database, temporary data will have the same collation as the database

19


PART ONE: users, migration, automation,

Naming, Logins, Users

Admin, administrator, guest, root, sa

Data Migration Support

You can use:

SQL Server 2008 Integration Services (SSIS)

The bulk copy utility (BCP.exe)

System.Data.SqlClient.SqlBulkCopy class

Scripts that use INSERT statements to load data into the database

SQL Data Compare from Red Gate

You can't use:

The RESTORE statement.

Attaching a database to the SQL Database server.

No SQL Server Agent (almost)

No SQL Server jobs

No SQL Server Browser

No Cross Database queries

20


PART ONE: Indexes

Indexes, of course indexes

You MUST use CLUSTERED INDEX with your (Azure)

SQL Database.

Heap tables are not supportes, so You MUST create

CLUSTERED INDEX before INSERT will be executed

error msg 40054:Tables without a clustered index are not supported in this version of

SQL Server. Create a clustered index and try again.

CREATE TABLE Source (Id int NOT NULLIDENTITY, [Name] nvarchar(max),CONSTRAINT [PK_Source] PRIMARY KEY CLUSTERED(

[Id] ASC))

21


PART ONE: SQL Database

Closed connections to the service… by the service

Because of multi-tenant :

Excessive resource usage

Connections that have been idle for 30 minutes or longer (Program

assuming your connection is going to fail)

Failover because of server failures

errors: 40197, 40501, 40544, 40549, 40550, 40551, 40552, 40553, 40613

22


Sample of SQL CompabilityIn Scope for v1

Constants

Constraints

Cursors

Index management and

rebuilding indexes

Local temporary tables

Reserved keywords

Stored procedures

Statistics management

Transactions

Triggers

Tables, joins, and table

variables

Transact-SQL language

elements such as

Create/drop databases

Create/alter/drop

tables

Create/alter/drop users

and logins

and so on.

User-defined functions

Views

Out of Scope for v1

Common Language Runtime (CLR)

Database file placement

Database mirroring

Distributed queries

Distributed transactions

Filegroup management

Global temporary tables

Spatial data and indexes

SQL Server configuration options

SQL Server Service Broker

System tables

Trace Flags

Physical server or catalog DDL and views

23


SQL Server 2005 {9.0} NON-Compability

Common Language Runtime (CLR) and CLR User-Defined Types

Database Mirroring

Service Broker

Table Partitioning

Typed XML and XML indexing is not supported. The XML data type is supported by SQL Azure.

24


SQL Server 2008 {10.0} NON-Compability

Change Data Capture

Data Auditing

Data Compression

Extended Events

External Key Management / Extensible Key Management

FILESTREAM Data

Integrated Full-Text Search

Large User-Defined Aggregates (UDAs)

Large User-Defined Types (UDTs)

Performance Data Collection (Data Collector)

Policy-Based Management

Resource Governor

SQL Server Replication

Transparent Data Encryption

25


SQL Server 2008R2 {10.50} NON-Compability

SQL Server Utility

SQL Server PowerShell Provider

Master Data Services

SQL Server Management Studiodoes not support Windows Azure SQL Database in versions

prior to

SQL Server 2008 R2

26


Connection Model

When writing applications for Windows Azure SQL Database, you can use the following drivers and libraries:

.NET Framework Data Provider for SQL Server (System.Data.SqlClient) from the .NET Framework 3.5

Service Pack 1 or later.

Entity Framework from the .NET Framework 3.5 Service Pack 1 or later.

SQL Server 2008 R2 Native Client ODBC driver. SQL Server 2008 Native Client ODBC driver is also

supported, but has less functionality.

SQL Server 2008 Driver for PHP version 1.1 or later.

An updated version of SQL Server JDBC Driver 3.0 that supports SQL Database.

Windows Azure SQL Database supports tabular data stream (TDS) protocol client version 7.3 or later.

Earlier versions of TDS protocol are not supported.

Connecting to Windows Azure SQL Database by using OLE DB or ADO is not supported

Support for ASP.NET controls

Clients connect directly to a database

27


Security Model

Uses regular SQL security model

Authenticate logins, map to users and roles

Authorize users and roles to SQL objects

Support for standard SQL Auth logins

Username + password

AD Federation, WLID – Windows Live ID, etc as alternate authentication protocols

[admin, administrator, guest, root, sa]

Security model is 100% compatible with on-premise SQL

28


Scenarios for V1• Departmental Applications

‒ Simple application built by individual or department

‒ Need simple deployment, self-management, IT: “Empowerment and Governance”

• Web Applications

‒ Small business or startup that uses the cloud as their IT

‒ Simple deployment, self-management, scale on demand

• ISV

‒ ISV hosting software on behalf of customer

‒ Multi-tenant support for billing and isolation

• Data Hub (Shortly After V1)

‒ Sharing and aggregating of data across tiers and across enterprises

‒ Centralized place for data, high scale, sync with existing data sources

29


PART ONE: SQL Database

Point of Difference On-premise SQL Server Windows Azure SQL Database

Where you manage server-level

security

The Security folder in SQL Server Management

Studio'sObject ExplorerThe master database

Server-level security role for

creating loginssecurityadmin fixed server role loginmanager database role in the master database

Commands for managing logins

CREATE LOGIN CREATE LOGIN

ALTER LOGIN ALTER LOGIN

DROP LOGIN DROP LOGIN

(There are some parameter limitations and you must be

connected to the master database)

View that shows all loginssys.syslogins (sys.sql_logins for SQL Server

authentication logins)

sys.sql_logins

(You must be connected to the master database)

Server-level role for creating

databasesdbcreator fixed database role dbmanager database role in the master database

Command for creating a

databaseCREATE DATABASE

CREATE DATABASE

(There are some parameter limitations and you must be

connected to the master database)

Dropping databases DROP DATABASE

DROP DATABASE

If a user is in the dbmanager role, they have permission

to DROP any database, regardless of which user originally created it.

View that lists all databases

sys.databases sys.databases

(view) (You must be connected to the master database)

30


PART ONE: SQL Database Federations

One or more tables within a database are split by row and portioned across multiple

databases (federation members) /sharding/

A federation is a collection of database partitions that are:

defined by a federation distribution scheme, known as the federation scheme.

federation scheme defines a federation distribution key,

(which determines the distribution of data to partitions within the federation)

federation distribution key must be an INT, BIGINT, UNIQUEIDENTIFIER, or VARBINARY

only one federation scheme and one federation distribution key for a federation.

31


PART ONE: SQL Database scale up/down

Scale up and out

10K records > 1DB >> INSERT, INSERT > 10M records

10K records > 10K max / DB >> INSERT, INSERT >> 100DB each 10K

Partitioning:

Horizontal / per row

Vertical / per column

32


PART ONE: Adventure with WASD

Adventure…

… of course Adventure Works 2012 for SQL Database

Adventure Works for SQL Server 2012

Release date: March 2012

Adventure Works for SQL Azure

Release date: April 2012

33


PART TWO:

Demo Mode

PART TWO: azure in the web

azure portal

management portal

management studio

third party tools

35


Don’t Ask Me…

37


PART THREE:

The Future of the DBA

PART THREE: Future of the DBA’sSQL Server on Premise

same job (almost like always)

same tasks, planning, architecture

same problems

same fun especially with new versions every… two years?


39

New horizons, new skills

SQL as a servis (SQL Azure Database)

not very new but still new concepto of work

some restriction (in minus)

some improvements (such a security)

PART THREE: Future of the DBA’s


40

Automation behind our business

someone (in MS DC) making half of our job

we can forget about few important features/tasks:

tempdb

mirroring

auditing

backup & restore

what is our role?

PART THREE: Future of the DBA’s


41

APPENDIX:

links & demos

Just Azure http://JustAzure.com43


http://justazure.com/

Most Important Article (in my private opinion)

http://bit.ly/WIHlDLAuthors: Conor Cunningham, Tobias Ternström, Silvano Coriani, Ewan Fairweather

Contributing Author: Ralph Squillace


44

MSDN SQL Database DistrictWindows Azure SQL Database (formerly SQL Azure) | http://bit.ly/H0Wdfx

What's New in Windows Azure SQL Database (formerly SQL Azure)

Backward Compatibility in Windows Azure SQL Database

Known Issues in Windows Azure SQL Database

Introducing Windows Azure SQL Database

Tutorials

Feedback and Community Information

Windows Azure Platform Management Portal

Windows Azure SQL Database Concepts

Administration

Development

Guidelines and Limitations

Management REST API Reference

Transact-SQL Reference

Errors and Exceptions Reference


45

http://bit.ly/H0Wdfx

Red Gate Cloud Ready Services

http://cloudservices.red-gate.com/

Windows Azure DataCenter Video

http://www.youtube.com/watch?v=JJ44hEr5DFE

General Guidelines and Limitations (Windows Azure SQL Database)

http://bit.ly/11wB6cK

Adventure Works for SQL Database (formerly SQL Azure)

http://bit.ly/125s0W0

Federations: Building Scalable, Elastic, and Multi-tenant Database Solutions

http://bit.ly/ZxzjmQ | http://bit.ly/15GzIEE

Error Messages (Windows Azure SQL Database)

http://bit.ly/17tnQtE

Compare SQL Server with Windows Azure SQL Database

http://bit.ly/15RyB5u

APPENDIX: additional links


46

http://cloudservices.red-gate.com/

http://www.youtube.com/watch?v=JJ44hEr5DFE

http://bit.ly/11wB6cK

http://bit.ly/125s0W0

http://bit.ly/ZxzjmQ

http://bit.ly/15GzIEE

http://bit.ly/17tnQtE

http://bit.ly/15RyB5u

Trust Center: Security, Privacy and Compliance Information for Office 365 and Microsoft Dynamics CRM Online

http://bit.ly/1q4i8bY

Announcing PCI DSS compliance and expanded ISO certification for Windows Azure

http://bit.ly/1tyt7vE

Windows Azure cloud services achieve critical federal security milestone

http://bit.ly/1osDHfi

INFOSYS – Microsoft Data Access Technologies SWOT

http://infy.com/1kgujKC

One year in, and the use cases for Microsoft's SQL Azure are becoming clear

http://bit.ly/TYWRRV



47

http://bit.ly/1q4i8bY

http://bit.ly/1tyt7vE

http://bit.ly/1osDHfi

http://infy.com/1kgujKC

http://bit.ly/TYWRRV

New Levels of Security via Machine Learning & Combined Data Sets

http://bit.ly/1s6seth

Predict What's Next: Getting Started on Azure Machine Learning - Part 1

http://bit.ly/1BLfWq1

Predict What's Next: How to Get Started with Machine Learning Part 2

http://bit.ly/1BLg099

Introducing Elastic Scale preview for Azure SQL Database

http://bit.ly/1xZk5sG

New D-Series Virtual Machine Sizes

http://bit.ly/10IapFg



48

http://bit.ly/1s6seth

http://bit.ly/1BLfWq1

http://bit.ly/1BLg099

http://bit.ly/1xZk5sG

http://bit.ly/10IapFg


Windows Azure IT Camp Resources–Slide Deck and Hands-on-Lab

http://blogs.technet.com/b/bobh/archive/2013/09/18/windows-azure-it-camp-resources-slide-deck-and-hands-on-lab.aspx#sthash.yFgxwPLv.dpuf

Microsoft Azure Free one month trial

http://azure.microsoft.com/en-us/pricing/free-trial/?WT.mc_id=AB7B32386

Microsoft Virtual Academy


Introducing Elastic Scale preview for Azure SQL Database

http://azure.microsoft.com/blog/2014/10/02/introducing-elastic-scale-preview-for-azure-sql-database/

Performance Best Practices for SQL Server in Azure Virtual Machines

http://msdn.microsoft.com/en-us/library/azure/dn133149.aspx?WT.mc_id=Social_SQL_General_DI&WT.mc_id=Social_FB_OutgoingPromotion_20140922_90669619_Microsoft%20SQL%20Server%20-%20sqlserver&linkId=9703972


49

http://blogs.technet.com/b/bobh/archive/2013/09/18/windows-azure-it-camp-resources-slide-deck-and-hands-on-lab.aspx#sthash.yFgxwPLv.dpuf



http://azure.microsoft.com/blog/2014/10/02/introducing-elastic-scale-preview-for-azure-sql-database/

http://msdn.microsoft.com/en-us/library/azure/dn133149.aspx?WT.mc_id=Social_SQL_General_DI&WT.mc_id=Social_FB_OutgoingPromotion_20140922_90669619_Microsoft SQL Server - sqlserver&linkId=9703972


SQL Database Migration Wizard v3.11 & v4.11 & v5.11

• Project Description

SQL Database Migration Wizard (SQLAzureMW) is designed to help you migrate your SQL Server

2005/2008/2012/2014 databases to Azure SQL Database.

Requirements

SQLAzureMW v3x and tools requires .NET Framework 3.5 and SQL Server 2008 R2 SP1

SQLAzureMW v4x and tools requires .NET Framework 4.5 and SQL Server 2012

SQLAzureMW v5x and tools requires .NET Framework 4.5 and SQL Server 2014

• AzureMWBatchBackup:

• This is used to create a backup of a database schema and data. The data is downloaded via BCP. The TSQL output

from SQLAzureMWBatchBackup can be used by SQLAzureMW or SQLAzureMWBatchUpload.

• SQLAzureMWBatchUpload:

• Takes the output from SQLAzureMW or SQLAzureMWBatchBackup and runs it against the target Azure SQL

Database (or SQL Server) server.


50

http://sqlazuremw.codeplex.com/


51

CONTACT:

MAIL: [email protected]

MSG: [email protected]

TWITTER/FACEBOOK/LINKEDIN: KoprowskiT

SLIDES FROM SESSION:

SQL Relay 2014 Page: https://www.sqlsaturday.com/311/schedule.aspx

SlideShare Profile: http://www.slideshare.net/Anorak

BLOGS:

ITPRO Anorak’s Vision: http://itblogs.pl/notbeautifulanymore/ [PL/EN]

Volume Licensing Specialites: http://koprowskit.eu/licensing/ [PL]

My MVP Blog: http://koprowskit.eu/geek/ [EN/ES/PL]

AFTER SESSION52


mailto:[email protected]

https://www.sqlsaturday.com/311/schedule.aspx

http://www.slideshare.net/Anorak

http://itblogs.pl/notbeautifulanymore/

http://koprowskit.eu/licensing/

http://koprowskit.eu/geek/

PLEASE REMEMBER:

Raffle at the End

in main ‚theatre’ auditorium

Feedback on site

53


Thank You Very Much

Q & A

54


Premier

Gold

Silver

Bronze

Our awesome sponsors

Technology

KoprowskiT_SQLRelay2014#6_Leeds_WADBForBeginners