Azure Data Warehouse OverviewJustin MunstersSystem Analyst - Holland

Microsoft Azure

Internet of Things

Audio / VideoLog Files

Text/Image

Social Sentiment

Data Market FeedseGov Feeds

Weather

Wikis / Blogs

Click Stream

Sensors / RFID / Devices

Spatial & GPS Coordinates

Modern WebMobileAdvertising

Collaboration

eCommerce

Digital Marketing Search MarketingWeb Logs

Recommendations

ERP / CRM

Sales Pipeline

PayablesPayroll

Inventory

ContactsDeal Tracking

Terabytes(10e12)

Gigabytes(1e9)

Petabytes(10e15)

Velocity | Variety | Variability

Volu

me

ERP / CRM / ECM

Modern Web

Internet of Things

What is Big Data?

Data Warehouse DiscussionSizeHow big is this thing?What about HA and redundancy?

PerformanceHow many cores, how much RAM?How do we handle peaks and bursts?

ScaleWhere is this thing going?Clusters, Rackspace, Build Up/Out

Why the cloud?Rapidly setup environments to drive business prioritiesScale to meet peak demands Increase daily activities, efficiency and reduced cost.

EconomicsScale Speed

Data Use & Computing Patterns

t

Com

pute

InactivityPeriod

On and Off – Standard Work Day/WeekOn & off workloads (e.g. batch job)Over provisioned capacity is wasted Time to market can be cumbersome

t

Unpredictable Bursting – Web demandUnexpected/unplanned peak in demand Sudden spike impacts performance Can’t over provision for extreme cases Co

mpu

te

t

Predictable Burst – Period EvaluationsServices with micro seasonality trends Peaks due to periodic increased demandIT complexity and wasted capacity

Com

pute

t

Growing Fast – New Product LaunchSuccessful services needs to grow/scale Keeping up w/ growth is big IT challenge Cannot provision hardware fast enoughCo

mpu

te

Azure Portal UX

Redundancy & Reliability Enterprise-ready SQL on a mature cloud platform for sophisticated DW scenarios

Redundancy

• Microsoft Azure Storage with replicas

• SQL Database built-in backup servers

• Failover with Microsoft Azure Traffic Manager

Reliability

• Auto recovery of crashed/nonresponsive instances

• Fault domain to scatter instances across racks

• Swap to allow promoting staging to production

The Cost To Do This On-Premise?

Azure

On-Premise Data Warehouse? • Hardware

• Software• Data Center• Off-Site

Backup• Staff

$$$$

$$$$

$$$$

Data Warehouse Unit (DWU)Simply buy the query performance you need, not just hardware

Quantified by workload objectives: how fast rows are scanned, loaded, copied

Measure of Power

Transparency

First DW service to offer compute power on demand, independent of storage

On Demand

Scan 1B rows

100 DWU = 297 sec400 DWU = 74 sec800 DWU = 37 sec1,600 DWU = 19 sec

Scan Rate xx M row/secLoading Rate xx K row/secTable Copy Rate

xx K row/sec

100 DWU

Flexible Configuration & Costing

Resize in minutes From Any Size to Any Size On-Demand

SQL Server CompatibilityFamiliar SQL Server scripts and tools you have been using just work

User-Defined Functions &

Stored Procedures

Modular programming(write once, execute multiple times)

Faster code execution Encapsulated programming logic

Table Partitioning

Easier maintenance of large tables Improves performance Enhanced scalability and availability

Collations

Allows proper use and comparisons of characters in different languages

Indices & Column-Store

Support

Mature Column-Store technology for best-in-class DW query performance

Complete, consistent, scalable data platformAzure Data Warehouse

1CONSISTENTPLATFORM

SQL Server 2014/16Mission-critical performanceFaster insights from any dataPlatform for hybrid cloud