Embed Size (px)
Citation preview
Deliver an elastic data warehouse as a service
• Introducing Azure SQL Data Warehouse• Under the hood – architecture overview• Demo 1 – Create a data warehouse in seconds• Demo 2 – Elastic scale and pause• Demo 3 – Enable real-time telemetry data
storage• Use case scenarios• Demo 4 – Integration with PowerBI
Agenda
Demand for data warehousing is rising
Information managers are seeking innovative DBMS’s which are able to handle large data volumes in new ways or to optimize existing products and processes. Gartner (April 2015)
Azure SQL Data WarehouseA relational data warehouse-as-a-service, fully managed by Microsoft.
Industries first elastic cloud data warehouse with proven SQL Server capabilities.Support your smallest to your largest data storage needs.
Elastic scale & performance
Scales to petabytes of data
Massively Parallel Processing
Instant-on compute scales in seconds
Query Relational / Non-Relational
Saas
Azure
PublicCloud
Office 365Office 365
Get started in minutes
Integrated with Azure ML, PowerBI & ADF
Powered by the Cloud
Market Leading Price & Performance
Simple billing compute & storage
Pay for what you need, when you need it with dynamic pause
AzureAzure
Under the hood of Azure SQL Data Warehouse
Azure SQL Data Warehouse Architecture
ControlNode
ComputeNode
ComputeNode
ComputeNode
ComputeNode
SQL DB
SQL DB
SQL DB
SQL DB
Blob storage [WASB(S)]
ComputeScale compute up or down
when required(SLA <= 60 seconds).
Pause, Resume, Stop, Start.
StorageAdd\Load data to WASB(S) without incurring compute
costs
Massively Parallel Processing (MPP) Engine
Azure Infrastructure and Storage
100 DWU < > 2000 DWU
Storage and Compute are de-coupled, enabling a true elastic service and
separate charging for both compute and storage
Application or User connection
HDInsight
Data Loading(SSIS, REST, OLE, ADO, ODBC,
WebHDFS, AZCopy, PS) DMS
DMS DMS DMS DMS
DMS (Data Movement Service) executes across all
database nodes
Azure SQL Data Warehouse – Control Node
ControlNode
SQL DB
ComputeNode
ComputeNode
ComputeNode
ComputeNode
SQL DB
SQL DB
SQL DB
SQL DB
Blob storage [WASB(S)]
Massively Parallel Processing (MPP) Engine
HDInsight
ControlNode
SQL DB
• Endpoint for connections• Regular SQL endpoint (TCP 1433)• Persists no user data (metadata
only)• Coordinates compute activity
using MPP
Azure SQL Data Warehouse - Compute Nodes
ControlNode
SQL DB
ComputeNode
ComputeNode
ComputeNode
ComputeNode
SQL DB
SQL DB
SQL DB
SQL DB
Blob storage [WASB(S)]
Massively Parallel Processing (MPP) Engine
HDInsight
ComputeNode(s)
Azure SQL Database
SQL DB
An increase of DWU will increase the number of
compute nodes
Azure SQL Data Warehouse – Blob storage
ControlNode
SQL DB
ComputeNode
ComputeNode
ComputeNode
ComputeNode
SQL DB
SQL DB
SQL DB
SQL DB
Blob storage [WASB(S)]
Massively Parallel Processing (MPP) Engine
HDInsight
• RA-GRS storage• +PB’s of storage• Ingest data without
incurring compute costs
Demo 1Creating a new Azure SQL Data Warehouse
Enabling the data-driven landscape
Any data Any size Anywhere
Massively Parallel Processin
g
Built for modern
data
Demo 2Elastic scale and pause
Business ScenariosRecommendations,
customer churn,forecasting, etc.
Personal Digital Assistant
Cortana
Dashboards and Visualizations
Power BI
Machine Learning and Analytics
Azure Machine Learning
Azure HDInsight (Hadoop)
Azure Stream Analytics
Integrates with existing processes
DATA
Business apps
Custom apps
IoT - Sensors and devices
INTELLIGENCE ACTION
People
Automated Systems
Data Store
Azure SQL Data Warehouse
Information Management
Azure Data Factory
Azure Data Catalog
Azure Event Hub
Azure Stream Analytics
Azure Blob Storage
????
Demo 3IoT - Vehicle telemetry
Azure SQLData
WarehouseeCommerce
RetailLogistics
Unlocking scalable solutions
Fraud Detection• Historical or batch-based
analysis• Fast pattern trending and behaviour analysis• MPP analysis of high output telemetry (gaming,
finance, manufacturing)• Monitor Distribution and Supply Chains• Perform mission critical intrusion analysis
Historical Transaction
Data
Azure SQL Data
Warehouse
Trend Analysis
Transaction Detail
Distribution Management
Intrusion AnalysisReference
data
Sales Forecasting• Historical sales data
analysis (P&L)• Product trending over
massive data volumes• Customer segmentation
based on new marketing or sales initiatives• Gap analysis & competitor
impactHistorical Sales Data
Azure SQL Data
Warehouse
P&L Analysis
Product trending
Market & Gap Analysis
Competitor impact
CHURN
• Reduce customer turnover
• Increase profitability• Maintain loyalty• Increase perception• Fast scenario testing
Data Source
s
Azure SQL Data
Warehouse
Reduce customer turnover
What’s hot and what’s not
Loyalty & Perception
Scenario SimulationsData
Sources
Data Source
s
Predictive Maintenance
• Identification of near-life assets
• Aggregation of historical telemetry
• Enhanced customer service and insight
• Product improvement using Machine Learning
• Execute workload simulations using trusted data
SL CARS1, Boulevard Pershing, 75017 Paris
PredictedRemaining Life
New Asset Data
Known Device Data
Location
External Data …..
MICROSOFT CONFIDENTIAL – INTERNAL ONLY
Data/ Workload
Azure Machine Learning
Analysis Services (MOLAP)
IaaS
Azure SQL Data
Warehouse
PaaS
Other Azure
components
Hadoop Azure Blob Storage
Supports data ingestion from literally anywhere…
Azure Data Factory
Demo 4Integrated with PowerBI
Thank you
Matt UsherSnr. Program ManagerSQL Data Warehouse Engineering
Azure SQL Data Warehouse
Enabling the era of modern data warehousing
• Main topic 1: size 40pt• Size 20pt for the subtopics• Size 20pt for the subtopics
• Main topic 2: size 40pt• Size 20pt for the subtopics• Size 20pt for the subtopics
• Main topic 3: size 40pt• Size 20pt for the subtopics• Size 20pt for the subtopics
Preferred text layout (no bullets)
Built on Azure.
© 2014 Microsoft Corporation. All rights reserved.
