Azure Big Data Story@LynnLangit

I love to…Learn & Build

Azure Big Data by the V’s

Value

Volume Velocity Variety Veracity

Big Data = Business Value ?

60% of Big Data projects

FAILto go beyond pilot and will be abandoned

(through 2017) - Gartner

Volume

How big is Big Data?

Variety

Is my Data rectangular?

Persistence Choices

Files

Hadoop

NoSQL

Relational

Azure Persistence Choices

• Storage• Store Simple

Files

• Data Lake• HDInsight• Cloudera

Hadoop• MANY…

NoSQL

• SQL Azure• SQL Azure DW• SQL Server on Azure

VM

Relational

Drilling In: Relational AND NoSQL

Azure Persistence Choices Detailed

• Storage• Store Simple

Files

• Data Lake• HDInsight• Cloudera

Hadoop• Redis Caching• DataStax Enterprise• Document DB• Mongo Labs• Graph Engine

NoSQL

• SQL Azure• SQL Azure DW• SQL Server on Azure VM

Relational

Velocity

How fast is my Data?

Veracity

How clean is my Data?

Load Choices

Load

Stream

Batch

Azure Load Choices

Load Libraries

StreamEvent Hub

BatchStream Analytics

Data Cleaning Choices

ETL

Client

Machine Learning

Azure Data Cleaning Choices

ETL

• SQL Server VM•Data Pipeline

Client

•Power BI•Power Query

ML

•Azure ML•Data Marketplace• SQL Server DQS

Data Pipelines

Azure Data Factory

Public Cloud

or

Hybrid Cloud

Data Model

On PremiseSQL Server+

CloudAzure+

Key-ValueQueues

NoneWindows Queues

Azure Redis CacheAzure Queues

Wide Sparse Columns

Columnstore IndexSSAS Tabular Models

Azure Tables DataStax Enterprise (Cassandra)

Files FileTable, FilestreamXML data type

Azure BLOB StoreStoreSimple

JSON or Graph

SQL Server 2016None

Azure DocumentDB / Graph Engine (beta)Hosted MongoDB or Neo4J

LargeRelational

SQL Server EnterprisePDWSQL Analysis Services

SQL Database (basic, standard, premium)APSSQL Data Warehouse

Hadoop Hortonworks HDInsight/ Data Lake,Hosted Cloudera

Other Stream Insight Event Hub, StreamAnalytics, MLMarketplace

Value

How useful is my Big Data?

Big Data = Business Value ?

60% of Big Data projects

FAILto go beyond pilot and will be abandoned

(through 2017) - Gartner

Architectural

Patterns

Architecture 1- File Storage / Backup

Architecture 2- Data Warehouse

Architecture 3 – Operational Database

Old

becomes

New

Architecture 4 – Small Big Data

MORE DATA

Architecture 5 – Big Data

www.TeachingKidsProgramming.org

Azure Big Data Story@LynnLangit