#MicrosoftAzure

Building your scalable & secure Internet-of-Things on Microsoft AzureTom Kerkhove

• Tom Kerkhove• Azure Consultant at Codit• AZUG Crew Member• Microsoft Azure MVP & Advisor

• You can find me• On Twitter - @TomKerkhove• On GitHub – github.com/tomkerkhove• And my adventures on tomkerkhove.ghost.io

Nice to meet you!

25 BILLIONConnected “things”will be in use by 2020

Gartner

$7.2 TRILLIONworldwide market for IoT solutionsby 2020

IDC: Worldwide and Regional Internet of Things (IoT) 2014–2020 Forecast

Platform Services

Infrastructure Services

OS/Server Compute Storage

Datacenter Infrastructure (36 Regions, 28 Online)

Web and Mobile

Web Apps

MobileApps

APIManagement

API Apps

Logic Apps

Notification Hubs

Media & CDN

Content DeliveryNetwork (CDN)

Media Services

Integration

BizTalkServices

HybridConnections

Service Bus

StorageQueues

HybridOperations

Backup

StorSimple

Azure SiteRecovery

Import/Export

Networking

Data

SQL Database

DocumentDB

RedisCache Azure

SearchStorageTables

DataWarehouse Azure AD

Health Monitoring

Virtual Network

ExpressRoute

BLOB Storage AzureFiles

PremiumStorage

Virtual Machines

AD PrivilegedIdentity Management

Traffic Manager

AppGateway

OperationalAnalytics

Services ComputeCloud Services

Batch RemoteApp

ServiceFabric

Developer Services

Visual Studio

AppInsights

Azure SDK

VS Online

ContainerService

DNS VPN GatewayLoad Balancer

Domain Services

Analytics & IoT

HDInsight MachineLearning

StreamAnalytics

Data Factory

EventHubs

MobileEngagement

Data Lake

IoT Hub

Data Catalog

Security & Management

Azure ActiveDirectory

Multi-FactorAuthentication

Automation

Portal

Key Vault

Store/Marketplace

VM Image Gallery& VM Depot

Azure ADB2C

Scheduler

Agenda• Device Connectivity• Device Management• Data Processing & Storage• Data Security• Data Management

Device Connectivity

HTTP POST /api/event?sensorId=123

Cloud Gateway

HTTPS POST /api/event?sensorId=123

HTTPS POST /api/event?sensorId=123

Securely store your credentials & keysUse a Trusted Platform Model (TPM) if possible

• High scale telemetry ingestion service• Ingres of millions of events per second• HTTP/AMQP protocol support• Each Event Hub supports• 1 million publishers• 1GB/s ingress

• Uses a data log-approach

Introducing Azure Event Hubs

Send over HTTPS/AMQP

Never expose an enpoint on the device for callbacksClose all ports that are not requiredFollow the Service Assisted Communication pattern

Send over HTTPS/AMQP

• How will the devices connect?• Will they be online all the time or once every day?• Are they capable of setting up a TLS connection?• What protocols are they speaking?• Are the connections stable?

Think about your devices

• Bi-directional device <-> cloud• Up to 10 million devices• Device Management• Device registry & identity• Support for bulk import/export of device identities

• Protocol Support• HTTP/AMQP/AMQP-WS/MQTT out-of-the-box• Extensibility support

• Operations Monitoring

Introducing Azure IoT Hub

• Endpoints for devices• Send endpoint for D2C• Receive endpoint for C2D• Endpoints for cloud-components• Process incoming messages a la Event Hubs• Send notifications to devices• Device Management

Internals of IoT HubIoT Hub

Device id

C2D queueendpoint

D2C send endpoint

Device …

Device …

Device …

D2C receive endpoint

C2D send endpoint

IoT Hub management

Send time-limited commands to devicesAssign a time-to-live (TLL) to your commands so that offline devices don’t perform actions while it’s no longer needed

Send over HTTPS/AMQP

Physically lock down your deviceThis avoids tampering with your devices or installing other softwareEncrypt device storageThis avoids potential information disclosure when hard drive is stolen

Send over HTTPS/AMQP

Device Management

Provision device XYZ

Activation Key ABC

Provision API

Register new inactive device

Store custo

m info & act. key

Securely store your credentials & keysBurn it into the silicone or use a TPM if possible

Device ID Connection Status Device State Auth Key

<Generated ID> Disconnected Inactive <Auth Key>

Activate device XYZ

Activation Key ABC

Provision API

Activates device

Verifies activation key

Securely store your credentials & keysBurn it into the silicone or use a TPM if possible

Returns auth key

Gets device-specific auth key

Device ID Connection Status Device State Auth Key

<Generated ID> Disconnected Active <Auth Key>

Command: Update

Blob Storage SAS URIApp v2

Blob Storage SAS URI

App v1

Data Processing & Storage

• What is the structure of the data?• Structured, unstructured, etc.• What do we need it for?• How will we process it?• What kind of data is it?• How sensitive is it?

Think about your data first

Determining Data CategoriesHot Data

Data coming in from devices near-real-time

Warm DataData that has been processed but gives an idea about the last couple of hours

Cold DataData from age for deep learning

Archived DataData that is stored but (almost) never used

Determining Processing MechanismsNear-Real-Time Processing

Azure Stream AnalyticsStorm on Azure Data Lake HDInsight

Batch ProcessingAzure Data Lake HDInsights / Analytics

Custom ProcessingService Fabric

Machine LearningAzure Machine Learning

Data “Storage” Overview

IoT Hub Table Storage Document DbEvent Hubs

Data Lake StoreBlob Storage

(Elastic) SQL Database

SQL Data Warehouse

Data Security

• Protect your data at rest, it’s free of charge!• Azure SQL Database Transparent Data Encryption (TDE)• Azure Blob Storage Encryption

• Protect your privates• Securely store encryption keys, authentication keys, passwords, etc.• Need to store sensitive data? Use Azure Key Vault!

• Only secure what needs to be secured• No need for publically available reference data

Protecting Your Data

• Understand how services store your data• Does it replicate the data by default? If so, how do I restore it?• Can I enable this with a feature?• How do I restore it?

• Replicate mission critical data to a different region

Securing Your Data

• Enabling replication in Azure• SQL Database: Setup Geo-Replication, built-in restore• SQL Data Warehouse: Geo Policies for daily-snapshots, built-in

restore• Storage account: Choose appropriate replication (GRS/RA-GRS)• Data Lake Store: Done by service

• Only want to replicate a subset?• Custom solution is required.

Securing Your Data

Data Management

• Keep track of all your data sources with Azure Data Catalog• Register a data sources in a centralized place• Annotate data sources describing what the data is, who’s in charge,

who to contact, etc.

Keeping Track Of Your Data

Moving & Transforming DataMove & transform you data with Azure Data Factory

Built for moving data stored in on-prem/cloudCreate & monitor data pipelines using out-of-the-box sources & sinks

• No high-level overview of your components in Azure• Document what components are interacting with each other• Document how your data flows• Document your data sets• Do this from the start

Architecture & Data Flow Documentation

40

• Explore the Pre-Configured Solutions• Set of scenarios implemented that you can provision• Azure IoT Suite for IoT focused scenarios (https://azureiotsuite.com)• Cortana Intelligence Gallery for data focused scenarios (https://gallery.cortanaintelligence.com)

• Perfect to get your feet wet & play around with

Learn From Microsoft

Learn From Microsoft

Source: Cortana Intelligence Gallery - IT Anomaly Insights

Learn from MicrosoftMyDriving Reference Implementation

Learn from MicrosoftMicrosoft Azure IoT Reference Architecture

• Service Assisted Communication by Clemens Vasters• Best Practices for Creating IoT Solutions

with Azure (//BUILD/ 2015)• Official Azure documentation about Security• Security from the ground up• Security Architecture• Security Best Practices

Learn from Microsoft

• Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications by Microsoft Patterns & Practices• Mastering Azure Analytics by Zoiner

Tejada• I <3 Logs by Jah Kreps

Recommended Books

47

• Think Big, Start Small• Start with ingestion, handle command & control later on• Design for high-scale while you might not need it yet• Think about Device Management from the

start• Know the limits and design around them• Build security from the ground up

Summary

49

50