CloudBrew 2016 - Building IoT solution with Service Fabric

Onsight

Building IoT solution with Service Fabric

Onsight

Teemu Tapanila Microsoft Azure MVP Architect, OnSight Helsinki oy Microsoft Certified Trainer

Teemu@onsight.fi@TapanilaT

Onsight

Topics Why Azure Service Fabric Device Registration Data Ingestion Data processing Data analysis

Onsight

Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications

Microservices

Application Programming Models

Azure

WindowsServer Linux

Hosted Clouds

WindowsServer Linux

Service Fabric

Private Clouds

WindowsServer Linux

High Availability

Hyper-Scale

Hybrid Operations

High Density

Rolling Upgrades Stateful

services

Low Latency Fast startup & shutdown

Container Orchestration & lifecycle management Replication &

FailoverSimple

programming models

Load balancing

Self-healingData Partitioning

Automated Rollback

Health Monitoring

Placement Constraints

Onsight

Battle-hardened for over 5 years

Azure Core Infrastructure

thousands of machines

Power BI

Intune

800k devices

Azure SQL Database

1.4 million databases

Bing Cortana

500m evals/sec

Azure Document

DB

billions transactions/wee

k

Skype for Business

Hybrid Ops

Event Hubs

20bn events/day

Onsight

What is a microservice? Is (logic + state) that is independently versioned,

deployed, and scaled Has a unique name that can be resolved

e.g. fabric:/myapplication/myservice Interacts with other microservices over well defined

interfaces and protocols like REST Remains always logically consistent in the presence of

failures Hosted inside a “container” (code + config) Can be written in any language and framework Developed by a small engineering team

Onsight

Types of microservices Stateless microservice

Has either no state or it can be retrieved from an external store

There can be N instances e.g. web frontends, protocol gateways etc.

Stateful microservice Maintain hard, authoritative state N consistent copies achieved through replication and

local persistence e.g. database, documents, workflow, user profile,

shopping cart etc.

Stateless microservice

Demo

A set of independent machines; physical or virtual

Typical datacenter

A set of machines that Service Fabric stitches together to form a cluster

Clusters can scale to 1000s of machines

Cluster: A federation of machines

Node

Node

Node

Node

Node

Node

Machine failure detection

Time = t1

83 76 50 4664 New Node arrived61

Time = t2

8361 50 46

Failures Detected

cluster reconfigured

83 76 64 50 46

Time = t0

Nodes failed

Cluster: System viewSystem Services Failover manager

Cluster manager

Naming

Image store

Service Fabric Explorer

Demo

Queues Storage

3-Tier service pattern

Front End(StatelessWeb)

StatelessMiddle-tierCompute

Cache

Scale with partitioned storage

Increase reliability with queues

Reduce read latency with caches

Manage your own transactions for state consistency

Many moving parts each managed differently

Load Balancer

StatefulMiddle-tierCompute

Stateful services: Simplify design, reduce latency

Front End(StatelessWeb)

data stores used for analytics and disaster recovery

Application state lives in the compute tier

Low Latency reads and writes

Partitions are first class for scale-out

Built in transactions

Fewer moving parts

Load Balancer

Onsight

Device Registration Device needs to be registered when it gets

connected to the system. This includes normally authentication keys and

serial number When device gets registered you should assing

it value which you will later use for sharding

Onsight

Data Ingestion Gathered data from 1 data source is not so

important normally Real-time data ingestion vs batch ingestion

Onsight

Data Ingestion

Onsight

Data processing Visibility to data averages on grouping level

(Machine, Building, Floor or etc) Logic based on value changes

StatefulMiddle-tierCompute

Data ingestion & Processing

Front End(StatelessWeb)

data stores used for analytics and disaster recovery

Load Balancer

Onsight

Data processing

Room Stateful Service

Floor Stateful Service

Building Stateful Service

Campus Stateful Service

Onsight

Data Analysis Data is stored outside of service fabric for

processing Most cases makes sense of using other tools to

build the analysis platform than Service Fabric

Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications

Microservices

Application Programming Models

Azure

WindowsServer Linux

Hosted Clouds

WindowsServer Linux

Service Fabric

Private Clouds

WindowsServer Linux

High Availability

Hyper-Scale

Hybrid Operations

High Density

Rolling Upgrades Stateful

services

Low Latency Fast startup & shutdown

Container Orchestration & lifecycle management Replication &

FailoverSimple

programming models

Load balancing

Self-healingData Partitioning

Automated Rollback

Health Monitoring

Placement Constraints

Onsight

Thank you!

Teemu@onsight.fi@TapanilaT