ICT infrastructure for AI Tools to accelerate Enterprise IOT

by Thavash Govender , Microsoft

We’ve observed three primary stages in the IoT customer journey

Ability to gain real time insights and improve business processes

Optimize usage of assets and/or deliver better products to customers

Develop new business opportunities, increase profitability, create more compelling business models

Connect

&

Monitor

Analyze

&

Improve

Transform &

Expand

Cu

sto

mer

So

ph

isti

cati

on

/ V

alu

e R

ealize

d

THE DIGITAL FEEDBACK LOOPS

A comprehensive

digital model that

includes products,

assets, and operations:

OPERATIONS

PEOPLE

PRODUCTSCUSTOMERS DATA

Our vision: to enable any organization to create digital feedback loops for all aspects of their business

The ability to track,

optimize, simulate and

predict the future

• People

• Places

• Things

• Systems

• Context

• Process

CONNECTING THE DOTS

Connected Building

Microsoft IoT for Smart Buildingshttps://www.youtube.com/watch?v=d55rBuB9D7s

Connected Office

Smart buildings & more productive peoplehttps://www.youtube.com/watch?v=uhfjeVNI9b0

Connected CarMicrosoft connected vehicle platformhttps://www.youtube.com/watch?v=rFo4l54vIw4

Connected Factory

Manufacturinghttps://azure.microsoft.com/en-us/features/iot-accelerators/connected-factory/

Connected Grid Smarter grid with Agder Energi in Norway https://www.youtube.com/watch?v=cxKphN8iVHE

Connected AgricultureFarmBeats tracks soil, moisture data 24/7https://www.youtube.com/watch?v=pDgjOHY7sMI

Enabling precision agriculture with AIhttps://www.youtube.com/watch?v=KTvdjcU0lf8

IOT & INTELLIGENT INFRASTRUCTURE

Citizen Engagement

Transportation, Traffic, Parking, Vehicles

Energy, Water & Utilities

Public Safety & Security

Housing,Buildings

Environment

Social

Economy

Municipal Command & Control Center

Responsive Store

Intelligent Supply Chain Devices

Connected Fleet Digital

Signage Automated Car System

TrafficSensors & Cameras

Logistics Optimization

Factory Optimization

Traffic Flow Optimization

Home Energy

Management

Building Optimization

Smart Grid

Comms NetworkOptimization

MODEL & INTERACT WITH THE REAL WORLD

THINGS INSIGHTS ACTIONS

RTOS, Linux, Windows, Android, iOS

Power BI

Mixed Reality

Logic Apps

Web Apps

Mobile Apps

SearchAzure Maps

Manage

View and manage solutions

Azure Sphere

Secured

MCU

Secured

OS

Cloud

Security

• Spatial intelligence graph

o People, Space, Device Twins

o Loadable Twin Object Models

o User Defined Functions

o Blob storage

o Nested tenancy

• Role-based access & control

• AuthN/AuthZ via AAD

• Notifications & egress routing

Cold Analytics

Find insights over historical data

AzureML

Azure Data Lake Analytics

Hot Analytics

Real-time monitoring

Azure Stream Analytics

HDInsight Spark & Storm

Warm Analytics

Time series data storage & analytics

Azure Time Series Insights

Business Integration

Connect to Business Processes

Office 365Dynamics 365

Flow

AI - Cognitive Services

Azure IoT Hub

• Bi-directional device-cloud

• Telemetry ingestion

• Command & Control

• Device Registry & Identity

• Device Provisioning

• Device Mgmt

• HTTP, AMQP, MQTT

Azure Digital Twins

People

Spaces

Devices

Edge Modules:• Protocol Adaptation

• Functions

• Stream Analytics

• Machine Learning

• AI

Azure IoT Edge

FPGAs

EFFICIENCY

Hardware Acceleration in the future

FLEXIBILITY

CPUs GPUsASICs

INFERENCINGCPUs, GPUS, FPGAs, GRAPHCORE?

TRAININGCPUs, GPUs + GRAPHCORE

Cloud Edge

INFERENCINGCPUs, GPUS, FPGAs, ASICs, Graphcore ?

TRAINING

(HEAVY EDGE)CPUs,GPUs, Graphcore ?

Understanding the Edge: Heavy Edge vs Light Edge

Cloud: Azure Heavy Edge Light Edge

DescriptionAn Azure host that

spans from CPU to GPU and FPGA VMs

A server with slots to insert CPUs, GPUs, and FPGAs or a X64 or ARM system that needs to be plugged in to work

A Sensor with a SOC (ARM CPU, NNA, MCU) and memory that can operate on batteries

ExampleDSVM / ACI / AKS /

Batch AI

- DataBox Edge

- HPE

- Azure Stack

- Intel Tank

- Ge Healthcare (Windows)

- DJI Drone PC

- Advantech- Dell Gateway- Intel UP2

-DLink Video Gateway-DVR

-Mobile Phones

-Peabody

-Dlink Camera

-Cisco Meraki

-Mobile Phones

-IP Cameras

-1st party devices* may move to NNA

-Azure Sphere - Appliances

What runs model

CPU,GPU or Arria 10 FPGA

Arria 10 FPGA

Intel HDDNN

Nvidia GPU

Other GPUx64 CPU Multi-ARM CPU

Hw accelerated NNA

CPU/GPU MCU

Model OS package

-Linux Container

-Native to Windows (x)

-Windows Native**will move to Linux Container

- Linux container- Windows Container + WinML

- Linux container

- Windows Container +WinML

- Linux container

- Android Native

- Linux container

- Android Native

- IOS Native

- Linux container

- Android Native

- Windows

- IOS Native

- Azure Sphere

- RTOS

Development Experience for AI on the Edge :VAIDK Vscode Experience.pptx.

Az ML + IoT Edge Lifecycle

The AML Development Life Cycle

© Microsoft Corporation

Microcontrollers (MCUs)

1970’s 1980’s 1990’s 2000’s 2010’s 2020’s 2030’s

Wave 2:Wave 1:The Microcontroller (MCU) Internet Connectivity

9 BILLION new MCU devices deployed every year

IN TOYS... IN APPLIANCES… IN EQUIPMENT…

FEWER THAN 1% ARE CONNECTED TODAY.

© Microsoft Corporation

Connected devices create profoundly

better customer experiences.

How does a consumer know the compressor

in their fridge needs to be replaced?

Option 1

Melted ice cream

Option 2

Predictive maintenance

It will change your relationships with your customers

© Microsoft Corporation

Highly-secured connected devices require 7 properties

Hardware Root of Trust

Is your device’s identity and software integrity secured by hardware?

Defense in Depth

Does your device remain protected if a security mechanism is defeated?

Small Trusted Computing Base

Is your device’s TCB protected from bugs in other code?

Dynamic Compartments

Can your device’s security protections improve after deployment?

Certificate-Based Authentication

Does your device use certificates instead of passwords for authentication?

FailureReporting

Does your device report back about failures and anomalies?

Does your device’s software update automatically?

Renewable Security

= Silicon support required = OS support required = Cloud Service support required

© Microsoft Corporation

Azure Sphere OS Architecture

App Containers for POSIX (on Cortex-A)

App Containers for

I/O (on Cortex-Ms)

On-chip Cloud Services

HLOS Kernel

Security Monitor

Azure Sphere MCUs

OS Layer 4

OS Layer 3

OS Layer 2

OS Layer 1

Hardware

The Azure Sphere OS is optimized for IoT, Security and MCU agility

On-chip Cloud Services

Provide update, authentication, and connectivity

Custom Linux kernel

Empowers agile silicon evolution and reuse of code

Security Monitor

Guards integrity and access to critical resources

Secure Application Containers

Compartmentalize code for agility, robustness & security

© Microsoft Corporation

Simplify development

Focus your device development effort

on the value you want to create

Streamline debugging

Experience interactive, context-aware

debugging across device and cloud

Simplify Azure connect

Connect your Azure Sphere devices

quickly and easily to Azure IoT

Modernize MCU development with Azure Sphere and Visual Studio

ENABLING PARTNERS