Deep Dive on AWS IoT

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Olawale Oladehin | Solutions Architect

July 13, 2016

Deep Dive on AWS IoT

What to expect from the session

1. Overview of AWS IoT

2. Use IoT rules engine to process data and move

data to AWS services

3. Use IoT shadow, lifecycle events, and Amazon

ML for fleet management

AWS IoT

Select an AWS IoT Device SDK

• Based on open standards like Eclipse Paho

• C

• iOS (Swift)

• Android

• Node.js

• Python

• Java

Securely connect device

• AWS IoT-generated certificates

• Use your own certificate

First steps with AWS IoT

=+

And then…

IoT rules engine

Rules engine

Rules engine: Extracting value from messages

• Filter messages with certain criteria

• Transform the payload of messages

• React based on messages

• Move messages to other topics

• Move messages to other systems

• Predict changes based on trends

What is a rule?

ruleArn: arn:aws:iot:<region>:<account_id>:rule/<uniq_rule_name>

rule:

• ruleName: human-readable name

• description: human-readable description

• ruleDisabled: true/false

• sql: e.g. SELECT * FROM ‘pm/topic’

• actions:

• action 1:

• action 2:

What is a rule?

• ruleName: human-readable name

• description: human-readable description

• ruleDisabled: true/false

• sql: e.g. SELECT * FROM ‘pm/topic’

• actions:

• action 1:

• action 2:

What is a rule?

• ruleName: human-readable name

• description: human-readable description

• ruleDisabled: true/false

• sql: e.g. SELECT * FROM ‘pm/topic’

• actions:

• action 1:

• action 2:

Rules engine: Filter and transform

• Filter messages with certain criteria

• Transform the payload of messages

• React based on messages

• Move messages to other topics

• Move messages to other systems

• Predict changes based on trends

Rules engine: Filter and transform

• SQL

• SELECT * FROM topic WHERE condition

• SELECT status.space_id AS room_id FROM

'iot/tempSensors/#' WHERE temp > 50

• Functions (in SELECT or WHERE)

• String manipulation (regex support)

• Mathematical operations

• Context-based helper functions

• Crypto support

• UUID, timestamp, rand, etc.

Rules engine: React and move

• Filter messages with certain criteria

• Transform the payload of messages

• React based on messages

• Move messages to other topics

• Move messages to other systems

• Predict changes based on trends

Rules engine: React (and move)

"rule": {

"ruleName": "republishPredictions”,

"ruleDisabled": false,

"sql": "SELECT no AS id, cycle AS c_number FROM 'pm/topic' WHERE failure = 1",

"description": "Republish ids of engines that are predicted to fail within 30 cycles",

"actions": [

{

"republish": {

"topic": "pm/failures",

"roleArn": "arn:aws:iam::012345678901:role/iot-actions-role”

}

}

],

}

Rules engine: React (and move)

"rule": {

"ruleName": "republishPredictions”,

"ruleDisabled": false,

"sql": "SELECT no AS id, cycle AS c_number FROM 'pm/topic' WHERE failure = 1",

"description": "Republish ids of engines that are predicted to fail within 30 cycles",

"actions": [

{

"republish": {

"topic": "pm/failures",

"roleArn": "arn:aws:iam::012345678901:role/iot-actions-role”

}

}

],

}

Rules engine: Move messages to other systems

• Filter messages with certain criteria

• Transform the payload of messages

• React based on messages

• Move messages to other topics

• Move messages to other systems

• Predict changes based on trends

Rules engine: Move messages to other

systems

Invoke a Lambda function

Put object in an S3 bucket

Insert, update a

DynamoDB table

Publish to an SNS topic

or endpoint

Publish to a Amazon Kinesis

stream

Publish to Firehose

Republish to AWS IoT

Publish to Amazon ES

Capture a CloudWatch

metric or change an alarm

Write to an SQS queue

Rules engine features

• Versioning

• 2015-10-08 – Original version

• 2016-03-23-beta – Beta version released on specific date

• beta – Latest beta version (breaking changes!)

{

"sql": "expression",

"ruleDisabled": false,

"awsIotSqlVersion": "2015-03-23-beta",

"actions": [{

"republish": {

"topic": "my-mqtt-topic",

"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"

}

}]}

Rules engine features

• JSON collections

• get(array, int) – get item at index of array

• get(string, int) – get character at position of string

• get(object, key) – get value of key

• SUB SELECT from collections

• SELECT (SELECT v FROM e WHERE n = 'temperature') as

temperature FROM 'topic'

Rules engine: Integrating data storage

Put object in an S3 bucket

Insert, update a

DynamoDB tablePublish to Firehose

Publish to Amazon ES

Publish to a Amazon Kinesis

stream

ElasticSearch Integration

Basic flow for using Amazon Elasticsearch

Service

1. Configure your Amazon ES domain

2. Use an IoT rule to send device data to the

Amazon ES domain you configured

3. Use Kibana to visualize your device data

Fleet management and IoT

operations

Fleet management

• How many devices are currently connected?

• How many successful firmware updates have occurred

over the last 24 hours?

• What was the state of a subset of devices 24 hours ago?

• Can I predict when a device might need maintenance?

AWS IoT Device Shadow

Shadow topics – request

• /update

• /get

• /delete

Shadow topics – response

• /update/accepted

• /get/accepted

• /delete/accepted

• /update/rejected

• /get/rejected

• /delete/rejected

Create operational metrics across your fleet of devices

Additional shadow topics

• /update/documents

• Previous shadow state

• Current shadow state

• History of devices

• /update/delta

• Difference between reported and desired

• Audit trail of desired attribute changes

Operational rules: Shadow metrics

"rule": {

"ruleName": ”deviceHealthMetrics”,

"ruleDisabled": false,

"sql": "SELECT state.reported.health AS health FROM ’$aws/things/+/shadow/update/accepted’

WHERE health = 1",

"description": ”Track the number of successful health checks from shadow topic",

"actions": [

{

”cloudwatchMetric": {

"roleArn": "arn:aws:iam::012345678901:role/iot-actions-role”,

”metricNamespace": ”IoTNamespace”,

”metricName": ”deviceHealth”,

”metricValue": ”1”,

”metricUnit": ”Count”

}

}

],

}

Operational rules: Device history

"rule": {

"ruleName": ”shadowDocumentHistory”,

"ruleDisabled": false,

"sql": " SELECT current.state AS state, cast(timestamp as number) * 1000 AS timestampMillis

FROM '$aws/things/+/shadow/update/documents' ",

"actions": [

{

”elasticsearch": {

”endpoint": ”https://search-fleet-xxxx.us-east-1.es.amazonaws.com",

"roleArn": "arn:aws:iam::012345678901:role/iot-actions-role”,

“index”:”devices”,

“type”:”sensor”,

“id”:”${newuuid()}”,

}

}

],

}

Lifecycle events

1. Publishes a message when a client connects or

disconnects

2. $aws/events/presence/connected/<clientId>

3. $aws/events/presence/disconnected/<clientid>

{

”clientId": ”a1sb2568s0d08s08d0sdkasldkfei",

”timestamp": 1460065233478,

”eventType": “disconnected”,

”clientInitiatedDisconnect": true,

”sessionIdentifier": “000000-000-0000-0000-0000000000”,

”principalIdentifier": “000000/ABCDEFGHIJKLM:user/user-id”,

}

Amazon ML predict function

Right balance depends on risk/cost ratio:

1. Predict a failure too soon = replacing a part

that doesn’t need repair yet – loss

2. No prediction = risk of sustaining a bigger loss

Amazon ML Predict Function

Basic flow for using predictions

1. Use an IoT rule to forward device data to S3

2. Train your Amazon ML model using the data from

S3

3. Use an IoT rule to:

• Obtain predicted value from real-time prediction

endpoint in Amazon ML

• Emit a CloudWatch metric (or trigger an alarm)

4. Use an IoT rule to emit details of a predicted failure

AWS IoT predict function for Amazon ML

machinelearning_predict(

'ml-XXXXXX',

'arn:aws:iam::<account_id>:role/<role>',

*).predictedLabel

ml-XXXXXX – Amazon ML predictor you trained

account_id – your account ID

role – a role in your account that:

• Has access to Amazon ML

• Part of a trust relationship b/w your account and IoT

Summary

• Extract value from device data: rules engine

• Visualize your data: rules engine with AWS

Services

• Use IoT shadow, lifecycle events, and

Amazon ML for fleet management

Thank You!@oladehin