Io t world_2016_iot_smart_gateways_moe

© 2016 IBM Corporation

IBM Analytics

IBM IoT Architecture and Capabilities at the Edge and Cloud

Shawn Moe – Architect, IBM AnalyticsMay 12, 2016

© 2016 IBM Corporation2

Agenda

IoT Architectural Components

Informix for the Intelligent Gateway

Informix for the Cloud

Performance

IoT Developer Resources


Simplified IoT Data Flow

Sensor Data History

Sensors

In-memory Analytics

Predictive Analytics

Publish / Subscribe

Cloud Infrastructure

Real-time Analytics

Real-time Analytics Operational Analytics Big Data Analytics

(no gateway)

(Gateways)

HDFS / Hadoop

Big Data Analytics

MessageSight / MQTT

Gateways for local analytics

InfoSphereStreams

Informix / Cloudant / Watson IOT Platform Service / TimeSeries Service

Informix Warehouse Accelerator / DashDB

PMQ / SPSS / Cognos

Softlayer / Bluemix

Watson / DashDB / BigInsights

Informix / Node-Red


• Reduces Cost

• Reduces cloud storage by filtering/aggregating/analyzing data locally

• Reduces cloud CPU requirements by precomputing values

• Reduces Latency

• Intelligent gateways can detect and respond to local events as they happen rather than waiting for

transfer to the cloud

• Performs 80% simple operations locally

• Security

• Gateways allow customers to capture and get value from their sensors without sending data to the

cloud

• Protocol Consolidation

• Cloud does not need to deal with the hundreds of IoT protocols

Over time more and more of the IoT processing will move from the cloud to gateway devices

How Do Smart Gateways Help IoT Solutions?


What are the Requirements for a Gateway Database?

The database management system must:

Have a small install footprint, less than 100 MB

Run with low memory requirements – less than 256 MB

Use lossless compression to minimize storage space

Have built-in support for common types of IoT data like time series and spatial/GIS data

Simple application development supporting NoSQL, SQL, and REST

Require absolutely no administration

Be able to network multiple gateways together to create a single distributed logical database

The database must be powerful enough to ingest, process and analyze data in real-time


What is a Time Series?

• A logically connected set of records ordered by time

What are the key strengths of Informix TimeSeries?

• Native data type resulting in significantly less space requirements

• Typically about 1/3 the space required by other vendors

• Queries run orders of magnitude faster

• Unique optimized storage means more data fits in memory

• Purpose built streaming data loader for sensor data

• Automatically run analytic and/or aggregate functions on new data

• Integrates structured (SQL) or unstructured (JSON) data to store metadata for each time value

• REST/ODBC/JDBC/JSON interfaces available to work with this

• API contains hundreds of manipulation & analytics functions + APIs to create your own

analytics

Sensor Data is TimeSeries Data


Traditional Table storage

Informix TimeSeries storage

Meter_ID Time KWH Voltage ColN

1 1-1-11 12:00 Value 1 Value 2 ……… Value N



… … … … ……… …




… … … … ……… …

Meter_ID Series

1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]




…

Traditional Sensor data storage vs. Informix TimeSeries storage


Many IoT applications have a spatial

component to them

• Vehicles, cell phones, even pets… tracking is

common

In these cases both location and time is

important. Typical uses:

• Show me the vehicles that have passed by

location X in the last hour

• Where has my car been over the last few hours?

Informix allows you to combine Time series

and Spatial data in the same query

IoT Requirements for SpatioTemporal Data


Simple to use

• Hands-Free operation – No administration

• Supports popular interfaces such as REST & MongoDB as well as traditional SQL

interfaces (ODBC/JDBC)

• Handles SQL and JSON data in the same database

• Seamless data replication and HA solutions to move or copy data where

needed

Performance

• Unique support for TimeSeries and Spatial data

• Stream data continuously into the database

• Run analytics as data arrives

• Dynamically add and update analytics when needed

• Storage is typically 1/3 the size compared to other vendors

Invisible

Agile

Informix is the only database management system perfectly suited to run in Gateways

IBM Informix: The Ideal Database for Gateways


What are the IoT Requirements for the Cloud?

• Requirements - similar to gateways (but for different reasons):

• Potentially 1000’s of servers means zero administration is a must

• Data volume adds up very quickly so low storage overhead is required

• Data flows into the cloud continuously and must be processed in real-time

• Must be able to handle time series, spatial, SQL and NoSQL data natively

• Additional requirements

• Must be able to scale-out

• Must be available as a service

The database must be able to ingest, process and analyze data in real-time


Why use Informix in the Cloud?

Simple to use

• Hands-Free operation

• Supports REST and MongoDB APIs as well as ODBC/JDBC

• Stores SQL and JSON collections in the same database

• Seamless data replication and HA solutions to move or copy data between Informix implementations in the IoT network

Highly Available

• Close to zero down time

• Partition your data across servers in the cloud

• Dynamically add/remove additional servers

Performance

• Continuous High Performance Analytics

• Specialized support for Time Series and Spatial data

• Build Hybrid applications to work with all types of stored data

• In-Memory query acceleration speeds analysis and analytics to near real time

Invisible

Agile

Resilient


Shards: Scale-out your Database across Servers or Gateways

• Distribute data among servers by range or hash partitioning

• Each shard can have an associated secondary server for high availability

• Run queries across all shards or a subset of the shards

• Only shards that could qualify are searched

• Shards are searched in parallel

• Ignores shards that are offline

Shards in a Cloud


Every IoT deployment will need to store time series data and possibly spatial data

Bluemix Cloud Services

Watson IoT Platform Service on Bluemix• Quickly and simply add new sensors

• Interface for continuously loading sensor data

• Adapters for MQTT and MessageSight

• Uses Informix for TimeSeries historian


Benchmark: Informix vs SQLite

Tests on Intel Quark Informix SQLite

Data loading – high-speed performance (records per

second)950 / 1050 secs

(DK100 / DK50)1

700 secs

(Average)2

Storage space that is required for 1 day of data 275 MB 1200 MB

Aggregation query (seconds) 2 secs 4-25 secs

Moving average (seconds) 25 secs 2592003 secs

Missing interval search (seconds) 2 secs 14-30 secs

1. The two figures for data loading with Informix reflect a slight difference in performance between the DK100 and DK50. DK100 had more

running components causing a drop in performance vs DK50

2. Data loading with SQLite had significant variations in load performance as the database size increased.

3. The moving average result for SQLite is a projected figure that is based on a partial result after 10 minutes.


Metric Competitor Informix

Daily processing time

Maximum number of cores used

11 hours

62

5 hour 50 min

32

Maximum amount of memory used 192GB 192GB

Size of database per month of data 15TB 5TB

# Records processed each day 2.88 Billion 2.88 Billion

Billing determinants creation (1/21 of the total meter population)

51,322 ~2 million reads per second

TimeSeries Meter Data Management Benchmark

- 30 million smart meters sending data every 15 minutes- 2.88 billion records inserted each day- Workload: data Ingestion, data cleanup, and a daily billing cycle


Smart Building Example

• Fire & smoke sensors

• HVAC sensors

• Door & window sensors

• Lighting sensors

• Water sensors

• Solar panels sensors

• Elevator sensors

• …Dashboard &

local analytics

On-premises

Smart Gateway &

database

Replication of selected

building data – all or part

Deeper or cross-building

analytics + employee data

Watch this demo video on YouTube! (https://youtu.be/q0D3acmXAjo)

https://youtu.be/q0D3acmXAjo


IBM IoT Smart Gateway Kit

• Designed to quickly get you up and running on ARM or Intel SoC devices, like

Raspberry Pi

• Works with TI SensorTag and Node-RED

• Simple install and configuration of:• Informix database

• Node.js

• Node-red

• TimeSeries nodes

• Bluetooth node.js application sample

• Visualization nodes and dashboard

Clone the GIT repository, run the install script, turn on your SensorTag and you are collecting TimeSeries data in your Gateway!


IBM Informix - best fit for IoT architecture

IoT Smart Gateway

IoT Cloud analytics

Supported on a wide array of platforms, including SoC computers

Best in class embeddability

Native support for sensor data - TimeSeries & Spatial data

Native support for unstructured (JSON) data

Ease of application development - REST access

Support to receive IoT data via MQTT protocol

High availability and dynamic scaling

In-memory query acceleration

Summary


IBM Smart Gateway kit - https://ibm.biz/BdXr2W

Code samples - https://ibm.biz/BdX4QV

Github - https://github.com/IBM-IoT/

Free Informix Developer Edition - https://ibm.biz/BdXp2g

Informix on Docker Hub https://registry.hub.docker.com/u/ibmcom/informix-

innovator-c/

Informix Developer Edition for Raspberry Pi (32bit)

https://registry.hub.docker.com/r/ibmcom/informix-rpi/

IoT Developers - Get Started!

Docker Hub

https://ibm.biz/BdXr2W

https://ibm.biz/BdX4QV

https://github.com/IBM-IoT/

https://ibm.biz/BdXp2g

https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/

https://registry.hub.docker.com/r/ibmcom/informix-rpi/


Shawn Moe – [email protected]

mailto:[email protected]


SolutionThe embedded solution adds to

Eurotech`s IoT gateway hardware and

Everyware Software Framework (ESF) ,

Informix TimeSeries technology, to

capture data and interact with the

devices, sensors and meters in the field.

Everyware Cloud, deployed in the

Cloud or on premise acts as an

M2M/IoT integration platform between

the distributed devices and the IT world.

Combined and integrated these

elements ensure best TCO and

performance in IoT solutions.

Encapsulating complexity of IoT/M2M solution, ensuring scalability and high performance for customers globally.

ChallengeReducing the cost, effort and risk

challenges in IoT/M2M projects.

Ensuring scalability, performance

and efficiency when developing,

deploying and running distributed

devices, sensors and meters in the

field.

Delivering highly integrated solutions

combining powerful database

technology at the edge / gateway and

the center of the network with state of

the art M2M/IoT software and

hardware building blocks.