IoT Use Cases with MapR

© 2015 MapR Technologies, confidential ®

®

Patric Uebele 15-06-21



Business Intelligence and the IoT


The value of BI

busi

ness

val

ue

reporting what happened?

OLAP why did it happen?

real-time analytics what and why is it happening, now?

predictive analytics what might happen?


What is the Internet of Things

Wearables

Connected Homes

Industrial Internet

Connected Cities

Connected Cars


IoT—a superset of the Internet

thin fat

stationary

mobile

Devices and their deployment



IoT use case categories and apps


Personal loT wearables smart phones clothes

Industrial loT smart factories agriculture retail manufacture

Group loT vehicles smart houses tourism education

Community loT smart cities smart roads parks


Internet of Things is Here Now and Growing

The New Essential Infrastructure


Operational Efficiency Waste Management

•  20K trucks, 30M GPS pings a day

•  Integrate 17 different systems that have operational significance related to driver, vehicle, and route performance management

•  Over 1.6M labor hours saved


Self Driving Cars Sensors Collaborate on the Road

Google’s Self-driving car generates 13 million laser measurements and calculates 20 driving decisions EACH second


© 2015 MapR Technologies ®

Connected Car Use Case I – Driver Profile and Automobile Status


APPROACH WHAT WE WANTED AND HOW WE STARTED

§  Open source project

§  Raspberry Pi based python script

§  Pre programed OBD Mapping

§  Bluetooth OBD-II connection integration

First approach

Solution: OBDPi

§  Access Motor data through OBD-II connection

§  Log key data points locally

Goal

Gain insights about driver profile and automobile status through telematics


OBT-II ACCESSING THE STATUS OF VARIOUS VEHICLE SUB-SYSTEMS

§  Required to be within 2 feet of steering wheel

§  Required to be built into all cars sold in California since

1991

§  Supplies Vehicle and sensor information to technicians

§  Supply diagnostic trouble codes (DTC)

§  OBD-II replaced the OBD standard in 1996

§  16 Pin Connector

On Board Diagnostics-II


RASPERRY PI CREDIT CARD-SIZED SINGLE-BOARD COMPUTER

§  Multi-functional microcontroller development board

§  Size of a credit card

§  Linux based OS Capabilities (Raspbian-Debian variant)

§  Initially created to promote computer science in schools

§  Also can be used as powerful development board

§  Features include: –  CPU: 700 MHz ARM Microcontoller –  GPU: Boradcom VideoCore IV @ 250 Mhz –  512 MB Memory (SDRAM) –  4 USB ports –  46 GPIOS (Model B+)

RAM

I/O CPU/GPU USB hub

Ethernet

2x USB


PY-OBD JOINING RASPERRY PI WITH OB-II

PY-OBD

Rasperry Pi OBD-II

§  Py-OBD is a project from GitHub User PeterH

§  Uses Data from OBD-II Port to display current Driving data

§  Project is GNU General Public Licensed and intended to be shared, modified and improved


SYSTEM OVERVIEW RASPBERRY PI DATA ACQUISITION

Car

OBD-II Dongle

Raspberry Pi B +

WiFi adapter for peripherals

GPS


HOW TO GAIN GREATER FUNCTIONALLITY

1

2

3

4

Stream Data into MapR Enterprise Database Platform using Fluentd

Data Storage and Manipulation with MapR Cluster

Data Aggregation with open source Apache Drill beta

Stream visualization with Grafana (real time time series with openTSDB on top of MapR-DB)

4

5 QlikView data summary visualization and Adhoc with Apache Drill SQL based view


SYSTEM OVERVIEW STREAM PROTOCOL

Stream Protocol Data Acquisition Data Store & Analytics Data Visualization

Raspberry PI

OBD2

fluentd fluentd

Grafana

QlikView

real-time dashboard

flat-files

OLAP Ad-Hoc

Realtime


FLUENTD TREASURE DATA BASED STREAM PROTOCOL

§  Unified logging layer

§  Fluentd allows you to unify data collection and consumption for a better use and understanding of data.

§  Plugin library allows for a variety of applications and platform use (more than flume)

§  Easier to Configure (rpm and start/stop scripts | ruby code for quick patches without the need for compilation)

§  Simple encryption plugins allow for the secure streaming of data

§  Only requires 40 MB of memory and can process 13,000 events/second/core

Rasberry PI

IN Rasberry PI

Out MapR Cluster

OpenTSDB on MapR-DB

MapR-FS tail_in plugin out_forward_plugin or

out_forward_secure_plugin in_forward_plugin or in_forward_secure_plugin

out_file_plugin

opentsdb_plugin

out_copy


Connecting Qlikview for Adhoc Queries using Apache Drill

Node Node Node Node MapR-Cluster

Mapr-FS Mapr-FS Mapr-FS Mapr-FS

Drillbit Drillbit Drillbit Drillbit

Drill Windows ODBC Driver

CSV Files from Fluentd

Drill Explorer Create View with Datatypes

Access View from Qlikview

ODBC

NFS

Data Analyst Windows Qlikview Server


PRACTICE DATA VISUALIZED IN OUR DASHBOARD

First incoming data

QlikView Dashboard


Connecting Grafana to Opentsdb

Node Node Node Node MapR-Cluster

Mapr-DB Mapr-DB Mapr-DB Mapr-DB

OpenTSDB OpenTSDB OpenTSDB OpenTSDB

Grafana

Fluentd

Rest

Opentsdb plugin


PRACTICE Realtime View using Grafana

Realtime stats using velocity, tachometer, load


© 2015 MapR Technologies ®

Connected Car Use Case II – Including Google Nest


APPROACH WHAT WE WANTED

§  High Arrival Data with Millions of Data Points per Second

§  Analyzing Car Location Data in Realtime

§  Start Heating and Cooling based on distance measuring between the car to home and to work

First approach

Solution: MapR-DB / OpenTSDB

§  Access Motor data through OBD-II connection

§  Log key data points locally

Goal

Optimize Heating and Cooling Costs in house by maschine to maschine communication


What is Google Nest?

Nest Learning Thermostat learns your schedule, programs itself Nest could be controlled remotely Nest Thermostat can lower your heating and cooling bills up to 20%




MapR Distribution including Apache Hadoop

MapR Data Platform (Random Read/Write)

Data Hub Enterprise Grade Operational

MapR-FS (POSIX)

MapR-DB (High-Performance NoSQL)

NFS HDFS API HBase API JSON API

Map

R C

ontr

ol S

yste

m

(Man

agem

ent a

nd M

onito

ring)

Security

YARN

Pig

Cascading

Spark

Batch

Spark Streaming

Storm

Streaming

HBase

Solr

NoSQL & Search

Juju

Provisioning &

Coordination

Savannah

Mahout

MLLib

ML, Graph

GraphX

MapReduce v1 + v2

APACHE HADOOP AND OSS ECOSYSTEM

EXECUTION ENGINES DATA GOVERNANCE AND OPERATIONS

Workflow & Data

Governance Tez

Hive

Impala

Spark SQL

Drill

SQL

Sentry Oozie ZooKeeper Sqoop

Knox* Whirr Falcon* Flume

Data Integration & Access

HttpFS

Hue


MapR Distribution for Hadoop

Theme Requirements Features Product

Enterprise Grade

•  Uptime service levels •  Site to site DR & Business

Continuity •  Backup/recovery •  Security •  High velocity data ingress

•  HW/SW HA •  Mirroring & Promotable

Mirrors •  Snapshots •  Authorization, Kerberos •  2X-5X performance

MapR Enterprise Edition

Data Hub

•  Hadoop •  Traditional applications •  Data of record •  Batch and interactive

•  HDFS •  POSIX •  Strong consistency •  MapReduce and SQL

MapR Enterprise Edition

Operational •  Real time •  NoSQL •  Operational analytics

•  HBase •  Update in place •  Concurrent read/write •  Timeseries

MapR Enterprise Database Edition

MapR Patent Pending – “Table Format for Map Reduce” “Map Reduce Ready Distributed File System”

Enterprise Grade

Operational

Data Hub

®© 2015 MapR Technologies 31

Global Real-time Synchronization

Multi-master (aka, active/active) replication

Active Read/Write

End Users

•  Faster data access – minimize network

latency on global data with local clusters

•  Reduced risk of data loss – real-time,

bi-directional replication for synchronized

data across active clusters

•  Application failover – upon any cluster

failure, applications continue via

redirection to another cluster



MapR Data Protection Advantages for IoT Advanced Business Continuity with Industry’s only mirroring & point-in-time consistent snapshots


MapR Security Advantages for IoT Strongest authentication, most flexible authorization, most powerful auditing


Q & A

@mapr maprtech

[email protected]

MapR

maprtech

mapr-technologies


Technology

IoT Use Cases with MapR