59
ArcGIS & the Internet of Things (IoT) Adam Mollenkopf Real-Time & Big Data GIS Capability Lead, Esri [email protected] @amollenkopf Josh Joyner GeoEvent Server, Product Manager [email protected]

ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS & the Internet of Things (IoT)

Adam MollenkopfReal-Time & Big Data GIS Capability Lead, Esri

[email protected]

@amollenkopf

Josh JoynerGeoEvent Server, Product Manager

[email protected]

Page 2: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Agenda:

What is the IoT?

Decomposing an IoT solution

IoT deployment patterns

ArcGIS as an IoT platform

Applying spatiotemporal analytics to IoT data

Handling IoT scale requirements

Additional resources

1

2

3

4

5

6

7

Page 3: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

1 Whatis the IoT?

Page 4: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Internet of Things overviewWhat is the IoT?

The network of physical objects that contain embedded

technology to communicate and sense or interact with their

internal states or the external environment.

Source: Gartner February 2016

Page 5: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Internet of Things overviewWhat Things?

Connected Cars

Connected Health

Connected Home

Precision

Agriculture

Smart City

Smart Community

Digital

Pipelines

Intelligent

Logistics

Page 6: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

2 Decomposingan IoT Solution

Page 7: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Blueprintfor IoT solutions

• The Internet of Things (IoT) is an integrated solution that senses & collects data from devices at the

edge, analyzes that data and takes action to accomplish the business goals of an enterprise.

• An IoT solution consists of the following layers:

- Edge: Embedded technology at the “edge” that senses, acquires & disseminates data.

- IoT platform: Accepts, ingests, stores, analyzes and shares intelligence gleaned from the data.

- Enterprise: Applications & processes that act upon the intelligence as a result of analytic results.

IoT PlatformEdge Enterprise

Page 8: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Edge of an IoT solutionenvironment, sensors, devices & actuators

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

• The Edge consists of:

- Environment: The “real world” that generates real-time observations which are sampled by sensors and

converted to information by devices.

- Sensors: Samples & collects data from the environment, typically 1,000 times per second to ensure sampling accuracy.

- Devices (aka Things): Applies basic analytics to sensor data to reduce it to a more limited & valuable

dataset for transmission. Typically devices emit data on set intervals (once per second), when certain events occur, or when polled.

- Actuators: Receive commands to adjust the behavior of devices in our environment.

IoT PlatformEdge Enterprise

Page 9: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Edge of an IoT solutionedge to IoT platform communication protocols

Sensors

Actuators

Devices(aka Things)

MQTT, HTTP,

AMQP, CoAP

En

vir

on

me

nt

• Edge communication to an IoT platform typically uses one of the following protocols:

- MQTT: Message Queuing Telemetry Transport

- HTTP: Hyper Text Transfer Protocol, pushed or polled

- AMQP: Advanced Message Queuing Protocol

- CoAP: Constrained Application Protocol

IoT PlatformEdge Enterprise

Page 10: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Edge of an IoT solutiongateways

• Optionally, the Edge may consist of one or more:

- Gateways (aka Aggregation Points): Accepts multiple protocols & data models from disparate devices &

translates that data into the IoT Platform’s protocol, API & data model. Serves as a primary interconnection to

the IoT Platform.

• Gateway communication to an IoT platform typically uses one of the following protocols:

- MQTT: Message Queuing Telemetry Transport

- HTTP: Hyper Text Transfer Protocol, pushed or polled

IoT PlatformEdge Enterprise

CoAP

MQTT

AMQP

HTTP

legacy

aggregates thousands

to millions of devices

Sensors

Actuators

Devices(aka Things)

Gateways

MQTT,

HTTP

En

vir

on

me

nt

Page 11: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT platformdevice management & ingestion

• An IoT platform consists of:

- Device Management: Authenticates & establishes secure communications with the Edge devices and/or

gateways to ensure incoming data can be trusted. Also, is responsible for keeping devices up to data and

provisioning software updates as appropriate.

- Ingestion: Accepts data from the Edge and prepares it to be consumed by one or more Streaming Analytics.

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Streaming

Analytics

Ingestion

Device

Management

Page 12: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT platformstreaming analytics, policy, orchestration, actions & intelligence

• An IoT platform consists of:

- Streaming Analytics: consumes a continuous stream of data from the Ingestion layer using Orchestration &

a Policy (aka Business Rule) to make decisions about what to do: trigger immediate actions and/or store the

data in a Data Store for Visualization and subsequent Analytics.

- Actions & Intelligence: takes action via secure access to Actuate Edge Devices and/or Enterprise

applications.

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Streaming

Analytics

Actions &

IntelligenceIngestion

Policy & Orchestration

Device

Management

Page 13: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT platformdata store & batch analytics

• An IoT platform consists of:

- Data Store: provides storage for data that has been Orchestrated for storage. Prepares stored data for

efficient visualization, interactive queries & Batch Analytics.

- Batch Analytics: performs descriptive, prescriptive and/or predictive analysis on data, processes resulting

events & applies policies to determine actions. Typically, can be ad-hoc or scheduled on a recurring basis.

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Device

Management

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

IntelligenceIngestion

Policy & Orchestration

Page 14: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT platformmanagement console, policy & orchestration

• An IoT platform consists of:

- Management Console: Handles a range of management tasks for the IoT platform such as managing the flow

of data, creating Policies, setting event thresholds & managing Data Stores.

- Policy (aka Business Rule): Processes resulting data from Analytics & compares thresholds/triggers

configured by data scientists to determine a course of action for the orchestration engine to execute.

- Orchestration: Coordinates steps necessary to execute an action.

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

Page 15: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT platformactuating devices

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

• An IoT platform consists of:

- Actuation Action: A special type of action that sends a command to a device to alter the behavior of it’s

environment.

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

Page 16: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

IoT PlatformEdge Enterprise

Sensors

Actuators

Devices(aka Things)

Gateways

En

vir

on

me

nt

IoT platformactuating devices with a gateway

• An IoT platform consists of:

- Actuation Action: A special type of action that sends a command to a device to alter the behavior of it’s

environment. When a gateway is present it is responsible for mediating the actuation command to the

appropriate device.

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

Page 17: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Enterprisevisualization & dashboards

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Visualization

&

Dashboards

• The Enterprise layer of an IoT consists of:

- Visualization: enables data scientists to spot key trends and patterns using exploratory visual analysis

techniques that have the ability to interactively query the data. Once trends are recognized policies can be

created to automatically detect the pattern and automatically trigger desired action(s).

- Dashboards: Dashboards can be configured to enable the monitoring of an IoT solution’s environment and

the behavior of it’s Devices/Things.

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

Page 18: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

3 IoTdeployment patterns

Page 19: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Blueprintfor IoT solutions

Sensors

Actuators

Devices(aka Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Visualization

&

Dashboards

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

- Ingestion

- Streaming Analytics & Policies

- Actions (including Actuation)

- Data Store

- Device Management

- Batch Analytics

- Management Console

- Visualization

- Dashboards

• An IoT Platform & Enterprise consists of the following capabilities:

Page 20: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Sensors

Actuators

Devices(or Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

Management

ArcGIS

Enterprise

GeoEvent

Server

ingestion

GeoAnalytics

Server

spatiotemporal

big data storeactions

analytics

policies & orchestration

management console

data store analytics

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

ArcGIS as an IoT Platformenabling geospatial insights with your IoT solution

• An ArcGIS based IoT Platform & Enterprise consists of the following capabilities:

- Ingestion: GeoEvent server input connectors

- Streaming Analytics & Policies: GeoEvent Services

- Actions (including Actuation): GeoEvent output connectors

- Data Store: spatiotemporal big data store

- Batch Analytics: GeoAnalytics Server

- Management Console: Portal & GeoEvent Manager

- Visualization: Map & Feature Services

- Dashboards: Operations Dashboard, Insights, Story Maps

Page 21: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Public Safetypolice fire

surveillance

Public Healthhospitals ambulances

Transitbuses taxis rail

trains crowds

Connected Carsautonomous driving traffic conditions holes

parking meters road conditions slippery areas

network improvements

Energy Usageelectricity gas smart meters

City Workerssanitation snow plows

Airportsflight status queues

plane location runway status

Buildingslighting hvac

occupancy counts

Internet of your Things

Weatherwarnings earthquakes

precipitation icy conditions

Peoplehealth monitoring

social activityTelecommunications

cell phone signals dropped calls

enabling geospatial insights with your IoT ArcGIS & the IoT

DesktopWeb Device

with real-time & big data capabilities

ArcGIS Enterprise

Environmentnoise co2 nitrates

gases temperature humidity

atmospheric pressure radiation

pesticides electromagnetic feedback

rain gauges water level gauges

water quality air quality

Page 22: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Sensors

Actuators

Devices(or Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

Management

ArcGIS

Enterprise

GeoEvent

Server

ingestion

GeoAnalytics

Server

spatiotemporal

big data storeactions

analytics

policies & orchestration

management console

data store analytics

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

ArcGIS as an IoT Platformenabling geospatial insights with your IoT solution

• An ArcGIS based IoT Platform & Enterprise consists of the following capabilities:

- Ingestion: GeoEvent server input connectors

- Streaming Analytics & Policies: GeoEvent Services

- Actions (including Actuation): GeoEvent output connectors

- Data Store: spatiotemporal big data store

- Device Management: for those requiring this functionality another IoT platform can be complemented with ArcGIS.

- Batch Analytics: GeoAnalytics Server

- Management Console: Portal & GeoEvent Manager

- Visualization: Map & Feature Services

- Dashboards: Operations Dashboard, Insights, Story Maps

Page 23: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Edge Enterprise

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

Management

ArcGIS

Enterprise

GeoEvent

Server

GeoAnalytics

Server

spatiotemporal

big data store

analytics

policies & orchestration

management console

data store analytics

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

Complementing an IoT platform with ArcGISenabling geospatial insights with your IoT solution

Sensors

Actuators

Devices(or Things)

Gateways

En

vir

on

me

nt

ingestion

actions

Device RegistryStore

Device StateStore

IoTHub

ingestion

actions

IoTEdge

• The Edge of an IoT broadcasts into an IoT platform such as: Azure IoT, Amazon IoT, Cisco IoT, IBM Bluemix, ...

• The IoT platform integrates with ArcGIS to expand it’s capabilities with spatiotemporal analytics,

visualization & dashboards.

Page 24: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Edge Enterprise

IngestionActions &

Intelligence

Policy & Orchestration

Spatiotemporal

Capabilitiesvia ArcGIS

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

Complementing an IoT platform with ArcGISenabling geospatial insights with your IoT solution

Sensors

Actuators

Devices(or Things)

Gateways

En

vir

on

me

nt

ingestion

actions

data store

• The Edge of an IoT broadcasts into an IoT platform such as: Azure IoT, Amazon IoT, Cisco IoT, IBM Bluemix, ...

• The IoT platform integrates with ArcGIS to expand it’s capabilities with spatiotemporal analytics,

visualization & dashboards.

streaming

analytics

batch

analytics

policies & orchestration

management console

Device

Management

Device RegistryStore

Device StateStore

IoTHub

ingestion

actions

IoTEdge

Page 25: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

4 ArcGISas an IoT Platform

Page 26: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

DesktopWeb Device

live & historic

aggregates & features

map & feature service

• Ingest high velocity real-time IoT

data into ArcGIS.

• Perform continuous analytics on

IoT events as they are received.

• Store IoT observations in a

spatiotemporal big data store.

• Visualize high velocity &

volume IoT data:

- as an aggregation

- or as discrete features.

• Notify about IoT patterns of

interest.

• Adjust behavior of things in our

environment through actuation.

stream service

live features

ArcGIS

Enterprise

GeoEvent

Server

ingestion

analytics

ArcGIS as an IoT PlatformArcGIS Enterprise with real-time & big data capabilities

GeoAnalytics

Server

spatiotemporal

big data store

storage analytics

visualization

actuation

Page 27: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS GeoEvent Server usage

1,273Organizations

88Countries

as of January 2018

Commercial- Financial Services

- Insurance

- Logistics / Trucking

- Manufacturing

- Media & Entertainment

- Real Estate

- Retail

Defense & Intelligence- Intelligence

- Military Operations

Public Safety- Emergency / Disaster Mgmt

- Fire, Rescue, EMS

- Homeland Security

- National Security

- Law Enforcement

- Special Events

Utilities- Electric & Gas

- Telco / Cable

- Public Works

Water- Water resources

- Water / wastewater /

stormwater

Professional Services- AEC

- Environmental Mgmt

- GIS & IT

Nonprofits & Education- Conservation

- Humanitarian

- Sustainable Development

- Higher Ed

- Research/Science Institutions

Natural Resources- Agriculture

- Forestry

- Mining

- Oil & Gas

- Pipeline

Transportation- Aviation

- DOT

- Railways

- Maritime & Ports

- Public Transit

Government- National

- State

- Local

Health & Human Services- Hospital & Health Systems

- Pharmaceuticals

- Public Health

Page 28: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

FlightAware Real-Time Flight Tracking

Page 29: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

exactEarth Real-Time AIS Vessel Feed

Page 30: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Waze Real-Time Traffic & Alert Feed

Page 31: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

California Wildfires ’17 – USGS & Waze Public Safety

Page 32: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Ingestion of real-time dataGeoEvent Server: input connectors

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

you can create

your own

connectors

Poll an ArcGIS Server for Features

Ou

t o

f th

e B

ox

Esri

Gallery

ActiveMQ

Part

ner

Gallery

Amazon IoT

Azure IoT

CompassLDE

enviroCar

exactEarth AIS

FlightAware

GNIP

Networkfleet

OSIsoft

Valarm

Waze

AIS

Common Alerting Protocol

Cursor-on-Target

Exploitation Support Data

GTFS

Instagram

Kafka

KML

MQTT

NMEA 0183

RabbitMQ

Sierra Wireless (RAP)

Trimble (TAIP)

Twitter

Poll an external website for GeoJSON, JSON, or XMLHTTP

Receive Features, GeoJSON, JSON, or XML on a REST endpointREST

Receive GeoJSON or JSON on a WebSocketWS

Receive RSS

Receive Text from a TCP or UDP Socket

Subscribe to an external WebSocket for GeoJSON or JSONWS

Watch a Folder for new CSV, GeoJSON, or JSON Files.csv

AIS

Part

ner

Gallery

Page 33: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Streaming analytics, policies & orchestrationGeoEvent Server: geoevent services

input1 output1buffer

geofences

filter(spatial)

input2 output2

geofences

geotagger

output3motion

calculator

geometry inside Zones/.*

geometry inside Zones/.*

• A GeoEvent Service configures the flow of real-time data,

- the filtering and geoevent processing steps to perform,

- the input(s) where data comes from and the output(s) to which results are sent.

Page 34: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

you can create

your own

processors

Streaming analytics, policies & orchestrationGeoEvent Server: processors

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

Geotagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Ou

t o

f th

e B

ox

Add XYZ

Esri

Gallery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

Page 35: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Actions & actuationGeoEvent Server: output connectors

you can create

your own

connectors

Ou

t o

f th

e B

ox

Add or Update a Feature

Esri

Gallery

Amazon IoT

Azure IoT

ActiveMQ

Cursor-on-Target

Hadoop

Kafka

MongoDB

MQTT

RabbitMQ

Twitter

Publish Text to a UDP Socket

Push GeoJSON or JSON to an external WebsiteHTTP

Push GeoJSON or JSON to an external WebSocketWS

Push Text to an external TCP Socket

Send a Text Message

Send an Email

Send an Instant Messageim

Send Features to a Stream Service

Write to a CSV, GeoJSON, or JSON File .csv

Add a Feature to a spatiotemporal big data store

Update a feature in a spatiotemporal big data store

Page 36: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

movingsomething that moves

• planes

• vehicles

• animals

• satellites

• storms

stationarystands still but attributes change

• water gauges

• weather stations

• traffic sensors

• air quality

• temperature

• wind

discretesomething that “just happens”

• crimes

• lightning

• accidents

Visualizationtypes of observation data

Page 37: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

• as data is written to a data source in the data store:

- up to four types of spatial indices are supported: geohash, square, pointy & flat hexagon/triangle

- this is in addition to a temporal index on the time field

- and an inverted index on each of the attribute fields

geohash

square

trianglepointy

triangleflat

hexagonflat

hexagonpointy

Storagespatial indexing to support on-the-fly aggregations

Page 38: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

• Map & Feature Services that make use data in the spatiotemporal store enable you to:

- visualize on-the-fly aggregations of data

- perform exploratory queries over any combination of space, time and attributes

- switch visualization from aggregations to raw features

- inspect feature level attributes while in aggregation or raw feature views

- replay via a time-slider historic observations in aggregation or raw feature views

Visualizationof observation data

Page 39: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

5 Applyingspatiotemporal analytics to IoT Data

Page 40: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

GeoAnalytics Server: analytic capabilities

Batch analysis

Analyze Patterns

Use Proximity

Summarize Data

Calculate Density

Find Hot Spots

Create Space Time Cube

Join Features

Aggregate Points

Summarize Within

Summarize Attributes

Reconstruct Tracks

Detect Incidents

Find Similar Locations

Geocode Locations

Create Buffers

Find Locations

“I want to…”

Manage DataCalculate Field

Copy To Data Store

GeoAnalytics

Server

Page 41: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Aggregate Points:

- “How does the spatial distribution of vehicle collisions change over time?”

- “What zip codes have the highest count of crimes incidents?”

- ”Where are there the most power outages?”

- ”What does my data look like?”

Page 42: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Aggregate Points:

- Input 1: Points, Input 2: Polygons (or generate bins)

- Output: Polygons

- Aggregates point data into either a square grid, a hexagonal grid, or user-supplied polygon features

- Users can choose to aggregate either spatially or spatiotemporally

- A count of points is returned for each grid cell (bins), in addition to optional attribute field statistics

Page 43: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Aggregate Points:

Methods of Aggregation

Spatial, into polygons:

Spatial, into bins:

Page 44: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Aggregate Points:

Methods of Aggregation

Spatiotemporal, into bins:

Spatiotemporal, into polygons:

Spatial, into polygons:

Spatial, into bins:

Page 45: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Join Features:

- “Which crime events occurred near sporting events, spatially & temporally?”

- “Which bodies of water intersect cities with populations greater than one million people?”

- “Which traffic jams occurred because of car accidents?”

Page 46: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Join Features:

- Input 1: Points, Polygons or Lines, Input 2: Points, Polygons or Lines

- Output: Points, Polygons or Lines

- Joins attributes from one layer to another based on spatial, temporal & attribute relationships, or any

combination of those relationships.

Page 47: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Join Features:

Spatial

• Intersects

• Equals

• Near

• Contains

• Within

• Touches

• Crosses

• Overlaps

Temporal

• Meets

• Met by

• Overlaps

• Overlapped by

• During

• Contains

• Equals

• Finishes

• Finished by

• Starts

• Started by

• Intersects

• Near

Attribute

Features are matched

when the field values

in the join layer are

equal to field values in

the target layer.

Near spatial and temporal relationships require a distance/length of time to be specified. For more information see:

https://pro.arcgis.com/en/pro-app/tool-reference/big-data-analytics/spatial-relationships-with-big-data.htm

Joins in GeoAnalytics Server

Page 48: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

summarize data

Batch analysis

• Join Features:

Temporal Operators

Page 49: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

6 HandlingIoT scale requirements

Page 50: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS

Enterprise

spatiotemporal

big data store

GeoAnalytics

Server

3 4

ArcGIS Enterprisewith real-time & big data GIS capabilities

IoT Big Data

GeoEvent

Server

2

MINIMUM environment4 machines

1

ArcGIS GeoEvent Server 10.2 10.3 10.4 10.5 10.6

Velocity throughputmeasured in events per second (e/s)

up to

500 e/s

up to

2,000 e/s

up to

3,000 e/s

up to

4,000 e/s

up to

6,000 e/s

Page 51: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS

Enterprise

spatiotemporal

big data store

GeoAnalytics

Server

6 75 9 108

ArcGIS Enterprisewith real-time & big data GIS capabilities

IoT Big Data

GeoEvent

Server

2 3 4

OPTIMIZED environment10 machines

1

ArcGIS GeoEvent Server 10.2 10.3 10.4 10.5 10.6

Velocity throughputmeasured in events per second (e/s)

up to

500 e/s

up to

2,000 e/s

up to

3,000 e/s

up to

4,000 e/s

up to

6,000 e/s

Resiliency & Scalabilityvia multi-machine site

no no no no yesup to 5 practically, so can

scale to up to 30,000 e/s

Page 52: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS Enterprisewith real-time & big data GIS capabilities on Microsoft Azure

ArcGIS

Enterprise

spatiotemporal

big data store

GeoAnalytics

Server

6 75 9 108IoT Big Data

GeoEvent

Server

2 3 4

1

ArcGIS Enterprise

Cloud Builder

for Microsoft Azure

Page 53: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

ArcGIS Enterprisewith real-time & big data GIS capabilities on Amazon EC2

ArcGIS

Enterprise

spatiotemporal

big data store

GeoAnalytics

Server

6 75 9 108IoT Big Data

GeoEvent

Server

2 3 4

1

ArcGIS Enterprise

Cloud Builder

for Amazon EC2

Page 54: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

7 Additionalresources

Page 55: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Esri User Conference 2017 pre-conference seminar on ‘Integrating IoT & ArcGIS’:

Additional Resources

ArcGIS & the Internet of Things

https://github.com/Esri/integrating-iot-arcgis

Page 56: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Please Attend Our Other Sessions!

• GeoEvent Server: An Introduction Tue, 2:30-3:30 pm, Primrose B

• GeoEvent Server: Applying Real-Time Analytics Tue, 5:30pm-6:30 pm, Primrose A

• Real-Time and Big Data GIS: Best Practices Wed, 10:30-11:30 am, Primrose B

Fri, 1:00-2:00 pm, Catalina/Madera

• ArcGIS and the Internet of Things (IoT) Wed, 2:30-3:30 pm, Primrose B

• Real-Time and Big Data GIS: Leveraging the Wed, 4:00-5:00 pm, Primrose A

Spatiotemporal Big Data Store

• Developing Real-Time Web Apps with the ArcGIS API Tue, 9:00-9:30 am, Demo Theater 1: Oasis 1-2

for JavaScript

• Real-Time GIS: Road Ahead Thu, 4:00-5:00 pm, Pasadena/Sierra/Ventura

• GeoEvent Server: Creating Connectors and Fri, 8:30-9:30 am, Mesquite B

Processors Using the GeoEvent SDK

Page 57: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Please Take Our Survey!

Download the Esri Events app

and find your event

Select the session you

attended

Scroll down to the

“Feedback” section

Complete Answers,

add a Comment,

and Select “Submit”

Page 58: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Esri.com Proceedings page

http://proceedings.esri.com

Where users can find PDF'ed

slides after DevSummit

Page 59: ArcGIS & the Internet of Things (IoT) - EsriMQTT, HTTP, nt AMQP, CoAP •Edge communication to an IoT platform typically uses one of the following protocols:-MQTT: Message Queuing

Adam MollenkopfReal-Time & Big Data GIS Capability Lead, Esri

[email protected]

@amollenkopf

Josh JoynerGeoEvent Server, Product Manager

[email protected]

Questions / Feedback?

https://github.com/Esri/integrating-iot-arcgis