ArcGIS and the Internet of Things - Esri...Complementing an IoT platform with ArcGIS enabling...

Thomas Paschke

ArcGIS and the

Internet of Things

Agenda

Introduction

ArcGIS as an IoT Platform

ArcGIS Analytics for IoT

1

2

3

Introduction1

Internet of Things overviewWhat is the IoT?

The network of physical objects that contain embedded

technology to communicate and sense or interact with their

internal states or the external environment.

Source: Gartner February 2016

Internet of Things overviewWhat Things?

Internet of Things overviewWhat Things?

Public Safetypolice fire

surveillance

Public Healthhospitals ambulances

Transitbuses taxis rail

trains crowds

Connected Carsautonomous driving traffic conditions holes

parking meters road conditions slippery areas

network improvements

Energy Usageelectricity gas smart meters

City Workerssanitation snow plows

Airportsflight status queues

plane location runway status

Buildingslighting hvac

occupancy counts

Internet of your Things

Weatherwarnings earthquakes

precipitation icy conditions

Peoplehealth monitoring

social activityTelecommunications

cell phone signals dropped calls

Environmentnoise co2 nitrates

gases temperature humidity

atmospheric pressure radiation

pesticides electromagnetic feedback

rain gauges water level gauges

water quality air quality

IoT & GISDynamic Mass Data (Velocity, Volume, Variety)

High Frequency

Sensor Data

Various Data

Sources

Massive Data Volumes

Commonality: Location!

ArcGIS as an IoT Platform2

Decomposing an IoT PlatformGeneric Blueprint of an IoT solution

• An IoT solution consists of the following layers & components:

Sensors

Actuators

Devices(or Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Ingestion

StreamingAnalytics

Data Store

BatchAnalytics

Actions &Intelligence

- Ingestion

- Streaming Analytics & Policies

- Actions (including Actuation)

- Data Store

- Batch Analytics

Sensors

Actuators

Devices(or Things)

En

vir

on

me

nt

IoT PlatformEdge Enterprise

Visualization

&

Dashboards

Ingestion

StreamingAnalytics

Data Store

BatchAnalytics

Actions &Intelligence

Policy & Orchestration

Device

ManagementManagement

Console

- Ingestion

- Streaming Analytics & Policies

- Actions (including Actuation)

- Data Store

- Batch Analytics

- Management Console

- Device Management

- Visualization

- Dashboards

GatewaysCoAP

MQTT

AMQP

HTTP

legacy

Decomposing an IoT PlatformGeneric Blueprint of an IoT solution

• An IoT solution consists of the following layers & components:

ArcGIS as an IoT Platformenabling geospatial insights with your IoT solution

IoT PlatformEdge Enterprise

Device

Management

Operations Dashboard for ArcGISInsights for ArcGIS

Esri Story MapsArcGIS Earth

ArcGIS OnlineArcGIS Pro

Collector for ArcGISWeb AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

• An ArcGIS based IoT platform consists of the following capabilities:

- Ingestion:

- Streaming Analytics & Policies:

- Actions (including Actuation):

- Data Store:

- Device Management: missing functionality in ArcGIS, 3rd party solution must be integrated such as Azure IoT

- Batch Analytics:

- Management Console:

- Visualization:

- Dashboards:

spatiotemporalbig data store

data store

GeoEventServer

ingestion

actions

analytics

GeoAnalyticsServer

analytics

ArcGISEnterprise

policies & orchestration

management console

visualization

GeoEvent input connectors

GeoEvent Services

GeoEvent output connectors

spatiotemporal big data store

GeoAnalytics Server

Enterprise (Portal & Server Manager)

Map & Feature Services, Stream Services

Operations Dashboard, Insights, AppBuilder

Sensors

Actuators

Devices(or Things)

En

vir

on

me

nt

GatewaysCoAP

MQTT

AMQP

HTTP

legacy

Edge Enterprise

Ingestion

Streaming

Analytics

Data Store

Batch

Analytics

Actions &

Intelligence

Policy & Orchestration

Device

Management

ArcGISEnterprise

GeoEvent

Server

GeoAnalytics

Server

spatiotemporal

big data store

analytics

policies & orchestration

management console

data store analytics

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

Complementing an IoT platform with ArcGISenabling geospatial insights with your IoT solution

Sensors

Actuators

Devices(or Things)

Gateways

En

vir

on

me

nt

ingestion

actions

Device RegistryStore

Device StateStore

IoTHub

ingestion

actions

IoTEdge

• The Edge of an IoT broadcasts into an IoT platform such as: Azure IoT, Amazon IoT, Cisco IoT, IBM Bluemix, ...

Edge Enterprise

IngestionActions &

Intelligence

Policy & Orchestration

Spatiotemporal

Capabilitiesvia ArcGIS

visualization

Operations Dashboard for ArcGIS

Insights for ArcGIS

Esri Story Maps

ArcGIS Earth

ArcGIS Online

ArcGIS Pro

Collector for ArcGIS

Web AppBuilder for ArcGIS

AppStudio for ArcGIS

dashboards

Complementing an IoT platform with ArcGISenabling geospatial insights with your IoT solution

Sensors

Actuators

Devices(or Things)

Gateways

En

vir

on

me

nt

ingestion

actions

data store

• The Edge of an IoT broadcasts into an IoT platform such as: Azure IoT, Amazon IoT, Cisco IoT, IBM Bluemix, ...

• The IoT platform integrates with ArcGIS to expand it’s capabilities with spatiotemporal analytics,

visualization & dashboards.

streaming

analytics

batch

analytics

policies & orchestration

management console

Device

Management

Device RegistryStore

Device StateStore

IoTHub

ingestion

actions

IoTEdge

Public Safetypolice fire

surveillance

Public Healthhospitals ambulances

Transitbuses taxis rail

trains crowds

Connected Carsautonomous driving traffic conditions holes

parking meters road conditions slippery areas

network improvements

Energy Usageelectricity gas smart meters

City Workerssanitation snow plows

Airportsflight status queues

plane location runway status

Buildingslighting hvac

occupancy counts

Internet of your Things

Weatherwarnings earthquakes

precipitation icy conditions

Peoplehealth monitoring

social activityTelecommunications

cell phone signals dropped calls

enabling geospatial insights with your IoT ArcGIS & the IoT

with real-time & big data capabilities

ArcGIS Enterprise

Environmentnoise co2 nitrates

gases temperature humidity

atmospheric pressure radiation

pesticides electromagnetic feedback

rain gauges water level gauges

water quality air quality

Apps

Desktop

APIs

ArcGIS

Enterprise

spatiotemporal

big data store

IoT

GeoAnalytics

Server

Big Data

GeoEvent

Server

enabling geospatial insights with your IoT ArcGIS & the IoT

• Integration of various data sources

- Connectors for different data formats and transports

- Common event based format

• Powerful processing & filtering

- Spatial analysis against geometries of interest

(GeoFences) that are kept in memory

- GeoEvent Services define

analysis

• Scalable architecture

• Dissemination and visualization of data

streams

- Storage in spatiotemporal big data store or external

application

- Immediate visualization via StreamLayer

IoT

GeoEvent

Server

enabling geospatial insights with your IoT ArcGIS & the IoT

ArcGIS

Enterprise

spatiotemporal

big data store

enabling geospatial insights with your IoT ArcGIS & the IoT

• The spatiotemporal

big data store is a

distributed, highly

available data store

• Visualize on-the-fly

aggregations of data

• Switch visualization

from aggregation to

raw features

• Perform exploratory

queries over any

combination of space,

time, and attributes

GeoAnalytics

Server

Big Data

enabling geospatial insights with your IoT ArcGIS & the IoT

Delimted Text, Shape, Parque, ORC

big datafile shares

featureservice

• Spatiotemporal Analysis

- Tools honor both dimensions

- Focus on aggregation and pattern detection

• Parallelized compute

- Spark as framework for parallelized

compute

• Connection to external data sources

- Use existing enterprise data via

big data file shares

• Run analysis as Python script

- Pipeline analysis

- Extent analysis via PySpark and SparkML

Demo:

ArcGIS as an IoT Platform

enabling geospatial insights with your IoT ArcGIS & the IoT

ArcGIS

Enterprise

storage

visualization

GeoEvent

ServerGeoAnalytics

Server

spatiotemporal

big data store

ArcGIS Enterprise withreal-time & big data capabilities

enabling geospatial insights with your IoT ArcGIS & the IoT

ArcGIS Online

storage

visualization

microservices

as containers

Analytics for IoT

ArcGIS Analytics for IoT3

ArcGIS Analytics for IoTreal-time & big data GIS capability for ArcGIS Online

ArcGIS Online

Analytics for IoT

FeedReal-Time

Analytic

Big Data

Analytic

Map

Layer

Feature

Layer

Stream

Layer

Apps

DesktopAPIs

• Extract location-based intelligence from your IoT

• Rapidly visualize, replay, & analyze IoT data

• Discover & use real-time feeds

• Detect patterns over space & time

• Capability of ArcGIS Online

- Web application

• Subscription-based business

model

• Managed by Esri

- No installation / overhead

- Capability is purchased and

automatically provisioned

• Runs in region of choice

- US (West US 2)

- Europe (West Europe)

- Others TBD

ArcGIS Analytics for IoTreal-time & big data GIS as a service

target use cases

Analytics for IoT

• Fleet & field crew tracking

- Snow plows, waste management, postal carriers, ships, park/facility

staff monitoring, container tracking

• Human movement analysis

- Site selection & planning, event monitoring

• System status, health, and response

- Grid/network outages & crew assignment, outage pattern

assessment, air quality status over clean-up sites

• Telematics

- Traffic counts for traffic signal optimization, roadway issue detection

& response, weather hazard alerting

FeedsReal-time, streaming data

• Real-time, streaming data

• Visualize directly in web maps

- Clients receive & display new data

immediately

- Behaves like stream layer

• Share as item

- Provides access to data for

visualization & analysis

FeedsSupported types and formats

• Types

• Formats

Website (Poll) Endpoint (Receive) Feature Layer Kafka Broker

Azure IoT Hub AWS IoT

Delimited Text JSON GeoJSON EsriJSON RSS

RabbitMQ

Cisco Kinetic EFM

MQTT

Stream Layer

Web Socket

SimulatorWaze

Azure Service Bus

Real-Time AnalyticsContinuous analysis

• Consume a Feed

• Analyze & detect patterns

of interest

• Alerts & actuation

• Store to map/feature layers

Big Data AnalyticsEasy configuration

• Access big data stores

• Analyze patterns

• Assess trends

• Store results

• Schedule analysis

Summarize Data

Aggregate Points

Reconstruct Tracks

Summarize Attributes

Summarize Within

Use Proximity

Create Buffers

Snap to Network

Calculate Distance

Analyze Patterns

Calculate Density

Find Hot Spots

Find Similar Locations

Detect Idle Locations

Detect Incidents

Calculate Journeys

Detect Gaps

Manage Data

Join Features

Find Locations

Enrich Data

Overlay Layers

Project

Dissolve Boundaries

Calculate Fields

Map Fields

Merge

Select Fields

Filter

Calculate Motion Statistics

real-time & big data analytic

big data analytic only

Analysis toolsSpatiotemporal capabilities

• Stream layers

- Underlying visualization for Feeds

- User-defined output from analytics

• Feature layers

- User-defined output from analytics

- Data stored & hosted in Analytics for IoT

- Support dynamic aggregation (custom Javascript apps)

• Map image layers

- Created with every feature layer

- Support dynamic aggregation

- Rendering configurable via Analytics for IoT app

VisualizationVisualizing observational data & analysis

• Observational data can be stored to

feature layers

• Data retention period chosen by

user

- E.g. N weeks, N months

• Older data can be automatically

transitioned to archive (‘cold store’)

- Archived data can be leveraged for

analysis via big data analytics

StorageObservational data archive

Demo:

Analytics for IoT

• Key Capabilities:

- Feeds: configure a real-time data stream & share it, visualize as live layer

- Real-Time Analytics: work on a Feed, identify activity of interest, take action

- Big Data Analytics: run on ArcGIS or other big data stores, assess spatiotemporal

patterns, take action

• Release: December 2019 (following ArcGIS Online 7.4)

• Landing Page: https://www.esri.com/en-us/landing-page/product/2018/arcgis-analytics-for-iot

• GeoNet Community: https://community.esri.com/community/arcgis-analytics-for-iot

ArcGIS Analytics for IoTSummary

Complete answers

and select “Submit”

Scroll down to find the

feedback section

Select the session

you attended

Download the Esri Events

app and find your event

Please Take Our Survey on the App