Geospatial Data Science & Artificial Intelligence€¦ · In ArcGIS: Spatially Constrained Multivariate Clustering, Multivariate Clustering, Density-based Clustering, Image Segmentation,

11/18/2019

1

Geospatial Data Science & Artificial Intelligence

Rohit Singh

Session Overview

• What is AI, Machine Learning & Deep Learning

• Machine Learning

- What is Machine Learning?

- Machine Learning Tools in ArcGIS

• Spatial Data Science

- Notebooks for data science workflows

- Spatial analysis using ArcGIS API for Python

• Geospatial Deep Learning

- Types of models and their applications

- Training and deploying deep learning models

- Scalable deep learning with Image Server

• Geo-AI projects showcase

1

2

11/18/2019

2

Data Science

ArcGIS integrates all types of data, from various sources

Imagery

LidarVector

Tabular

Real-Time (IoT)

Big Data

Unstructured

BIM

3D

Network

Maps, Scenes, Layers

ArcGIS Enterprise

Supports interactive and batch analysis workflows

Scalable

Raster analysis Realtime analytics Big Data analysis GeocodingSpatial analysis Notebooks

Data Science

3

4

11/18/2019

3

• Clustering

• Prediction

• Classification

• Regression

• Interpolation

• Object Identification

New and Improved

• Feature Extraction

• Site Selection

• Event Prediction

• Image Analysis

Coming

Data Science, Machine Learning & Deep Learning

ArcGIS is a Data Science platform . . .and includes Machine Learning & Deep Learning

Empirical Bayesian KrigingRegression Prediction

Training Data Preparation

Density-BasedClustering

Forest-Based Classificationand Regression

Spatial Analysis

Transportation

Feature Identification

Deep Learning

Feature Extraction

Survey

Python Notebook Integration

ArcGIS

• PyTorch

• TensorFlow

• scikit-learn• Microsoft

• IBM Watson

• Amazon

R Integration

Data Science

SAS Jupyter

pandas

What is AI?

5

6

11/18/2019

4

Neural Networks

TensorFlow

CNTK

Natural Language Processing

Cognitive Computing

GeoAI

Computer Vision

Dimensionality Reduction

Object Detection

Support Vector Machines

Object Tracking

Keras

PyTorch scikit-learn

fast.ai

Random Forest Machine Learning

Deep Learning

Artificial IntelligenceCaffe

Machine Learning

Deep Learning

Artificial Intelligence

7

8

11/18/2019

5

Machine Learning


Deep Learning

Machine Learning


Deep Learning

Data Science

9

10

11/18/2019

6

Machine Learning

Labelled Data Sets: Features + Output

Call Drops

# Complains Subscribed Package

Call Rate Decline

Churned?

4 5 ABC 20% Yes

6 2 ABC 5% No

9 4 XYZ 12% Yes

Features Output (label)

Training Data(Historical)

For Learning..

New Data Prediction

Learning Models

Trained Model

Problem: Predicting Churn

7 5 KLM 8% Yes (75%)

12

13

11/18/2019

7

Labelled Data Sets: Features + Output

Segment TypeProximity to Intersection

Time of Day

Weather Accident

Highway 0.1 M Morning Raining Injury

Tunnel 0.3 M Evening Sunny Property

Inner 0.2 M Noon Foggy Injury

PipeAge

Depth Temperature Pressure Break

20 3 m 95 F 20 P Yes

15 4 m 5 F 35 P Yes

6 3.5 m 2 F 17 P Yes

IS Indirect Fire

IS Explosive Attack

IS Vehicle Attack

Coalition Air Ops

ISIS Camp

0.2 M 2 M 3 M 0.7 M Yes

4 M 9 M 6 M 1 M No

1.2 M 5 M 5 M 0.5 M Yes

# Females 35 - 50

F&B Sales% Urban

Chic#

CompetitorsSales

35,000 $400M 55% 15 $20M

14,000 $150M 15% 27 $5M

27,000 $210M 26% 9 $12M

Road Accidents Prediction Water Leakages Prediction

ISIS Location Prediction Retail Sales Prediction

Python API

R-Bridge

14

15

11/18/2019

8

Machine Learning in ArcGIS

ArcGIS has Machine Learning Tools

ArcGIS

Classification

Clustering

Prediction

16

17

11/18/2019

9

Using the known to estimate the unknown

Use Case: Accurately predict impacts of climate change on local temperature using global climate model data

Prediction

In ArcGIS: Empirical Bayesian Kriging, Areal Interpolation, EBK Regression Prediction, Ordinary Least Squares Regression and Exploratory Regression, Geographically Weighted Regression

The process of deciding to which category an object should be assigned based on a training dataset

Use Case: Classify impervious surfaces to help effectively prepare for storm and flood events based on the latest high-resolution imagery

Classification

In ArcGIS: Maximum Likelihood Classification, Random Trees, Support Vector Machine

18

19

11/18/2019

10

The grouping of observations based on similarities of values or locations

Use Case: Given the nearly 50,000 reports of traffic between 5pm and 6pm in Los Angeles (from Traffic Alerts by Waze), where are traffic zones that can be used to elicit feedback from current drivers in the area?

Clustering

In ArcGIS: Spatially Constrained Multivariate Clustering, Multivariate Clustering, Density-based Clustering, Image Segmentation, Hot Spot Analysis, Cluster and Outlier Analysis, Space Time Pattern Mining

The grouping of observations based on similarities of values

Use Case: Use different economic indicators to define economic regions automatically

Spatially-Constrained Clustering

Similar In ArcGIS: Multivariate Clustering, Density-based Clustering, Image Segmentation, Hot Spot Analysis, Cluster and Outlier Analysis, Space Time Pattern Mining

20

21

11/18/2019

11

Notebooks for data science workflows

What is ArcGIS Notebooks?

A new capability that lets you create hosted Python notebooks using a Juypterscripting environment that has been built into ArcGIS.

• Let’s break this down…

23

24

11/18/2019

12

Python is one of the most widely used programming languages. It is easy to learn and interpret and can be applied to many disciplines.

Python

Web GIS Administration

Content Management

Analysis and Data Science

Python + GIS use cases

Python Notebook / Jupyter

• Jupyter is open source software designed to support interactive data science and scientific computing.

• It is browser based and lets you write, run, and document your code all in one place.

• Esri has included Jupyter alongside ArcGIS Pro for several years and the ArcGIS API for Python was developed to make the most of Jupyter’s functionality.

25

26

11/18/2019

13

ArcGIS Notebooks sits at the intersection of ArcGIS and open data science

*Tools only available with a license of GeoAnalytics Server and Image Server, respectively

ArcGIS

Open SourcePython & Data Science

• ArcGIS API for Python• ArcPy• Analytics

• Geoprocessing • Web GIS analysis• GeoAnalytics*• Raster Analytics*

How is ArcGIS Notebooks different than “traditional” Jupyter?

• Built within ArcGIS rather than sitting beside it

• Leverages an integrated UI

- Automatically insert Python snippets

- Quickly access a gallery of samples

• Notebooks as items

• Use ArcPy without ArcGIS Pro

• Centrally manage libraries and packages

27

28

11/18/2019

14

- ArcGIS Pro

- R-ArcGIS Bridge, R Notebooks

- ArcPy Spatial machine learning

- Machine learning GP tools

- Deep learning tools for imagery and rasters

- ArcGIS Notebooks (coming soon)

- ArcGIS Enterprise

- GeoAnalytics Server

- Custom Python tools

- Machine learning tools

- Raster Analytics (Image Server)

- Deep learning

- Raster functions

- ArcGIS Notebook Server

- ArcGIS Online

- ArcGIS Notebooks (now in beta)

- ArcGIS API for Python

- arcgis.learn

Spatial data science across ArcGIS

ArcGIS Notebooks

29

30

11/18/2019

15

Features of ArcGIS Notebooks - Recap

• Jupyter interface is integrated with ArcGIS

- Use UI to browse for analysis tools and data and insert automatic Python snippets

• Access to content in your organization, the Living Atlas, and ArcGIS Online

• Create and run scripts with ArcPy

• Private workspace for data, interim results, and scratch environments

• Python libraries and packages are synchronized across the organization

- Customize what comes out of the box by modifying the notebook runtime

• Python notebooks are saved as items

- There is a new item type called Notebook

• Read-only previews of notebooks

• Secure identity-based access

• Esri Sample Notebooks

…and more

Esri sample notebooks

31

32

11/18/2019

16

Notebook Samples

• ArcGIS Notebooks includes end-to-end sample notebooks for:• Web GIS administration• Content management• Data Science and Analysis

• Samples can be used to learn something new or to jumpstart your own analysis

• Some samples are ready-to-run, while others require you to configure parameters

Spatial Analysis using Python

33

34

11/18/2019

17

Why Python?

• Popularity

• Productivity

• Interoperability

• Solves the “two-language” problem

• Scientific Python ecosystem

• Community

Source: Gateway Data Sciences Courses Reach Enrollment Milestones

35

36

11/18/2019

18

ArcGIS API for Python

gis

geometry

network

schematics

features

realtime

widgets

mapping

env

geocoding

geoenrichment

geoprocessing

raster

geoanalytics

Spatially Enabled

DataFrame

37

38

11/18/2019

19

39

40

11/18/2019

20

41

42

11/18/2019

21

Spatial Analysis Tools

• Summarize Data

• Find Locations

• Data Enrichment

• Analyze Patterns

• Use Proximity

• Manage Data

Summarize DataCalculate summary statistics for features and attributes

43

44

11/18/2019

22

Add detailed demographic data and statistics to your analysis

Enrich Layer

“What is near what?”

Use Proximity

45

46

11/18/2019

23

Use Proximity“What is near what?”

Manage DataManage geographic data, overlay layers

47

48

11/18/2019

24

Analyze PatternsIdentify, quantify, and visualize spatial patterns in your data.

Geocoding, Batch geocoding, reverse geocodingarcgis.geocoding module

Geocoding

49

50

11/18/2019

25

GeocodeSingle line or multi field addresses

GeocodePoints of Interest

• Cultural or geographic landmarks

• Businesses by name or category

• Administrative divisions

51

52

11/18/2019

26

Batch GeocodingGeocode an entire list of single line or multi field addresses

Reverse GeocodeDetermines address at a particular x/y location

53

54

11/18/2019

27

Routing and directions, location allocation, service areas…

arcgis.network.analysis module

Network Analysis

Network analysis capabilities and toolsCommercial grade, traffic aware routing and directions for multiple travel modes

55

56

11/18/2019

28

Network analysis – Service Areas

Enrich your analysis with demographic and business data

arcgis.geoenrichment module

GeoEnrichment

57

58

11/18/2019

29

GeoEnrichment

• Get facts about a location or area

- Street addresses

- Points, lines and polygon geometries

- Within a drive time or service area

- Named geographical areas

- Counties or block groups in California

- Districts and subdistricts in India

• Create charts and choropleth maps

GeoEnrichment – create reports

59

60

11/18/2019

30

Fast distributed spatio-temporal analysis of large vector and tabular data

arcgis.geoanalytics module

GeoAnalytics

GeoAnalytics Tools

• Summarize Data

• Manage Data

• Use Proximity

• Find Locations

• Analyze Patterns

61

62

11/18/2019

31

DemoAnalysis of crime patterns in Houston

63

64

11/18/2019

32

65

66

11/18/2019

33

Imagery and raster analysis

arcgis.raster module

Raster Analysis

67

68

11/18/2019

34

Image Processing

• Correction

• geometric and radiometric corrections

• the curved shape of the Earth,

• the imperfectly transparent atmosphere,

• daily and seasonal variations in the amount of solar radiation

• imperfections in scanning instruments, among other things.

• Visualization and Appearance

• brightness and contrast

• sharpen, blur, detects edges

• pansharpen

Image Processing (cont.)

• Band Math and Indices

• Image indices are computed from multiband images to emphasize a specific phenomenon.

• The normalized difference vegetation index (NDVI) is a standardized index allowing you to display greenness, also known as relative biomass.

• High values represent forest areas and lush vegetation.

• Moderate values represent areas of shrubs and grassland

• Very low values represent areas of little to no vegetation, such as concrete, rock, or bare soil.

• Extremely low or negative values represent areas with no vegetation at all, such as cloud, water, snow.

• The NDVI Colorized function applies the NDVI function on the input image, and then uses a color map or color ramp to display the result.

69

70

11/18/2019

35

Image Processing (cont.)

• Image Segmentation and Classification

• Perform object based image classification on image segments or pixels using a variety of supervised and unsupervised techniques through an easy-to-use wizard workflow.

• Supervised/unsupervised classification

• Object based and pixel based segmentation

• Accuracy assessment

• SVM, random trees, ISO clustering, MLC

Raster Analysis

• Distance Analysis

• distance, cost distance, shortest path, optimal travel path…

• Suitability Modeling

• Determine an ideal location for operations

• Combine categorial (land use, soil type) and continuous (slope, property value) variables to create a suitability map.

• Hydrological Analysis

• surface flow patterns, create watersheds, stream networks, flow distance, and other hydrological characteristics. Summarize landscape data as input to hydrologic and hydraulic models, as well as flood inundation and erosion models.

71

72

11/18/2019

36

Raster Analysis (contd.)

• Interpolation and Density

• Predicts values and creates a continuous surface from sample data points, such as elevation, rainfall, chemical concentrations, and noise levels.

• Derivation of land surface characteristics

• Create hillshade, slope, aspect, curvature, solar insolation, and visibility to help solve spatial problems

• Statistical Analysis

• Perform calculations on a per-cell basis between multiple datasets, based on local, neighborhood, or zonal functions.

• Generalization

• Either clean up small erroneous data in the raster or generalize the data to get rid of unnecessary detail for a more general analysis.

• Map Algebra

• An easy-to-use and powerful way to perform raster analysis as algebraic expressions.

Image Processing & Raster Analysis Functions

Math

CalculatorAbsDivideExpExp10Exp2FloatIntLn, Log10 Log2MinusModNegatePlusPowerRound DownRound UpSquareSquare RootTimes

ACosACosHASinASinHATanATan2ATanHCosCosHSinSinHTanTanH

ConSet Null

Data Management & Conversion

Raster to VectorVector to RasterColormapColormap To RGBComplexGrayscaleRemap / Reclass Spectral ConversionUnit Conversion Vector FieldLAS to RasterLAS Dataset to RasterClip CompositeExtract BandsMaskMosaic RastersRasterize FeaturesReprojectNibble

Interpolation

Interpolate Irregular DataNearest NeighborIDWEBKSwath

Correction

Apparent ReflectanceGeometric CorrectionSpeckle Filtering (Lee,Frost,Kuan)Thermal NoiseRadiometric Calibration

Analysis: Image Segmentation & Classification

Segmentation (Mean Shift)Training (ISO, SVM, ML)Supervised Classification

Visualization & Appearance

Contrast and Brightness Convolution Pansharpening Resample Statistics and Histogram Stretch

Surface Generation & Analysis

Aspect Curvature Elevation Void Fill Hillshade Shaded Relief Slope ViewshedContour

Analysis: Overlay

Weighted OverlayWeighted Sum

Analysis: Band Math & Indices

NDVI / NDVI Colorized SAVI / MSAVI / TSAVIGEMIGVI (Landsat TM)PVITasseled Cap (Kauth-Thomas)Binary Thresholding

Analysis: Distance & Density

Euclidean Distance Cost DistanceLeast Cost PathKernel Density

Statistics:

Zonal StatisticsCell StatisticsArgStatistics

Python

Custom Algorithms

Analysis: Hydrology

FillFlow AccumulationFlow DirectionFlow DistanceStream LinkWatershed

Multiband Math

Arithmetic Band Arithmetic

BitwiseAndLeft ShiftNotOrRight ShiftXor

BooleanAndNotOrXor

Equal ToGreater ThanGreater Than EqualIs Null Less ThanLess Than EqualNot Equal

73

74

11/18/2019

37

Applications

Effects of Climate Change

Clark Labs

Change Analysis

DigitalGlobe

Forest Acidification

Midwestern US Loss of Habitat

Central Africa

Blue Raster

Farm Management

New Zealand

Crop Rotation

USA

FEMA

Assessing Vulnerable Populations

Wildfire Risk

FranceCaroviz

Volcanic Monitoring

JapanJAXA

Watersheds

arcgis.learn moduleDeep Learning using ArcGIS Python API

75

76

11/18/2019

38

arcgis.learn

The arcgis.learn module in ArcGIS API for Python enables GIS analysts and data scientists to easily adopt and apply deep learning in their workflows.

It enables training state-of-the-art deep learning models with a simple, intuitive API.

Beyond Detections: End to End GeoAI Lifecycle enabled by ArcGIS Imagery AI Capabilities

ImageryAccess

Imagery Prep

Training Data Prep

Train & ConsumeModels

Run Inference at SCALE

Feedback Loop

TakeAction

Deploy Models to Production

Before After

• Installing External DL

Frameworks

• Dozens of lines of

Code

• HARD!

• No Installation

(Notebooks)

• 3-5 lines

• EASY

77

78

11/18/2019

39

Not just “Training”!

Exporting Training Data arcgis.learn.export_training_data

Training DL Modelsarcgis.learn.SingleShotDetectorarcgis.learn.UnetClassifierarcgis.learn.FeatureClassifierarcgis.learn.PSPNetClassifierarcgis.learn.RetinaNetarcgis.learn.MaskRCNNarcgis.learn.EntityRecognizer

Preparing Data (Augmentation)arcgis.learn.prepare_data

Model Managementarcgis.learn.list_models

arcgis.learn.Model

Model.install

Model.uninstall

Model.query_info

Inference APIsarcgis.learn.detect_objects

arcgis.learn.classify_pixels

Things you can do today with ArcGIS.LearnObject Detection using SSD, Pixel Classification using Unet, Feature Classification

Damaged Structures

Roads

Swimming Pools

Building Footprints

Oil Pads

Land Cover

Road Cracks

Cars Palm Trees

Pipeline Encroachment

79

80

11/18/2019

40

Geospatial Deep Learning

81

82

11/18/2019

41

Why use Deep Learning for Imagery

ImageNet Visual Recognition Challenge error rate

• Better than human accuracy at vision tasks

Deep Learning Workflowin ArcGIS

83

84

11/18/2019

42

Train DL ModelTrain DL Model

Detect ObjectsDetect Objects

Export

Training

Data For DL

Export

Training

Data For DL ModelModel

Training SamplesTraining Samples

ArcGIS Pro ArcGIS Proarcgis.learn

Classify Pixels

Collect Samples Export Training

SamplesTrain

Perform Inference

ArcGIS Enterprise ArcGIS Enterprise

ArcGIS Deep Learning Workflow

Data Labeling: Training Samples Manager

• Add Labels

• Quickly Collect Samples

• Save Samples to a Feature Class

Collect Samples

Export TrainingSamples

Train Perform Inference

85

86

11/18/2019

43

Export Training Data for Deep Learning Tool

Collect Samples


TrainPerform Inference

• Exports Samples to Training Images• Each Image has Labels• Supports various formats

Train Model

• arcgis.learn module in ArcGIS API for Python

- No installation (ArcGIS Notebooks)

- Easy to use - 3-5 lines of code

- Models:

- Object Detection

- Pixel Classification

- Feature Classification

• External Deep Learning Frameworks

- Tensorflow

- PyTorch…

Collect Samples



87

88

11/18/2019

44

Consume Deep Learning Models

Perform Inference

Model DefinitionModel Definition

ArcGIS UserArcGIS User

Inference results

Inference results

Input ImagesInput Images

InferenceTools

InferenceTools

• ArcGIS Image Analyst in Pro

• ArcGIS Image Server on Enterprise

Inference ToolsInference Tools

• Classify Pixels Using Deep Learning

• Object Detection Using Deep Learning

Non Maximum Suppression

Collect Samples



Classify Pixels Using Deep Learning

Runs the model on an input raster to product a classified

raster, each valid pixel has an assigned class label.

• Mini-batch support

• Processor type: CPU or GPU

• Parallel processing in ArcGIS Pro and distributed

raster analysis on Enterprise

Inference Tools



89

90

11/18/2019

45

Detect Objects Using Deep Learning

Runs the model on an input raster to produce a feature class

containing the objects it finds.

• Batch processing

• Optional Non Maximum Suppression

• Processor type: CPU or GPU

• Parallel processing in Pro and distributed raster analysis

on Enterprise

Inference Tools



ArcGIS in use for each step of the deep learning workflow

End-to-end from raw imagery to structured information products

ArcGIS Deep Learning Workflow

Imagery Labelling Data Prep

Train Model

Detect Objects

AnalysisField

Mobility, Monitoring

92

93

11/18/2019

46

Computer Vision TasksApplied to GIS

Semantic SegmentationAssign a label to each pixel

Cat

Ground

Sky

Turf/Grass

Building

Water

Pixel Classification

Applications:

- Land Cover Classification

- Pervious/Impervious mapping…

Model Architectures:

- U-Net

- PS Net…

94

95

11/18/2019

47

Object DetectionFind objects and their location (bounding boxes)

Applications:

- Detect trees, cars, airplanes, …


- Single Shot Detector (SSD)

- YOLO

- RetinaNet

Image ClassificationAssign a label to a given image

Cat

Applications:

- Damaged building classification

- Clean or ‘green’ pools…

Undamaged Damaged

Feature ClassificationAssign a label to a given feature


- Inception

- ResNet

- VGG…

96

97

11/18/2019

48

Instance SegmentationFind objects and their precise locations (masks or polygonal features)

Applications:

- Building footprint extraction


- Mask-RCNN

- DeepMask…

Impervious Surface Classification

Coconut Tree Detection

Building Footprint Extraction

Damaged House Classification

Applications of Deep Learning to Imagery

Pixel Classification Object Detection Instance Segmentation Image Classification

End to End Deep Learning – Wide spectrum of deep learning models

98

99

11/18/2019

49

Scalable Deep Learning

60,000 buildings

ArcGIS Pro: 1 GP100 GPU (16 GB):

4.5 hours

ArcGIS Enterprise:

4 nodes RA server with 3 x P40

GPUs (24GB):

20 minutes

100

101

11/18/2019

50

Geo-AI projects showcase

ETA PredictionPredict Arrival Times accurately to Optimize Supply Chain & Boost Customer Satisfaction

102

103

11/18/2019

51

What if we want to accurately Predict Arrival Times per Fleet Car, taking into consideration:

• Weekend VS Weekday

• Winter VS Summer

• Holidays VS Traditional Days

• Rush-Hour VS Traditional Hours

• Type of Vehicle

• Time of the Day

• Type of Products

• .. Other variables

Traditional Rout-Solvers wan’t do the job, we need Machine Learning to:

• Mine the deep patterns in historical delivery data (Origin, Destination, Total Time, Time o the Day, Vehicle Type, Weather, Type of Products..etc)

• Build Predictive Models that take such seasonality and patterns into consideration, to predict arrival times with high accuracy

ETA Prediction

Nvidia GPU:

2560 CUDA cores

300K Predicted

Points/Second

(VS 4-5 points/sec NA)

Network Analyst (Blue) VS Predictions (Red)

Training Data: 320M Network Analyst OD Points

104

105

11/18/2019

52

106

107

Documents

Geospatial Data Science & Artificial Intelligence€¦ · In ArcGIS: Spatially Constrained Multivariate Clustering, Multivariate Clustering, Density-based Clustering, Image Segmentation,