Embed Size (px)
Citation preview
Mansour Raad [email protected] Kim [email protected]
Geospatial Analytics and AI at Scalewith Big Data Toolkit
• Esri Professional Services solution that allows customers to analyze, aggregate, and enrich big data within their existing big data analytics platform
• [email protected] for inquiries
What is Big Data Toolkit (BDT)?
Target Audience
Unifying Big Data Analytics Platform + ArcGIS Platform together
HDFSHiveSpark Massive Analytics
Dissemination
Advanced AnalyticsBig Data Toolkit
• No Spatial Indexing
ArcGIS Enterprise
Spatiotemporal
Enterprise Geodatabase
OMG!Really?
Automatic publishing of Map/Feature services
……
Open-sourcegeo tools
• No Input Data Prep / Movement
Capabilities
Nearest Coordinate
Point in Polygon
Distance Relation
GeneralizeAdd
Calculate Field
Web Mercator
Hex
Extent Filter
Point Exclude
Feature Filter
Point Include
Smart
Snapping(Map
Matching)
Time Filter
Project
Stats
Delete
Dissolve
Geocoding & Reverse
Geocoding
Routing(requires ArcGIS
Enterprise)
Service Area
Calculation(requires ArcGIS
Enterprise)
Smart Similarity Analysis
Standard Distance
Clustering
GWR
Moving Averages Clip Future
Capability
Geodetic Area
Simplify
Intersection
SQL
File Geodataba
se
Shapefiles
Enterprise
Geodatabase
csv tsv
parquet
Hive
Enterprise Geodatabase
Native Geometry
Format
ArcGIS
Spatiotemporal
Big Data Store
csv
tsvparquet
Sources Processors Sinks
Smart Snapping (Map Matching)
Big Data Toolkit Stack
- NYC Taxi 35M- ZIPCODE 360 Polygons with 2100+ Demographic Variables
- Add New Column- Point-in-Polygon- Summary Statistics based on 200m & 500m hexagons- Persist to Enterprise Geodatabase (SQL Server)- Visualize Hexagon Aggregations in ArcGIS
Demo 1
- Sneak Peek at New Big Data Toolkit 2.0- New Target Audience: Data Scientists- Use Big Data Toolkit in Notebook
Demo 2
• RDD -> DataFrame, SQL• Project Tungsten
- Memory Management and Binary Processing- Cache-aware computation- Code generation- https://databricks.com/blog/2015/04/28/project-tungsten-bringing-spark-closer-to-bare-
metal.html
New Big Data Tookit v2.0
