GeoSpatial Tools and AnalysisDavid Currie

GeoSpatial Tools and Analysis

Data for Good Sept 23, 2014

• Focus on open source tools• Bias towards remote sensing topics• Some examples from completed projects

David Currie, P.Eng

Open Source Geo Tools

A good place to start: http://www.osgeo.org/

• GUI tools for GIS and Image Analysis

• QGIS, OpenJUMP, Grass

• Command line utilities

• GDAL/OGR

• GEOS

• Proj4

• Ossim

• Databases and Web Services

• PostGIS/PostgreSQL

• Mapserver, OpenLayers/GeoMoose, Leaflet

1

• Django/GeoDjango

• Scripting tools

• R

• Python

Quantum GIS (QGIS)

• Useful for Raster and Vector Exploration• Basic mapping capabilities• Scriptable using Python

Grass

• Very capable raster and vector processing• Good performance• Steep learning curve

GDAL/OGR

http://www.gdal.org/

• Raster and Vector format translations• Datum and projection transformations• many useful utilities, eg gdalwarp

2

GEOS

http://trac.osgeo.org/geos/

• topology library• computes useful things like INTERSECTION and CONVEXHULL• included in PostGIS

Web Services

• Mapserver• custom map generation on demand

3

• can also function as a tileserver• GeoDjango• Python based CMS with PostGIS in the background• Client-side tools (javascript)• Openlayers/GeoMoose• Leaflet

PostGIS

• Spatial engine running on PostgreSQL• handles vector and raster data• provides Proj4 and GEOS functionality for projections and topology

library("RPostgreSQL")v<-dbSendQuery(con,"select gid, length, width from feature_polygon where featuretype_id=22")exfeat<-fetch(v,n=-1)hist(exfeat$length,main="Length of Extreme Features",xlab="Length (m)",breaks="FD",col="lightblue")

Length of Extreme Features

Length (m)

Fre

quen

cy

0 10000 20000 30000

050

150

4

Scripting: R

• R-project http://www.r-project.org/• free and open source• powerful statistical capabilities• package system to link external capabilities, including GDAL, GEOS, GRASS, Raster, Spatial, SQL . . .

- RStudio IDE http://www.rstudio.com/

5

Scripting: Python

• free and open source https://www.python.org/• lightweight, useful for web servers• lower level programming interface than R, can give higher performance

6

• useful GIS libraries available: fiona, shapely, pyProj, Rtree

Open Geo Data

Worldwide Data - OpenStreetMap http://www.openstreetmap.org/ - Natural Earth http://www.naturalearthdata.com/ - NASA http://visibleearth.nasa.gov/ - USGS http://www.usgs.gov/

Canada Specific - GeoBase http://www.geobase.ca/ - GeoGratis http://geogratis.cgdi.gc.ca/ - Altalis http://www.altalis.com/ - Calgary Open Datahttp://data.calgary.ca/

7

8

Project Examples: Web Data Capture

The problem: capture dimensions of thousands of sea ice features from hundreds of images in order to quantify the variation in size by season andlocation.

The deliverable: Probability of exceedence statistics for each zone and season

9

Area 2 − Iceberg

Diameter (m) Note Log Scale

Exc

eede

nce

Pro

babi

lity

n:6470 m:0

10 100 1000

0.0

0.4

0.8

Web Data Capture

Tools

• PostGIS/PostgreSQL• Python with GDAL for data loading• Grass for image enhancement• Mapserver with GeoMoose for web based digitizing• Django for project administration and reporting• R and QGIS for generation of results

10

Web Data Capture

11

12

Web Data Capture

13

14

Web Data Capture

15

Web Data Capture

Results: - Selected 408 images from a set of 2073 (Landsat and Aster) - 45,578 icebergs detected and measured - 3,292 other significant featuresdetected - Multiple dates between 1999 - 2011 allowed movement to be detected and mapped

16

Project Examples: Raster Data Mining

17

The problem: Arctic polynyas are ephemeral features that are critical to wildlife and human survival. Where and when they occur is difficult to mapdue to access and weather conditions.

The deliverable: Spatio-temporal analysis of open water occurrence for period July 2002 - June 2013

Tools:- R with spatial and raster packages

18

Raster Data Mining

Starting from MODIS sea surface temperature grids. - 1 km resolution - Two grids produced per day

19

Raster Data Mining

• Day and Night grids give up to 6 observations of each square kilometer• Cloud and lack of sunlight affect available data

20

21

Raster Data Mining

• Over 110K images collected, registered, classified, and stacked

22

23

Raster Data Mining

• Although daily data is available, the coverage was not continuous• Daily data was stacked and filtered to generate a weekly time series• A set of open water composite maps were generated for each week of the 11 year span

24

25

Raster Data Mining

• Landsat imagery was used to tune the analysis and quality control the result.

26

Raster Data Mining

• Summary statistics from the resulting grids were extracted and analyzed• Open water fraction gives a normalized comparison

27

28

Raster Data Mining

• Polynyas are most easily recognizable in the early Spring• Comparing maps from specific week between years gives a good indication of variation

29

30

31

Raster Data Mining

• Specific locations were of interest, including the Last Ice Area

32

33

34

Thanks!

35