Bulk Interpolation using R Environment

Jiří Kadlec – Aalto University, Finland Pavel Treml – Masaryk Water Research Institute and Charles University, Czech Republic

20.9.2013 FOSS4G Conference - Nottingham

Fields with Observations in time and spaceWhat, Where, When (irregular sampling)

Space, S

Time, T

Variables, V

s

t

Vi

Dc

“Where”

“What”

“When”

A data value

4.2

Praha

2013-09-20

Air Temperature (C)

Image created by CUAHSI, 2010

Type of data in this tutorial

• Tens of variables• Hundreds of stations• Hundreds of observations

per station• Incomplete data

Observationsstation longitude latitude variable time value

Praha 14.430 50.0156 temperature 2013-03-31 -0.2

Praha 14.430 50.0156 temperature 2013-04-01 1.1

Praha 14.430 50.0156 snow 2013-04-01 3.0

Brno 16.254 49.147 temperature 2013-03-31 -0.1

Brno 16.254 49.147 snow 2013-03-01 12.0

TASKFor each time-step, Examine how one orMore variables changeIn space

Interpolation• Deterministic methods• Geostatistical methods• Repeated over many time steps

• How to automate?

?

time

value lat

lon

?

Interpolation - Time Interpolation - Space

the R EnvironmentFree statistical softwareWindows, Mac, LinuxCreate High-quality graphics

Many input data formats– Text file – Vector data (shapefile)– Raster data (grid)

Many output data formats– Picture– Text file– Raster, vector

Scripting language for automating repeated tasks

Case study: Maps of air temperature andsnow, Czech Republic

• Year 2013 (365 maps)

• Require identical color-scale

• Need to show rivers, boundaries

• Need to show point values (to assess interpolation)

Load Data sets

Select next time

Select matching

observations

Run Interpolation

Create mapcartography

More times

?END

START

NO

YES

Load Data Sets – Raster (SRTM elevation)Library(“sp”)library(“raster")library(“rcolorBrewer")

srtm <- raster("srtm_utm.asc")

colors=brewer.pal(6, "YlOrRd")intervals = c(0, 250, 500, 750, 1000, 1600)

spplot(srtm_proj, col.regions=colors, at=intervals)

Select packagesWith needed functions

Color ramp setting

Raster:Load from local file

Visualize raster usingspplot

Note: You can get DEM for your area in R using:

Load Data Sets – Vector (boundaries, rivers)library(“maptools")library(“sp")

border = readShapeLines(“border.shp")border_layer = list("sp.lines", border, lwd=2.0,col=“black")

rivers = readShapeLines(“rivers.shp")proj4string(rivers) <- CRS("+proj=krovak +lat_0=49.5 +lon_0=24.83333333333333")rivers_proj <- spTransform(rivers, CRS("+proj=utm +zone=33"))river_layer = list("sp.lines", rivers_proj, lwd=1,col=“blue")

layout = list(river_layer, border_layer) spplot(srtm, sp.layout=layout)

Select packagesWith needed functions

Rivers:Load shapefile,Reproject vector fromKrovak to UTM system

Borders:Load shapefile

Visualize vector datasetson top of raster

Load Text File – Point Data (Stations)At first, we can use a subset (for first date/time in the dataset)

data = read.table(“data.txt”, header = TRUE, sep = "\t", dec = ".")

st = subset(data, DateTimeUTC == ‘2012-08-12’ & VariableCode==“SNOW")

Coordinates(st) = ~Long + Latproj4string(st) = CRS("+init=epsg:4326")stations <- spTransform(st, CRS("+proj=utm +zone=33"))

stations_layer = list("sp.points", stations, pch=19, cex=1.0, col="black")

labels = list("panel.text", coordinates(stations)[,1], coordinates(stations)[,2], labels=stations$value, col="black", font=1, pos=2)layout = list(stations_layer, labels)

Reproject from Lat/Lon to UTM system

All Layers together (raster, vector, points)(We use different color ramp in this example)

Interpolation: IDW and Kriging Methods

Interpolation: Covariate (Elevation)model = lm(value ~ elev, data)intercept = model$coefficients[1]slope = model$coefficients[2]

plot(value~elev, data)abline(model)

TEMP = a * ELEVATION + b

In our case temperature is often negativelycorrelated with elevation

Interpolation: Elevation as covariate

TEMP = a * ELEVATION + b

residuals

Instead of interpolating temperature directly,we create grid using regression equation andwe only interpolate the residuals

Using TEMP = a * ELEVATION + b

Interpolated residuals

Combination (regression + interpol.residuals

Color Breaks, Color Ramps

#user-defined color breakscolors = rev(brewer.pal(8, "YlGnBu"))

brk <- c(-40,-35,-30,-25,-20,-15,-10,-5,0)

plot(grid,breaks=brk,col=palette(colors)

One color ramp and set of user-defined color breaks easily re-used for different grids

RColorBrewer packages providesPre-defined color ramps

Example Final Map: One time stepCombines the previous steps …

Snow depth (cm): 2013-01-01

Saving map to file

figure = “2012-02-07.jpg”

png(filename = figure, width = 1500, height = 1000, pointsize = 25, quality = 100, bg = "white", res = 150)

DO THE PLOT COMMANDS HERE

dev.off()

• Set image size• Set image resolution• Other options (margins, multiple maps in one image)

PNG pictureJPEG picturePDF fileWMF file.asc ascii grid file (for GIS softwares)……..

Bulk Interpolation: “FOR” loop(multiple time steps)

for (j in 1:length(timesteps)) {

# select subset of observations # run interpolation # create map # export map to file (picture, pdf, …)

}

Schema of the Loop Load Data sets

Select next time

Select matching

observations

Run Interpolation

Create mapcartography

More times

?END

START

NO

YES

Final Map: Multiple time steps

R as Compared with Desktop GIS

R Statistical Software EnvironmentDesktop GIS

Maps are highly interactive

Create small number of maps withgraphical user interface

Automated cartography (label placement…)

Map-Layout, model builder tools

ArcGIS, QGIS, SAGA, MapWindow,…IDV, Panoply, …

(-) Maps are static

(-) Some commands counter-intuitive for novice user

(+) Create very large number of mapswith same cartographic symbology

(+) Task is easy to automate and reproducible

(+) Good documentation and large user base

Desktop GIS: project file (.mxd, .qpj, …) R: script (.R)

References

• Bivand, Roger S., Edzer J. Pebesma, and Virgilio Gómez Rubio. Applied spatial data: analysis with R. Springer, 2008.

• Hengl, Tomislav. A practical guide to geostatistical mapping of environmental variables. Vol. 140. No. 4. 2007.

BOOKS

• www.spatial-analyst.net• www.r-project.org• http://hydrodata.info/api/ (source of data in our demos)

WEBSITES

R-PACKAGES USED

RColorBrewer, maptools, rgdal, sp, raster

Try the scripts yourself!

hydrodata.info/R/

• Sample scripts for bulk interpolation• This presentation• Source data { central Europe meteorological

observations }