Introducing GSIF (seminar at Lamont campus)

Global Soil Information Facilities

(A methodological framework for Open Soil

Information)

Tomislav Hengl

ISRIC � World Soil Information, Wageningen University

Seminar at CIESIN, Sept 14 2011

Key issues

I What do we know about world soils?

I What do you know about the GlobalSoilMap.net project?

I How to produce complete GlobalSoilMape.net propertymaps?

I How will soil information �t into the Global LandInformation System?


My backgrounds

I Senior researcher at ISRIC � World Soil Information;

I PhD in pedometric mapping @ ITC (GIS institute in Enschede)in 2003;

I 2 years university assistant; 2.5 years JRC Ispra; 2 yearsUniversity of Amsterdam;

I My expertise: Geostatistics, Digital Soil Mapping, spatialdata analysis, geomorphometry (vice-chair);

I Global Soil Information Facilities


http://globalsoilmap.net/data/poster_GSIF.pdf

My publications


http://spatial-analyst.net/wiki/index.php?title=Publications

What am I doing in USA?


AfricaSoils.net

Thank you!

1. Markus Walsh (Keith Shepherd)

2. Sonya Ahamed & Pedro Sanchez


My main inspirations / principles of work

I Open Source software for education and research

I Crowd sourcing systems for environmental datacollection

I Publicly accessible (soil) data products


Important assumptions

My research philosophy

is based on 4 important assumptions:


Assumption #1

Humans (companies and governments)

need to be closely monitored


Did you know?

I Global biodiversity has been heavily degraded due to humanactivities. The Living planet index has dropped from 1970sto 60% and will continue to do so (source: MillenniumAssessment project).

I By 2048 we will run out of �sh (your children will leave on aplanet where there are hardly any visible �sh in the oceans).

I 15-35% of global irrigation withdrawals are estimated to beunsustainable (source: WBCSD).

I Every year, 9.4 million ha of forests are lost (source: FAO�World agriculture: towards 2015/2030�).


http://www.millenniumassessment.org/proxy/Document.354.aspx

http://www.millenniumassessment.org/proxy/Document.354.aspx

http://endoftheline.com/campaign/fish_facts/

http://www.wbcsd.org

Population trends


Decline of species (biodiversity)


Forests and croplands


Assumption #2

Soils (and hence information on soils)

will become more and more important


Critical areas (irrigation)


http://www.wbcsd.org

Food price index (FAO)


http://www.fao.org/worldfoodsituation/wfs-home/foodpricesindex/en/

Soil threats

Soils are also more important because we are slowly loosing them:

I 305 million ha of land has been completely degraded (nolonger suitable for agriculture).

I 10-50% irrigated land a�ected by salinization (source:GLASOD).

I For a forest to return takes maybe 100 years; it takes100�400 years to produce 1 cm of topsoil � are soilsrenewable resource at all?


http://ec.europa.eu/agriculture/envir/report/en/inter_en/report.htm

http://ec.europa.eu/agriculture/envir/report/en/inter_en/report.htm

Soils might become precious in future

Reports by FAO (2002) show that, in future, 80 percent ofincreased crop production in developing countries will have to comefrom intensi�cation � higher yields, increased multiple croppingand shorter fallow periods.Any agricultural or environmental management modelrequires soil data as an input to estimation of yields, waterand nutrient dynamics.World demand for cereals has jumped from 39 million tones (in1970) to 103 million tones (in 2000) (source: FAO �Worldagriculture: towards 2015/2030�).


Assumption #3

Soil Information (global)

is one of the poorest GIS layers


What do we know about world soils?

I Harmonized World Soil Database: 1 km resolution griddedsoil property maps (16 properties for top and sub-surface soil).

I 1:5M scale FAO-UNESCO Soil Map of the Word: fromwhich ISRIC has produced 5 by 5 arc-minutes global soilproperty maps (for 0�20, 20�40, 40�60, 60�80 and80�100 cm) in combination with the ISRIC-WISE soil pro�ledatabase.

I The Distributed Active Archive Center (DAAC) soilproperty maps

I USGS-produced soil property maps

I Atlas of the Biosphere soil maps


http://www.fao.org/nr/water/news/soil-db.html

http://globalsoilmap.net/data/HWSD_.zip

http://daac.ornl.gov/SOILS/guides/igbp-surfaces.html

http://daac.ornl.gov/SOILS/guides/igbp-surfaces.html

http://soils.usda.gov/use/worldsoils/mapindex/

http://www.sage.wisc.edu/atlas/data.php

HWSD vs GlobCov

GlobCover HWSD


http://commons.wikimedia.org/wiki/Category:Publicly_available_global_data_sets

Should soils follow political boundaries?


HWSD vs ISRIC SIS (753 pro�les)


http://soilprofiles.org/sites/default/files/isis_sampling_bias.zip

The agreement plot (kappa <10%)


Assumption #4

Global Resource Planning System

can do much better than a local one


GLIS

GLOBAL

LAND INFORMATION

SYSTEM

Soil properties (soil information system)

- physical and chemical soil properties, nutrient

capacity, water storage, acidity/salinity…

Live weather channel (meteorological forecasting)

- anticipated temperature (min, max), rainfall, frost

hazard, drought hazard, flood hazard…

Plant monitoring channel (MODIS/ENVISAT)

- current biomass production, biomass anomalies

(pest and diseases), plant health…

Socio-economic data (site-specific)

- administrative units, new laws and regulations,

market activity, closest offices, agro-dealers…

Spatial location (site)

Query site

attributes

Information

incorrect?Update with

ground truth data

Fertilization

Irrigation

Pest treatment

Best crop calendar

Yield estimates

Environmental risks

Suggest the best

land use practice

Model library


GRMS (see �Zeitgeist moving forward� 1:34h)


http://www.youtube.com/watch?v=4Z9WVZddH9w

GlobalSoilMap.net

I An international initiative to make soil property maps (7+3) atsix depths at 3 arcsecs (100 m).

I the leitmotif is to �assemble, collate, and rescue as much of

the worlds existing soil data� ;

I The soil-equivalent of the OneGeology.org, GBIF, GlobCoverand similar projects.

I The biggest DSM project ever!


http://onegeology.org

http://data.gbif.org

http://ionia1.esrin.esa.int/

GlobalSoilMap.net in comparison with other projects

SRTM GADM

1990 1995 2000 2005 2010 2015 2020

2.0

2.5

3.0

3.5

4.0

Year

Re

so

lutio

n (

m)

in lo

g-s

ca

le

GPWv3

MOD13C2

MOD12C1

CHLO/SST

GLWD

DMSP-OLSv4

WorldClim

GlobCov2

FRA

5.6 km

HWSDv1EcoRegions

GlobalSoilMap?

OneGeology?


World soils in numbers

I Total land area: 14.8 billion ha

I Estimated total productive soil area: 10.9 billion ha (73.6%)

I Drylands (deserts, semi-deserts): 3.6 billion ha (24.3%)

I Wetlands (swamps, marshes, and bogs): 440 million ha (3%)

I Arable and permanent crops: 1.5 billion ha (11%)

I Potential areas suitable in varying degrees for the rainfedproduction of arable and permanent crops: 2.8 billion ha


Global Soil Mapping (in numbers)

I The total productive soil areas: about 104 million squarekm.

I To map the world at 100 m (1:200k), would cost about5 billion EUR (0.5 EUR per ha) using traditional methods.

I We would require some 65M pro�les according to the strictrules of Avery (1987).

I World map at 0.008333333 arcdegrees (ca.1 km) resolution isan image of size 43,200Ö21,600 pixels.

I 27 billion pixels needed to represent the whole world in100 m (productive soil areas).






























Productive soil areas

Figure: Soil productive area mask derived using the MODIS LAI images.Projected in the Transverse Mercator system used e.g.in Google Maps.


Maybe GlobalSoilMap.net will not cost as much?

Technology might be the solution!

I Automated mapping

I Global soil covariates � SRTM DEM GDEM TanDEM-X,MODIS LST, Meteo images (SMOS), TRMM

I Downscaling methods

I Soil spectroscopy (rapid soil sampling)


http://www.esa.int/esaLP/ASE5HBUG0SC_LPsmos_0.html

http://trmm.gsfc.nasa.gov/

The 3(4) bottles of vine

At the GSM2011.org meeting at JRC Ispra several people haveo�ered to award the DSM team that delivers a completecountry/continent size GlobalSoilMap.net product:

I 1 bottle if it contains complete list of soil properties;

I 1 bottle if it includes uncertainty estimates;

I 1 bottle if its accuracy is satisfactory;

I (1 bottle if it is being used by agronomist);


ISRIC's response to the GSM initiatives

Global Soil Information Facilities

a set of open tools and data portals


GSIF components

1. Cyber infrastructure for input, analysis and visualizationof data.

2. Global databases (legacy data, gridded covariates) thatare main inputs to global soil mapping.

3. Software tools (modules and packages) and manuals forcreation of geoinformation, for instance, according tothe GlobalSoilMap.net speci�cations.

4. Standards and protocols for data entry, map generationand data sharing.


Overview

Open Soil Profiles

(GSIF Servers) cyber infrastructure

Soil variables

Soil site info

Soil analytical data

Descriptive properties

Soil covariates (worldgrids)

5.6 km repository

Global

1 km repository

Continental scale

100 / 250 m repository

Country/state-level

R packages

GSIF package

Map import module

Data entry module

Harmonization module

Spline fitting

Spatial analysis module

plotKML

Data import to R

Data visualization

Data export

Soil property maps

100 m (250 m, 1 km and 5.6 km)

Global coverageSix+four key soil parameters

(organic carbon, pH, clay, silt,

sand, coarse fragments)

at six standard depths (0-5, 5-

15, 15-30, 30-60, 60-100, 100-

200 cm)

and with included upper and

lower 95% probability ranges

Webmapping API

Real-time spatial prediction

(Google Maps)

GlobalSoilMap.net functionality

for web-applications

Geo-serving and geoprocessing

functionality


Proposed implementation

1. Produce a suite of utilities to import, re-format, analyzeand visualize spatial soil data

2. Design them so they �t the needs of operational globalsoil mapping

3. Focus on using R+OSGeo

4. Get the whole DSM community involved (in design, indevelopment, in use)

5. Provide training in development and use to countries andnodes


List of utilities

1. Global soil mapping (core) package � GSIF

2. Soil visualization package � plotKML

3. Soil Reference Library � SRL

4. Geo-services (PythonWPS, Geoserver, RServe, GDAL utilities)


https://r-forge.r-project.org/projects/gsif/

https://r-forge.r-project.org/projects/plotkml/

Main principles of programming

1. Hide complexity from the users (scale, e�ective precision,3D geostat)

2. Deliver data and results so that no software training is requiredto open it (KML)

3. Link to R+OSGeo community (do not invent functionalitythat already exists and is operational)


The software triangle

GIS analysis

Browsing of

geo-data

Statistical

computing

KML

GDAL

ground

overlays,

time-series

GRASS GIS


Functionality (plotKML)

I Visualize soil pro�les measurements (using the original soil

colors);

I Visualize soil pro�le photographs;

I Plot results of prediction (soil property maps) using standard

color schemes;

I Visualize uncertainty in the soil property maps;


http://plotkml.r-forge.r-project.org/

Soil pro�le


http://globalsoilmap.net/content/soil-monolith-prepared-using-google-sketchup

Soil pro�le attribute plot


Soil grids as transparent polygons


http://globalsoilmap.net/content/predictions-organic-matter-meuse-data-visualized-using-polygons-kml

Multiple layers (above each other)


Animations


http://globalsoilmap.net/content/animated-display-uncertainty-mapping-soil-classes

Why KML? (1)

Google Earth is #1: >350 millions of downloads!


Why KML? (2)

People that made Google Earth understand

(space-time) statistics


What is Global Soil Mapper?

Global Soil Mapper

is an automated system (R+OSGeo) for

generation of soil property maps

that meet the GlobalSoilMap.net specs


Global Soil Mapper: the main principles

1. Put emphasis on inputs (point data, soil polygon maps,covariates) and tools (GSIF)

2. Fit model parameters per soil property for the wholeworld

3. Map the world block-by-block (automated mapping)

4. Update the maps as soon as the new point / covariatesarrive (while you sleep)


GSIF function predict

predict.gsm ( target.var = "ORCDRC", observations = soilprofiles.org,

+ covariates = worldgrids.org, model = GMN-RK,

+ newdata = boundingbox )

model = GMN-RK is the default global model (�tted using theglobal data);


GMN-RK

Global Multiscale Nested RK =

a 3D spatial prediction method

based on a four-level nested Regression-Kriging


Nested RK

z(sB) = m0(sB−k) + e1(sB−k|sB−[k−1]) + . . .+ ek(sB−1|sB) + ε(sB)

where m0(sB−k) is the value of the target variable estimated at thecoarsest global scale (B− k), B−1, . . . ,B−k are the higher order

components, ek(sB−k|sB−[k−1]) is the residual variation from scalesB−k to a �ner resolution scale sB−[k−1], and ε is the spatially

auto-correlated residual soil variation dealt with ordinary kriging.


Multiscale signal

S4

+ S

3 +

S2

+ S

1 +

eS

4 +

S3

+ S

2 +

S1

S4

+ S

3 +

S2

S4

+ S

3S

4

Figure: Based on McBratney (1998): Some considerations on methodsfor spatially aggregating and disaggregating soil information.


65k soil pro�les

Figure: USDA NCSS Characterization Database, CSIRO National SoilArchive, ISRIC WISE, SPADE, Iran National soil pro�le database,Canadian Soil Information System, and African soil pro�les.


http://globalsoilmap.net/data/OSP_sgdb.kmz

Data sets available for Malawi


Gridded maps for Malawi

5.6 km

1 km

250 m

100 m

BiomesClimateParent

material

General

land use

Erosion

deposition

MODIS-based long term Land

Surface Temperature (day/night)

Land

management

Rainfall map of the world

Elevation

Geologic Provinces of Africa

MODIS (MCD12Q1) land cover dynamics

ENVISAT Land Cover map (GlobCov)

MODIS (MCD13Q1) Enhanced Vegetation

Index (EVI) and medium infrared band (MIR)

TWI, TRI, Slope,

Surface roughness,

Insolation

Landsat ETM

thermal band

Soil polygon map (FAO classes)


The downscaling approach

Figure: Predictions of soil organic carbon for top depth at various scales.By running a multiscale global model we can �ll in the large gaps in thedata (interpolate instead of extrapolate).


Organic carbon (6 depths)


http://maps.google.com/maps?q=http://globalsoilmap.net/data/E33_S14_ORCDRC_1_gif.kml

One done, 18 thousand to go. . .


http://globalsoilmap.net/content/1-degree-tiles-world

Lessons learned

Conclusions


Conclusions

I Value of soil information is likely to grow.

I GSIF is a methodological framework for continuousproduction of Open Soil Information.

I Advantage of using a GMN-RK is that we can employ adiversity of predictors (CLORPT factors work at di�erentscales).

I Global is now (local statistical models will become extinct?).


Conclusions






Conclusions






Conclusions






Soils of Mars

�Astrophysists are selling something very abstract for a

high price. Soils are the basic of human survival, yet we

manage to acquire much less research funds.�

Neil McKenzie (CSIRO)

�We know more about soils of Mars than about soils of

Africa.�

Pedro Sanchez (Earth Institute)


Next steps

I Next step: Re-implement the method using a `clean' dataset (USA data) and write up step-by-step guidelines.

I Publish the GSIF package and WPS for GSM (anyone canbecome a digital soil mapper).

I Complete and publish plotKML and GSIF R packages.

I Map the whole of Africa at 100 m (end of 2012).
