A Sptatial-statistical Approach to the Visualisation of Uncertainty in Land Suitability Analysis

This article was downloaded by: [201.67.109.167]On: 05 April 2013, At: 18:00Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Spatial SciencePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tjss20

A spatial-statistical approach to thevisualisation of uncertainty in landsuitability analysisK. K. Benke a & C. Pelizaro aa Department of Primary Industries – Parkville Centre, PO Box4166, Parkville, Victoria, 3052, AustraliaVersion of record first published: 24 Nov 2010.

To cite this article: K. K. Benke & C. Pelizaro (2010): A spatial-statistical approach to thevisualisation of uncertainty in land suitability analysis, Journal of Spatial Science, 55:2, 257-272

To link to this article: http://dx.doi.org/10.1080/14498596.2010.521975

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representationthat the contents will be complete or accurate or up to date. The accuracy of anyinstructions, formulae, and drug doses should be independently verified with primarysources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand, or costs or damages whatsoever or howsoever caused arising directly orindirectly in connection with or arising out of the use of this material.

http://www.tandfonline.com/loi/tjss20

http://dx.doi.org/10.1080/14498596.2010.521975

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

A spatial-statistical approach to the visualisation of uncertainty

in land suitability analysis

K.K. Benke* and C. Pelizaro

Department of Primary Industries – Parkville Centre, PO Box 4166, Parkville,Victoria 3052, Australia

A formal GIS-based procedure known as land suitability analysis (LSA) is used todetermine the most appropriate crops for cultivation in different geographical locations.The approach is based on multi-criteria decision analysis utilising biophysical measure-ments (including rainfall, temperature, pH) and expert opinion captured from regionalworkshops. The issue of uncertainty in model predictions and its importance is discussed,and a method is described for its analysis and visualisation in LSA maps. Experimentalresults using Monte Carlo simulation are presented for ryegrass/sub-clover and winterwheat crops grown in south-western Victoria. It was found that uncertainty in theprediction of land suitability, as described by the coefficient of variation (CV), ranged from0.13 to 0.18 for ryegrass, and much higher at 0.28 to 0.30 for the winter wheat crop.Results showed that, for close matches between crop type and production location, over 90percent of the standard deviation in the prediction was accounted for by uncertainty inexpert opinion rather than uncertainty in biophysical data.

Keywords: land suitability analysis; uncertainty analysis; visualisation; multi-criteriaanalysis; spatial variability

1. Introduction

Estimation of land suitability and its spatialvariability represents a continuing challengeto resource planning and policy develop-ment. Optimisation of agricultural produc-tion is possible by matching crops to themost appropriate regional areas, althoughclimate-based predictions can be subject tomany uncertainties (Dessai et al. 2007;CAWCR 2008; Karoly 2008). Improve-ments are necessary in the supply andaccuracy of modelling and data sets relatingto land suitability in local regions, such asthe South West region in Victoria, wheredata distribution is required to informfarmers and policy-makers alike.

Development of agricultural manage-ment and policy, in the face of changingenvironmental conditions, requires plan-ning within a defined risk assessmentframework. Uncertainty originates frommany sources, including decision-supportmodels, biophysical models, productiondata, management and stakeholder deci-sions, and political and socio-economicfactors (Giorgi 2005; Solomon et al. 2007).Discussions on the importance of un-certainty analysis, and its relevance todecision analysis, can be found in theliterature (Helton 1993; Regan et al. 2002;Oberkampf et al. 2004; Burgman 2005;Benke et al. 2007).

*Corresponding author. Email: [email protected]

Journal of Spatial Science

Vol. 55, No. 2, December 2010, 257–272

ISSN 1449-8596 print/ISSN 1836-5655 online

� 2010 Surveying and Spatial Sciences Institute and Mapping Sciences Institute, Australia

DOI: 10.1080/14498596.2010.521975

http://www.informaworld.com

Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

Very little research has been reported inuncertainty analysis and its visualisation inmodel predictions of land suitability forregional agricultural production. This isdespite the recognised advantages of visua-lisation in GIS and the human-computerinterface (Coffey et al. 2007). In this paper,the concepts, application and importance ofuncertainty analysis are discussed. A de-scription is given of a computationalprocedure for the statistical analysis ofuncertainty and its visualisation by a GIS-based model for land suitability analysis.Results are presented for ryegrass andwinter wheat, which are two importantcrops grown in the South West region ofVictoria.

2. Land suitability analysis

Multi-criteria decision analysis

The land suitability prediction model usesmulti-criteria decision analysis (MCDA) toproduce an expert system with a decisiontree (or hierarchy) composed of weights andratings reflecting inputs from subject matterexperts and biophysical measurements, re-spectively (see Hossain et al. 2006). TheMCDA approach is common in naturalresource management, with applicationssuch as land suitability analysis (LSA),which is a GIS-based method developedfor the prediction of agricultural yields(Herath and Prato 2006). The particularform of MCDA used in this study features adecision tree in addition to criterion weightsassigned by expert opinion using the analy-tic hierarchical process (AHP) described bySaaty (1994a, 1994b). Dawes (1979) refersto such multi-criteria approaches, based onexpert opinion, as strictly speaking ‘impro-per models’ to distinguish them from‘proper models’ based on objective mathe-matical approaches, such as statisticalregression analysis. Dawes notes that suchexpert system models can be effective androbust in performance when predicting a

numerical criterion from numericalpredictors.

Alternatives to AHP-based approachesto weight-assignment in MCDA are cited inthe literature (see, for example, Kujawski2003; Lahdelma and Salminen 2006). Sub-ject to the type of MCDA applied, thevariables can be expressed as degrees ofpreferences, numerical scores or levels ofachievement for each criterion. In the AHPapproach, alternatives are compared se-quentially by subject-matter experts withrespect to performance against a set ofdecision criteria. A set of ratings (biophy-sical measurements) are used as inputs, anda hierarchy of weightings is derived fromconsideration of expert opinion gainedfrom regional workshops. A matrix ofjudgements (given in numerical form) isprovided by decision-makers in a workshopsetting against a set of alternatives in aprocess of repeated pair-wise comparisonsand prioritisation checked for consistencyby a process of eigenvalue decomposition(Saaty 1994a). The procedure yields a set ofpriorities or relative weights indicating thedecision-makers’ preferences for alterna-tives, and a decision tree is produced withnumerical values assigned to the criterionweights (Saaty; 1994b; Hahn 2003). Anexample of a decision tree producing aprediction of land suitability is shown inFigure 1, and applications in a GIS-basedcontext are provided by Hossain et al.(2006) and Rahman and Saha (2008).

The AHP differs from a statisticalparametric model, based on regressionanalysis, where one compares a set of modelpredictions with a set of measurements in apre-existing training set (for error minimi-sation and parameter estimation). WithAHP, estimates reflect predictions basedon using experts systematically weightingthe inputs using the AHP process describedin the previous paragraph. In some cases,AHP may be the only analysis possible, ifbiophysical models are not available or not

258 K.K. Benke and C. Pelizaro

Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

Figure

1.

Example

ofatypicalAHPdecisiontree

forlandsuitabilityanalysis.

Experim

entalconditionsforproducingthemodel

forryegrass

pasture

are

described

inSposito

etal.(2010).

Journal of Spatial Science 259

Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

validated, or under development, or cannotbe used state-wide within a reasonable time-frame (Sposito et al. 2010). Furthermore,the AHP methodology provides scope forcombining expert opinion with measure-ments, or numerical predictions from bio-physical models in order to provide a moreintegrated or systems-based approach toresource management (Belton and Steward2002). In summary, MCDA with weight-assignment by AHP provides the capabilityto produce a decision-support tool that (i)blends or integrates both subjective andobjective data, (ii) models consensus forgroups of subject-matter experts or stake-holders, and (iii) for decision criteria,accounts for inconsistent or possibly in-complete judgements and preferences in theanalysis of the possible alternatives.

Uncertainty in decision analysis

A problem with traditional deterministicMCDA models is that the criterion weightsin reality represent imprecise point esti-mates (i.e. with no indication of error orconfidence). The model prediction itself isalso a point estimate with no indication oferror or confidence (Benke et al. 2008).Attempts have been made at suggestingalternative stochastic formulations forthe AHP decision-making process usedfor weight-assignment, see, for example,the literature (Haines 1998; Hahn 2003;Jablonsky 2005; Mustajoki et al. 2006).Although such proposals are promising,they are often complex in their description,and may be at variance or in conflict withthe fundamental mechanisms behind theoriginal AHP methodology. A desirableobjective in MCDA is to gain a sense ofthe error or uncertainty in the predictions,given the uncertainty in the criterionweights. In the current investigation, amore direct and transparent posterior sta-tistical approach is described, which isdescriptive in nature and links uncertainty

with visualisation of uncertainty in the landsuitability map itself.

Analysis for South West Victoria

A spatial-based MCDA within a geo-graphic information system (GIS) frame-work was designed and implemented for thecurrent study (and also used in a parallelstudy by Sposito et al. 2010). The GIS-based land suitability analysis was con-ducted in the South West region of Victoriato assess possible impacts of climate changeon the agricultural system. Both currentand potential land suitability estimates weremapped for grain crops (barley, oats andwinter wheat), pasture (ryegrass/sub-clover,lucerne and phalaris) and forest species(blue gum and Pinus radiata). Current landsuitability refers to the agricultural landsuitability given current climate conditions.The results obtained will provide support inthe design of an appropriate policy frame-work and strategic plan at regional level forutilisation of available land for sustainableagricultural production. In this paper,current baseline data were used for landsuitability predictions for ryegrass/sub-clo-ver and winter wheat crops, and uncertaintyanalysis was investigated and combinedwith the LSA modelling approach.

3. Uncertainty analysis

Taxonomy of uncertainty

In the past, uncertainty has been attributedto a variety of sources, such as theconsequences of statistical variation orinherent randomness, measurement errors,systematic errors in judgement, linguisticimprecision or lack of agreement amongstexperts (Helton 1993; Burgman 2005). Inparticular, Burgman (2005) has summarisedsources of uncertainty as being due to thelimitations of measurement devices, insuffi-cient data, extrapolations and interpola-tions, linguistic ambiguity and variability


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

over space or time. The actual classificationof uncertainty is often reduced to achoice between two primary cases: eitheraleatory uncertainty or epistemic uncertainty(Helton 1993; Helton and Burmaster 1996;Oberkampf et al. 2004).

Aleatory uncertainty refers to variability,irreducible uncertainty, or inherent or sto-chastic uncertainty (Helton 1993). It isassociated with inherent variation in physicalsystems and is related to statistical (popula-tion) variability. This type of uncertaintymay be modelled using an appropriatestatistical technique, such as a probabilitydensity function (Helton and Burmaster1996). Epistemic uncertainty refers to redu-cible uncertainty, or subjective uncertainty(Oberkampf et al. 2004), and is associatedwith ignorance or incomplete information(lack of knowledge). For example, ecosys-tems may evolve in unexpected patterns, dueto natural events, such as volcanic eruptionor drought, leading to changes in the numberof species, their location and interaction.

The binary distinction represents con-venience in terminology – referring simplyto variability versus lack of knowledge.Vose (2000) uses the terms variability(aleatory) or lack of knowledge (epistemic),but notes that they are often combined inanalysis in many situations into totaluncertainty. The approach of Helton andBurmaster (1996) is convenient for model-ling and simulation in quality control andcomputational assessment of uncertainty inlarge-scale natural or artificial systems.Sources of uncertainty can also be docu-mented individually as a categorical list(Morgan and Henrion 1990).

Uncertainty and risk analysis

Uncertainty analysis is sometimes confusedwith risk analysis, when in practice theyrefer to very different types of assessment.Risk management is a process of dealingwith the management of hazards to the

environment on the basis of estimatingprobabilities of adverse events. Qualitativerisk assessment is based on subjectiveestimates of probabilities, whereas quanti-tative risk assessment is based on prob-ability theory and statistical analysis (Vose2000; Burgman 2001, 2005). Human psy-chology and linguistic imprecision areparticularly difficult to incorporate intoany risk assessment framework (Reganet al. 2002; Bomford and Hart 1999).

Uncertainty analysis embraces issuesbeyond traditional risk definitions. Thesebroader issues include the propagation oferrors in predictive environmental models,which may occur in the inputs, outputs,parameters and model structure (Oberkampfet al. 2004). Such issues require analysis ofprobability distributions rather than a riskspecification based on a single imprecise pro-bability or consequence. For example, therelationship between risk and uncertainty issuch that, if risk is the probability of anadverse event occurring, then uncertainty isthe bound or confidence in the point esti-mate of that probability (Benke et al. 2007).

4. Uncertainty in MCDA

Uncertainty in model weights and predictions

The decision tree produced by the determi-nistic AHP approach for the MCDA modelcontains weights with discrete values, but inpractice these values represent imprecisepoint estimates (i.e. with no indication oferror or confidence). The model predictionis also a point estimate without indication oferror or confidence. In Figure 1, the weightsin the decision tree have a possible range of[0,1]. If we arbitrarily set xmin ¼ 0 for allweights at the lower bound, and set xmax ¼ 1for all weights at the upper bound, and thenmultiply all the weights through the hier-archy, in both cases (lower and upperbounds), the output range of the modelwould be the maximum possible. It wouldrepresent only the maximum theoretical


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

range that values could take for the specifiedratings data. This result would have verylimited use for uncertainty analysis.

The alternative of replacing the existingweights in the model with centre-weightedprobability distributions (where the existingdeterministic weight is the mode value), andthen randomly sampling for each iterationof the model in a Monte Carlo simulation,would produce a final output probabilitydistribution significantly reduced in range incomparison to the case in the last para-graph. From this distribution, uncertaintymetrics can then be derived, such as stand-ard deviation or confidence interval. Thisresult would provide the scatter or statisticalvariation needed for uncertainty assessmentof the model output. Published research alsodemonstrates that it is not necessarily thelower and upper bounds on a parameterdistribution, but also the distribution and itsshape, that affect uncertainty in the modeloutput (Benke et al. 2008). Some attemptshave been made at producing stochasticformulations for the AHP decision-makingprocess for criterion weights; see, for exam-ple, the recent literature (Haines 1998; Hahn2003; Jablonsky 2005).

Probabilistic characterisation of thepair-wise comparisons (of expert judge-ments) has been examined experimentallyin the past. For example, a weighted hierar-chical multinomial logit model was used toobtain priorities (weight relativities or ra-tios), followed by inference on the prioritiesby Markov Chain Monte Carlo methods(Hahn 2003). At the cost of significantlyincreased computational complexity, theapproach was argued to be congruent withthe eigenvector method comparing prioritieswith respect to statistical inference ondistributions. Despite its current complex-ity, the approach offers an attempt ataccounting for some aspects relating toconfidence in the weight determinations.

Another strategy for addressing un-certainty involves replacing single-valued

judgements by interval judgements using aprobabilistic method with defined distribu-tions (Haines 1998). Once again priorities orweight ratios were treated as interval esti-mates. Distributions were constrained in thiscase by the feasible region (interval boundon weights), and the simple standard uni-form distribution was used to model varia-bility. Uncertainty in the weights was ofparamount interest and the extraction ofmarginal distributions was the prime objec-tive. Computational expense was very sig-nificant, as both statistical and mathematicalprogramming techniques were used exten-sively. Replacement of weights with intervalshas also been explored in funds managementwith encouraging results (Jablonsky 2005).

For LSA, the challenge is to incorporatesome measure of uncertainty or confidencein the weight assignments produced by theAHP model. In the past, various analyses ofuncertainty in the criterion weights havebeen characterised by a priori interventionsthat modified the AHP process itself, andwere therefore arguable on theoreticalgrounds (see Haines 1998; Jablonsky 2005).The primary interest in such studies wasconfined mainly to the errors in the weights.

Despite their analytic complexity, theseprevious studies provided insights intothe problems faced by risk analysts in theinterpretation of expert opinion from work-shop sessions. One criticism levelled is thatthe numerical complexity of the interven-tions actually introduced further significantuncertainty and computational expense tothe model predictions. A theme of thecurrent investigation is to carry out aposterior sampling strategy on the feasibleweights, and to produce a quantitative (andvisual) representation of possible uncer-tainty in model predictions.

Probability distributions

Once the weights have been assigned by thechosen method, such as the AHP method, or


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

an alternative approach, the values andassociated uncertainties can be propagatedthrough the model by Monte Carlo simula-tion. The weight values are subject to variousconstraints, including magnitudes, boundsand the unit-sum condition at each layer (orlevel) in the decision tree. Whenmoving froma deterministic approach to a probabilisticapproach, these constraints must also beobserved when sampling from the probabil-ity density function (PDF) used to representuncertainty. The actual ratings at the base ofthe tree have associated uncertainty and arealso sampled from a probability distribution.

The PDF for variability in the weightswas represented by the PERT probabilitydistribution, which has been used to modelexpert opinion in the past, and is described instatistical textbooks (Vose 2000). The PERTdistribution is distinguished by its smooth-ness and continuity, and its greater weightingto the most likely values rather than to thetails. Other advantages include computa-tional flexibility, and greater suitability formodelling expert opinion due to its non-linear nature. When the deterministic weightvalue in the decision tree is replaced by aprobabilistic value, the sampling distributionof possible weight values is defined by the set{xmin, xmode, xmax}, where xmode is the pointestimate from the workshop experts, and thebounds are prescribed by the AHP.

Constraint satisfaction requires non-negative lower and upper bounds onweights and ratings, as defined by theAHP model inputs, and also the unit-sumnormality condition on weights, wk, to holdfor each layer in the AHP hierarchy, i.e. fork ¼ 1, 2, . . . , m criterion weights, xk,

xk ¼�xk 2 Rþ

�� ak � xk � bk and

Xk¼mk¼1

xk ¼ 1

�ð1Þ

Note that ak and bk represent lower andupper bounds, respectively, and together

with wk are members of the set of positivereal numbers, Rþ. The ratings values at thebase of the decision tree are not subject to thenormality condition. The PERT probabilitydistribution is a special case of the Betadistribution. The probability density func-tion for the Beta distribution is describedsuccinctly by Croarkin and Tobias (2006):

fðxÞ ¼xv�1ð1�xÞw�1

Bðv;wÞ 0 � x � 1 ; v;w > 0

0 Otherwise

8<:

ð2Þ

where the term in the denominator is theBeta function,

Bðv;wÞ ¼Z 1

0

tv�1 ð1� tÞw�1dt ð3Þ

The shape parameters, v and w, are given by

n ¼ xmean � xmin

xmax � xmin

� �2xmode � xmin � xmax

xmode � xmean

� �

ð4Þ

w ¼ nxmax � xmean

xmean � xmin

� �ð5Þ

and the mean value, xmean, is defined as

xmean ¼ ð1=ðlþ 2ÞÞ ðxmin þ l xmode þ xmax Þð6Þ

The scale parameter for the height of thedistribution has default value l ¼ 4 (seereferences cited in Benke et al. 2008). ThePERT distribution is centre-weighted butcan be either symmetric or highly skeweddepending on the parameters. The ability tosimulate a single-peaked asymmetric distri-bution with pre-defined mode and rangemakes this a convenient model for repre-senting uncertainty in the weights (asthe original deterministic weights may be


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

located near the boundaries of the feasibleregion).

5. Experimental method

The AHP decision tree was constructed foreach agricultural commodity under consid-eration using information gained fromregional workshops with subject-matterexperts in agronomy, soil science, produc-tion (crop growers) and local authorities.Each agricultural commodity had specificgrowth requirements characterised by dataavailable on soil, climate and landscape(biophysical factors), and consensus wasgained on the set of factors (criteria) andtheir level of importance (weights andratings). The AHP pair-wise comparisontechnique was used to assign the criteriaweights. A total of three workshops wereorganised, with the participation of 12 to 17experts per workshop session. At the end ofthe process, a hierarchical model of criteriaweights and ratings was produced for eachagricultural commodity (see Sposito et al.2010). In this paper, the results for landsuitability and uncertainty analysis will bedemonstrated for designated high-prioritycrops ryegrass/sub-clover and winter wheat.

The original determination of point-estimates of weights in workshops dependson information available to the experts, sothat there is a component of ‘lack ofknowledge’ or epistemic uncertainty, aswell as ‘variability’ or aleatory uncertainty.As the uncertainty analysis uses a prob-ability distribution to cover the full range ofthe weight values possible, both cases areincluded in the Monte Carlo simulation (seedetailed discussion on this issue in thetextbook by Vose 2000). Metrics for quan-tification of aleatory uncertainty includestandard deviation, variance, the confidenceinterval and percentile data from the prob-ability distribution (see applications byBenke & Hamilton 2008 and Benke et al.2008). Uncertainties in the AHP decision

tree occur in the weights, ratings and landsuitability estimates (i.e. expert opinion,data and predictions). The uncertaintyassociated with land suitability predictionsis achieved by sequential application ofMonte Carlo simulations to produce aseries of model predictions for land suit-ability, where the corresponding uncer-tainty metric in each case is computedfrom the simulation output PDF.

The computational aspects of MonteCarlo simulation for risk assessment anduncertainty have been documented in text-books (e.g. Vose 2000). The method used toquantify uncertainty in land suitabilitypredictions is detailed in Figure 2. In thecase of weight-assignment by AHP, themodelling approach in land suitabilityanalysis is based on GIS-based spatialanalysis (Hossain et al. 2006; Sposito2006). In the current investigation, theuncertainty in the prediction associatedwith the model derives from the modelweights and ratings captured from thesubject matter experts and biophysicalmeasurements (such as rainfall and tem-perature). The principal sources of uncer-tainty in the model are due to errors andvariability in human judgement encapsu-lated in the criterion weights (in the tradi-tional deterministic approach, these errorsare ignored). The method described inFigure 2 for uncertainty analysis is basedon simulation by repeated sampling fromthe probability distributions used for thecriterion weights – with the aim of buildingup a distribution at the model output fromwhich the various uncertainty metrics canbe extracted.

6. Results and discussion

Monte Carlo simulation

The LSA model was implemented usingModel Builder in ArcView GIS (�ESRI).The input consists of a set of mapsdepicting the spatial representations of the


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

various criteria used in the AHP. Each map(representing a specific criterion) was re-classified according to the AHP rating andmultiplied by the AHP weight. The criter-ion maps were overlaid and added togetheraccording to the hierarchy level to provide afinal composite index map. The final map isa ranking of areas in terms of percentagesuitability for the growth of the commodityunder consideration and has an index rangeof 71 to 10, where 71 means restrictedand 10 represents the best available site forplant growth (i.e. 100% suitable). Formodelling and simulation, the land suit-ability scale is 0.0 to 1.0 (for non-restrictedland). The model was constructed so thatproductivity of a particular area can beestimated by multiplying the maximumyield defined at the apex of the AHPhierarchy by the suitability index (Hossainet al. 2006; Sposito et al. 2010).

Monte Carlo simulations were carriedout using the software package @RISK(Palisade Corporation). Sampling from in-put probability distributions was subject to

the Latin Hypercube method, which exe-cutes stratified random sampling withoutreplacement across the full range of eachparameter (Vose 2000). The Latin Hyper-cube approach provides more uniformsampling in the tails of the chosen distribu-tion in comparison with traditional randomsampling and generally produces muchfaster convergence for a specified numberof iterations (McKay et al. 1979; Iman andConover 1983).

Each Monte Carlo simulation experi-ment in the series was executed over 10,000iterations. Skewness and kurtosis of theoutput distributions approximated that ofthe normal distribution in each case. MonteCarlo simulation outputs can have skeweddistributions, although this has no effect onthe estimation of the confidence interval inthis study as it is based on using percentiledifferences from the cumulative probabilitydistribution, a measure that is independentof the shape of the distribution (see Benkeand Hamilton 2008). The simulation pro-cess is repeated (Figure 2), incrementing the

Figure 2. Determination of uncertainty by simulation.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

mean values of the ratings vector (where theuncertainty in each rating is represented bya probability distribution).

In each simulation in the series, adifferent set of inputs (represented by theratings vector) was applied to the model (atthe base level of the AHP hierarchy), and acorresponding probability distribution pro-duced for land suitability predictions. Boththe weightings and ratings were representedby PERT probability distributions. Thisprocess provided a set of model predictionsfor different levels of land suitability to-gether with a corresponding set of proba-bility distributions. This latter step providedthe required linkage between land suitabilityestimates and corresponding probabilitydistributions (from which the uncertaintymetrics were derived). A graph of theuncertainty metric against LS produces acalibration plot, and a continuous functioncan then be fitted by regression analysis.

Locus of uncertainty

In the case of ryegrass/sub-clover, separateMonte Carlo simulations were used toproduce separate LS distributions for plot-ting the uncertainty metrics as a function ofland suitability, with all linear regressionfits statistically significant. The reason forapplication of linear regression was toconnect the points to produce a continuouscalibration plot so that any LS value has anassociated uncertainty value. Once this stepwas completed, graphical representation ofuncertainty was completed by addressingthe calibration plot directly in order toassign an uncertainty metric to any discretevalue of land suitability associated with themap legend. In Figure 3a and 3b, forryegrass and winter wheat data, uncertainty(variability) is shown for the LS values inthe form of a range of percentile plots,which provide quantitative data for riskanalysis. For example, the 95th percentileresult is often used in traditional risk

assessment. The regression analysis usedto produce the percentile plots was statisti-cally significant in all cases, as confirmed bythe t-test on correlation coefficients, whichwere r 4 0.99 in all cases. Uncertaintymetrics in the form of the standard devia-tion (Std Dev), 90 percent confidenceinterval (CI) and the coefficient of variation(CV) were also extracted and plotted inFigure 4a and 4b.

Figure 3. (a) Uncertainty in ryegrass landsuitability can be plotted as standard percentileplots (Sposito et al. 2008), with 95th percentilerisk results. (b). Uncertainty in modelpredictions for the winter wheat data.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

The CV is the ratio of the standarddeviation to the mean value of LS, and is anormalised measure analogous to compar-ing the magnitude of the noise to themagnitude of the signal. For the CVmetric, the correlation coefficients for theregression fits were r ¼ 70.99 andr ¼ 70.89, for ryegrass and winter wheat,respectively. As a measure of variability,the CV provides an alternative to thestandard deviation and confidence interval,although these measures are more familiarto decision-makers and policy analysts. Aninformal survey of stakeholders revealed

greater receptivity to the standard devia-tion as a convenient measure ofuncertainty.

Figure 4a for ryegrass data reveals thatas LS increases in value, absolute uncer-tainty (Std. Dev. or CI) also increases invalue, i.e. there is a direct proportionalrelationship. In contrast, relative uncer-tainty (CV) decreases with increasing LS,i.e. there is an inverse relationship. Figure4b for winter wheat, however, reveals thatboth absolute and relative uncertaintymetrics are fairly constant across the rangeof LS values for this crop. These resultsindicate that trends in uncertainty are noteasily predicted without recourse to model-ling expert opinion and experimental data.Comparison between Figure 4a and 4b alsoreveals that the uncertainty in the LS valuesis much higher for winter wheat for allmetrics considered. Differences in uncer-tainty could be attributed to variations inthe uncertainty in expert opinion andbiophysical data for each crop. The appor-tionment of uncertainty between expert

Table 1. Uncertainty in land suitability, (a)Ryegrass, (b) Winter wheat.

Ryegrass

Uncertainty Metrics

Prediction90%CI

20thperc

80thperc C.V.

Std.Dev.

0.70 +0.22 0.59 0.81 0.18 0.130.80 +0.24 0.68 0.92 0.16 0.150.90 +0.26 0.77 1.00 0.13 0.16

Winter Wheat

Uncertainty Metrics

Prediction90%CI

20thperc

80thperc C.V.

Std.Dev.

0.70 +0.34 0.52 0.88 0.30 0.210.80 +0.39 0.59 1.00 0.29 0.240.90 +0.43 0.66 1.00 0.29 0.271.00 +0.48 0.73 1.00 0.28 0.29

Figure 4. (a) Uncertainty metrics for ryegrassdata. (b) Uncertainty metrics for winter wheatdata.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

opinion and input data was explored in thefollowing experiment.

An issue of interest was uncertaintywhere land suitability prediction was highor near-maximum in value (LS ¼ 0.85–1.0),as this indicates the best fit between croptype and location. To gain an indication ofthe relative contributions to total uncer-tainty due to weights, as opposed to ratings,the uncertainty analysis (Figure 2) wasrepeated but with deterministic values forthe ratings (pH, temperature, rainfall, etc.as listed in Figure 1). The experiments werecarried out for both ryegrass and winterwheat crops. It was observed that thedifference in the standard deviation of themodel output, with and without probabilitydistributions on the ratings, was about 10percent for ryegrass (average from threeseparate experiments) and 3 percent forwinter wheat (average from four separateexperiments). These results suggest thatvariability in criterion weights accountedfor 90 percent and 97 percent of uncertaintyin model outputs for ryegrass and winterwheat, respectively. As the criterion weightsrepresent the contributions from expertopinion derived from regional workshops,the implication is that expert opinion is themain contributor to uncertainty in the LSpredictions.

Table 1 shows uncertainty metrics ex-tracted from the data used for the functionplots shown in Figure 4a,b. These discretevalues appear in the legend of the landsuitability map as an indication of theuncertainty in each category. The structureof this representation is that of a look-uptable (LUT), which is necessary for trans-fer of the uncertainty data to the mapfor visualisation. In the final step, the

uncertainty metric (in this case standarddeviation) is depicted in the land suitabilitymap itself, along with the LS values, asshown in Figure 5a and 5,b (where thevalues have been converted to percentages).The entire process provides both a statis-tical representation and a spatial represen-tation of uncertainty for data visualisation.

These results are conservative estimatesof uncertainty because the full feasibleregions of the weight ranges were used inthe simulation, providing an indicator ofthe uncertainty possible in the model out-put. With respect to model uncertainty, thisis effectively the upper bound on theuncertainty in predictions. A stronger focuson the capture of the weight distributionsexperimentally, and their bounds in parti-cular, will help to reduce the uncertaintyestimate. One ploy may be to elicit lowerand upper bounds for the weight estimatesin workshops. If the AHP method is used,then there are difficulties in modifyingweight assignments, due to the nature ofthe workshop scenarios used with subject-matter experts. For example, replicatingworkshops in order to estimate the disper-sion of modal values in the weights (andfitting appropriate distributions) may incursignificant additional costs and deployedresources. In reducing the uncertaintyinterval, repeated sampling from uncer-tainty distributions will need to be cogni-sant of the AHP consistency criteria andeigenvalue analysis.

The current investigation provided aframework for consideration of uncer-tainty analysis in land suitability predic-tions. Modelling and simulation producedconservative values on the uncertainty inpredictions for land suitability for two

Figure 5. (a) Visualisation of uncertainty in ryegrass map, where uncertainty in land suitability isrepresented by the standard deviation. This type of representation has been incorporated into amodelling framework for optimisation of commodity production (see Sposito et al. 2010). (b) As in (a),but for winter wheat land suitability predictions.

3


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

important agricultural products. Possiblefuture extensions could place greateremphasis on more accurately defining theuncertainty in the weights. Researchreported elsewhere has explored the pos-sibility of probabilistic interpretations ofthe AHP methodology, suggesting somepossibilities in this respect, but thisremains the object of ongoing research(see also Rosenbloom 1997; Manasseroet al. 2004).

7. Conclusion

Model predictions for GIS-based landsuitability analysis are often cited withoutany indication of error or confidence in theresults. The investigation reported in thispaper provides a background on uncer-tainty in modelling, together with itssignificance and quantitative analysis. Anapproach is described for the statisticalanalysis, graphical representation and vi-sualisation of uncertainty in land suitabilitymaps. The multi-criteria decision analysis(MCDA) model used for land suitabilitypredictions produces a decision tree withcriterion weights determined by an appro-priate method, such as the AHP approach,which is a widely used strategy in naturalresource management (Herath and Prato2006; Hossain et al. 2006).

Assessment of uncertainty in the resultswas carried out by representation of modelweights by probability distributions. Aseries of Monte Carlo simulation experi-ments produced a range of land suitabilitypredictions and the chosen uncertaintymetric was then depicted spatially on theland suitability map, providing visualisationin addition to statistical plots. Uncertaintymaps for land suitability were described forthe crops ryegrass and winter wheat grownin the South West region of Victoria.

Uncertainty in the ryegrass data wasrevealed to be much less than that occurringin the winter wheat data, irrespective of the

metric used. Normalised uncertainty esti-mates showed that that the CV ranged from0.13 to 0.18 for ryegrass, and 0.28 to 0.30for winter wheat. For high land suitabilityvalues (i.e. LS ¼ 0.85–1.0), representingclose matches between crop type andproduction location, more than 90 percentof uncertainty (standard deviation) in pre-diction was due to the weights in the model(representing expert opinion) rather thanthe ratings (representing biophysical data).

In workshop conditions, ambiguity dueto linguistic responses may introduce furtherepistemic uncertainty, which can be reducedsubject to improved experimental design.The quantification of uncertainty in thecurrent case is conservative due to the rangeimposed on weight values for the simulation.More accurate estimates may be possible byincorporating uncertainty analysis directlyinto the AHP process itself, rather than as aposterior analysis, but at the risk of greatercomplexity in theory and operation.

References

Belton, V., & Steward, T. (2002) MultipleCriteria Decision Analysis, An IntegratedApproach, Kluwer, Boston.

Benke, K.K., & Hamilton, A.J. (2008) Quanti-tative microbial risk assessment: uncertaintyand measures of central tendency for skeweddistributions. Stochastic Environmental Re-search and Risk Assessment, vol. 22, pp. 533–539.

Benke, K.K., Hamilton, A.J., & Lowell, K.E.(2007) Uncertainty analysis and risk assess-ment in the management of environmentalresources. Australasian Journal of Environ-mental Management, vol. 14, pp. 16–22.

Benke, K.K., Lowell, K.E., & Hamilton, A.J.(2008) Parameter uncertainty, sensitivityanalysis and prediction error in a water-balance hydrological model. Mathematicaland Computer Modelling, vol. 47, pp. 1134–1149.

Bomford, M., & Hart, Q. (1999) Risk assessmentfor importing and keeping exotic vertebratesin Australia. Australasian Journal of Envir-onmental Management, vol. 6, pp. 149–156.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

Burgman, M.A. (2001) Flaws in subjective assess-ments of ecological risks and means forcorrecting them. Australian Journal of Envir-onmental Management, vol. 8, pp. 219–226.

Burgman, M.A. (2005) Risks and Decisions forConservation and Environmental Manage-ment, Cambridge University Press,Cambridge.

CAWCR (2008) Centre for Australian Weatherand Climate Research. Available at: http://www.cawcr.gov.au.

Coffey, J., Beard, D.J., & Ryan, D.A. (2007)Visualising coastal seabed characteristics:using VMRL models to present three dimen-sional spacial data via the web. Journal ofSpatial Science, vol. 52, pp. 133–143.

Croarkin, C., & Tobias, P. (Eds) (2006) NIST/SEMATECH e-Handbook of StatisticalMethods. Available at http://www.itl.nist.gov/div898/handbook.

Dawes, R.M. (1979) The robust beauty ofimproper linear models in decision making.American Psychologist, vol. 34, pp. 571–582.

Dessai, S., O’Brien, K., & Hulme, M. (2007)Editorial: on uncertainty and climatechange. Global Environmental Change, vol.17, pp. 1–7.

Giorgi, F. (2005) Interdecadal variability ofregional climate change: implications forthe development of regional climate changescenarios. Meteorology and AtmosphericPhysics, vol. 89, pp. 1–4.

Hahn, E.D. (2003) Decision making with un-certain judgements: a stochastic formulationof the analytic hierarchy process. DecisionSciences, vol. 34, pp. 443–466.

Haines, L.M. (1998) A statistical approach to theanalytic hierarchy process with intervaljudgements. (I) Distributions on feasibleregions. European Journal of OperationalResearch, vol. 110, pp. 112–125.

Helton, J.C. (1993) Uncertainty and sensitivityanalysis techniques for use in performanceassessment for radioactive waste disposal.Reliability Engineering and System Safety,vol. 42, pp. 327–367.

Helton, J.C., & Burmaster, D.E. (1996) Guesteditorial: treatment of aleatory and epistemicuncertainty in performance assessments forcomplex systems. Reliability Engineering andSystem Safety, vol. 54, pp. 91–94.

Herath, G., & Prato, T. (2006) Using Multi-Criteria Decision Analysis in Natural Re-source Management, Ashgate, Aldershot.

Hossain, H., Sposito, V., & Evans, C. (2006)Sustainable land resource assessment in

regional and urban systems. Applied GIS,vol. 2, pp. 24.1–24.21.

Iman, R.L., & Conover, W.J. (1983) A ModernApproach to Statistics, Wiley, New York.

Jablonsky, J. (2005) Measuring efficiency ofproduction units by AHP models. ISAHP2005, Honolulu, Hawaii.

Karoly, D. (Convenor) (2008) Future Directionsfor Australian Climate Change Science, Cli-mate Change Workshop, The AustralianAcademy of Science, Shine Dome, ANU,Canberra, 12–13 March 2008. Available at:http://www.science.org.au/.

Kujawski, E. (2003) Multi-Criteria Decision Ana-lysis: Limitations, Pitfalls, and Practical Diffi-culties, Engineering Division, LawrenceBerkeley National Laboratory, Berkeley, CA.

Lahdelma, R., & Salminen, P. (2006) Classifyingefficient alternatives in SMAA using crossconfidence factors. European Journal ofOperational Research, vol. 170, pp. 28–240.

Manassero, G., Semararo, Q., & Tolio, T. (2004)A new method to cope with decision makersuncertainty in the equipment selection pro-cess. CIRP Annals - Manufacturing Techno-logy, vol. 53, pp. 389–392.

McKay, M.D., Conover, W.J., & Beckman, R.J.(1979) A comparison of three methods forselecting values of input variables in theanalysis of output from a computer code.Technometrics, vol. 21, pp. 239–245.

Morgan, M.G., & Henrion, M. (1990) Uncer-tainty: A Guide to Dealing with Uncertaintyin Quantitative Risk and Policy Analysis,Cambridge University Press, Cambridge.

Mustajoki, J., Hamalainen, R.P., & Lindstedt,M.R.K. (2006) Using intervals for globalsensitivity and worst-case analyses in multi-attribute value trees. European Journal ofOperational Research, vol. 174, pp. 278–292.

Oberkampf, W.L., Helton, J.C., Joslyn, C.A.Wojtkiewicz S.F., & Ferson, S. (2004)Challenge problems: uncertainty in systemresponse given uncertain parameters. Relia-bility Engineering and System Safety, vol. 85,pp. 11–19.

Rahman, M.R., & Saha, S.K. (2008) Remotesensing, spatial multi criteria evaluation(SMCE) and analytic hierarchy process(AHP) in optimal cropping pattern planningfor a flood prone area. Journal of SpatialScience, vol. 53, pp. 161–177.

Rosenbloom, E.S. (1997) A probabilistic inter-pretation of the final rankings in AHP.European Journal of Operational Research,vol. 96, pp. 371–378.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

http://www.cawcr.gov.au

http://www.cawcr.gov.au

http://www.science.org.au/

Regan, H.M., Colyvan, M., & Burgman, M.A.(2002) A taxonomy and treatment of uncer-tainty for ecology and conservation biology.Ecological Applications, vol. 12, pp. 618–628.

Saaty, T.L. (1994a) Fundamentals of DecisionMaking, RWS Publications, Pittsburgh.

Saaty, T.L. (1994b) How to make a decision: theanalytic hierarchy process. Interfaces, vol.24, pp. 19–43.

Solomon, S., Qin, D., Manning, M., Chen, Z.,Marquis, M., Averyt, K.B.,Tignor, M., &Miller, H.L. (2007) Climate Change 2007:The Physical Science Basis, Cambridge Uni-versity Press, Cambridge.

Sposito, V.A. (2006) A strategic approach toclimate change impacts and adaptation.Applied GIS, vol. 2, pp. 23.1–23.26.

Sposito, V., Pelizaro, C., Benke, K., Anwar, M.,Rees, D., Elsley, M., O’Leary, G., Wyatt, R.,& Cullen, B. (2008) Climate Change Impactson Agriculture and Forestry Systems inSouth West Victoria, Australia, Departmentof Primary Industries – Parkville Centre,Victoria, Australia, 3052.

Sposito, V., Benke, K., Pelizaro, C., & Wyatt, R.(2010) Adaptation to climate change inregional australia: a decision-making frame-work for modelling policy for rural produc-tion. Geography Compass, vol. 4, pp. 335–354.

Vose, D. (2000) Risk Analysis, John Wiley,Chichester.


Dow

nloa

ded

by [

201.

67.1

09.1

67]

at 1

8:00

05

Apr

il 20

13

Documents

A Sptatial-statistical Approach to the Visualisation of Uncertainty in Land Suitability Analysis