Boundary Crossings

A policymaker’s puzzle, or how to cross theboundary from agent-based model to land-usepolicymaking?

Nick Green

School of the Built Environment, University of Salford, Greater Manchester M5 4WTEmail: n.green@salford.ac.uk

Revised manuscript received 14 March 2012

We are comfortably familiar with our towns and citiesand villages, and the land in which they sit and onwhich they depend, but that casual acquaintance canblind us to the complexity of such places. The policy-maker charged with steering the evolution of suchplaces is thus in the unenviable position of having touse relatively blunt instruments (policies) to steer andmediate the enormously complex and interconnectedissues that distinguish these human–geographical sys-tems. They have received plenty of help to do this,but it seems that where those efforts to assist haveinvolved creating models of land use, they have notalways been successful. As Couclelis put it three dec-ades after Lee’s famously despondent prognosis forlarge-scale models (Lee Jr 1973):

Land-use models, most of which were explicitly developedto help forecast the future states of the systems of interest,have done little to enhance the future-oriented, strategicmission of planning. (Couclelis 2005, 1353)

Geographical information systems (GIS) (arguably atype of land-use model) have of course been usedsuccessfully to support decisionmaking in land-useplanning (Longley et al. 2005; Nyerges and Jankowski2010). So the story is rather depressing, perhaps; butnot completely hopeless.

In this essay I want to look at the potential role ofa relatively untested (compared with GIS) decisionsupport tool, the agent-based model (ABM). This is atype of computational model which, for reasons wewill come to, is likely to become more common. SinceABMs tend to broaden the scope of a problematiquerather than narrowing things down to a simple solu-tion, misunderstandings over their purpose seemlikely to arise. My point is that if those misunder-standings can be forestalled, then ABMs can becomeuseful thinking tools that can serve, along with GIS(or in combination with GIS), as effective decisionsupport tools of use to the policymaker.

‘Agent-based model’ is really a generic term for abroad class of computational models known also as‘multi-agent models’. As the name suggests, themodel consists of a number of different ‘agents’ thatrepresent real-world entities such as households, busi-nesses, individuals and so forth. Each agent follows aset of relatively simple rules (an algorithm) control-ling its interactions with other adjacent agents and itsimmediate environment over time.

Crucially, these interactions generate a systemicbehaviour that is emergent: in other words, the aggre-gate behaviour of the agents cannot be predicted sim-ply by extrapolating the rules being followed by theindividual agents. Rules may be probabilistic (e.g. anagent has a 60% chance of renting a house and 40%chance of buying one) so that multiple ‘runs’ of anABM may produce different outcomes for the sameinput parameters. This is because the agents maythemselves behave differently in successive runs, andso we can see whether some outcomes occur moreoften than others (Batty 2005; Coveney and Highfield1995). A more sophisticated version is the ‘dynamicnetwork multi-agent model’ (DNMA), which includesnetwork characteristics so that agents can influenceone another from afar (Louie and Carley 2007). Ele-ments of these models can of course be combinedwith other types of model: a system dynamics model,for example, might be combined with a multi-agentmodel (Kabisch et al. 2010). In the remainder of thisessay, I shall simply use the generic term ‘agent-basedmodel’ (ABM) to refer to this broad class of compu-tational models.

It is probably fair to say that ABMs are still in therealm of what Couclelis (2002) calls ‘research models’rather than ‘policy models’, although recent work bythe International Panel on Climate Change (IPCC2007) and the UK Climate Programme (UKCP09)(Murphy et al. 2009) straddles the two. A notable

Citation: 2012 doi: 10.1111/j.1475-5661.2012.00532.xISSN 0020-2754 � 2012 The Author.

Transactions of the Institute of British Geographers � 2012 Royal Geographical Society (with the Institute of British Geographers)

example of what might be considered a policy model,set out in The limits to growth project (Meadows et al.2004) famously demonstrated how computer modelsoriginally intended to inform debate can end up pola-rising it instead. More recently, ABMs have beenused to explore housing in East Anglia, UK (Fontaine2011); the use of eco-system services in Belgium(VOTES Project 2011); and peri-urban land useacross Europe (PLUREL 2008). An ABM (MO-LAND) has also been used to explore policy-relevantscenarios for the rapidly urbanising city of Wuhan,China (RIKS 2011). The MOLAND model has alsobeen used to explore possible scenarios for land-usechange in a variety of contexts: tourism (Petrov et al.2009); sustainable city development (Lavalle et al.2002); and even the growth of mega-cities (Lavalleet al. 2001).

Such models are increasingly commonplace inurban analysis, driven by ever more powerful comput-ers and relatively user-friendly programming environ-ments such as Repast ⁄ S and NetLogo (Castle andCrooks 2006; Crooks 2006 2007; Eclipse Foundation2009; Wilensky 2011). The current policymaking ethosdemands a strong evidence base (Batty et al. 2003;Couclelis 2005; Haase and Nuissl 2007), and modelsintended to contribute to that evidence base need tobe reliable, transparent, easy to operate and able toaccommodate policy variables (Couclelis 2002);clearly a tall order. Furthermore, some argue thatpoliticians tend to pick and choose the science thatjustifies how they want to act anyway: dispassionatetreatment of the evidence might simply be absent(Marmot 2004; Schmidt 2011).

These issues are especially germane, for the emer-gent behaviour of ABMs brings a particular problem:the impossibility of knowing which rules are resultingin which bits of systemic behaviour. We can tinkerwith them and observe the behaviour over multipleruns of the simulation to get an idea of which ruleshave what effects (Wilensky 2011). For example, ifchanging a particular parameter from X to Y consis-tently generates a particular outcome, then we mightreasonably conclude that our parameter is somehowaffecting the model’s behaviour.

However, the changed parameter might be inter-acting with other parameters to produce the changedbehaviour, or it might be the only influence: we can-not know. True, the original algorithms may containclues; here, we would be aiming to trace the decisionpaths that make up the algorithm. But if there aremultiple types of agent each with its own set of deci-sion paths, we will quickly find many different routesfrom cause to effect, any of which might cause themodel’s changed behaviour. These relationships arefurther complicated by the nature of emergence:being a function of time, it shares time’s arrow; you

cannot simply roll it backwards (Manson 2007). None-theless, through a process of repeated runs, each withsmall variations of a single parameter, we can narrowdown the options and produce a more focusedaccount of how the model’s systemic behaviouremerges (Wilensky 2011).

That still leaves the issue of calibration and valida-tion. The usual process is to use historical data as thebasis for the calibration and then to adjust the modelso that it consistently reproduces historical events(O’Sullivan 2004). The assumption here is that if themodel reproduces historical events most of the time(however ‘most’ is defined), then it is a reasonablyaccurate model of those events, and hence valid.Besides, it is as well to keep in mind that a ‘modelrepresents a theory about the world, rather than theworld itself’ (O’Sullivan 2004, 291) a key point forpolicymakers hoping to use ABMs.

Now, different models may each be calibrated togive reasonably accurate representations of pastevents, but might nonetheless produce very differentrepresentations of future events. Since ‘validationdoes not establish the truth of a model’, conventionsfor validation of the model need to be agreed upon(O’Sullivan 2004, 290). This is more easily said thandone. Many questions of validation will be concernedwith practical issues: the scale of the agents (shouldagents be villages or people?); or how to reconcilesimplicity and complexity; or how to handle disputedmoral or ethical values (Evans and Manson 2007) toname just three. The answers to such questions willvary from model to model, and so generalising is dif-ficult. Nonetheless, the practical nature of these con-cerns – locating a city-region’s boundaries, forexample – leaves the possibility of achieving a work-able consensus amongst those actors being modelled.In other words, conventions for validation can beagreed pragmatically on a per model basis, althougheven that may be difficult. After all, experts can dis-agree with one another, or agree and still be wrong(Louie and Carley 2008).

This leaves the policymaker who is interested insuch models in an awkward position: how can theybe (reasonably) sure that an ABM is trustworthy?One argument is that an ABM can be trustworthy ifits internal rules generate results consistent withobservations of what happened; it remains legitimateuntil the rules are found to be wrong (Johnson 2001).An ABM can ‘be wrong’ in a variety of ways, but ifwe take ‘wrong’ to mean inaccurate, we can link accu-racy to notions of validity and path dependence(Brown et al. 2005). In essence, accuracy, in the con-text of ABMs, has two core definitions. ‘Predictiveaccuracy’ refers to the model’s ability to consistentlyreproduce historical events, such as patterns of landuse; ‘process accuracy’ is to do with how consistent

2 Boundary Crossing

Citation: 2012 doi: 10.1111/j.1475-5661.2012.00532.xISSN 0020-2754 � 2012 The Author.Transactions of the Institute of British Geographers � 2012 Royal Geographical Society (with the Institute of British Geographers)

the model’s processes for determining land use arewith those of the real world (Brown et al. 2005).

But we must be careful. The greater the accuracyof the model, the greater the risk that it will produceresults that are very consistent with one particularcase, but from which it may be impossible to general-ise (Brown et al. 2005). This may result in the com-plete abandonment of any claims to being able topredict future states (Brown et al. 2005), a situationprofoundly unhelpful to the policymaker charged withimproving that future. So what to do?

One solution is to make comparisons across abroad range of different models and land-use pat-terns. This should, in theory, offer the researcher dee-per insights into the processes that result in accurateor inaccurate predictions (Brown et al. 2005). In thisschema, then, ‘predictive accuracy’ is founded on thenotion that what actually happened, or somethingclose, was the most likely turn of events.

Another solution lies in the fact that ABMs do nothave to be tested using only empirical data; they canalso be used to show how simple local rules can pro-duce complex macro-level phenomena (Janssen andOstrom 2006). Such ‘laboratory experiments’ can becombined with empirical data to enable the use ofABMs as a ‘tool to examine the theoretical conse-quences of more complex assumptions’ (Janssen andOstrom 2006). These empirical data can serve eitheras a model’s input, or for validation and calibration.

Our policymaker, for whom these models areintended, seems unlikely to find such arcane discus-sions helpful. How, they might ask, are they to tellwhether or not a model is a ‘good’ one? After all, westill need to define ‘good’ (Janssen and Ostrom2006). As we saw above, a more complex model mayprovide a good fit to one particular data set, but thismakes generalisation problematic (Janssen andOstrom 2006). ‘The best model’, then, ‘balances thegoodness of fit and the ability to generalize’ (Janssenand Ostrom 2006, np). Other criteria for judging thequality of a model become important, too: plausibil-ity; understanding why the model (apparently) workswell; reaching a better understanding of empiricalobservations; the extent to which the observed behav-iour of the model coincides with stakeholders’ percep-tions of the system in question (Couclelis 2002;Janssen and Ostrom 2006).

This last is a crucial issue, for the patterns arisingfrom a model may be conflated with the underlyingprocesses that drive that model, and in particular theassumption that complex patterns are sufficient toindicate underlying complexity (Manson 2007). Infact, complex patterns may arise from simple underly-ing processes, or because of the way the model works(which may or may not accurately represent reality)(Manson 2007).

An example of this intrinsic uncertainty can befound in a review of a proprietary ABM tool (Com-munityViz Policy Simulator) which explores generalquestions about the use of ABMs as ‘laboratories forspatially explicit planning policies’ (Ligmann-Zielinskaand Jankowski 2007). Intriguingly, little changed afterentering some exaggerated policies into the model soas to force a reaction from the agents (property taxeswere raised by 20%) (Ligmann-Zielinska and Jankow-ski 2007). Now this could mean that either the agentsdid not mind paying higher property taxes for somereason, or it means that the model got it wrong. Thelesson? There is more to calibrating a model thanfeeding it the right data. A measure of intuition, com-mon sense, expert judgement (or what you will) isneeded, since ultimately, the usefulness of a modeldepends on its plausibility. It if looks implausible, it isunlikely to be taken seriously, above all by policymak-ers, who must persuade the public of the soundnessof their judgements and decisions. So just how ‘good’must a model be to be useful? Another examplemight help.

The ancient historical settlement patterns of theAnasazi people in the Long House Valley in northeast Arizona, USA, are notable for a clustering of set-tlements along the boundaries of the valley. Compari-son of the historical patterns (from 400 to 1450 ce)with those generated by an ABM found that althoughthe model’s outputs were not an exact match to thehistorical settlement patterns, the general pattern wasindeed reproduced by the model (Grimm et al. 2005).

Might such a model be ‘good enough’? Perhaps.Presumably, the actual settlement patterns in theLong House Valley were not the only possible settle-ment patterns, just one possible outcome amongmany. Equally, the output from the model simply rep-resents a possible (and plausible) alternative history.The model is deemed ‘accurate’ because the alterna-tive histories it produces are broadly similar to theactual historical settlement patterns, and because theyare not implausible.

So there is a balance to be struck. We want amodel that allows general exploration (i.e. it is notrestricted to modelling a single pattern), that is nottoo complex and that is transparent (Couclelis 2002;Grimm et al. 2005). These are broad criteria, and amodel is more likely to fit them if the ‘process ofmodel development is guided by multiple patternsobserved at different scales and hierarchical levels’(Grimm et al. 2005, 988), a point also made by Brownet al. (2005).

This view, it seems to me, is likely to make senseto anyone involved in land-use planning, not leastbecause that is probably how they think of things any-way. Indeed, exploration of some of these issues inmodels of north-west England revealed that

Boundary Crossing 3

Citation: 2012 doi: 10.1111/j.1475-5661.2012.00532.xISSN 0020-2754 � 2012 The Author.

Transactions of the Institute of British Geographers � 2012 Royal Geographical Society (with the Institute of British Geographers)

In addition to the sheer number of parameters that arenecessary to achieve an adequate picture of regional devel-opment processes, many are also inherently ‘fuzzy’ in nat-ure . . . (Lindley 2001, 150)

This further highlights the need to involve the policy-maker in the process of creating the model, andmight forestall yet another problem.

The problem is this: simply producing results thatmimic historic events risks giving to policymakers asense of false security in the model’s projections ofthe future. Worse yet, these projections may be inter-preted as predictions, and so encourage the erroneousbelief that the model is (or could be) an accurateforecasting tool.

For those seeking certainty, and politicians andstakeholders usually do, such a belief is especiallybeguiling. But an ABM, however good it may be,does not provide certainty. Instead, it is a means ofexploring possible scenarios, a thinking tool thatoffers alternative perspectives on the world (Battyand Torrens 2001; Ligmann-Zielinska and Jankowski2007). A classic example is Schelling’s model of segre-gation, in which a slight preference in agents for thefamiliar over the unfamiliar is enough to generatepatterns of segregation in, for example, differentneighbourhoods (Schelling 1969). It is an elegantillustration of how commonly shared preferences forthe familiar can generate an outcome unsought by theagents and unwanted by policymakers.

Explorations of this ‘model–policy interface’ arelacking. Some examples can be found from the medi-cal, environmental and disaster management litera-ture, which raise questions about it, although theseare often based on multi-criteria analysis rather thanABMs (Elliott and Popay 2000; Kiker et al. 2005;McNown 1986; Zerger and Smith 2003), althoughthere is an ongoing research project exploring therole of transport models in policymaking (TransportStudies Unit 2011). The north-west of England Regio-nal Spatial Strategy (GONW 2008, 12) took thingsone step further, and used maps showing flood risk tosupport a general policy approach to climate changeadaptation (2008, 33). However, there appears to belittle analysis of whether such approaches have actu-ally worked in practice, especially with regard toABMs which present the policymaker with a broaderview of the problems, rather than offering a numberof potential solutions in the way of multi-criteriaapproaches.

We cannot claim that there is a problem with usingABMs because we simply don’t know yet; but as thesemodels drift inevitably out of the hands of their cre-ators, and into the realm of the end-user, it wouldseem sensible to think more carefully about how theuse of such models might work out in practice.

In the end it may prove to be less about technicali-ties and more about expectations. The IPPC qualifiesits projections, speaking in terms of likelihood on ascale that goes from ‘not very likely’ to ‘virtually cer-tain’: for example: ‘Warmer and fewer cold days andnights over most land areas . . . based on projectionsfor 21st century using SRES scenarios . . . [are] . . .virtually certain’ (IPCC 2007, 7). While such languagecan be calibrated against statistical probabilities basedon model outputs, its usefulness lies in the fuzzy qual-ity that makes it better matched to the fuzziness ofreal-world policymaking.

Our perception of ABMs, then, is inevitably boundup with how we expect them to work. An agent-basedmodel is no oracle. If we see ABMs as forecastingtools, and create policy on that basis, we will sufferthe inevitable disappointment of one caught out bytheir own unreasonable expectations. The trick then,is actually to ask for less. For while an ABM has littlepower to give us answers, it can reveal the questionsthat help us better understand the complexities ofthose towns, cities, villages and landscapes with whichwe remain so comfortably familiar.

Acknowledgements

This essay is the better for the comments and sugges-tions of the referees.

References

Batty M 2005 Cities and complexity MIT Press, CambridgeMA

Batty M and Torrens P M 2001 Modelling complexity: the lim-its to prediction Centre for Advanced Spatial Analysis(UCL), London

Batty M, Besussi E and Chin N 2003 Traffic, urban growthand suburban sprawl Centre for Advanced Spatial Analysis(UCL), London

Brown D G, Riolo R, Robinson D T, North M and Rand W2005 Spatial process and data models: toward integrationof agent-based models and GIS Journal of GeographicalSystems 7 25–47

Castle C J and Crooks A T 2006 Principles and concepts ofagent-based modelling for developing geospatial simulationsCentre for Advanced Spatial Analysis (UCL), London

Couclelis H 2002 Modeling frameworks, paradigms, andapproaches in Clarke K C, Parks B E and Crane M P edsGeographic information systems and environmental modelingPrentice Hall, Upper Saddle River, NJ 36–50

Couclelis H 2005 Where has the future gone? Rethinkingthe role of integrated land-use models in spatial planningEnvironment and Planning A 37 1353–71

Coveney P and Highfield R 1995 Frontiers of complexity: thesearch for order in a chaotic world Faber and Faber, London

Crooks A T 2006 Exploring cities using agent-based modelsand GIS Centre for Advanced Spatial Analysis (UCL),London

4 Boundary Crossing

Citation: 2012 doi: 10.1111/j.1475-5661.2012.00532.xISSN 0020-2754 � 2012 The Author.Transactions of the Institute of British Geographers � 2012 Royal Geographical Society (with the Institute of British Geographers)

Crooks A T 2007 The repast simulation ⁄ modelling system forgeospatial simulation Centre for Advanced Spatial Analysis(UCL), London

Eclipse Foundation 2009 Eclipse IDE for Java programmersEclipse Foundation (http://www.eclipse.org) Accessed May2011

Elliott H and Popay J 2000 How are policy makers using evi-dence? Models of research utilisation and local NHS policymaking Journal of Epidemiology andCommunityHealth 54 461

Evans T P and Manson S M 2007 Space, complexity, andagent-based modeling Environment and Planning B: Plan-ning and Design 34 196–9

Fontaine C 2011 Scenarios of urban change in East Anglia,UK, produced by an Agent-based model (ABM) Centrefor the study of Environmental Change and Sustainability,University of Edinburgh (http://www.geos.ed.ac.uk/research/cecs/research/cecs/research/Land_Use.html#anglia) Accessed14 November 2011

GONW 2008 North West of England plan – regional spatialstrategy to 2021 Government Office for the NorthWest ⁄ The Stationery Office, London

Grimm V, Revilla E, Berger U, Jeltsch F, Mooij W M, Rails-back S F, Thulke H, Weiner J, Wiegand T and DeAngelisD L 2005 Pattern-oriented modeling of agent-based com-plex systems: lessons from ecology Science 310 987–91

Haase D and Nuissl H 2007 Does urban sprawl drive changesin the water balance and policy? The case of Leipzig (Ger-many) 1870–2003 Landscape and Urban Planning 80 1–13

IPCC 2007 Climate change 2007: the physical science basis,summary for policymakers IPCC, Geneva

Janssen M A and Ostrom E 2006 Empirically based, agent-based models Ecology and Society 11 37 (http://www.ecologyandsociety.org/vol11/iss2/art37/) Accessed October 2011

Johnson J 2001 The ‘can you trust it?’ Problem of simulationscience in the design of socio-technical systems Complexity6 34–40

Kabisch N, Haase D and Haase A 2010 Evolving reurbanisa-tion? Spatio-temporal dynamics as exemplified by the EastGerman city of Leipzig Urban Studies 47 967

Kiker G A, Bridges T S, Varghese A, Seager T P and LinkovI 2005 Application of multicriteria decision analysis inenvironmental decision making Integrated EnvironmentalAssessment and Management 1 95–108

Lavalle C, Demicheli L, Turchini M, Casals-Carrasco P andNiederhuber M 2001 Monitoring megacities: theMOLAND ⁄ MURBANDY approach Development inPractice 11 350–7

Lavalle C, Niederhuber M, McCormick N and DeMicheli L2002 The MURBANDY ⁄ MOLAND methodology and itspotential to support sustainable city development, Occa-sional paper (http://enviroinfo.isep.at/UI%20200/LavalleC_110700.el.ath.pdf) Accessed October 2011

Lee Jr D B 1973 Requiem for large-scale models Journal ofthe American Institute of Planners 39 163–78

Ligmann-Zielinska A and Jankowski P 2007 Agent-basedmodels as laboratories for spatially explicit planning policiesEnvironment and Planning B: Planning and Design 34 316–35

Lindley S J 2001 Virtual tools for complex problems: anoverview of the AtlasNW regional interactive sustainabilityatlas for planning for sustainable development ImpactAssessment and Project Appraisal 19 141–51

Longley P A, Goodchild M F, Maguire D J and Rhind D W2005 Geographical information systems and science JohnWiley & Sons Ltd, Chichester

Louie M A and Carley K M 2007 The role of dynamic-net-work multi-agent models of socio-political systems in policyTechnical report CMU-ISRI-07-102 Carnegie-Mellon Uni-versity Institute for Software Research

Louie M A and Carley K M 2008 Balancing the criticisms:validating multi-agent models of social systems SimulationModelling Practice and Theory 16 242–56

Manson S M 2007 Challenges in evaluating models of geo-graphic complexity Environment and Planning B: Planningand Design 34 246–60

Marmot M G 2004 Evidence based policy or policy basedevidence? BMJ 328 906

McNown R 1986 On the uses of econometric models: a guidefor policy makers Policy Sciences 19 359–80

Meadows D H, Randers J and Meadows D L 2004 The limitsto growth: the 30-year update Earthscan, London

Murphy J M, Sexton D M H, Jenkins G J, Booth B B B, BrownC C, Clark R T, Collins M, Harris G R, Kendon E J, Betts RA, Brown S J, Humphrey K A, McCarthy M P, McDonaldR E, Stephens A, Wallace C, Warrent R, Wilby R and WoodR A 2009 UK climate projections science report: climatechange projections Met Office Hadley Centre, Exeter

Nyerges T L and Jankowski P 2010 Regional and urbanGIS: a decision support approach The Guilford Press,New York

O’Sullivan D 2004 Complexity science and human geographyTransactions of the Institute of British Geographers 29 282

Petrov L O, Lavalle C and Kasanko M 2009 Urban land usescenarios for a tourist region in Europe: applying theMOLAND model to Algarve, Portugal Landscape andUrban Planning 92 10–23

PLUREL 2008 PLUREL: peri-urban land use relationships –strategies & sustainability assessment tools for urban–rurallinkages University of Copenhagen (http://www.plurel.net)Accessed May 2012

RIKS 2011 Wuhan urban accessibility planning supportsystems (http://www.riks.nl/projects/Wuhan%20UAPSS)Accessed 3 October 2011

Schelling T C 1969 Models of segregation The AmericanEconomic Review 59 488–93

Schmidt G 2011 ‘Gavin Schmidt talks about the politics of cli-mate change’ (YouTube Video) (http://www.youtube.com/watch?v=krXF9Icfa6k&feature=related) Accessed Octo-ber 2011

Transport Studies Unit 2011 The use of state-of-the-artmodels by policy makers to address global energy andenvironment challenges: the case of transport policy Trans-port Studies Unit (http://www.tsu.ox.ac.uk/research/models/)Accessed 24 October 2011

VOTES Project 2011 Welcome to the VOTES project —FUNDP (http://www.fundp.ac.be/en/sci/geography/votes/)Accessed 14 November 2011

Wilensky U 2011 NetLogo 4.1.3 user manual Center for Con-nected Learning and Computer-Based Modeling, North-western University, Evanston IL

Zerger A and Smith D I 2003 Impediments to using GIS forreal-time disaster decision support Computers, Environmentand Urban Systems 27 123–41

Boundary Crossing 5

Citation: 2012 doi: 10.1111/j.1475-5661.2012.00532.xISSN 0020-2754 � 2012 The Author.

Transactions of the Institute of British Geographers � 2012 Royal Geographical Society (with the Institute of British Geographers)