Technological Forecasting & Social Change 77 (2010) 917–923

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

Addressing deep uncertainty using adaptive policies: Introduction tosection 2

Warren E. Walker a,⁎, Vincent A.W.J. Marchau a, Darren Swanson b

a Delft University of Technology, Delft, The Netherlandsb International Institute for Sustainable Development, Winnipeg, Manitoba, Canada

a r t i c l e i n f o

⁎ Corresponding author. P.O. Box 5015, 2600 GA DeE-mail address: w.e.walker@tudelft.nl (W.E. Walke

0040-1625/$ – see front matter © 2010 Elsevier Inc.doi:10.1016/j.techfore.2010.04.004

Article history:Accepted 31 March 2010

1. About uncertainty

In a broad sense, uncertainty can be simply defined as missing knowledge; i.e., the absence of information. With respect topolicymaking, uncertainty refers to the gap between available knowledge and the knowledge policymakers would need in order tomake the best policy choice. This uncertainty clearly involves subjectivity, since it is related to satisfaction with existingknowledge, which is colored by the underlying values and perspectives of the policymaker (and the various actors involved in thepolicymaking process). Uncertainty can be associated with all aspects of a policy problem (e.g., the system of interest, the worldoutside the system, the outcomes from the system, the weights stakeholders place on the various outcomes, etc.).

There are myriad examples of how uncertainty plays havoc with public policies and policymaking. For example, in 1995, after atwo-year deliberative process, some decisions were made by the Dutch Parliament that were intended to guide the growth of civilaviation in the Netherlands to the year 2015. One of the outcomes of the process was the decision to constrain the number ofpassengers at Schiphol to no more than 44 million passengers per year. This constraint was supposed to be more than enough toaccommodate the most optimistic estimates of passenger growth until at least the year 2015. This limit was actually reached in2004. And the noise limits, also expected to be reached no sooner than 2015, were reached in 1999. As a result, policymakers wereforced to revisit their air transport policy (something they thought they would not have to do until 2015). Swanson et al. [1]describe a long-standing rate control agreement for transporting grain produced on the Canadian Prairies by rail. Known as theCrow Rate, the freight rate was engraved in legislation in 1925 and remained fixed for sixty years. The unanticipated effects ofinflation combined with the fixed rate eroded the railway's revenues leading to a significant deterioration of the rail system over aspan of several decades. Moench [2] articulates a startling case of the Kosi River in Nepal (a tributary to the Ganges) in whichembankments failed due to high sediment loads causing the river to want to shift course. Over sixty thousand people weredisplaced in Nepal and a half million in India. The failure was not in response to an extreme event, but a complex array of social,political, and environmental relationships that built up over time.

The above examples indicate that policy failures often follow from a failure to take uncertainties into account in making policy,and suggest that taking into account uncertainty can be essential for successful long-term policymaking. It is clear that uncertaintyis at the heart of the very nature of long-term policymaking. In long-term policymaking, decision makers must make decisionsabout the future. The future is impossible to predict. But, that is no reason to throw up one's hands and ignore uncertainty. Quitethe opposite. Ignoring uncertainty could lead to large adverse consequences for people, countries, and the earth's ecosystems, and

lft, The Netherlands.r).

All rights reserved.

918 W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

policymakers have an interest in minimizing the possibility that these would happen. So, policy analysts and policymakers mustaccept, understand, and manage uncertainty, since:

• crystal balls do not exist, so uncertainties about the future cannot be eliminated;• ignoring uncertainty can result in poor policies, missed chances and opportunities, and lead to inefficient use of resources; and• ignoring uncertainty could mean that we limit our ability to take corrective action in the future and end up in situations thatcould have been avoided.

In order to manage uncertainty, one must be aware that an entire spectrum of different levels of knowledge exists, rangingfrom the unachievable ideal of complete deterministic understanding at one end of the scale to total ignorance at the other. Therange of levels of uncertainty, and their challenge to decisionmakers, was acknowledged by Donald Rumsfeld, who famously said:

1 Don

“As we know, there are known knowns – these are things we know we know. We also know there are known unknowns –that is to say we know there are some things we do not know; but there are also unknown unknowns – the ones we don'tknow we don't know…. It is the latter category that tends to be the difficult one.”1

For purposes of determining ways of dealing with uncertainty in developing public policies or business strategies, Courtney [3]and Walker et al. [4] have distinguished two extreme levels of uncertainty (determinism and total ignorance) and fourintermediate levels. In Fig. 1, the four levels are defined with respect to the knowledge assumed about the various aspects of apolicy problem: (a) the future world, (b) themodel of the relevant system for that future world, (c) the outcomes from the system,and (d) the weights that the various stakeholders will put on the outcomes. The levels of uncertainty are briefly discussed below.

Determinism is the ideal situation in which we know everything precisely. It is not attainable, but acts as a limitingcharacteristic at one end of the spectrum.

Level 1 uncertainty is any uncertainty that can be described adequately in statistical terms. In the case of uncertainty about thefuture, Level 1 uncertainty is often captured in the form of a (single) forecast (usually trend based) with a confidence interval. Anexample of Level 1 uncertainty is the measurement uncertainty associated with all data. Measurement uncertainty stems from thefact that measurements can practically never precisely represent the “true” value of what is being measured.

Level 2 uncertainty implies that there are alternative, trend-based futures and/or different parameterizations of the systemmodel, and some estimate can be made of the probability of each of them. In the case of uncertainty about the future, Level 2uncertainty is often captured in the form of a few trend-based scenarios based on alternative assumptions about the driving forces(e.g., three trend-based scenarios for air transport demand, based on three different assumptions about GDP growth). Thescenarios are then ranked accordingly to their likelihood.

Level 3 uncertainty represents deep uncertainty about the mechanisms and functional relationships being studied. We knowneither the functional relationships nor the statistical properties, and there is little scientific basis for placing believableprobabilities on scenarios. In the case of uncertainty about the future, Level 3 uncertainty is often captured in the form of a widerange of plausible scenarios.

Level 4 uncertainty implies the deepest level of recognized uncertainty; in this case, we only know that we do not know.Recognized ignorance is increasingly becoming a common feature of our existence, because catastrophic, unpredicted, surprising,but painful events seem to be occurring more often. Taleb [5] calls these events “Black Swans”. He defines a Black Swan event asone that lies outside the realm of regular expectations (i.e., “nothing in the past can convincingly point to its possibility”), carriesan extreme impact, and is explainable only after the fact (i.e., through retrospective, not prospective, predictability). One of themost dramatic recent Black Swans is the concatenation of events following the (2007) subprime mortgage crisis in the UnitedStates. The mortgage crisis (which some had forecast) led to a credit crunch, which led to bank failures, which led to a globalrecession (in 2009), which was outside the realm of regular expectations. A more recent example is the aftermath of the eruptionof the volcano in Iceland.

Total ignorance, which is the other extreme from determinism on the scale of uncertainty, acts as a limiting characteristic atother end of the spectrum.

Makridakis et al. [6] make a similar distinction among levels of uncertainty. They call Level 1 and Level 2 uncertainties ‘SubwayUncertainty’ and Level 3 and Level 4 uncertainties ‘Coconut Uncertainty’. As they explain: “‘Subway’ uncertainty refers towhatwe canmodel and reasonably incorporate in probabilistic predictions that assume, for example, normally distributed forecasting errors.‘Coconut’ uncertainty pertains to events that cannot bemodeled, and also to rare and unique events that are simply hard to envision.”

2. Deep uncertainty

There are many policy analysis approaches to deal with Level 1 and Level 2 uncertainties. In fact, as pointed out byMcDaniel andDriebe [7], most of the traditional applied scientific work in the engineering, social, and natural sciences has been built on thesupposition that the uncertainties result from a lack of information, which “has led to an emphasis on uncertainty reduction throughever-increasing information seeking and processing”, or from random variation, which has concentrated efforts on stochasticprocesses and statistical analysis. However, most of the important policy problems currently faced by policymakers are

ald Rumsfeld, Department of Defense news briefing, Feb. 12, 2002.

Fig. 1. The progressive transition of levels of uncertainty from determinism to total ignorance.

919W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

characterized by the higher levels of uncertainty, which cannot be reduced by gathering more information. The uncertainties areunknowable at the present time, but will be reduced over time. They can involve uncertainties about all aspects of a policy problem—

external developments, the appropriate (future) system model, the parameterization of the model, the model outcomes, and thevaluation of the outcomes by (future) stakeholders. Such situations have been characterized as having ‘deep uncertainty’—definedas “the condition in which analysts do not know or the parties to a decision cannot agree upon (1) the appropriate models todescribe interactions among a system's variables, (2) the probability distributions to represent uncertainty about key parameters inthe models, and/or (3) how to value the desirability of alternative outcomes” [8].

New approaches are needed to deal with conditions of deep uncertainty, since the approaches used for handling Level 1 and 2uncertainties are inadequate for policymaking. Quade [9] (p. 160) writes: “Stochastic [Level 1 and Level 2] uncertainties aretherefore among the least of our worries; their effects are swamped by uncertainties about the state of the world and humanfactors for which we know absolutely nothing about probability distributions and little more about the possible outcomes.”Goodwin and Wright [10] (p. 355) demonstrate that “all the extant forecasting methods—including the use of expert judgment,statistical forecasting, Delphi and prediction markets—contain fundamental weaknesses.” And Popper et al. [11] state that thetraditional methods “all founder on the same shoals: an inability to grapple with the long-term's multiplicity of plausible futures.”

Traditionally, the past has been seen as a reasonably reliable predictor of the future. Such an assumption works well when change isslowandsystemelements arenot too tightly connected. It alsoworkswellwhen there arenot toomanyBlackSwans coming towardus. Infact, assuming that the futurewill be fairly closely related to thepast is thebasis of classical statistics andprobability, and the foundation forthe way we live our lives. However, if this assumption is not a valid one, policies based on it may be deeply flawed.

3. Need for adaptive policies to handle deep uncertainty

The shortcomings of the traditional approaches for dealing with uncertainty mentioned above suggest that new approaches topolicymaking under conditions of deep uncertainty are needed—approaches that protect against and/or prepare for unforeseeabledevelopments. It is clear from experience that a static policy designed for a best estimate future is unlikely to survive in a complexand dynamic policy setting. A static policy that is crafted to be robust under a range of plausible futures is a better starting point,but is still not likely to perform under conditions of deep uncertainty given the inevitability of Black Swans.

Broadly speaking, although there are differences in definitions, and ambiguities in meanings, the literature offers three(overlapping, not mutually exclusive) ways for dealing with deep uncertainty in making policies [12]:

• resistance: plan for the worst possible case or future situation• resilience: whatever happens in the future, make sure that you can recover quickly• adaptation: prepare to change the policy, in case conditions change

920 W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

This special section deals mainly with the last category. A policy that can adapt to changing conditions is well suited tosituations involving deep uncertainty. An adaptive policy is aware of themultiplicity of plausible futures that lie ahead, is designedto be changed over time as new information becomes available, and leverages autonomous response to surprise. The adaptivepolicy approachmakes adaptation explicit at the outset of policy formulation. Thus, the inevitable policy changes become part of alarger, recognized process and are not forced to bemade repeatedly on an ad-hoc basis. Under this approach, significant changes inthe system would be based on an analytic and deliberative effort that first clarifies system goals, and then identifies policiesdesigned to achieve those goals and ways of modifying those policies as conditions change.Within the adaptive policy framework,individual actors would carry out their activities as they would under normal policy conditions. But policymakers andstakeholders, through monitoring and corrective actions, would try to keep the system headed toward the original goals. McCrayet al. [13] describe it succinctly as keeping policy “yoked to an evolving knowledge base.”

There is a definite appetite for such approaches, as demonstrated by the papers in this section. For example, Swanson et al.describe how Canada's National Round Table on the Environment and the Economy recently recommended that governmentpolicies for reducing greenhouse gas emissions should “incorporate adaptive management practices and have built-in monitoringand assessment mechanisms to allow for regular reviews to ensure efficiency and effectiveness [14].” The Round Table even wenton to say that such an adaptive approach would “ensure that progress is monitored, compliance issues are addressed, and policiesare adjusted to match the required level of abatement effort, and will minimize and mitigate unanticipated adverse outcomes.”But, while there is certainly an appetite for adaptive policy approaches in many governments and supporting institutions, it can beviewed as a destabilizing influence by those closest to the policy formulation process. Evidence from the United States revealedthat in most observed examples, planned policy adaptation was imposed from outside the executing agencies themselves,suggesting many challenges and barriers to implementing adaptive policies [13].

Another example comes from the Dutch Advisory Council for Transport, Public Works and Water Management [15]. In theiradvice to the Transport Ministry, the Council recommends a prompt and cost-effective adaptation of the infrastructure to copewith climate change, giving systematic consideration to the possible effects of climate change when weighing up infrastructureinvestment decisions. In the English summary of their report, the Council says: “This implies that a fundamentally differentapproach to the uncertainties associated with climate change must be adopted in policymaking and in government. A change ofoutlook is needed: the pursuit of certainty should be replaced by the acceptance of and allowance for uncertainty. Instead ofbasing policy on what is or appears to be certain, uncertainties should be explicitly covered by the policy analysis andproactively accommodated in the policies that are formulated. One possibility is to use ‘planned adaptation’ or ‘adaptivepolicies’” [15].

4. Adaptive approaches for handling deep uncertainty

This section presents a state-of-the-art overview of different adaptive approaches for policymaking and their applicationswithin different domains. The literature on adaptive policies is relatively sparse and diffuse, and a typology of approaches has yetto emerge. As a starting point for advancing an understanding of adaptive policies, it is helpful to structure the array of adaptiveresponses that have been documented in the context of environmental change. Basic dimensions to distinguish different types ofadaptation are presented in Table 1 and are based upon [16]:

1. Purposefulness, divided into:• Planned adaptation, which is the result of deliberate policy decisions, based on an awareness that conditions might change orhave changed and that action is required to return to, maintain, or achieve a desired state.

• Autonomous adaptation, which is adaptation that is not a planned external response to a situation, but is an internal systemreaction due to changes within the system.

2. Timing, divided into:• Anticipatory adaptation, which takes place before negative impacts are observed.• Reactive adaptation, which takes place after negative impacts are observed.

The examples of terms used in Table 1 are descriptive and supportive in distinguishing different types of adaptation fromanother. All of the above types of adaptive responses have a role to play in helping policymakers navigate deeper uncertainties inpolicy design and implementation.

able 1ases for differentiating adaptations [16].

General differentiating concept or attribute Examples of terms used

Purposefulness AutonomousSpontaneousAutomaticNatural

PlannedPurposefulIntentionalStrategic

Timing AnticipatoryProactiveEx-ante

ResponsiveReactiveEx-post

TB

921W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

The papers contained in this section can all be related to some degree to the above typology. The first paper, by Swanson et al.[1], describes observed tools from agriculture and water resources management that can be used singly or in combination todevelop adaptive policies. These tools and their relation to the adaptation typology of Table 1 are as follows:

1. Integrated and forward-looking analysisThis involves participatory scenario planning for policymaking and is an effective tool for anticipatory/planned types of policyadaptation.

2. Built-in policy adjustmentSome necessary policy improvements can be anticipated in advance and signposts monitored to trigger their implementation.This is a tool that facilitates anticipatory/planned adaptation.

3. Formal policy review and continuous learningSystematic review of policy is an important tool for anticipatory/planned adaptation. Treating policies as hypotheses for whichassumptions must be continually tested also makes it more likely to detect surprises before they occur.

4. Multi-stakeholder deliberationDialogue among stakeholders strengthens policy design in many ways. This tool facilitates anticipatory/planned adaptation, aswell as autonomous/reactive adaptation undertaken by stakeholders most directly affected by policy.

5. Enabling self-organization and social networkingA policy that does not undermine existing social capital and actively facilitates the sharing of good practices strengthens thepotential for autonomous adaptation in the face of deep uncertainty.

6. Decentralization of decision makingAutonomous adaptation can also be enabled by placing the authority and responsibility for decision making at the lowesteffective and accountable unit of governance.

7. Promoting variationImplementing a variety of policies to address the same issue increases the likelihood of achieving desired outcomes and isillustrative of planned/anticipatory adaptation. Additionally, such diversity creates opportunity for autonomous response tosurprise.

The second paper, by Marchau et al. [17], specifies a structured, stepwise procedure for defining an adaptive policy that is bothanticipatory and reactive. (They call such a policy a dynamic adaptive policy.) Their approach allows implementation of a ‘basicpolicy’ to begin prior to the resolution of all major uncertainties, with the policy being adapted over time based on new knowledge.Most importantly, the approach advocates not only the development of a monitoring system but also the pre-specification ofresponses when specific trigger values are reached. The approach is illustratedwith adaptive policies for solving various long-termproblems in the fields of road, rail, and air transport. It is similar to what Eriksson and Weber [18] call Adaptive Foresight.

The third paper, by McCray et al. [13], documents examples of planned/reactive policy adaptation approaches observed inregulations designed and implemented in the United States. Specifically, these researchers are interested in “fostering policyadjustment without the ruckus”, or, more specifically, “can governments bring new knowledge to old policies in amore thoughtfulway, and one in which the underlying uncertainties are successively reduced—or at least better characterized—over time?” Theyconclude that, although rare in U.S. rule-making, planned adaptation is feasible as a decision strategy, even in tough policy areas.Their policy adaptation is focused on making changes based on a regularly scheduled review of the situation. So, it is planned, butreactive.

The fourth paper, by Lempert and Groves [19], discusses a Robust Decision Making (RDM) approach for identifying andevaluating adaptive policy responses. The paper presents an application to water management and climate change in theAmerican West. RDM supports making decisions under conditions of deep uncertainty and uses simulation models to assess theperformance of a water agency's plans over thousands of plausible futures. Statistical ‘scenario discovery’ algorithms then identifya small number of policy-relevant scenarios that concisely summarize those futures in which the plans fail to meet the agency'sperformance goals. These scenarios are then used to help decisionmakers understand the vulnerability of their plans and to assessthe options for ameliorating these vulnerabilities. In relation to the different types of adaptation approaches, RDM facilitatesanticipatory/planned adaptation.

The fifth paper, byMoench [2], presents a case study of embankment failures on the Kosi River in Nepal and addresses the arrayof anticipatory/planned adaptive responses as well as autonomous/reactive responses. Moench argues that, “at the policy level,approaches to identify and plan for anticipated changes need to be linked with approaches that enable local populations to evolvetheir own response strategies in response to the opportunities and constraints they facewithin their location specific contexts.”Hestresses that local populations are often “the only ones that have the intimate long-term knowledge of local conditions (whether itbe drainage patterns, the status of flood control structures, or opportunities for livelihood diversification) that are central to floodrisks and responses.” As a result, Moench concludes that “policy frameworks that enable action to be taken at multiple levels by thesets of actors who are most directly affected and engaged at that level are essential. This includes enabling local communities andregions to form institutions for responding to emerging problems at any scale necessary to respond, whether that be highlylocalized or across the basin as a whole.”

The final paper in this section, by Wardekker et al. [20], explores the utility of six resilience principles from the ecological andsystem dynamics literature in the context of identifying strategies for an urban delta in the Netherlands in response to climatechange. These principles indicate the close relationship (and, indeed, overlap) between the use of resilience and adaptation in

922 W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

formulating policies to deal with deep uncertainty. In fact, the principles offer insight into possible system adaptations that makethe system more resilient. The six principles and their relation to the adaptation typologies are as follows:

1. Homeostasis: Incorporating feedback loops that stabilize the system to external perturbations. This is an example of planned/anticipatory adaptation. For example, insurance incentives accepting water-related problems; early flood warningmechanisms; floating houses.

2. Omnivory: Different approaches that can be used in combination to fulfill one's needs. This is also an example of the planned/anticipatory adaptation approach.

3. High flux: Allowing for quick responses to threats and changes. This principle as articulated in the paper supports both theplanned/anticipatory and planned/reactive adaptation types of adaptive responses. For example, planning green areas or otherquickly modifiable land use areas where future changes may be required.

4. Flatness: Limiting excessive decision making hierarchies—“in top-heavy systems, early warning signals observed at the bottomreach higher levels too slowly”. This is an example of facilitating autonomous/reactive adaptation.

5. Buffering: Enhancing the ability to absorb disturbances to a certain extent. For example, having certain non-essential low lyingareas serve as water retention areas for a limited period. This is an example of planned/anticipatory adaptation.

6. Redundancy: Having multiple instances of something available in the event that one fails. For example, having multiple roadsand ferry services and electrical and sewage services. This is an example of planned/anticipatory adaptation.

5. Conclusions: cross-cutting findings on adaptive policy approaches

Discussion of adaptive policy approaches is, for themost part, in a relatively early phase of application and exploration. As such,we have not attempted to forge a consensus definition among the contributing authors of this special section about what anadaptive policy is or should be. That said, there are definitely clear threads of consistency and agreement among the papers in thissection. These can best be viewed through the general typology of adaptive responses described earlier (see Table 1).

Marchau et al., Lempert and Groves, Swanson et al., andMcCray et al. each stress the importance of anticipatory/planned policyadaptation through the systematic update of key policy parameters in light of new information, and the idea that policy should bein a continuous learning state to enable a systematic intake of new information as opposed to an ad-hoc process of policy reviewand change. In this context, the first three papers recommend scenario planning as an important element in developinganticipatory/planned adaptive policies.

Moench, Swanson et al., and (to a degree) Wardekker et al., all propose that adaptive policies be linked to local capacity fordecision making, where stakeholders are the first to detect change and can self-organize to cope with emergent stress and takeadvantage of opportunities.

The ‘omnivory’ and ‘redundancy’ principles explored by Wardekker et al. map closely to the adaptive policy tool of ‘promotingvariation’ recommended by Swanson et al. Similarly, between these two papers, the ‘flatness’ principle mirrors the‘decentralization of decision making’ tool, and the ‘high flux’ principle mirrors the spirit of the ‘enabling self-organization andsocial networking’ tool.2

Finally, some cross-cutting challenges for future research on adaptive policies can be identified from a review of the paperscontained in this section. These are summarized below:

Tools and methods for developing adaptive policies: Adaptive policymaking is a way of dealing with deep uncertainty that fallsbetween too much precaution and acting too late. While the need for adaptation is increasingly acknowledged, it is still adeveloping concept, and requires the further development of specific tools and methods for its operationalization.

Costs and benefits of adaptive policies vs. traditional static policies: A key issue in adaptive policies is how to cope with the risk ofunder-investing and over-investing (or wrongly investing) in future policies. Also, the initial costs of an adaptive policy are oftenhigher than those of a static policy. An in-depth analysis of the costs and benefits of adaptive policies (as compared to traditionalstatic policies) is required.

The efficacy of adaptive policies vs. traditional static policies: There are few examples of the actual implementation of adaptivepolicies. One important reason for this lack of application is that their efficacy has not yet been established [21]. Evidence on theefficacy of adaptive policies compared to traditional static policies can help to convince policymakers and stakeholders to adoptadaptive policies.

Institutional implications of adaptive policymaking: The development and execution of adaptive polices is influenced by theinstitutional framework. An important question is whether institutions have the capability to cope with deep uncertainty. Guptaet al. [22] argue that institutions are traditionally conservative and lack the capability to develop robust response strategies. Theimplementation of adaptation can be hindered by several institutional and social complexities, often felt as barriers for change,such as too many policy domains, too many administrative levels, too fragmented and rigid regulation and budgets, too detailedplanning and budget allocation procedures, lack of awareness, insufficient learning capacity of key players, etc. (e.g. [23]). AsLempert and Light [24] say, “today's policymakers generally lack the tools and the institutions that can…identify priority long-

2 The similarities between the concepts presented in these two papers are not surprising because the conceptual foundation for the recommendations made bySwanson et al. comes from a review of the complex adaptive systems literature and those from Wardekker et al. originate from the ecological and systemdynamics literature—a largely common pool of literature, which includes the resilience literature.

923W.E. Walker et al. / Technological Forecasting & Social Change 77 (2010) 917–923

term decisions”. A major research challenge, therefore, appears to be about the institutional arrangements that are necessary foradaptive plans to be developed and implemented. Addressing this issue will be of crucial importance for the success or failure ofadaptive policy approaches.

References

[1] D. Swanson, S. Barg, S. Tyler, H. Venema, S. Tomar, S. Bhadwal, S. Nair, D. Roy, J. Drexhage, Seven tools for creating adaptive policies, Technol. Forecast. Soc.Change 77 (6) (2010) 924–939 (this issue).

[2] M. Moench, Responding to climate and other change processes in complex contexts: challenges facing development of adaptive policy frameworks in theGanga Basin, Technol. Forecast. Soc. Change 77 (6) (2010) 975–986 (this issue).

[3] H. Courtney, 20/20 Foresight: Crafting Strategy in an Uncertain World, Harvard Business School Press, Boston, 2001.[4] W.E. Walker, P. Harremoës, J. Rotmans, J.P. van der Sluijs, M.B.A. van Asselt, P. Janssen, M.P. Krayer von Krauss, Defining uncertainty: a conceptual basis for

uncertainty management in model-based decision support, Integr. Assessment 4 (1) (2003) 5–17.[5] N.N. Taleb, The Black Swan: The Impact of the Highly Improbable, Random House, New York, 2007.[6] S. Makridakis, R.M. Hogarth, A. Gaba, Forecasting and uncertainty in the economic and business world, Int. J. Forecasting 25 (2009) 794–812.[7] R.R. McDaniel, D.J. Driebe (Eds.), Uncertainty and Surprise in Complex Systems: Questions on Working with the Unexpected, Springer, 2005.[8] R. Lempert, S. Popper, S. Bankes, Shaping the Next One Hundred Years: New Methods for Quantitative Long-term Policy Analysis, MR-1626-RPC, The RAND

Pardee Center, Santa Monica, 2003.[9] E.S. Quade, Analysis for Public Decisions, Third EditionElsevier Science Publishers B.V, Amsterdam, 1989.

[10] P. Goodwin, G. Wright, The limits of forecasting methods in anticipating rare events, Technol. Forecasting Soc. Change 77 (3) (2010) 355–368.[11] S.W. Popper, J. Griffin, C. Berrebi, T. Light, E.Y. Min, Natural Gas and Israel's Energy Future: A Strategic Analysis Under Conditions of Deep Uncertainty, TR-747-

YSNFF, RAND, Santa Monica, 2009.[12] A. Leusink and H.A. Zanting, Naar een afwegingskader voor een klimaatbestendig Nederland, met ervaringen uit 4 case studies : samenvatting voor

bestuurders [Towards a trade-off framework for climate proofing the Netherlands, with experiences from 4 case studies: executive summary], (2009). http://edepot.wur.nl/15219 (accessed 22 March 2010).

[13] L.E. McCray, K.A. Oye, A.C. Petersen, Planned adaptation in risk regulation: An initial survey of US environmental, health, and safety regulation, Technol.Forecast. Soc. Change 77 (6) (2010) 951–959 (this issue).

[14] National Round Table on the Environment and the Economy, Getting to 2050: Canada's Transition to a Low-emission Future, Report of the National RoundTable on the Environment and the Economy (NRTEE), Government of Canada, 2007, http://www.nrtee-trnee.com/eng/publications/getting-to-2050/2-1-getting-to-2050.php, retrieved 31 March 2010.

[15] RVW (Raad voor Verkeer en Waterstaat), Witte zwanen, zwarte zwanen: Advies over proactieve adaptatie aan klimaatverandering [White Swans, BlackSwans: Advice on Proactive Adaptation to Climate Change], 2009 The Hague.

[16] B. Smit, I. Burton, R.J.T. Klein, R. Street, The science of adaptation: a framework for assessment, Mitig. Adapt. Strat. Glob. Change 4 (3–4) (1999) 199–213.[17] V.A.W.J. Marchau, W.E. Walker, G.P. van Wee, Dynamic adaptive transport policies for handling deep uncertainty, Technol. Forecast. Soc. Change 77 (6)

(2010) 940–950 (this issue).[18] E.A. Eriksson, K.M. Weber, Adaptive foresight: navigating the complex landscape of policy strategies, Technol. Forecasting Soc. Change 75 (2008) 462–482.[19] R.J. Lempert, D.G. Groves, Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American

West, Technol. Forecast. Soc. Change 77 (6) (2010) 960–974 (this issue).[20] J.A. Wardekker, A. de Jong, J.M. Knoop, J.P. van der Sluijs, Operationalising a resilience approach to adapting an urban delta to uncertain climate changes,

Technol. Forecast. Soc. Change 77 (6) (2010) 987–998 (this issue).[21] R.J. Hansman, C. Magee, R. De Neufville, R. Robins, D. Roos, Research agenda for an integrated approach to infrastructure planning, design, and management,

Int. J. Crit. Infrastruct. 2 (2–3) (2006) 146–159.[22] J. Gupta, K. Termeer, J. Klostermann, S. Meijerink, M. van den Brink, P. de Jong, S. Nooteboom, Institutions for Climate Change, Report W-08/12, Institute for

Environmental Studies, Amsterdam, 2008.[23] M. Hulme,W.N. Adger, S. Dessai, M. Goulden, I. Lorenzoni, D. Nelson, L.-O. Naess, J. Wolf, and A.Wreford, Limits and Barriers to Adaptation: Four Propositions,

Tyndall Briefing Note No 20, London, 2007.[24] R.J. Lempert, P.C. Light, Evaluating and Implementing Long-term Decisions, in: R.J. Lempert, S.W. Popper, E.Y. Min, J.A. Dewar (Eds.), Shaping Tomorrow

Today: Near-term Steps Towards Long-term Goals, CF-267-RPC, RAND, Santa Monica, 2009.

Warren E. Walker is Professor of Policy Analysis in the Faculty of Technology, Policy and Management at the Delft University of Technology. He has a Ph.D. inOperations Research from Cornell University, and has 40 years of experience in public policy analysis. Previously, he was a member of the staff of RAND Europe,where he oversawmuch of the organization's work on transport and infrastructure. Hewas an advisor to the Director for Freight Policy in the NetherlandsMinistryof Transport, PublicWorks andWaterManagement, and has carried out strategic planning studies for that Directorate. His recent research has focused on transportpolicy and the treatment of uncertainty in policymaking. Dr. Walker is recipient of the 1974 Lanchester Prize from the Operations Research Society of America, andthe 1997 President's Award from the Institute for Operations Research and the Management Sciences (INFORMS) for his “contributions to the welfare of societythrough quantitative analysis of governmental policy problems.”

VincentMarchau is an Associate Professor in the Transport Policy and Logistics Organisation (TLO) section of the Faculty of Technology, Policy andManagement atthe Delft University of Technology (TUD). He completed his study in Applied Mathematics at TUD in 1992. He received his PhD on Technology AssessmentAutomated Vehicle Guidance at TUD in 2000. His work focuses on research and lecturing in the field of transport innovations and decision making underuncertainty. He has been involved as a principal researcher within various national and international projects on transport policy development regardinginnovations. Marchau is a member of the Editorial Board of the European Journal of Transport and Infrastructure Research (EJTIR), guest-editor of different scientificjournals, a Research Fellow of the Dutch Research School on TRAnsport, Infrastructure and Logistics (TRAIL), and director of education of the MSc program of theTransport, Infrastructure and Logistics (TIL) Research School. Currently, his main interest involves the development and evaluation of long-term transport policieswithin the context of deep uncertainty.

Darren Swanson is a Senior Project Manager working with the Measurement and Assessment Program at the International Institute for Sustainable Developmentin Winnipeg, Canada. He is a sustainable development policy specialist and professional engineer with 18 years of consulting and research experience. He workswith governments at all levels and from around the world on strategic processes for organizational and societal-wide learning and adaptive management,including sustainable development strategies, indicator information systems, integrated assessment methods and adaptive policymaking approaches. He holds aMaster of Public Administration degree in international development from the Kennedy School of Government at Harvard University, a Master of Geo-environmental Engineering degree from the University of Saskatchewan, Canada, and a Bachelor's degree in civil engineering.