1 Robert S. Webb, Roger S. Pulwarty, Kristen Averyt Defining and Expressing Uncertainty Or, how not to do the wrong thing more precisely "In this world

1

Robert S. Webb, Roger S. Pulwarty, Kristen Averyt

Robert S. Webb, Roger S. Pulwarty, Kristen Averyt

Defining and Expressing Uncertainty

Or, how not to do the wrong thing more precisely

Defining and Expressing Uncertainty

Or, how not to do the wrong thing more precisely

"In this world nothing can be said to be certain, except death and taxes." - Benjamin Franklin

2

Desire for scientific certainty and unanimity among "experts" a barrier decisions to

mitigate or adapt to climate change

Desire for scientific certainty and unanimity among "experts" a barrier decisions to

mitigate or adapt to climate change

3

Uncertainty in climate predictionsUncertainty in climate predictions

Uncertainty - a fundamental characteristic of weather, seasonal climate, and hydrological prediction, and no forecast is complete without a description of its uncertainty

Uncertainty - an overarching term that refers to the condition whereby the state of a system cannot be known unambiguously. Probability is one way of expressing uncertainty.

Probabilistic forecasts convey uncertainty in the prediction. The converse are deterministic forecasts, which provide single predictions of the future state of a system, with no information regarding uncertainty

4

• “….The most important questions of life are, for the most part, really only problems of probability”(Laplace, Théorie Analytique des Probabilités, 1812)

• "The research community should be focused on delivering a "coherent" product without major disagreements, to the user communities if it seeks to have the information acceptable and used wisely" (Changnon, 1994)

• "Incorporating" or "building in" forecast information into a decision-making process does not automatically result from an explicit demonstration of “value” (Stewart, 1994)

• Value is a complex, user-dependent function (Murphy, 1994)

Uncertainty, managing risk, hedging strategiesUncertainty, managing risk, hedging strategies

5

Epistemology: Are we asking the right question?

Risk: odds known Technical uncertainty: odds unknown, but

main parameters may be known Ignorance: don’t know what we don’t know Indeterminacy: causal chains/networks are

open (physical outcomes depend on social and institutional behavior)

Disagreement: divergence over observation, framing and interpretation of issues.

Epistemology: Are we asking the right question?

Risk: odds known Technical uncertainty: odds unknown, but

main parameters may be known Ignorance: don’t know what we don’t know Indeterminacy: causal chains/networks are

open (physical outcomes depend on social and institutional behavior)

Disagreement: divergence over observation, framing and interpretation of issues.

Types of uncertainty

6

Different types and sources of uncertainty characterizations

Different types and sources of uncertainty characterizations

Morgan and Henrion Funtowicz and Ravetz

(1990) (1985)

1. Empirical quantities 1. Technical uncertainties

2. The functional form of models 2. Methodological3. Disagreements among experts 3. Epistemological

Morgan and Henrion Funtowicz and Ravetz

(1990) (1985)

1. Empirical quantities 1. Technical uncertainties

2. The functional form of models 2. Methodological3. Disagreements among experts 3. Epistemological

Decision analysis vs Decision-making

Decision Event vs Decision Process

7

Sources of uncertaintySources of uncertainty

ObservationsInstrumental errorSpatial/temporal sampling

Methodology/TechnicalVariable quantities are uncertain because their

value varies, seemingly stochasticallyUncertainty in model parameters based on expert

judgment or on empirical estimates of a parameter's value

Uncertainty in model functional formsUncertainty in reduced model forms are based

on comparisons of full-form with reduced-form model results

Behavior/EnvironmentalPersonal/societal interactions and feedbacks

ObservationsInstrumental errorSpatial/temporal sampling

Methodology/TechnicalVariable quantities are uncertain because their

value varies, seemingly stochasticallyUncertainty in model parameters based on expert

judgment or on empirical estimates of a parameter's value

Uncertainty in model functional formsUncertainty in reduced model forms are based

on comparisons of full-form with reduced-form model results

Behavior/EnvironmentalPersonal/societal interactions and feedbacks

8

Minimizing uncertainty:Minimizing uncertainty:

Where is the uncertainty? Where is the uncertainty?

--Problem DomainProblem Domain

-science-science

-organizational-organizational

-community-community

-political-political

- adequate theory- adequate theory-multiple hypotheses-multiple hypotheses & congruent management & congruent management actions.actions.-tractability (complexity) -tractability (complexity) -confronting models w/data-confronting models w/data-independence/ rigor-independence/ rigor-novelty-novelty

Gunderson and others

9

Managing Uncertainty:Managing Uncertainty:

Where is it? Where is it? --Problem DomainProblem Domain

-science-science

-organizational-organizational-community-community-cognitive-cognitive-political-political

- expressions of power- expressions of power- multiple equilibria- multiple equilibria

paths not takenpaths not taken- NONE are scale invariant - NONE are scale invariant - stability of institutions- stability of institutions

novelty of approachesnovelty of approaches-role of epistemic groupsrole of epistemic groups-perception of riskperception of risk-multiple discourses-multiple discourses-juggling domains-juggling domains

10

Uncertainty in Weather vs. Climate

Uncertainty in Weather vs. Climate

Challenge....

What works for weather may not work for climate

“The entire [weather and climate] enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information” NRC, 2006

11

Climate & UncertaintyClimate & Uncertainty

Virtually certain

Extremely likely

Very likely

Likely

More likely than not

About as likely as not

Unlikely

Very unlikely

Extremely unlikely

Exceptionally unlikely

>99%

>95%

>90%

>66%

>50%

33–66%

<33%

<10%

<5%

<1%

IPCC AR4 WGI Terminology

12


FAR: IPCC (1990) Broad overview of climate change science, discussion of uncertainties and evidence for warming.

SAR: IPCC (1995) “The balance of evidence suggests a discernible human influence on global climate.”

TAR: IPCC (2001) “Most of the warming of the past 50 years is likely (>66%) to be attributable to human activities.”

AR4: IPCC (2007) “Warming is unequivocal, and most of the warming of the past 50 years is very likely (90%) due to increases in anthropogenic greenhouse gases.”

Did it work?

13

Climate & UncertaintyClimate & UncertaintyGlobal Warming ‘very-likely’ Man-Made AP, Feb 1 2007

Science Panel calls global warming ‘unequivocal’

NY Times, Feb 2 2007

World urged to act on definitive report

Financial Times, Feb 2 2007

New tack on global warming: Report’s dire forecast seen as call to action

Chicago Tribune, Feb 4 2007

14


Virtually certain

Extremely likely

Very likely

Likely

More likely than not

About as likely as not

Unlikely

Very unlikely

Extremely unlikely

Exceptionally unlikely

>99%

>95%

>90%

>66%

>50%

33–66%

<33%

<10%

<5%

<1%

IPCC AR4 WGI Terminology

Very high confidence

High Confidence

Medium Confidence

Low confidence

Very low confidence

At least 9 out of 10

About 8 out of 10

About 5 out of 10

About 2 out of 10

Less than 1 out of 10

IPCC AR4 WGII Terminology

CCSP SAP 4.3

One size does not fit all!

15

Regional Climate Assessments: Uncertainty Language


Quote & explain IPCC likelihood statements

Uses and explains IPCC terminology but scientists make own judgments

No uncertainty framework

“Modeling projections...indicate that by the 2080s the most probable amounts of change in NYC and its Watershed Region will be 7.5ºF to 8.0ºF....”

“Climate models project Colorado will warm 2.5ºF [1.5ºF to +3.5ºF by 2025....”

“...annual average temperatures across Pennsylvania are projected to increase by 2.5ºF....”

16



Quotes & explains IPCC likelihood statements



“It is extremely likely (>95%) that global temperatures and temperatures over Chicago are expected to warm further over coming decades....”

17



Quotes & explains IPCC likelihood statements



“also likely, though less certain, are increases in precipitation quantity (particularly in winter and spring), precipitation intensity, intensity of tropical and extratropical cyclones (though their frequency may decrease), and sea-level variability....”

18

Regional Climate Assessments: Education

Regional Climate Assessments: Education

All had detailed explanations of

models, scenarios, downscaling, and

climate projections

19

Moving ForwardMoving Forward

Do we need a best practices for regional climate assessments, including how best to develop the data (model results) that go into the them?

Can an uncertainty framework be developed that can be used in all regions by all sectors for all products?

Can we take the IPCC model and develop a model framework?

Can a typology approach work with different frameworks for different users?

Or should regions be left to their own devices?

Do we need a best practices for regional climate assessments, including how best to develop the data (model results) that go into the them?

Can an uncertainty framework be developed that can be used in all regions by all sectors for all products?

Can we take the IPCC model and develop a model framework?

Can a typology approach work with different frameworks for different users?

Or should regions be left to their own devices?

The need for climate information to inform decisions is immediate…and everyone is trying to fill that gap.

Different uncertainty frameworks are being used depending on the type of climate information, the intended use for planning, and the scale.

20

Climate ScienceSociety:

Impact andresponse

Historical model of communicating uncertainty

Traditional Science and Policy InteractionsTraditional Science and Policy Interactions

21

Participatory assessments:

Pathways for Communicating UncertaintyPathways for Communicating Uncertainty

Is the research compatible with existing

decision models?

Is the research accessible to policy/decision maker?

Are policymakers receptive to the

problem and to research?

Is the research relevant for decisions?

Analysis of usefulness for policy/decision making arena

Goals, Criticality, time frame, basis for decisions, usability, entry points, experience

Are the sources/providers of information credible

to the decision maker?

22

1. Bring the delivery persons, research community etc, to an understanding of what has to be done to translate current information into usable information: assess this process

2. Interacting with actual and potential users to better understand the true informational needs, desired formats of information, timeliness of delivery etc.: How do people know what they’re missing?

3. Identify areas of agreement and divergence between scientists engaged in prediction/forecasting, the community that carries out research in impacts and human resources, and the community at large

1. Bring the delivery persons, research community etc, to an understanding of what has to be done to translate current information into usable information: assess this process

2. Interacting with actual and potential users to better understand the true informational needs, desired formats of information, timeliness of delivery etc.: How do people know what they’re missing?

3. Identify areas of agreement and divergence between scientists engaged in prediction/forecasting, the community that carries out research in impacts and human resources, and the community at large

Process for empowering decision making under uncertainty

23

1. Classic decision analysis,2. Traditional scenario planning,3. Robust decision making,4. Real options, and5. Portfolio planning.

Decision Support Planning Methods: Incorporating Climate Change

Uncertainties into Water Planning


Uncertainties into Water Planning

24

• probability-based• systematically catalogs information and

mathematically evaluating and ranking decision alternatives against multiple, potentially conflicting, decision objectives

• illustrates the process with a decision tree or influence diagram

• handles uncertainty through the use of probabilities.• used to find a preferred plan with the best value


Uncertainties into Water PlanningClassic decision analysis


Uncertainties into Water PlanningClassic decision analysis

25

• scenario-based • objective is development of a plan that best prepares

for a plausible range of uncertain circumstances• scenarios are developed through the identification of

critical uncertainties and driving forces • goal is to develop a range of future conditions that go

beyond extrapolation of current trends and represent surprising but plausible conditions

• scenarios are treated as equally likely to occur


Uncertainties into Water PlanningTraditional scenario planning


Uncertainties into Water PlanningTraditional scenario planning

26

• combines features of both classic decision analysis and traditional scenario planning

• systematic way of developing a strategy to best adapt to a wide range of plausible future conditions

• uses existing or modified decision models to evaluate candidate strategies against large sets of quantitative scenarios that reflect future uncertainty.

• used to identify major vulnerabilities within these strategies (unacceptable consequences).

• hedging options and alternative strategies are develop to address vulnerabilities

• Successive iterations lead to robust strategies


Uncertainties into Water PlanningRobust decision making


Uncertainties into Water PlanningRobust decision making

27

• helps identify strategies that adjust over time and balance risks.

• determines sets of strategies that maximize value by using traditional discounted cash flow approaches.

• flexible investment strategies are sought that can be• risk-adjusted with time and deferred into the future• Uncertainties are handled through the use of

probabilities• Results are flexible in that they may incorporate

delaying and phasing of decision implementation


Uncertainties into Water PlanningReal options


Uncertainties into Water PlanningReal options

28

• used in the financial world to select a portfolio containing a mix of assets or strategies that minimize exposure due to future scenarios

• uncertainty is handled through the use of probabilities and Monte Carlo simulations

• exposure to uncertainty is minimized through hedging• used extensively in the electric utility area


Uncertainties into Water PlanningPortfolio planning


Uncertainties into Water PlanningPortfolio planning

29

___________________________________________________________________________

Approach Assumptions and actions

Development and From the risk expert to the public--finite and

delivery of a risk message uni-directional

Aimed at bringing public views into line with expert views

Assumes expert view has more validity for decision-making

Dialogue about risk Interactive exchange of risk information--continuous

Aimed at balancing the content of risk message

Assumes both views contribute to decision-making

Social processes Engage in a process that addresses concerns about risk

of risk communicationAimed at enhancing understanding among stakeholders (DECISIVE AND NON-DECISIVE). Assumes the process is as important as the product

_____________________________________________________________________________• Problem-definition and Framing: Mapping decision processes and information needs

across temporal and spatial scales

Approaches to risk communication and associated assumptions

Approaches to risk communication and associated assumptions

30

Decisions are made every day under the cloud of uncertaintyDecisions are made every day under the cloud of uncertainty

3131

Backup SlidesBackup Slides

32

Managing an Uncertain Future: Climate Change Adaptation Strategies for

California’s Water

Managing an Uncertain Future: Climate Change Adaptation Strategies for

California’s Water• array of adaptive water management

strategies must be implemented to better address the risk and uncertainty of changing climate patterns

• precipitation and runoff patterns are changing, increasing the uncertainty for water supply and quality, flood management, and ecosystem functions

• existing systems has some capacity to cope with climate variability, but extreme weather events resulting in increased droughts and floods will strain that capacity to meet future needs

• since some uncertainty will always exist, agencies need to perform sensitivity analyses of preliminary planning studies, and risk-based analyses for more advanced planning studies.

33

2012 Central Valley Flood Protection Plan

Climate Change Scope DefinitionWork Group Summary Report

2012 Central Valley Flood Protection Plan

Climate Change Scope DefinitionWork Group Summary Report

• planning frameworks need to be flexible enough to incorporate uncertainties related to climate change in managing risks.

• planning approaches incorporating climate change probabilities, robust decision making, and adaptive management allow decisions to be more flexible and consider future advances in scientific understanding

• planning starts by identifying key system vulnerabilities and/or critical risk thresholds

• regional downscaled climate projections can be used to assess the likelihood of projected climate crossing system vulnerabilities or environmental thresholds

• as climate science, global modeling, and downscaling methods improve, better information can be used to assess if probability of occurrence of critical climate condition thresholds have change.

34

Regional Climate Reports & Assessments

Regional Climate Reports & AssessmentsNOAA Near-Term Opportunities

Team: Assessing Assessments

R. Webb, C. McNutt, D. Easterling, N. Cyr, K. Averyt, D. Goodrich, D. Kluck, K. Broughton

– Provide guidance on the extent for downscaling as supported by the science

– Provide datasets in easy to use formats for the non-climate scientist

– Accurate portrayal of uncertainty (known unknowns)

– Create an online climate assessment and climate impact assessment toolkit

Documents

1 Robert S. Webb, Roger S. Pulwarty, Kristen Averyt Defining and Expressing Uncertainty Or, how not to do the wrong thing more precisely "In this world