Friday, March 30, 2007UK Cambridge presentation1 Risk, Reasonableness, and Rationality Robin Gregory Decision Research, University of Victoria and UBC

Friday, March 30, 2007 UK Cambridge presentation 1

Risk, Reasonableness, and Rationality

Robin GregoryDecision Research, University of Victoria and UBC

Office address: Galiano Island, B.C. CanadaTel: 250-539-5701

E-mail: [email protected]

Presentation to Risk & Rationality ConferenceQueen’s College, Cambridge

March 29-31, 2007


Risk, Reasonableness, and Rationality

Introduction: Rationality vs. reasonableness

Public engagement : Structured decision making

Example 1: Treating interface fires

Example 2: Facing taboo trade-offs

Example 3: Recognizing invisible values

Discussion: What should be done?


Management of risks is a central concern of society Highly skilled people – risk managers, scientists, elected

officials – view their job as helping citizens to make / support more rational choices across competing risk-reduction & mitigation possibilities.

In 1970’s, work of Paul Slovic, Sarah Lichtenstein, & Baruch Fishchhoff – my Decision Research colleagues -- instrumental in initiating a dialogue about the standard of rationality that guides people’s perceptions of risk.

Expanded set of factors considered as contributors to risk: voluntariness of exposure, familiarity, dread.

Risk, Reasonableness, and Rationality: Introduction


How might rational risk choices be defined? Ignoring philosophical debates, I suggest three criteria:

Concern for risk management is higher among those experiencing adverse consequences

People think carefully about consequential risk topics and seek out information to become better informed

The factual base for estimating risk impacts is repeatable and reflects observations based in “good science”

In my experience, these minimum criteria are repeatedly violated. What does this mean?

People are irrational - no There are other standards besides rationality - yes Risk managers and policies need to reflect more than just

rational / cognitive responses to risk - yes

Risk, Reasonableness, and Rationality: Introduction


Risk and rationality: Introduction

In this talk, I will focus on public engagement as input to risk managers and decision makers

Context: Risk problems involving multiple dimensions: economic,

social, environmental, cultural, health & safety

Participants: Diverse stakeholders (public, government, industry,

academics, interest groups, indigenous populations)

Management dilemma: Increases on one dimension (protection of biodiversity,

fewer illnesses ) mean decreases on another (fewer jobs, higher costs)


Risk and rationality: Introduction to Structured Decision Making

One useful approach for public engagement in risk management: Structured decision making

Based in insights from decision analysis and behavioral decision theory

Widely used in stakeholder deliberations when problems are complex and involve tough choices across multiple dimensions of value

Goal is insight for decision makers: show pros and cons of risk management alternatives


Define Issues, Objectives & Measures

Develop Alternatives

Estimate Consequences

Make Trade-Offs and SelectImplement and Monitor

Define Problem1

2

3

4

5

6

Steps in a structured decision making approach

Iterate as required


Terminology

Objectives The things that matter in the decision

Performance Measures (or attributes) The specific information used to report the consequences of the

alternatives on the objectives

Alternatives (or options) The means of achieving the objectives

Trade-offs (or balancing across objectives) Differences in performance among objectives; the key to

understanding tough choices that decision makers have to make

Constructed preferences The recognition that for many policy options, we don’t really know

what the components are or how we feel. So we construct our situation-dependent preferences in relation to cues.


Example: Use of SDM to help address management of Cultus Lake sockeye

(with G. Long, Compass Resource Mgt)

Worked with multi-stakeholder committee (approx. 20 people) over 1 month period (April, 2006) – but issues contentious for over a decade!

Key risk tradeoff: Environmental protection (of endangered stocks) vs. Economic

(commercial fishing revenues)

Multiple interests: High visibility species, High importance to Conservation

(COSEWIC listed) Commercial fishers, and First Nations

Data quality variable (and controversial)

Multiple management options (exploitation rate, captive breeding, predator removal) but uncertainty about consequences of actions


SDM Cultus Sub-committeeObjectives & Performance Measures

Sockeye conservation Probability of meeting Recovery Plan objectives 1

and 2 Returns in years 2010 and average of 2016-19 Probability of extirpation by 2036 % Enhanced in 2010 and average of 2016-19

Costs Total costs over 12 years, levelized No cost allocation attempted

Catch Traditional commercial catch Commercial TAC available upstream of Vedder Total First Nations FSC

Jobs Employment opportunities directly related to

enhancement and freshwater projects


SDM Cultus Sub-committeeAlternatives

Alternatives created by assembling ‘blocks’ of options:

Cultus Exploitation Rate % Lates (Upstream) Exploitation Rate % Enhancement options Freshwater projects options(Note: these included those factors considered

to be most important; fine tuning to occur at a later time)

Make use of strategy tables to encourage creative thinking. Two examples:


Building Alternatives – A Strategy Table

Cultus Exp Rate Lates Harvest Differential Location Enhancement FW Projects

5 0, As Cultus SQ – Downstream Only None None

10-12 (2005) 10 Mixed Current: Captive Brood Current Milfoil

20 20 UpRiver (Vedder) Current Ongoing Moderate Milfoil

30 30 Double Current Smolt Full Milfoil

40 Unconstrained Max Enhancement Current Pikeminnow (<5%)

Ongoing dbl current capModerate Pikeminnow (5-

20%)

Full Pikeminnow (+20%)

Hire Stewardship Co-ordinator

Alternative 1: “Status Quo 2005”


Building Alternatives – A Strategy Table

Alternative 6: “Spread the Pain 2”

Cultus Exp Rate Lates Harvest Differential Location Enhancement FW Projects

5 0, As Cultus SQ – Downstream Only None None

10-12 (2005) 10 Mixed Current: Captive Brood Current Milfoil

20 20 UpRiver (Vedder) Current Ongoing Moderate Milfoil

30 30 Double Current Smolt Full Milfoil

40 Unconstrained Max Enhancement Current Pikeminnow (<5%)

Ongoing dbl current capModerate Pikeminnow (5-

20%)

Full Pikeminnow (+20%)

Hire Stewardship Co-ordinator


SDM Cultus Sub-committeeAlternatives

Exploration of alternatives through iterative SDM process: creation, analysis, elimination

Step 1 created 6 alternatives

Step 2 Reviewed these 6 and created 3 more

Step 3 reviewed all 9, eliminated 6 because they were

dominated (others the same or better on objs) agreed on several key components for all altsf Focused on one, favoured alternative and created 6

new simpler variations

Use of VISTA (G. Long): Displays objectives & alternatives in consequence matrices, highlighting tradeoffs (pros and cons)


SDM Cultus Sub-committeeTrade-Off Analysis


Cultus Lake: SDM trade-off analysis

Participants examined how well different alternatives “satisfied” the objectives

Recognition that some objectives are easily & cheaply met – common sense to retain these “low-hanging fruit” in all alternatives

Recognition of need to simplify the decision problem through elimination of relevant objectives and alternatives. Do this via exploration of

Redundancy: where performance measures do not vary across alternatives

Dominance: where one alternative is better than or equal to all (or, by collective agreement, nearly all) aspects of another


SDM Cultus Sub-committeeTrade-Off Analysis

Three alternatives remained at the end of this process:


Establish/communicate areas of agreement and disagreement

One alternative favoured by group (8)

Agreement on common features: Full predator-removal program Full habitat-improvement program Employ habitat stewardship coordinator

Bound & deliberate on remaining issues: Degree of enhancement (100-150k) Exploitation rate (20% - 40%)

Provide information on areas of agreement and disagreement to decision makers, also provide process recommendations

Current status: discussions continuing !!!


Risk and rationality: SDM and insights

SDM / risk engagement assumes there is a problem to be addressed, that it can be characterized in terms of values and measurable attributes, and that people’s preferences for management options will reflect their values.

New research in behavioral decision making suggests otherwise:

The role of emotions (pre-cognitive) (Damasio & Loewenstein & Hsee)

The importance of affect (judgements of good and bad) (Slovic & Peters)

The role of judgemental heuristics (Kahenman & Tversky, Thaler & Knetsch)


Risk v. reasonableness: Role of affect

Four roles of affect (Peters, 2006) As Information: feelings act as information to

guide the judgment or decision process As a spotlight: focusing us on different

information (e.g., numerical cues) As motivator: we can take action or devote

more attention to aspects of tasks As common currency: facilitates the

comparison of unlike options

Affect is thus an active component in the construction of preferences:

conditions more delayed cognitive responses

works in parallel with cognition


Rationality v. reasonableness: Examples

This tension between affect and cognition is a challenge for decision aiding approaches that seek to assist in the development of risk policies through public engagement (prescriptive)

Three examples: Interface fires (public engagement) - Canada Facing taboo choices (experimental) – US Recognizing invisible values (NSF, science

and local / aboriginal knowledge) – Canada, US, Thailand, New Zealand


Example 1: Interface forest fires -- the role of emotions (with Joe Arvai)

Context: public groups and surveys in aftermath of severe forest

fires in western Canada

Goal: aid risk managers in making fire management

decisions (fuels, “speed bumps,” evacuation plans, replanting, harvest timing, equipment)

Analysis: starting point was structured dialogue with

stakeholders, asking: What do you think are key sources of risk? What might be done to reduce risks?


Risk and rationality: Interface fires

Factual result: Most high risk areas remain so, even after burning. In some cases,

risks increase.

Expected finding: Wake-up call people in high-risk fire areas, having already experienced losses, will

seek added protection & risk reduction (e.g., US after 9/11)

Actual result: Letdown people in high-risk areas feel safer – the low-probability event already

has happened, and “lightning doesn’t strike twice in the same place.”

Source: Arvai, Gregory et al. 2006. Letdowns, wake-up calls, and constructed preferences: People’s responses to wildfire risks. Journal of Forestry 104: 173-181.


Risk and rationality: Interface fires

Post exposure wake-up call vs. post-exposure letdown

Homeowners’ self-ratings of affect (items 1-5; midpoint = “neutral” in all cases), and levels of concern and motivation in an

exposed (Kelowna) and unexposed community (Vernon). Responses were provided on seven-point Likert scales.Comparisons between the two samples were made using a 2-sample t-test.

SURVEY ITEMKelowna Vernon

P

X SE X SE

1. Calm vs. Upset (1=very upset, 7=very calm) 2.80 0.16 2.62 0.26 ns

2. Safe vs. At Risk (1=very much at risk, 7=very safe) 3.58 0.25 2.62 0.30 0.05

3. Confident vs., Unconfident (1=not at all confident, 7=very confident) 3.51 0.22 2.86 0.27 0.05

4. Optimistic vs. Pessimistic (1=very pessimistic, 7=very optimistic) 5.67 0.21 4.08 0.32 <0.0001

5. At Ease vs. Worried (1=very worried, 7=very at ease) 3.80 0.24 2.68 0.22 0.001

6. Level of Concern about Future Fires (1=low, 7=extreme) 5.51 0.19 6.16 0.19 0.05

7. Level of Motivation (1=low, 7=high) 4.89 0.21 6.11 0.20 <0.0001


Risk and Rationality: Interface Fires

Practical implications of affective response Little demand for risk mitigation in many

high-risk communities – people feel more safe than they really are

Homeowners in high-risk areas not willing to undertake risk mitigation – not at threshold of risk identification

High level of trust (unfounded) in protection efforts

Emphasis on context & history-specific construction of preferences for management actions: what? and who?


Example 2: Facing taboo choices (with J. Irwin and S. Lichtenstein)

Context: some decision situations are so difficult or repugnant

that people dislike or refuse to make a choice.

Literature refers to these as “taboo” choices (Tetlock), “protected values” (Baron), or “protest votes”’ (CVM)

From standpoint of SDM / DA, it’s important to distinguish choices that are difficult or confusing – provide help? -- from those that truly are taboo – respect (and leave alone?).

Source: S. Lichtenstein, R. Gregory, J. Irwin. What’s Bad is Easy: Taboo Values, Affect, and Cognition. Ms. under editorial review.


Risk and rationality: facing taboo choices as part of public engagement

Examples: Do not eat foods containing modified genes Do not make choices that will increase risks to

one’s own children Do not jail citizens on basis of DNA profiling

Characteristics of such values: Quantity insensitivity: 10 deaths = 20 Agent relativity: it matters who makes the TO Moral obligation: a social, as well as personal,

reference (effects beyond the individual)


Risk and rationality: Taboo choices

Study goals: distinguish mere disapproval from protest and from

taboo values understand reasons for disapproval develop rules for public engagement: when to probe

or educate vs. when to leave alone

Study design: 22 brief scenarios, covering range of issues. Ex:

Cut old-growth trees to assist in development of new drugs

Transplant organs from patients in deep coma to help otherwise terminal patients.

16 reasons, both affective (this plan disgusts me) and cognitive (this plan is complex)



Note U-shape of figure: scenarios judged most difficult to think about are intermediate in affect, whereas scenarios judged lowest and highest in affect are judged easy to think about.



The worst and the bestscenarios are theeasiest to think about.Thus it may be difficultto get people to acceptassistance in thinkingthrough tough decisions:What’s bad is easy! Leads to polarization as

part of engagement.


Example 3: Recognizing invisible values (with L. Failing, M. Harstone, N Turner, T. Satterfield)

Risk management choices should reflect the “best available knowledge” – few would disagree

But in the context of public deliberations about risks, What does this mean?

Typically, three sources of knowledge: Science (technical) Community (local) Aboriginal (indigenous – First Nations / First

Peoples)Source: L. Failing, R. Gregory, M. Harstone. Integrating Science and

Local Knowledge in Environmental Risk Management: A Decision-Focused Approach. Ecological Economics. (In press)


Recognizing invisible values – the limitations of science

Typical link is from knowledge to facts: a “rational” approach to identifying risk consequences leads to the examination of factual evidence.

But this assumption may be too limiting: Knowledge of local communities includes

concerns such as pride, integrity, lifestyle Knowledge of Aboriginal communities

includes spiritual and ceremonial concerns, also impacts on ancestors

Access to knowledge and values remains closed until basic level of trust is established


Recognizing invisible values-- the limitations of science

Effective risk management thus must also look to values as a source of knowledge. In some cases, key impacts may be unobservable by “neutral” participants. Examples:

In Alaska: impacts of oil/gas exploration on visible walrus populations vs. “Great” walrus

In British Columbia, impacts of development on grizzly bear populations vs. “Spirit” bears

In New Zealand, impacts of GMO foods on Maori culture (T. Satterfield)

In Thailand, impacts of development on “spirits” of forest and on “ghosts” of historical inhabitants


Invisible values: the limitations of science

Practical implications for risk engagement Requires expansion of measures of value in

assessing risks. Food contamination (GMOs) – health impacts, also spiritual, ceremonial, social relations impacts

Brings focus to affective and emotional considerations: worry, shame, fear, reverence

Brings focus to constructed preferences / TOs Leads to questions about tradeoffs: cultural

survival, transmission of knowledge across generations

Leads to questions about elicitation: how to conduct “public” engagement with Elders


Rationality v. Reasonableness in Risk Management: What should be done?

Issues of fact are typically resolvable: new studies, more deliberation, better models

Issues of value are usually not resolvable: no reason to expect that values will align

Usual tools – rational, decompositional, analytical – may not work well in contexts where emotions and culture underlie risk exposure and risk management choices

This leaves those of us working on public engagement in risk management with several (wonderful & perplexing) challenges


Challenge 1: Developing useful performance measures / attributes

Best measures of risk are natural, easily understood: e.g. lives, dollars, kilometers. But often need to use case-specific “constructed” measures:

ecological health Inter-generational knowledge transmission degree of cultural or lifestyle change worry or shame or fear

How can different measures best be used to capture dimensions of value without overstepping limitations of analysis (compartmentalization)?

What are criteria for distinguishing successful from inappropriate measures / attributes?


Challenge 2: Public engagement in highly affective situations

“Rational” risk management may make sense to experts but not to stakeholders:

don’t want too little risk reduction: initiatives overturned by post-exposure letdown

don’t want too much risk reduction: initiatives inflated by affective worry or fear

Reasonable suggestions: Employ affect in risk communication (markers) Use non-traditional measures of risk Use risk language of local participants Address affective issues as prelude to risk

reduction Recognize influence of affect on other judgements

(e.g., assessment of probabilities & consequences)


Challenge 3: Applying RA or SDM techniques in other cultures

Engagement need to reflect culturally “reasonable” tools of analysis – stories, qualitative analyses, narrative

Engagement needs to be flexible Where cultural worldviews are more holistic Where spiritual values are more important Where trust in “outside” experts is missing Where key tradeoffs include invisible beings Where stakeholder expression of values is

unusual (taboo to speak to outsiders) Where key populations are marginalized or

unapproachable (thus speaking out is unsafe)


Risk, Rationality and Reasonableness: Conclusions

Reasonable risk tradeoffs are informed by both rationality and emotion / affect

Preferences for risk management options will be informed by culture and values as well as by factual knowledge – science plus community/local knowledge

Structured engagement methods need to work with expanded set of concerns and make use of constructed attributes / measures to incorporate diverse values

Successful public engagement needs to recognize constructed nature of preferences

Successful facilitators need to adapt tools (probability assessments with Elders?) and learn new ways to probe or question (stories? narrative? walks?)

Establishment of trust may be precursor to actions

Documents

Friday, March 30, 2007UK Cambridge presentation1 Risk, Reasonableness, and Rationality Robin Gregory Decision Research, University of Victoria and UBC