GSA-WA Perth 2006

Oliver Kreuzer

Centre for Exploration TargetingThe University of Western Australia

Risk, Uncertainty and Bias:

Rulers over Exploration

Success and Failure

Acknowledgements

Mike Etheridge, Maureen McMahonGEMOC Key Centre, Macquarie University

Colin Wastell, Gillian LucasDepartment of Psychology, Macquarie University

Presentation outline

Aspects of our business Performance, low base rate situation, low probability of success

Risk, uncertainty and decision analysis Definitions of risk and uncertainty

What is decision analysis?

The psychology of decision-making Common heuristics and biases

What is their impact on the process of decision-making?

Outlook What can we learn from the petroleum industry?

Mineral exploration Business aspects

Randolph (2002)

Mineral exploration Business aspects

Bosma (2003)

Mineral exploration Business aspects

Economic activity

As such expected to provide acceptable returns to investors

However, probability of success so low and geological uncertainty so

high that it has proven difficult to manage for financial success

Mineral exploration Business aspects

At best a break-even proposition

Schodde (2003, 2004)

• Compiled NPVs of 109 major Australian gold projects (1985–2003)

• NPVs = $4.74 billion; costs of finding / evaluating $4.64 billion

• Average return of $1.02 per $1 dollar spent on exploration

Leveille & Doggett (in press, Economic Geology Special Publication)

• Measured costs + returns from 65 Chilean copper projects (1950–2004)

• Only 14 generated sufficient returns to offset their exploration costs

• Overall return below breakeven

Mineral exploration Business aspects

Problem: Low base rate situation

Exploration is an example of a low base-rate situation, i.e. there is a

low rate of occurrence of ore deposits in individual targets

High number of drill holes per discovery

Based on Schodde (2003)

Data exclude follow-up drilling!

Mineral exploration Business aspects

Low chance of proceeding to the next stage

Rio Tinto

100%

10%

0.3%

0.06%

0.03%

Kennecott

10%

10%

10%

Project generation

Build an expert team for the belt

Establish data base and management system

Define prospect risks

Test presence of mineralizing system

Test geologic and mineralization models

Test geologic information

Test potential of mineralizing system

Establish size and grade potentail

Establish controls on grade distribution

Feasibility

Determine project costs

Determine NPV

Stage Milestones / Aims

Prospect definition(reconnaissance)

Systematic drilltesting of targets

Resource delineation

Select and acquire ground in well endowed belts

Build area knowledge

Test continuity

Establish economic / metallurgical parameters

Mineral exploration Business aspects

Measuring Exploration Success in a Brownfields Environment(Laverton District, WA)

0.00

0.20

0.40

0.60

0.80

1.00

A B C D E F

Stage

Pro

ba

bili

ty

0.0

5.0

10.0

15.0

20.0

25.0

30.0

A$

Mill

ion

Total expenditure Average cost per prospect Probability of advancing from previous stage

Lord et al. (2001)

Mineral exploration Business aspects

Parry (2001)

Mineral exploration Business aspects

Parry (2001)

Observation 1

For a some companies exploration has been very lucrative; huge

profits were made when they reached the ultimate goal of mining

success

However, on average, mineral exploration appears to be a break-

even proposition – or worse…

The studies of Schodde and Leveille & Doggett illustrate that we

need to measure exploration performance if we want to improve it

• E.g. Schodde (2003): As a rule of thumb, we should aim to find gold for

less than A$15/oz. This is twice as good as the current average.”

Risk

Variability of possible returns

As measured by their standard deviation

• Risk includes but is not limited to chance of making a loss

• Risk equals opportunity

Probability of failure

PFailure = 1 – PSuccess

• Risk can be estimated if we can assign a value to PSuccess

• Risk can be reduced if we can find ways of improving our PSuccess

e.g. Singer & Kouda (1998), Guj (2005)

)()(( 21 iPmeanxSUMSD ini

Uncertainty

Definition

A measure of our inability to assign a single value to risk

Types of uncertainty

Inherent natural variability of geologic objects and processes

Conceptual and model uncertainty

Errors / inaccuracies / biases that occur when we sample,

observe, measure or mathematically evaluate geological data

e.g. Bardossy & Fodor (2001), Purvis (2003)

Uncertainty

Most decisions we make in mineral exploration are

made under conditions of significant uncertainty

the performance of ourtargeting tools

Imperfect knowledge ofof geological systems

the tenement or prospectgeology

our exploration / ore depositmodels

interpretation of targetingparameters

the grade, continuity orextent of mineralisation

controls on localisation ofmineralisation

Inherent variability of geo-logical objects / processes

Limitations and biases ofgeological investigations

Uncertainty about

Uncertainty

Uncertainty has rarely been estimated or quantified for our models,

maps or sections

In fact, many geological products imply a level of certainty that is

simply unrealistic

This is a major impediment to mineral exploration If we don’t estimate or determine uncertainty we won’t be able to

quantify and evaluate exploration risk

Figures from Shatwell (2003)

Decision analysis

Identify what choices or alternatives are available

Identify the possible outcomes for each alternative

Estimate the value of each possible outcome

Estimate the probability of each possible outcome

Calculate the weighted average value for each choice

Make the decision

e.g. Newendorp & Schuyler (2000)

Decision analysis

Does not eliminate or reduce risk

Helps us to evaluate, quantify and understand risk

Helps us choose the alternative that offers the best risk / reward ratio

Does not replace professional judgment

Helps us to communicate geological risks and uncertainties

• without ambiguity, and

• in terms of probabilistic and monetary values

e.g. Newendorp & Schuyler (2000)

Decision analysis

Is decision analysis only for the majors?

To expensive (software, consultant fees) and too time consuming

(compilation of input values) to be practical for juniors?

In my opinion – No.

Juniors face the same risk and uncertainty as the majors

The junior business model is even more vulnerable to gambler’s ruin

(limited risk capital, limited diversity of portfolio, few projects)

A quick and dirty analysis is still better than failure to manage risk

Observation 2

Mineral exploration is a business bedeviled by uncertainty

Yet, many of our outputs and decision-making processes imply a

level of confidence that is simply unrealistic

For effective, formal risk management to take place we have to

estimate, measure or calculate geological uncertainties

Decision analysis provides us with simple, effective tools for choosing

the best course of action under conditions of uncertainty

Psychology of decision-making

Intuitive

The inherent geological complexities and uncertainties in exploration

clash with rational decision-making

Hence, we tend to rely extensively on intuitive thinking and judgment

Biased

This Intuitive thinking is subject to a well understood set of mental

short cuts (heuristics) and systematic errors (biases)

Psychology of decision-making

The Two-Systems View Recognizes that we use 2 main types of cogitive process

System 1

IntuitionSystem 2

Reasoning

FastParallelAutomaticEffortlessAssociativeSlow-learningEmotional

SlowSerialControlledEffortfulRule-governedFlexibleNeutral

Pro

ce

ss

e.g. Kahneman (2003)

Psychology of decision-making

A stamp and an envelope cost $1.10 in total.

The stamp costs $1 more than the envelope.

How much does the envelope cost?

e.g. Kahneman (2003)

Psychology of decision-making

Most people intuitively answer 10 cents

$1.10 separates naturally into $1 and 10 cents

10 cents is about the right magnitude

But, envelope = 5 cents, stamp = $1.05

Implications of such cognitive tests

Monitoring of System 1 by System 2 is generally quite lax

We tend to offer answers without checking them

We are not used to thinking hard and often trust a plausible judgment

that quickly comes to mind

e.g. Kahneman (2003)

Heuristics

What are heuristics? Rules of thumb or mental shortcuts

Pros Very effective, automatic processes

Reduce the time and effort of decision-making

Lead to reasonable decisions in many situations

Cons Frequently bias our perception impact on System 1

Cause severe and systematic errors of judgment

Worse when we are under time pressure / multitasking

e.g. Kahneman (2003)

Heuristics

Common types of heuristics

Representiveness

Framing

Anchoring and adjustment

Availability

e.g. Kahneman (2003)

Heuristics Representativeness

Representativeness heuristic

Our tendency to overgeneralize from a few characteristics or

observations

We often judge whether an object (X) belongs to a particular class

(Y) by how representative (or similar) X is of Y

Source of multiple biases

Base rate neglect

Gambler’s ruin

Overconfidence

e.g. Kahneman (2003)

Heuristics Representativeness

Base Rate Neglect: an example

We know that 1 in 100 targets delivers a gold discovery

A new targeting method has been developed

It is practical only over small areas (i.e. known targets)

• Generates an anomaly in 90% of test cases over known deposits

• Delivers a null result in 90% of test cases in barren areas

Exploration companies run it over a total of 1,000 targets

What is the likelihood that it will correctly identify a deposit?

Example based on Nick Hayward (BHP Billiton), 2003 AIG Symposium

Example based on Nick Hayward (BHP Billiton), 2003 AIG Symposium

Heuristics Representativeness

Answer: 8.3%

True Positives : Total Positives = 9 : 108 = 0.083

# of Targets

= 1000

Targets = deposit

= 10

Target ≠ deposits

= 990

Anomalies

= 108

True positives

= 9

False positives

= 99

No anomaly

= 892

False negatives

= 1

True negatives

= 891

Heuristics Representativeness

Exploration: example of a low base rate situation

Base rates should be the main factor in our estimations

However, we tend to ignore prior probabilities when other targeting

parameters seem more relevant

Consequences

Our targeting models need to focus on those parameters that have

relatively low false positive rates

Wasting time and money on false positives is one of our industry’s

main contributors to poor performance

e.g. Hronsky (2004), Etheridge (2004)

Heuristics Representativeness

Gambler’s ruin (gambler’s fallacy)

Wins are perceived more likely after we suffered a string of losses

Example: tossing a fair coin

After H turned up 9× in a row, is it more likely that T will turn up next?

No, the odds are exactly the same for every single toss

Each toss of the coin is an independent event

The coin has no memory of the past 9 tosses

e.g. Busenitz & Barney (1997), Roney & Trick (2003)

Heuristics Representativeness

Small sample of tosses very likely for the number of H and T

outcomes to be unequal

Only in the long run will those outcomes equalize

Example: probability of gambler’s ruin

Sufficient capital for 5 trials, each @ Psuccess = 0.1 (or 10%)

What is the probability of at least 1 success in 5 trials?

Equation:

e.g. Busenitz & Barney (1997), Roney & Trick (2003); Example by Guj (2005)

xnxnx

nx PPCP )1()(

Heuristics Representativeness

• Where (Cnx) = n! / [x! × (n – x)!]

• (P15 ) = [(5! / 1! × 4!) × 0.1 × 0.94 + … + (5! / 4! × 1!) × 0.14 × 0.9 = 0.4099

• PGambler’s ruin = 1 – 0.4099 = 0.5901 or 59% chance of going bust!

• If PSuccess = 0.01 PGambler’s ruin = 0.9509 or 95% chance of failure!

Consequences

Spending too much on too few prospects is extremely risky

A streak of bad luck does not mean that we are due for successExample by Guj (2005)

Heuristics Framing

Framing heuristic

Our tendency to process information depending on how this

information is presented (or framed)

Consequences

Most judgements and decisions are guided by information derived

from the rarest events in our business – discoveries

We should start thinking outside the box by framing decisions with

information derived from the bulk of our projects – those that failed

e.g. Kahneman (2003)

Heuristics Anchoring and adjustment

Anchoring and adjustment heuristic

We tend to base our initial estimates on any value we have at

hand (anchor), regardless of its relevance

We then adjust our estimate until we reach a final value

Our adjustments are typically insufficient, narrow and biased

towards the value of the anchor

e.g. Kahneman (2003), Welsh et al. (2005)

Heuristics Anchoring and adjustment

Consequences

Strong anchoring to specific exploration models means we

are less likely to find something that is different

We drill our best target in a project first; but when it fails, we

often lower our standards to justify drilling lesser quality

targets

Observation 3

Even after decades of cognitive research we continue to assume that

our intuition, experience and intelligence will guide us toward the best

possible decision under conditions of uncertainty

Yet, the opposite is true: we are prone to cognitive biases that

frequently prevent us from choosing the optimal course of action

Moreover, the situations of greatest uncertainty are the ones where

poor judgment is most likely to result in failure

Awareness of our limitations is the first critical step in developing

good decision-making procedures

cf. Bratvold et al. (2002), Purvis (2003)

Outlook The petroleum example

So, where should we go from here?

We could, for example, look at how our colleagues in petroleum

exploration have changed the fortunes of their industry

What can we learn from the petroleum example?

• That disciplined management of risk and uncertainty can generate value

and turn an industry around

• That prediction and visualization of subsurface geology can improve

success rates

• That holistic geological models that focus on “where” rather than “how”

can reduce uncertainty

Outlook The petroleum example

BP exploration 1983–2002

0%

10%

20%

30%

40%

50%

60%

70%

80%

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

Economic success rate

High-risk wells

Late 90’s

High-risk wells ~ 10%

Success rate > 50%

Onset of formalrisk assessment

Late 80’s

High-risk wells > 50%

Success rate < 20% Glenn McMaster (BP), 2003 SPE Distinguished Lecturer

Before

After

Outlook The petroleum example

Figures from Jones & Hillis (2003), Etheridge (2004), Cockcroft (2005)

Management ofrisk and uncertainty

Process-basedmodels

Visualization of subsurface geology

Outlook Probabilistic ore systems models

Risk management and ore deposit modeling

Holistic, flexible and process-based

• build on the petroleum and mineral systems approach (Geoscience Australia)

Probabilistic

• assign probabilities to critical success factors

• multiplication rather than addition of critical success factors to eliminate those areas

where one or more of these factors are absent

• value distributions instead of single values

Calibrated

• multiple realizations

• statistical assessment of sensitivity of outputs

Outlook Probabilistic ore systems models

Link models to decision structures + GIS E.g. decision trees, Monte Carlo simulation

0.3 0.5 0.7 0.9 1.1

5% 90% 5% .4824 .9174

0.3 0.5 0.7 0.9 1.1

Mean=0.700001

Distribution for Probability of trap being in the positi...

0.000

0.500

1.000

1.500

2.000

2.500

3.000

Mean=0.700001

0.3 0.5 0.7 0.9 1.1

EV outcome for comparison of potential project risks and rewards, regardless of project type, stage or location

“The successful explorers over the next decade will be those that

embrace effective risk management”

Marcus Randolph

President Diamonds and Specialty Products, BHP-Billiton, 2003

“After all, the risk in discovery is still the greatest single risk”

Siegfried Muessig

The Art of Exploration: SEG Presidential Address, 1978

“What we need in all our endeavors … is responsible risk taking

and what we want are the rewards of such responsibility”

Paul Bailly

Risk and the Economic Geologist: SEG Presidential Address, 1982