Presenting: How to Value Imperfect Information (?)...Imperfect information must be risked. Must take into account the possibility of an untrue (inaccurate) prediction. The magnitude

Presenting:

How to Value Imperfect Information (?)by Ron Allred and Jon Anker

DAAG Conference 2003

DAAG is the annual conference of the SDP. To find out more about SDP or to become a member, visit

www.decisionprofessionals.com

Decision & Risk Analysis

How to Value Imperfect Information

( ? )

Some very suspicious thinking by Ron and Jon


Information about Ron and JonRon AllredConocoPhillips NorwayStrategy and Portfolio CharacterizationLeader – Decision QualityResponsibilities: Strategy planning, project support, D&RA training

Jon Christian AnkerConocoPhillips NorwayEconomics Group / Developing PropertiesBusiness AnalystResponsibilities: Economic analysis and modeling


Topics covered in presentation

• Decision Analysis• Value of Information• Case Study• Decision Tree Solution• Simulation Solution• Observations and Conclusions


Decision Analysis

Recognizing a “Value of Information” situation


Nearly all important decisions, business or personal, are made under conditions of uncertainty.

We lack information about factors that could significantly affect the outcomes of our decisions.

The decision maker must choose one course of action from all that are available.

The difficulty is in understanding the consequences or outcomes of the different courses of action.

Decision making under uncertainty


Understanding the differences between alternatives (value drivers)

Alternative ‘A’ Alternative ‘B’

Valu

e

Additional Capital

Increasedproduction Delay


Two general patterns with regards to decision-making

A general EMV pattern, where the decisions occur up front and then all the uncertainties occur after those decisions are made.

A phased decision pattern, where the decisions are interspersed with the uncertainties.

A phased decision pattern is indicative of a “Value of Information” situation


Value of Information

Some background information


History lesson

• Thomas Bayes 1701 - 1761• Mathematician & ordained

minister• Bayes Theorem published 1763• Taken from his ‘theory of logic and

reasoning’.

“a method by which we might judge concerning the probability that an event has to happen, in given circumstances, upon supposition that we know nothing concerning it but that, under the same circumstances, it has happened a certain number of times, and failed a certain other number of times.”

Bayes’ TheoremA statistical method to

revise probability estimates from new

information.


Value of informationgeneral principles

There must be a decision which can change as a result of the information

Confidence has no intrinsic value. Value is added by making better, higher EMV decisions

The state of the world can not change w/out new information Value of information is the difference between the project with the information and the project without information


Acquire Info?

ActualUncertaintyResolved

PursueProject?

Decision MeasuredUncertainty

Decision

Acquire Info?


PursueProject?

Decision MeasuredUncertainty

Decision

DecisionMeasuredUncertainty Decision

Outcome ofUncertainty

Acquire info?

Pursueproject?

Indicateduncertainty(from info)

Actual uncertainty

resolved

Acquire info?

Pursueproject?

Indicateduncertainty(from info)

Actual uncertainty

resolved

Phased decision patterns:

Perfect information

Imperfect information

Perfect information --completely resolve uncertainty before

making the decision.

Imperfect information –cannot completely

resolve uncertainty. The prediction may be wrong, uncertainty

remains.

Pursue Project?


Decision Actual OutcomeThe baseline, what is

the value of the project without the information?

Just Make the Decision – no effort to resolve uncertainty before making the decision.

Perfect / Imperfect information


Most of the information we deal with is imperfect information

Imperfect information sources:

Market research or surveys

Analysis of historical data (past trends)

Testing or pilot projects

Indirect measurements

Expert opinion

Past experiences (gut feel)


Why worry about imperfect information?

The value of perfect information can be calculated, but actually acquiring this type of information is rare.

Imperfect information must be risked. Must take into account the possibility of an untrue (inaccurate) prediction.

The magnitude of the difference between the value of perfect and imperfect information relates to the risk of untrue predictions from imperfect information.

Failure to take into account the impact of imperfect information can result in incorrect estimations of value.


Imperfect information“Bayes’ Theorem”

Three types of probabilities we need to be concerned with:

Prior probabilities - the probabilities established for some actual event before we gather additional information

Conditional probabilities - the probabilities predicted by some test if an actual event really happens

Posterior probabilities - the probabilities of the outcome of an actual event (with some prior probability) following a test with known conditional probability


“Bayes’ Theorem” the basics

Event1 , E2 , E3 ,…………Enpossible states of nature

Probability(of it being Ei)probability of each of them being the true state of

nature (prior probability)

Probability(of eventB|given Ei)probability of B happening given that event Ei is the

true state of nature (conditional probability)

n

iEiPEiBP

EiPEiBPBEiP1

)()|()()|()|(

The probability of Ei given the outcome of event B

(posterior probability)


What is the probability?

1 person out of 1000 will have the rare “buga” disease.

A test is available to determine if you have the disease, it is 99% accurate.

Given a positive test result, what is the probability that you actually have the disease?

)()|()()|()()|()|(

2211

111

EPEBPEPEBPEPEBPBEP

09.0)999.0(*)99.0()001.0(*)99.0(

)001.0()99.0()|( 1 =+

*=BEP


Calculating the value of information

• Value of information is the difference between the project with the information and the project without information

• The value of both perfect and imperfect information can be calculated.

Specific Scenario EMV

A) Value of the project without information $MM

B) Value of the project - Perfect Information $MM (B>A)

C) Value of the project - Imperfect Information $MM (C>A, C<B)


Case Study


Lunar Oil Company has made a discovery – should they appraise or go straight to development?

What is the value of acquiring appraisal information?

Lunar Oil Co.Big DrillOil Co.

Viking Oil Co.Play Area

Individual Prospect

Discovery

*


Reserve uncertainty

You are evaluating whether or not you should drill an appraisal well before developing an oil discovery. The key uncertainty for this development is oil reserves. Your reservoir engineer has provided you with the following lognormal reserve estimates:

p10 (Low) 80 MMbbls (prob .3)p50 (Medium) 130 MMbbls (prob .4)p90 (High) 200 MMbbls (prob .3)

Concept SelectionFixed Platform Development Greater than 180 MMbblsFloating Production, Storage andOfftake (FPSO)

Greater than 110 MMbbls, but less than 180 MMbbls

Tie-back to Existing Facility Less than 110 MMbbls

Discovery Well

*


Information from appraisal Well• Appraisal drilling will tell you net effective pay and thus provide some

information on reserves. The decision that might change as a result of the information is the concept selection.

• Data from the expert:If actual reserves are 200 MMBO (Fixed Platform)

75% chance of predicted reserves > 180 MMbbls (Fixed Platform)20% chance of predicted reserves > 110 MMbbls (FPSO)5% chance of predicted reserves < 110 MMbbls (Tie-back)

If reserves are 130 MMBO (FPSO development)15% chance of predicted reserves > 180 MMbbls (Fixed Platform)75% chance of predicted reserves > 110 MMbbls (FPSO)10% chance of predicted reserves < 110 MMbbls (Tie-back)

If reserves are 80 MMBO (Tie-back development)5% chance of predicted reserves > 180 MMbbls (Fixed Platform)10% chance of predicted reserves > 110 MMbbls (FPSO)85% chance of predicted reserves < 110 MMbbls (Tie-back)


Decision Tree Solution

Case Study


Advantages of decision trees

• A chronological sequence of decisions to be made and the uncertainties which affect them

• A graphical means of displaying key alternatives and options available to the decision maker

• A diagnostic tool to map how outcomes are generated.

• Communicates the decision-making process to others in a clear and concise succinct manner

• Outcome values easily obtained (hand solution feasible)


No appraisal drillingEMV = 163 $MM

Assumptions: Development decision based on FPSO solution

Cum. Prob.0.3

Reserves = 200 MMbbls257 $MM 0.3

257

0.4Develop - Yes Reserves = 130 MMbbls

163 $MM 0.4163.00 163.00

0.3EMV Reserves = 80 MMbbls

1 69 $MM 0.3163.00 69

Develop - No0 $MM

0

NPV

FPSO


Appraisal drilling (perfect info.)EMV = 179 $MM

Assumptions: Cost of appraisal program equals 5 $MM

Cum. Prob.Develop = Yes

0.3 276 $MM 0.3Reserves = 200 MMbbls 276

1276

Develop = No-20 $MM

-5

Develop = Yes0.4 163 $MM 0.4

Appraisal Drilling - Yes Reserves = 130 MMbbls 1631

178.9 163Develop = No

-20 $MM-5

Develop = YesEMV 0.3 103 $MM 0.3

1 Reserves = 80 MMbbls 103178.9 1

103Develop = No

-5 $MM-5

Appraisal Well - No163 $MM

163

Tie-back

Perfect Information NPV

Fixed Platform

FPSO


Cum. Prob.0.8036Reserves = 200 MMbbls

0.225(.225 / .28)

0.280 0.1429> 180 MMbbls Reserves = 130 MMbbls

0.04(.04 / .28)

0.0536Reserves = 80 MMbbls

0.015(.015 / .28)


0.06(.06 / .39)

0.390 0.7692>110 MMbbls Reserves = 130 MMbbls

0.3(.06 + .3 + .03) (.3 / .39)


0.03(.03 / .39)


0.015(.015 / .33)

0.33 0.1818<110 MMbbls Reserves = 130 MMbbls

0.06(.06 / .33)


0.255(.255 / .33)

Predicted Actual

(.225 + .04 + .015)

(.015 + .06 + .255)

Cum. Prob.0.75

> 180 MMbbls0.225

0.3 0.20Reserves = 200 MMbbls >110 MMbbls

0.06

0.05<110 MMbbls

0.015

0.1> 180 MMbbls

0.04


0.3

0.15<110 MMbbls

0.06

0.05> 180 MMbbls

0.015


0.03

0.85<110 MMbbls

0.255

Actual Predicted

Bayes’ Theorem - Inversion of Probabilities


Appraisal drilling (imperfect info.)EMV = 172 $MM

0.8036 Cum. Prob.Reserves = 200 MMbbls

276 $MM 0.2250276

0.1429Develop = Yes Reserves = 130 MMbbls

138 $MM 0.0400242.3571 138

0.28 0.0536Prediction > 180 MMbbls Reserves = 80 MMbbls

1 16 $MM 0.0150242.3571 16

Develop = No-5 $MM

-5


257 $MM 0.0600257

0.7692Develop = Yes Reserves = 130 MMbbls

163 $MM 0.3000170.2308 163

0.39 0.0769Appraisal Well - Yes Pridiction > 110 MMbbls Reserves = 80 MMbbls

1 69 $MM 0.0300171.93 170.2308 69

Develop = No-5 $MM

-5


177 $MM 0.0150177

0.1818EMV Develop = Yes Reserves = 130 MMbbls

1 146 $MM 0.0600172 114.1818 146

0.33 0.7727Prediction < 110 MMbbls Reserves = 80 MMbbls

1 103 $MM 0.2550114.1818 103

Develop = No-5 $MM

-5

Appraisal Well - No163 $MM 1.0000

163

FPSO

Tie-back

Prediction ActualNPV

Fixed Platform


Decision tree - VOI calculations


No Appraisal Drilling 163 $MM

Appraisal Drilling - Perfect Information 179 $MM

Appraisal Drilling - Imperfect Information 172 $MM

Bayes’ Theorem is the basis for revising the original perceptions of the possible states of nature, given the new information that we have acquired.

VOI = project value w/info - project w/out infoThe state of the world can not change


Simulation Solution

Case Study


Crystal Ball

• Forecasting and risk analysis program• Excel add-in• User friendly tool for modelling uncertainty in you excel

spreadsheet• Simulation by the Monte Carlo technique• Applicable to all kinds of decision involving uncertainty• Easy to use, easy to misuse


Modeling parameters

100 000450Fixed installation

50 000300FPSO

20 00080Tie in to existing platform

Production rateBBLS/DAY

CapitalExpenditure MMUSD


50 000300FPSO




Development Costs and Minimum Rates


50 000300FPSO





50 000300FPSO




Development Costs and Minimum Rates

Frequency Chart

MMBBLS

Mean = 141,000

,011

,021

,032

,042

0

10,5

21

31,5

42

36 137 239 340 441

1 000 Trials 0 OutliersForecast: Total oil reserves

Logic of model – fit development concept to reserve level

Example: Low reserves=> the appropriate development solution would be a tie-in facility to an existing platform


-$100

$0

$100

$200

$300

$400

$500

$600

$700

0.00 50.00 100.00 150.00 200.00 250.00 300.00 350.00 400.00 450.0

Million barrels

NPV

Mill

ion

Tie inFPSOFixedLog. (Tie in)Log. (FPSO)Linear (Fixed)

Reserves versus NPV

Value of InformationValue from optimized solutionIncrease your EMVReduce your riskChoose the optimal path

$144 MM

$180 MM

$163 MM

$144 MM


Optimum development solution

Reserve distribution with development “thresholds”

35% Tie-in, 50% FPSO, 15% Fixed platform

Frequency Chart

MMBBLS

Mean = 141,000

,011

,021

,032

,042

0

10,5

21

31,5

42

36 137 239 340 441


Tie inFPSO

Fixed Platform

Frequency Chart

MMBBLS

Mean = 141,000

,011

,021

,032

,042

0

10,5

21

31,5

42

36 137 239 340 441


Tie inFPSO

Fixed Platform

110 180


Incorporating imperfect information into simulation modeling

Bayes' theorem provides the correct reasoning for incorporating imperfect information into models. Decision trees and Monte Carlo runs are simply methodologies for implementing the solution.

Mathematically, posterior distributions can be calculated. We are presenting a more “practical” approach in solving for the value of imperfect information.


Model overview

Model Layout

(take from input page, no reference to PL203)

Choose concept based on reservesLink to production profile generator, costs and NPVSimulate with crystal ball

Panel Case studyCOP equity Imperfect information

100 %

Discount rate10 %

22 Appraisal wells

### 1Tie in FPSO Fixed platform ### 1

2006 2006 2007 ### 1Start up year 2006 ### 0

3

0 0Concepts Concepts

Trigger MMBBLS Plateau rate BBLS/day22 000

110 50 000 180 100 000

2 2

Well 1

Well 2

Well 3

Well 4

Tie in

FPSOFixed platform

Consepts

Exchange rate

NOK

USD

Run mode

on

off


Modeling imperfect information

• Adds some uncertainty around the reserves estimate

• Will not always choose the optimal solution• Important to understand the uncertainty estimate• Reduces the value of information compared to perfect information

Picture of the moduleValue from crystal ball run 125

Uncertainty in estimate 25 %Uncertainty range Min Mean Max

94 125 156Value used for conspet selection 108

Imperfect information


Simulation - VOI calculations


No Appraisal Drilling 163 $MM

Appraisal Drilling - Perfect Information 180 $MM

Appraisal Drilling - Imperfect Information 174 $MM

Sampling routine built to simulate Bayes’ TheoremVOI = project value w/info - project w/out info

The state of the world can not change


Conclusions / Observations

Hopefully our thinking was not too suspicious– The importance of valuing information and taking into

account imperfect information– Bayes’ Theorem is more easily applied using decision

tree analysis in comparison to Monte Carlo simulations– We presented a simple methodology for the application

of Bayes’ Theorem in simulation modeling– Using the case study presented, the VOI comparison

between decision tree analysis and simulation modeling is very similar


Acknowledgements

• ConocoPhillips• Phil Kerig• Andrew Burton• TreePlan• Crystal Ball

Documents

Presenting: How to Value Imperfect Information (?)...Imperfect information must be risked. Must take into account the possibility of an untrue (inaccurate) prediction. The magnitude