2. Decision making under uncertainty: Decision trees

Case Study Minerals exploration handout [from P.G. Moore and H. Thomas The anatomy of decisions Penguin, second edition 1988, page 89]

Wedding anniversary? Julian Sizer is about to leave his office for home. He looks again at his desk calendar. It is March 15. This, he begins to feel, is an important date for him. Is it his wedding anniversary? He better buy his wife some flowers (cost $25). But what if it is not their anniversary and he arrives home with the present? She will become suspicious. He would have to take her out to dinner to explain (cost $125). Worse still if he does

not bring anything home and it is their anniversary there will be trouble; perhaps $500 would be needed to help patch things up. What should he do? Solution: State-of-the world decision-making model: Let us consider the set, the state of the world: S={anniversary, no anniversary}, and a set A of possible actions: A={buy, don't buy}

Payoff table anniversary no

anniversary buy -25 -150 don’t buy -500 0 We now discuss four decision criteria that can be used to choose an action assuming we want to maximise the payoff. • The maximin criterion chooses the action with the "best" worst outcome. Find the worst possible outcome for each

decision. Select the decision for which the worst outcome is better than the other worst outcomes. In the above case. worst outcome buy -150 don’t buy -500 We select "buy". That is, we firstly find the minimum possible payoff foe any one decision and then select the decision which is the maximum of these minima. This is a pessimistic or conservative approach, but may prevent the decision maker from taking the advantage of good fortune. The maximin criterion is concerned with making the worst possible outcome as pleasant as possible.

• The maximax criterion chooses the action with the "best" best outcome. Find the largest value in the payoff table and choose the corresponding decision. In the above the largest payoff is "0", so "don't buy". Definitely the optimist's criterion, but open to disastrous possibilities. • The minimax regret criterion uses the concept of opportunity cost to arrive at a decision (or ...if only I'd.....). If for example you "buy" and it is not the anniversary you suffer an opportunity loss of $150 ("if only I didn't buy") or if you don't buy and it is the anniversary you suffer the opportunity loss of $475 (“if only I bought..."). So the opportunity loss table is

anniversary no anniversary

Maximum

buy 0 150 150 don’t buy 475 0 475 We select the decision which minimises the maximum possible regret (or opportunity loss). Hence "buy". Decision making under risk Wedding anniversary? (continued) Giving the matter some further thought Julian decides that it is in fact unlikely that March 15 is the date of his anniversary. He estimates the chance that it is, at only 10%. How would this estimate affect his decision? We shall use the subjective probability of 0.1.

•Maximise the expected monetary value (EMV). Anniversary

p1 = 0.1 no anniversaryp2 = 0.9

buy -25 -150 don’t buy -500 0 expected monetary

value buy 0.1*(-25)+0.9*(-150) =

-137.5 don’t buy 0.1*(-500)+0.9*0 = -50

50)2(

5.137)1(

−=

−=

dEMV

dEMV

Since the EMV is higher for we select "don't buy", if we use EMV criterion.

d 2

How high would need to be for Julian to select ?

p1d 1

Suppose the switch-over from to occurs when

d 2 d1

p p1= . Then

pdEMV

ppdEMV

500)2(

)1(15025)1(

−=

−−−=

.24.0500150125

=∴−=−∴

ppp

In general if V d si i( , ) denotes the pay off associated with decision i and sate j, then

EMV d p s V d si jj

( ) ( ) ( ,=∑

09.04751.0)2(

1509.001.0)1(

=×+×=

=×+×=

dEOL

dEOL

d 2

i j ) • Minimise the expected opportunity loss (EOL)

5.47

135

So minimises EOL. In fact the decision which maximises EMV always minimises EOL. Note that EOL + EMV = -2.5 for each decision! Why? Decision Trees For more complex situations it is helpful to draw a diagram, called a decision tree. In the above case it is

-25 yes -137.5 0.1 Buy no 0.9 -150 Don’t buy -50 yes 0.1 -500 no 0.9 0

We work backwards through the tree Bayes' rule and value of additional information. Wedding anniversary? (again) In desperation Julian is about to ring his mother-in-law. What is the expected value of perfect information (EVPI) he is about to obtain? EVPI = EVWPI-EVWOI Where

EVWPI - Expected value with perfect information, EVWOI - Expected value with original information. Before information: EVWOI(best decision)=-50. After information: If anniversary - buy (p=0.1), if not - don't buy (p=0.9) So -2.5=00.9 + (-25)0.1=EVWPI ×× hence the EVPI = -2.5-(-50) = 47.5. This is always the EOL (best decision without information).

More typically any possible information is imperfect (for example market survey). Medical test Two percent of the population suffers from a disease. A new test is +ve for 96% of those having the disease and 3% of those not having it. A person chosen at random returns a positive result. What is the probability that he has the disease? P(+ve/ disease) = 0.96 P(+ve/ no disease) = 0.03 Question P(disease/+ve)= ?

+ve

+ve

0.02 0.98

0.04

0.96

0.97

0.03

disease no disease have

0.98 x 0.03

0.02 x 0.96

40.003.098.096.002.0

96.002.0)ve|disease( =×+×

×=+P

We have implicitly used Bayes' rule to revise our probabilities.

A more formal statement of Bayes' rule. If we denote possible outcomes (states of nature) by j( j=1..n). The states of the world include all possibilities. The prior probabilities will be denoted by

s

P s j( ). To obtain more information about state of the world, the decision maker may observe the outcome of an experiment. If extra information is obtained the possible indicators will be written as I k . We use the conditional probabilities P I sk j( | )

)

and the prior probabilities to obtain the revised probabilities

P sj(P s Ij k( | ).

P I S P I Sk j k( ) (∩ = and j ) So )(/)()|( jjkjk SPSIPSIP ∩=

)(/)()|( kjkkj IPSIPISP

Also ∩=

)()|()()()( 21 iii

kkkk SPSIPSIPSIPIP ∑=+∩+∩= K Now

)(/)()|()(/)()|( kjjkkjkkj IPSPSIPIPSIPISP =∩= So

)()|(

)()|()|(∑

=

iiik

jjkkj SPSIP

SPSIPISP

The equation above is called Bayes' rule.

Utility, risk analysis. Lottery Which would you prefer (i) a lottery which gives you $1000 for certain or (ii) a lottery which gives 50:50 chance of $10000 or a loss of $5000?

1000

10 000

-5000

(i)

(ii)0.5

0.52500 Using the EMV criterion we would select (ii). Most people would choose (i) because (i) offers a relatively large payoff, whereas (ii) yields a substantial chance of loosing $5000. Our goal is to determine a method that a person can use to choose between lotteries.

"The idea that choices among alternatives involving risk can be explained by the maximization of expected utility is ancient..." (The utility analysis of choices involving risk, J. Political Economy 56 (1948), 279-304, M.Friedman and L.J. Savage). "...the value of an item must not be based on its price, but rather on the utility it yields. The price of the item is dependent only on the thing itself... the utility, however, is dependent on the particular circumstances of the person making the estimate" (Daniel Bernoulli, 1738).

Quantifying utility The measure of a decision-maker's preference pattern for the alternative outcomes arising from different courses of action is commonly referred to as utility. If we attach a utility of 1 to $10000 (the most favorable outcome), U(10000)=1, and a utility of 0 to -$5000 (the least favorable outcome) U(-5000)=0 and 'average out' the expected utility value (EUV) of (ii) is 0.5 (0 5 1 0 5 0. .× + × ). If we prefer (i) we are saying that U(1000) > 0.5. Now we ask what amount in (i) would make you indifferent between (i) and (ii)? If you say $-1000 then U(-1000)=0.5

Now using $-1000 and the most favorable outcome $10000 as possible outcomes, we can construct a lottery:

10 000

-1000

(i)

(ii)0.5

0.5

?

And again we ask for which? would be indifferent between (i) and (ii)? If you say $3000 then U U U( ) ( ) (

. .

3000 12

10000 12

1000

12

1 12

0 5 0 75

= + −

= × + × =

)

Similarly $-5000 and $-1000 can be used to find

an x, such as U(x)=0.25. If you say $-4000 then U U U( ) ( ) ( )

.

− = − + −

= × + × =

4000 12

5000 12

1000

12

0 12

12

0 25

In this way we can build up a utility curve.

00.10.20.30.40.50.60.70.80.9

1

-5000 0 5000 10000 Note that the curve depends on the individual (or company), the particular situation.

Typical utility curves

risk seekingrisk-neutralrisk-averse

In reality, many people exhibit both risk-seeking (gambling) and risk-averse (buying insurance) behaviour.