View
220
Download
1
Category
Preview:
Citation preview
1
CHAPTER 6 (handout)Decision Trees
2
6.1. Introduction
Sequential decision making sequence of chance-dependent decisions presentation of analysis can be complex
Decision Trees Pictorial device to represent problem &
calculations Useful for problems with small no. of sequential
decisions
3
6.3. Another Decision Tree Ex.
2 boxes, externally identicalMust decide which box a1: box 1: 6 black balls, 4 white balls a2: box 2: 8 black balls, 2 white balls
Correct guess Receive $100 Wrong guess Receive $0
Prior Probability P(1) = 0.5 P(2) = 0.5
4
Decision Tree
A connected set of nodes & arcs
Nodes: join arcs Arcs: have direction (L to R) Branch: arc & all elements that follow it
2 branches from same initial node cannot have elements in common
2 nodes cannot be joined by > 1 arc
5
Example of a Decision Tree
6
A diagram which is not a tree
7
Types of nodes
Decision point• Choosing next action (branch)
Chance node• Uncontrollable probabilistic event
Terminal node• Specifies final payoff
8
Example of Sequential Decision Problem
Car Exchange ProblemA person must decide whether to keep or exchange his car in a showroom. There are 2 decisions:
a1: keep cost = 1400 SR
a2: exchange, has 2 possibilities: • good buy P(G) = 0.6 cost = 1200 SR• bad buy P(B) = 0.4 cost = 1600 SR
Good or bad buy can be identified only after buying and using the car. What he should do to minimize his cost?
9
Car Exchange Problem (no information)
Payoff (Cost) Matrix
P() a1: keep a2: exchange
1: Good 0.6 1400 1200
2: Bad 0.4 1400 1600
EV 1400 1360
10
Car exchange decision tree
Keep
Exchange
G: 0.6
B: 0.4
$1400
$1400
G: 0.6
B: 0.4
$1200
$1600
11
Car exchange decision tree
Keep
Exchange
G: 0.6
B: 0.4
$1400
$1400
G: 0.6
B: 0.4
$1200
$1600
$1400
$1360
12
6.2. A Sequential Test Problem
Car Exchange ProblemAssume the person has 5 options for deciding whether
to keep or exchange his car.
(i) Decide without extra information(ii) Decide on basis of free road (driving) test(iii)Decide after oil consumption test costing $25(iv)Decide after combined road/oil test costing $10(v) Decide sequentially: road test then possibly oil test
costing $10
In (iv), both tests must be takenIn (v), oil test is optional, depending on road test
13
Car Exchange Problem (with information)
• The decision tree is complicated
• Cannot fit in 1 slide
• 5 branches: 5 options
• Probabilities after extra information are conditional (posterior)
• To illustrate, we choose the branch of option (v)
• Road test then, depending on result, possible oil test costing $10
14
Car Exchange Problem (with information)
Result of road test:
• y1 : fair p(y1) = 0.5
• y2 : poor p(y2) = 0.5
Result of oil consumption test:
• Z1 : high p(Z1|y)
• Z2 : medium p(Z2 |y)
• Z3 : low p(Z3 |y)
15
Car exchange decision tree (with information)
y1: 0.5
y2: 0.5
No test
Oil testZ3
Z1
Z2
No test
Oil testZ3
Z1
Z2
Road test
16
Car exchange decision tree with information (y1 branch)
y1: 0.5
No test
Z1: 0.28
Oil test
a2
a1
0.60.4
1400
1400
0.60.4
1200
1600
a2
a1
0.430.57
1410
1410
0.430.57
1210
1610
a2
a1
0.50.5
1410
1410
0.50.5
1210
1610
a2
a1
0.750.25
1410
1410
0.750.25
1210
1610
Z2: 0.24
Z3: 0.48
17
Car exchange decision tree with information (y1 branch)
y1: 0.5
No test
Z1: 0.28
Oil test
a2
a1
0.60.4
1400
1400
0.60.4
1200
1600
a2
a1
0.430.57
1410
1410
0.430.57
1210
1610
a2
a1
0.50.5
1410
1410
0.50.5
1210
1610
a2
a1
0.750.25
1410
1410
0.750.25
1210
1610
Z2: 0.24
Z3: 0.48
1400
1360
1410
1439
1410
1410
1410
1310
1360
1410
1410
1310
1362
18
Car exchange decision tree with information (y2 branch)
y2: 0.5 No test
Z1: 0.32Oil test
a2
a1
0.60.4
1400
1400
0.40.6
1200
1600
a2
a1
0.250. 75
1410
1410
0.250.75
1210
1610
a2
a1
0.310.69
1410
1410
0.310.69
1210
1610
a2
a1
0.570.43
1410
1410
0.570.43
1210
1610
Z2: 0.26
Z3: 0.42
19
Car exchange decision tree with information (y2 branch)
y2: 0.5 No test
Z1: 0.32Oil test
a2
a1
0.60.4
1400
1400
0.40.6
1200
1600
a2
a1
0.250. 75
1410
1410
0.250.75
1210
1610
a2
a1
0.310.69
1410
1410
0.310.69
1210
1610
a2
a1
0.570.43
1410
1410
0.570.43
1210
1610
Z2: 0.26
Z3: 0.42
1400
1440
1410
1510
1410
1487
1410
1381
1400
1410
1410
1381
1398
20
Decision Tree Calculations
• Tree is developed from left to right
• Calculations are made from right to left
• Many calculation are redundant• For inferior solutions
• Not needed in final solution
• Probabilities after extra information (road or oil tests) are conditional (posterior)
• Calculated by Bayes’ theorem
21
Initial Payoff Data (no information)
Payoff (Reward) Matrix
P() a1: Box 1 a2: Box 2
1: Box 1 0.5 100 0
2: Box 2 0.5 0 100
EV 50 50
22
Initial Probability Data (no information)
Prior Probability Matrix
P() B: Black W: White
1: Box 1 0.5 0.6 0.4
2: Box 2 0.5 0.8 0.2
23
Decision tree without information
Box 1
Box 2
1 : 0.5
2: 0.5
$100
$0
1: 0.5
2: 0.5
$0
$100
$50
$50
24
Decision Tree Example with information
Samples from box can be taken Ball is returned to the box Up to 2 samples are allowed Cost = $3 per sample
What is the optimal plan?
25
Posterior probabilities for sample 1
Probability Calculations
P() P(B) P(W) Joint Posterior
1: 0.5 0.6 0.4 0.3 0.2 0.43 0.67
2: 0.5 0.8 0.2 0.4 0.1 0.57 0.33
1.0 0.7 0.3 1.00 1.00
26
Decision tree with information
No sample
Sample 1
B: 0.7
$50
$
W: 0.3
$
$
a1 or a2
$
Sample 2
Sample 2
No sample
No sample
No information
27
Posterior probabilities for sample 2when sample 1 is Black
Probability Calculations
P() P(B) P(W) Joint Posterior
1: 0.43 0.6 0.4 0.26 0.17 0.36 0.61
2: 0.57 0.8 0.2 0.46 0.11 0.64 0.39
1.0 0.72 0.28 1.00 1.00
28
Sample 1 Black, No Sample 2
No 2nd sample
Sample 2
1: 0.43
2: 0.57
$97
$-3
B: 0.72
W: 0.28
$
$
1: 0.43
2: 0.57
$-3
$97
a1
a2Black sample 1
40
54
54
29
Samples 1 & 2 Both Black
Black sample 2
Sample 2
1: 0.36
2: 0.64
$94
$-6
B: 0.72
W: 0.28
$
1: 0.36
2: 0.64
$-6
$94
a1
a2
Black sample 1
30
58
No Sample
$54
58
30
Sample 1 Black, Sample 2 White
White sample 2
Sample 2
1: 0.61
2: 0.39
$94
$-6
W: 0.28
B: 0.72
$58
1: 0.61
2: 0.39
$-6
$94
a1
a2
Black sample 1
55
33
No Sample
$54
55
57.16
31
Posterior probabilities for sample 2when sample 1 is White
Probability Calculations
P() P(B) P(W) Joint Posterior
1: 0.67 0.6 0.4 0.40 0.27 0.61 0.79
2: 0.33 0.8 0.2 0.26 0.07 0.39 0.21
1.0 0.66 0.34 1.00 1.00
32
Sample 1 White, No Sample 2
No 2nd sample
Sample 2
1: 0.67
2: 0.33
$97
$-3
B: 0.66
W: 0.34
$
$
1: 0.67
2: 0.33
$-3
$97
a1
a2White sample 1
64
30
64
33
Sample 1 White, Sample 2 Black
Black sample 2
Sample 2
1: 0.61
2: 0.39
$94
$-6
B: 0.66
W: 0.34
$
1: 0.61
2: 0.39
$-6
$94
a1
a2
White sample 1
55
33
No Sample
$64
55
34
Samples 1 & 2 Both White
White sample 2
Sample 2
1: 0.79
2: 0.21
$94
$-6
W: 0.34
B: 0.66
$55
1: 0.79
2: 0.21
$-6
$94
a1
a2
White sample 1
73
15
No Sample
$64
73
61.12
35
Decision tree summary of results
No samples
Sample 1
B: 0.7
$50
$55
W: 0.3
a1 or a2
$54
Sample 2
No 2nd sample
No information
$64
Sample 2
No 2nd sample
$58
$55
W, 0.28: a1
B, 0.72: a2
a2
B, 0.66: a1
W, 0.34: a1
a1
$73
57.2
61.1
64
57.2
59.2
a1: 6B, 4W
a2: 8B, 2W
36
Decision Tree with Fixed Costs
Example of fixed cost: • sampling cost = 3/sample in previous example
If objective is to maximize expected payoff, Constant costs can be deducted either from:
• Terminal node payoffs
• Expected values
37
Example: Including fixed costs
Sample 1 Black, cost = $3
1: 0.43
2: 0.57
$100
$0
43 – 3a1
Sample 1 Black, cost = $3
1: 0.43
2: 0.57
$97
$– 3
40a1
Recall Slide 9
38
Fixed Costs & Utilities
Utilities can be used instead of payoffs If objective is to maximize expected utility
• Constant costs must be deducted from terminal node payoffs
• Net payoffs are converted to net utilities• Expected values are taken of utilities of net
payoffs
39
Including fixed costs
Sample 1 Black, cost = $3
1: 0.43
2: 0.57
U(100)
U(0)
EU–U(3)a1
Sample 1 Black, cost = $3
1: 0.43
2: 0.57
U(97)
U(– 3)
EUa1
Incorrect
Correct
40
Allowing an optional 3rd sample
Suppose now a 3rd sample is allowed Sample cost = $3 Assume the decision whether or not to
take sample 3 depends on results of samples 1 and 2
What is the optimal plan?
41
Posterior probabilities for sample 3
After 2 blacks (slide 8)
P() P(B) P(W) Joint Posterior
1: 0.36 0.6 0.4 0.22 0.14 0. 3 0.52
2: 0.64 0.8 0.2 0.51 0.13 0. 7 0.48
1.0 0.73 0.27 1.00 1.00
42
Decision tree with optional sample 3
Sample 1 B: 0.7
$50
W: 0.3
$54
Sample 2
No 2nd sample
No sample
$
$57.2
Sample 3
No 3rd sample
$64
Sample 2
No 2nd sample
$
$61.1
Sample 3
No 3rd sample
43
Fixing the number of samples
Suppose now a 3rd sample is allowed Sample cost = $3 Assume we must decide the number of samples
in advance:
0, 1, 2, or 3
What is the optimal plan?
44
Zero samples
a1: Box 1
1 : 0.5
2: 0.5
$100
$0
1: 0.5
2: 0.5
$0
$100
$50
$50a2: Box 2
50No samples
45
One Sample
B: 0.7
W: 0.3
1: 0.43
2: 0.57
$97
$-3
1: 0.43
2: 0.57
$-3
$97
a1
a2
Sample once
40
54
54
1: 0.67
2: 0.33
$97
$-3
1: 0.67
2: 0.33
$-3
$97
a1
a2
64
30
64
57
46
Posterior probabilities for 2 samples
Examples: P(BB|1) = P(BB) = 0.6(0.6) = 0.36
P(BW|1) = P(BW) + P(WB) = 0.6*0.4 + 0.4*0.6 = 0.48
P(WW|1) = P(WW) = 0.4(0.4) = 0.16
P() BB BW WW Joint
1: 0.5 0.36 0.48 0.16 0.18 0.24 0.08
2: 0.5 0.64 0.32 0.04 0.32 0.16 0.02 0.50 0.40 0.10
Post
1: 0.36 0.60 0.80
2: 0.64 0.40 0.20
47
Two Samples
BB: 0.5
WW: 0.1
1: 0.36
2: 0.64$94
$-6a1
a2
Sample twice
30
58
58
1: 0.36
2: 0.64$-6
$94
58
1: 0.6
2: 0.4$94
$-6a1
a2
54
541: 0.6
2: 0.4$-6
$94
34
1: 0.8
2: 0.2$94
$-6a1
a2
74
741: 0.8
2: 0.2$-6
$94
14
BW: 0.4
48
Posterior probabilities for 3 samplesP(BBB|1) = 0.6(0.6)(0.6) = 0.216
P(BBW|1) = P(BBW) + P(BWB) + P(WBB)= 3*0.6*0.6*0.4 = 0.432
P(BWW|1) = P(BWW) + P(WBW) + P(WWB)= 3*0.6*0.4*0.4 = 0.288
P(WWW|1) = 0.4(0.4)(0.4) = 0.064
P BBB BBW BWW WWW Joint
1:0.5 0.216 0.432 0.288 0.064 0.108 0.216 0.144 0.032
2:0.5 0.512 0.384 0.096 0.008 0. 256 0.192 0.048 0.004
0.364 0.408 0.192 0.036Post
1: 0.30 0.53 0.75 0.89
2: 0.70 0.47 0.25 0.11
49
Three Samples
BBB: 0.36
WWW: 0.04
1: 0.3
2: 0.7$91
$-9a1
a2
Sample 3 times
21
61
55.7
1: 0.3
2: 0.7$-9
$91
61
BBW: 0.41
1: 0.53
2: 0.47$91
$-9a1
a2
44
441: 0.53
2: 0.47$-9
$91
38
1: 0.75
2: 0.25$91
$-9a1
a2
66
661: 0.75
2: 0.25$-9
$91
16
1: 0.89
2: 0.11$91
$-9a1
a2
80
801: 0.89
2: 0.11$-9$91
2
BWW: 0.19
50
Summary of results with fixed number of samples
$50
$57
1 sample
0 samples
$55.7
$58
2 samples
3 Samples
51
Value of Sample (new information Results of previous example
• With sequential samples (slide 23)
• With fixed no. of samples (slide 31)
3rd Sample is never needed Questions:
• How many samples should be taken?
• Is it better to decide immediately or after more information?
52
Expected Value of Information
Assume P(1) = p, P(2) = 1 – p ThenP() P(B) P(W) Joint
1:p 0.6 0.4 0.6p 0.4p
2:1–p 0.8 0.2 0.8(1-p) 0.2(1-p) 1.0 (4-p)/5 (1+p)/5
Posterior3p/(4-p) 2p/(1+p)
4(1-p)/(4-p) (1-p)/(1+p)
53
Expected payoff Best payoff if Black = 100[ max{3p/(4-p), 4(1-p)/(4-p)} ] Best payoff if White = 100[ max{2p/(1+p), (1-p)/(1+p)} ]
Expected outcome F(p) = 100 (4-p)/5 [ max{3p/(4-p), 4(1-p)/(4-p)} ]
+ 100 (1+p)/5[ max{2p/(1+p), (1-p)/(1+p)} ]
F(p) = 100[ max{0.6p, 0.8(1-p)} + max{0.4p, 0.2(1-p)} ] F(p) = max{60p, 80(1-p)} + max{40p, 20(1-p)} F(p) = max{a, b} + max{c, d}
54
Graph of expected payoff
p
100
1
80
4/71/3
55
Maximum Expected Payoff
To maximize F(p) on 0 < p < 1, Graphical solution gives
• 0 < p < 1/3 F(p) = 100(1 – p) b + d
• 1/3 < p < 4/7 F(p) = 80 – 40p b + c
• 4/7 < p < 1 F(p) = 100p a + c
For 1st and 3rd ranges, solution is same as expected payoff given only P(1) = p, P(2) = 1 – p.
Only 2nd range has improvement in expected payoff Sample should be taken only if: 1/3 < p < 4/7
56
Expected Value of Sample Information Value of sample information
= Expected improvement in payoff
= 80 – 40p – (100 – 100p), 0 < p < 0.5
= 80 – 40p – (100p), 0.5 < p < 1
Or
= 60p – 20, 0 < p < 0.5
= 80 – 140p, 0.5 < p < 1
57
Range of p for sample cost = 3
For sample cost = 3 Sample should be taken only improvement is > 3
• 60p – 20 > 3• p > 0.383
• 80 – 140p > 3• p < 0.55
Thus, 0.383 < p < 0.55
58
For fixed no. of samples
Posteriors after 2 samples (slide 27)
BB BW WW
P(1) = p 0.36 0.60 0.80
Since all probabilities are outside the range
(0.383 < p < 0.55)
A 3rd sample should not be taken
59
How many samples?
So far, analysis is for the value of 1 sample We can estimate value of several samples
Max. no. of samples• Expected payoff with no information = 50 • Payoff with perfect information = 100• Max. no. of samples = (100 – 50)/3 = 16
Recommended