Embed Size (px)
DESCRIPTION
ST3236: Stochastic Process Tutorial 5. TA: Mar Choong Hock Email: [email protected] Exercises: 6. Question 1. Consider the MC with transition probability matrix Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and - PowerPoint PPT Presentation
Citation preview
Question 1Consider the MC with transition probability matrix
Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
Question 1Denote by wi1 the mean time the process spends in
state 1 starting in state i prior to the absorption. We have,
w01 = 0w11 = 1 + 0.1w01 + 0.2w11 + 0.5w21 + 0.2w31
w21 = 0 + 0.1w01 + 0.2w11 + 0.6w21 + 0.1w31
w31 = 0
The solution isw01 = 0, w11 = 1.8182, w21 = 0.9091, w31 = 0.
Question 1Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w02 = 0w12 = 0 + 0.1w02 + 0.2w12 + 0.5w22 + 0.2w32
w22 = 1 + 0.1w02 + 0.2w12 + 0.6w22 + 0.1w32
w32 = 0
The solution isw02 = 0, w12 = 2.2727, w22 = 3.6364, w32 = 0.
Question 1Denote by vi the mean time to the absorption
starting in state i prior to the absorption. We have,
v0 = 0v1 = 1 + 0.1v0 + 0.2v1 + 0.5v2 + 0.2v3
v2 = 1 + 0.1v0 + 0.2v1 + 0.6v2 + 0.1v3
v3 = 0
The solution is v0 = 0, v1 = 4.0909, v2 = 4.5455, v3 = 0.We have verified that,
v1 = w11 + w12
Question 2Consider the MC with transition probability matrix
Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
Question 2Denote by wi1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have,
w01 = 0w11 = 1 + 0.5w01 + 0.2w11 + 0.1w21 + 0.2w31
w21 = 0 + 0.2w01 + 0.1w11 + 0.6w21 + 0.1w31
w31 = 0
The solution is,w01 = 0, w11 = 1.290, w21 = 0.3225, w31 = 0.
Question 2Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w02 = 0w12 = 0 + 0.5w02 + 0.2w12 + 0.1w22 + 0.2w32
w22 = 1 + 0.2w02 + 0.1w12 + 0.6w22 + 0.1w32
w32 = 0
The solution isw02 = 0, w12 = 0.3230, w22 = 2.5808, w32 = 0.
Question 2Denote by vi the mean time to the absorption starting in state i prior to the absorption. We have,
v0 = 0v1 = 1 + 0.5v0 + 0.2v1 + 0.1v2 + 0.2v3
v2 = 1 + 0.2v0 + 0.1v1 + 0.6v2 + 0.1v3
v3 = 0
The solution is v0 = 0, v1 = 1.613, v2 = 2.9033, v3 = 0.We have verified that,
v1 = w11 + w12
Question 3
Consider the MC in question 1. Starting in state 1, determine the probability that the process is absorbed into state 0. Compare this with the (1,0)th entry in the matrix powers P2, P4, P8 and P16.
Question 3
Denote by ui the probability that the MC is absorbed by 0 starting in state i. We have,
u0 = 1u1 = 0.1u0 + 0.2u1 + 0.5u2 + 0.2u3
u2 = 0.1u0 + 0.2u1 + 0.6u2 + 0.1u3
u3 = 0
The solution is,u0 = 1, u1 = 0.4091, u2 = 0.4545, u3 = 0.
Compare:u1 = 0.4091
Consider a (4 x 4) transition probability matrix,
By definition,
1
01 10t
t XXPu
Question 3
10
...
...
...
...
....
....
....
;
...
...
...
023013201210111000210
30
20
10
00
131211102
30
20
13121110
00
XXPppppppppp
p
p
p
p
pppp
p
p
pppp
p
P
P
But for our case, p00 = 1, p03 = 0. Question 3
2
10
2012101110
3013201210111000210
10t
t XXP
ppppp
ppppppppp
1 21 p11 p22
p10
p20
30 1
p21
p12
p13
p23
Let:
1.F(t) be the set of t-step first passage paths from state 1 to state 0
2.G(n-t) be the set of (n-t)-step paths from state 0 to state 0
3.H(t) be the set of paths that is formed jointly by F(t) followed by G(n-t). Note: paths are n-step.
Question 3-Optional
Question 3-Optional
01 0G(n-t)F(t)
H(t)
Let L(n) be the set of n-step paths from state 1
to state 0. s.t.
Question 3-Optional
n
ttn
n
t
n
t
n
t
n
t
n
t
n
t
n
t
XXPtf
tnGPtFPtnGPtFP
tnGtFPtHP
tHPtHPnLP
tHnL
10,1
11
11
11
1
00
,)(
)(
npnLP 10
f1,0(t) is the t-step first passage probability from
state 1 to state 0.
If state 0 is an absorbing state,
Also, trivially,
Question 3-Optional
100 tn XXP
10 00,1 XXPtf t
n
tt
n
ttn
n
XXP
XXPtf
pnLP
10
10,1
10
10
00
Question 3-Optional
10
101 10
p
XXPut
t
Question 4
Which of the following MC is regular:
a)
b)
Question 4
a) YES, because
(all entries are greater than 0)
b) NO, because it has absorbing states
