Transcript ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email:
ST3236: Stochastic Process Tutorial 5
TA: Mar Choong Hock Email: [email protected]
Exercises: 6
Question 1
Consider the MC with transition probability matrix Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
Question 1
Denote by
w i
1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have,
w
01
w
11
w
21
w
31 = 0 = 1 + 0
.
1
w
01 = 0 + 0
.
1
w
01 = 0 + 0
.
2
w
11 + 0
.
2
w
11 + 0
.
5
w
21 + 0
.
6
w
21 + 0
.
2
w
31 + 0
.
1
w
31 The solution is
w
01 = 0
, w
11 = 1
.
8182
, w
21 = 0
.
9091
, w
31 = 0
.
Question 1
Similarly, denote by
w i
2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w
02
w
12
w
22
w
32 = 0 = 0 + 0.1
w
02 = 1 + 0.1
w
02 = 0 + 0.2
w
12 + 0.2
w
12 + 0.5
w
22 + 0.6
w
22 + 0.2
w
32 + 0.1
w
32 The solution is
w
02 = 0,
w
12 = 2.2727,
w
22 = 3.6364,
w
32 = 0
.
Question 1
Denote by
v i
the mean time to the absorption starting in state i prior to the absorption. We have,
v
0
v
1
v
2
v
3 = 0 = 1 + 0.1v
= 0 0 = 1 + 0.1v
0 + 0.2v
1 + 0.2v
1 + 0.5v
2 + 0.6v
2 + 0.2v
3 + 0.1v
3 The solution is
v
0
v
3 = 0
.
= 0, v We have verified that, 1 = 4.0909, v 2 = 4
.
5455,
v
1 =
w
11 +
w
12
Question 2
Consider the MC with transition probability matrix Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
Question 2
Denote by
w i
1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have,
w
01
w
11
w
21
w
31 = 0 = 1 + 0
.
5
w
01 = 0 + 0
.
2
w
01 = 0 + 0
.
2
w
11 + 0
.
1
w
11 + 0
.
1
w
21 + 0
.
6
w
21 + 0
.
2
w
31 + 0
.
1
w
31 The solution is,
w
01 = 0
, w
11 = 1
.
290
, w
21 = 0
.
3225
, w
31 = 0
.
Question 2
Similarly, denote by
w i
2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w
02
w
12
w
22
w
32 = 0 = 0 + 0.5
w
02 = 1 + 0.2
w
02 = 0 + 0.2
w
12 + 0.1
w
12 + 0.1
w
22 + 0.6
w
22 + 0.2
w
32 + 0.1
w
32 The solution is
w
02 = 0,
w
12 = 0.3230,
w
22 = 2.5808,
w
32 = 0
.
Question 2
Denote by
v i
the mean time to the absorption starting in state i prior to the absorption. We have,
v
0
v
1
v
2
v
3 = 0 = 1 + 0.5v
= 0 0 = 1 + 0.2v
0 + 0.2v
1 + 0.1v
1 + 0.1v
2 + 0.6v
2 + 0.2v
3 + 0.1v
3 The solution is
v
0
v
3 = 0
.
= 0, v We have verified that, 1 = 1.613, v 2 = 2.9033,
v
1 =
w
11 +
w
12
Question 3
Consider the MC in question 1. Starting in state 1, determine the probability that the process is absorbed into state 0. Compare this with the (1,0)th entry in the matrix powers
P
2 ,
P
4 ,
P
8 and
P
16 .
Question 3
Denote by
u i
the probability that the MC is absorbed by 0 starting in state i. We have,
u
0
u
1
u
2
u
3 = 1 = 0 = 0
.
1
u
0 = 0.1
u
0 + 0.2
u
1 + 0.2
u
1 + 0.5
u
2 + 0.6
u
2 + 0.2
u
3 + 0.1
u
3 The solution is,
u
0 = 1
, u
1 = 0
.
4091
, u
2 = 0
.
4545
, u
3 = 0
.
Compare: u
1 = 0
.
4091
By definition,
Question 3
u
1
t
1
P
X t
0
X
0 1 Consider a (4 x 4) transition probability matrix,
P
p p p
00 10 20
p
30 .
p
11 .
.
.
p
12 .
.
.
.
.
p
13 ;
P
2
p
10
p
10 .
.
.
p
00 .
p
11 .
.
p
10 .
p
12 .
.
p
11
p
10 .
.
.
p
13
p
00
p
10
p
20
p
30
p
12
p
20 .
.
.
.
.
.
.
.
.
.
.
.
p
13
p
30
P
X
2 0
X
0 1
1
Question 3
But for our case, p 00 = 1, p 03 = 0.
p
10
p
10
p p
11 00
p p
10 10
p p
12 11
p p
10 20
p
12
p
20
p
13
p
30
t
2 1
P
X t
0
X
0 1 p 20 p 23 p 21 0 p 11 1 2 p 22 p 12 3 p 10 p 13 1
Question 3-Optional
Let: 1. F(t) be the set of t-step first passage paths from state 1 to state 0 2. G(n-t) be the set of (n-t)-step paths from state 0 to state 0 3. H(t) be the set of paths that is formed jointly by F(t) followed by G(n-t). Note: paths are n step.
Question 3-Optional 1 F(t) 0 G(n-t) 0 H(t)
Question 3-Optional
Let L(n) be the set of n-step paths from state 1 to state 0. s.t.
P
L
p
10
L
t n
1
H P
L
P
t n
1
H
t n
1
P
H
(
t
)
t n
1
P
H
(
t
)
t n
1
P
F
n
t
t n
1
P
F
P
G
n
t
t n
1
P
F
P
G
n
t
t n
1
f
1 , 0
P
X n
0
X t
0
Question 3-Optional
f 1,0 (t)
is the t-step first passage probability from state 1 to state 0
.
If state 0 is an absorbing state, Also, trivially,
f
1 , 0
P
X t
0
X
0
P
1
X n
0
X t
0 1
P
L
t n
1
p
10
f
1 , 0
P
X n
t n
1
P
X t
0
X
0 0
X t
1 0
Question 3-Optional
u
1
t
1
P
X t
p
10 0
X
0 1
Question 4
Which of the following MC is regular: a) b)
a) YES, because
Question 4
(all entries are greater than 0) b) NO, because it has absorbing states