ST3236: Stochastic Process Tutorial 7 TA: Mar Choong Hock Email: [email protected]

Exercises: 8

Question 1

From purchase to purchase, a particular customer switches brands among products A, B and C according to a MC whose transition probability matrix is In the long run, what fraction of time does this customer purchase brand A?

Question 1

Let  = (  A

,

 B

,

 C ) be the limiting distribution, we have The solution is,  A = 0.2

,

 B = 0

.

3

,

 C = 0

.

5 In the long run, the fraction of time that the customer purchase brand A is 0.2

Question 2

A MC has transition probability matrix For which integers

n

= 1

,

2

, … ,

20 is it true that what is the period of the MC?

Question 2

Note: means that we can find a n-step return path from state 0 to state 0. The probability is 0 if we cannot find a return path.

Question 2

We draw the diagram of state associated with transition probabilities and observe, 1 1 2 1 3 1 1 0.5

0 4 0.5

1 7 5 6 1 1

Question 2

Observe that starting at 0, the earliest it can return to 0 is either at the fifth step or the eighth step.

Therefore, we have the period (greatest common divisor between 5 and 8):

d

(0) = 1

.

Example:{5,8,10,13,

15,16

,…} (aperiodic, because smallest different between two consecutive n is one)

Question 2

Note that all the states are communicating. In general, a sufficient condition to determine that a group of states, G={0, 1, …,j,…,n} are communicating is to find a return path from j to j that passes through all the states in set G.

Question 3

Which states are transient and which are recurrent in the MC whose transition probability matrix is

Question 3

4 1 1 1/4 2 1/4 1 1/3 1/2 1/3 0 1/3 1 5 1/4 1/4 1/4 1/4 3

Question 3

Because state 0 is transient.

Because state 1 is transient.

Question 3

Because state 3 is transient.

Because state 2 is recurrent

Question 3

Since states 2 and 4 communicate, state 4 is recurrent Because state 5 is recurrent

Question 4

Determine the communicating classes and period for each state of the MC whose transition probability matrix is

Question 4

From the state diagram, it is easy to see that

{

0

}, {

1

}, {

2

,

3

,

4

,

5

}*

are communicating classes.

1/3 5 1 1/3 1/3 2 1 0.5

0 0.5

4 1 Note*: Return path in red.

3 1 1

Question 4

d

(0) = 1,

d

(1) = 0,

d

(2) =

d

(3) =

d

(4) =

d

(5) = 1** Note**: A quick way to determine is to find two consecutive n in say, state 2 and determine the smallest differences.

Question 5

Consider the MC whose transition probability matrix is

Question 5

(a) Determine the limiting probability  0 process in state 0 that the (b) By pretending that state 0 is absorbing, use a first step analysis and calculate the mean time

m

10 for the process to go from state 1 to state 0.

(c) Because the process always goes directly to state 1 from state 0, the mean return time to state 0 is

m

0 = 1 +

m

10 . Verify  0 = 1

/m

0 .

Question 5

(a) Let  0

,

 1

,

 2

,

 3 be the limiting distribution. Then The solution is  0  3 = 0

.

1449

,

= 0

.

1530

.

 1 = 0

.

4140

,

 2 = 0

.

2880

,

Question 5

(b) By pretending that state 0 is absorbing, we consider a MC with transition probability matrix Let

m i

0 be the mean time to be absorbed starting from state i, i = 0, 1, 2, 3. Then using the first step analysis

Question 5

the solution is

Question 5

(c) Because the process always goes directly to state 1 from state 0, the mean return time to state 0 is

m

0 = 1 +

m

10 = 6

.

9 By the basic limiting theorem, we have

m

0 = 1

/

 0 = 6.9