1. ST3236: Stochastic Process Tutorial 7
TA: Mar Choong Hock
Exercises: 8

2. Question 1 From purchase to purchase, a particular customer
switches brands among products A, B and
C according to a MC whose transition probability
matrix is
In the long run, what fraction of time does this
customer purchase brand A?

3. Question 1 Let  = (A, B, C) be the limiting distribution, we have
The solution is,
A = 0.2, B = 0.3, C = 0.5
In the long run, the fraction of time that the customer purchase brand A is 0.2

4. Question 2 A MC has transition probability matrix
For which integers n = 1, 2, … , 20 is it true that
what is the period of the MC?

5. Question 2 Note: means that we can find a n-step
return path from state 0 to state 0. The probability is
0 if we cannot find a return path.

6. Question 2 We draw the diagram of state associated with
transition probabilities and observe,

7. Question 2 Observe that starting at 0, the earliest it can return
to 0 is either at the fifth step or the eighth step.
Therefore,
we have the period (greatest common divisor between 5 and 8):
d(0) = 1.
Example:{5,8,10,13,15,16,…} (aperiodic, because smallest different between two consecutive n is one)

8. Question 2 Note that all the states are communicating. In
general, a sufficient condition to determine that a
group of states, G={0, 1, …,j,…,n} are
communicating is to find a return path from j to j that
passes through all the states in set G.

9. Question 3 Which states are transient and which are recurrent in
the MC whose transition probability matrix is

10. Question 3

11. Question 3
Because
state 0 is transient.
Because
state 1 is transient.

12. Question 3
Because
state 3 is transient.
Because
state 2 is recurrent

13. Question 3 Since states 2 and 4 communicate, state 4 is recurrent
Because
state 5 is recurrent

14. Question 4 Determine the communicating classes and period
for each state of the MC whose transition probability
matrix is

15. Question 4
From the state diagram, it is easy to see that {0}, {1}, {2, 3, 4, 5}* are communicating classes.
Note*: Return path in red.

16. Question 4 d(0) = 1, d(1) = 0, d(2) = d(3) = d(4) = d(5) = 1**
Note**: A quick way to determine is to find two consecutive n in say, state 2 and determine the smallest differences.

17. Question 5
Consider the MC whose transition probability matrix is

18. Question 5
Determine the limiting probability 0 that the process in state 0
(b) By pretending that state 0 is absorbing, use a first step analysis and calculate the mean time m10 for the process to go from state 1 to state 0.
(c) Because the process always goes directly to state 1 from state 0, the mean return time to state 0 is m0 = 1 + m10. Verify 0 = 1/m0.

19. Question 5 (a) Let 0, 1, 2, 3 be the limiting distribution. Then
The solution is
0 = , 1 = , 2 = ,
3 =

20. Question 5 (b) By pretending that state 0 is absorbing, we
consider a MC with transition probability matrix
Let mi0 be the mean time to be absorbed starting from state i, i = 0, 1, 2, 3. Then using the first step analysis

21. Question 5
the solution is

22. Question 5
(c) Because the process always goes directly to state 1 from state 0, the mean return time to state 0 is
m0 = 1 + m10 = 6.9
By the basic limiting theorem, we have
m0 = 1/0 = 6.9