1. ST3236: Stochastic Process Tutorial 6
TA: Mar Choong Hock
Exercises: 7

2. Question 1 Consider the MC with transition probability matrix
Determine the limiting distribution.

3. Question 1 Let  = (0, 1, 2) be the limiting distribution, we have
deleting one of the first three equations, we have the solution as
0 = , 1 = , 2 =

4. Question 2 Consider the MC with transition probability matrix
What fraction of time, in the long run, does the
process spend in state 1?

5. Question 2 Let  = (0, 1, 2) be the limiting distribution, we have
deleting one of the first three equations, we have the solution as
0 = , 1 = , 2 =

6. Question 2
With frequency 1 = , in the long run, does the process spend in state 1

7. Question 3 Consider the MC with transition probability matrix
Every period that the process spends in state 0
incurs a cost of 2$. Every period that the process
spends in state 1 incurs a cost of 5$. Every period
that the process spends in state 2 incurs a cost of
3$. What is the long run average cost per period
associated with this Markov chain.

8. Question 3 Let  = (0, 1, 2) be the limiting distribution, we have
deleting one of the first three equations, we have the solution as
0 = , 1 = , 2 =

9. Question 3
The long run average cost per period associated with this Markov chain is
x x x 3 = $

10. Question 4 Suppose that the social classes of successive
generations in a family follows a Markov chain with
transition probability matrix given by
What fraction of families are upper class in the long
run?

11. Question 4 Let  = (L, M, U) be the limiting distribution, we have
deleting one of the first three equations, we have the solution as
L = , M = , U =

12. Question 4
The fraction of families are upper class in the long run is U =