1. CSE 3504: Probabilistic Analysis of Computer Systems Topics covered: Discrete time Markov chains (Sec. 7.2-7.3)

2. Absolute probability distribution  Definition:  Mathematical representation:

3. Absolute probability distribution (contd..)  How to compute:

4. Steady-state or limiting probabilities  Definition:  Classification of DTMCs based on limiting probabilities:

5. Absorbing chains  Example:  Transient states:  Absorbing states:

6. Absorbing chains (contd..)  Number of visits to transient state 1 before reaching state 0:  What is the “average” number of visits to each transient state before being absorbed?

7. Absorbing chains (contd..)  Arrange the entries of matrix P:  Partition the matrix P:

8. Absorbing chains (contd..)  Fundamental matrix M:  Mean number of visits:

9. Absorbing chains (contd..)  Example:

10. Irreducible chains  Example:  Definition:  Steady-state or equilibrium state:

11. Irreducible chains (contd..)  Computation of steady-state probabilities:

12. Irreducible chains (contd..)  Example: