1. CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Discrete time Markov chains (Sec. 7.1)

2. Introduction  Example: State of a component, at every clock tick.  Family of random variables:  Classification of the process:

3. Memoryless property  Definition of memoryless property:  Mathematical representation of memoryless property:

4. pmf and conditional pmf  pmf of random variable Xn:  Conditional transition probability:  Homogenous chains:

5. Transition probabilities  One-step transition probabilities:  Representation as a matrix:

6. Properties of transition probability matrix  Stochastic matrix:

7. Graphical representation

8. Examples  Count the number of cars in a service station, at the point of departure of each car.

9. Examples (contd..)