CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Discrete time Markov chains (Sec. 7.1)

Introduction  Example: State of a component, at every clock tick.  Family of random variables:  Classification of the process:

Memoryless property  Definition of memoryless property:  Mathematical representation of memoryless property:

pmf and conditional pmf  pmf of random variable Xn:  Conditional transition probability:  Homogenous chains:

Transition probabilities  One-step transition probabilities:  Representation as a matrix:

Properties of transition probability matrix  Stochastic matrix:

Graphical representation

Examples  Count the number of cars in a service station, at the point of departure of each car.

Examples (contd..)