1. CSE 3504: Probabilistic Analysis of Computer Systems Topics covered: Continuous time Markov chains (Sec. 8.1-8.2)

2. Introduction  Example: Counting the number of cars at a service station each time a car departs.  Discrete-space, continuous-time:

3. Introduction (contd..)  Time spent in each state:

4. Introduction (contd..)  Transition rate:  Generator matrix:  Diagonal entries:

5. Introduction (contd..)  State probabilities:

6. Example of CTMC  Example: State of a component  Generator matrix:  How to compute the probabilities of being in each one of the states:

7. Example of CTMC (contd..)  Balance equations:  Steady-state probabilities:  Numerical example:

8. Birth-death process  Example: Number of cars in a service station, each time a car arrives and departs.  Transitions in the state-space:

9. Birth-death process (contd..)  Generator matrix:  Balance equations:

10. Birth-death process (contd..)  Steady-state or limiting probabilities

11. Birth-death process (contd..)  Finite state space:  Transitions in the state space:

12. Birth-death process (contd..)  Generator matrix:  Balance equations:

13. Birth-death process (contd..)  Steady-state or limiting probabilities: