1. Lecture 7  Poisson Processes (a reminder)  Some simple facts about Poisson processes  The Birth/Death Processes in General  Differential-Difference Equations  The Pure Birth Process (Generating our own Poisson process from scratch)  How are queues related to Markov chains  The M/M/1 queue  A Poisson arrival and Poisson departure queue modelled using Markov chains

2. The Poisson Distribution (reminder)  A random variable X is said to be Poisson if it has the following distribution:  We can calculate the mean as follows: beginner exercise – prove it IS a distribution

3. A Poisson Process  A Poisson process is a process where the number of arrivals in a time interval (size  ) has a Poisson distribution:  Where A(t) is the number of events (arrivals) up to time t.  Note that this is a Poisson distribution with mean t  The parameter is known as the rate of the process – because in t time units, t arrivals will occur.

4. Poisson Interarrival Times  t n is the time of packet n and  n = t n+1 – t n  How is  n distributed? The probability  n > s is the probability that there are 0 arrivals in the period t n to t n +s  Note that a similar derivation proves the “memoryless” property of the Poisson process. The distribution of the time to next arrival starting from any time t where t n < t < t n+1 would be just the same as if we start counting from the previous arrival.

5. Approximating a Poisson Process  For every t  0 and   0: The third property follows from the first two.

6. The Birth Death Process  A birth-death process is a Markov process in which transitions from state k can only be made from the adjacent states k-1 and k+1  Think of a transition from k to k+1 as a birth and in the reverse direction as a death.  More importantly, we could consider it as arrivals and departures from a queue where the arrivals and departures are Poisson processes.  Firstly, we must consider why a Markov chain is appropriate to the modelling.

7. Continuous Time Markov Chains  Note that the Markov chains we have talked about before were “discrete time” – there were discrete steps which occurred at given times.  Here we need to think about continuous time Markov chains – those where transitions between states could occur at any time.  The technicalities of continuous time Markov chains are beyond the scope of this course.  Therefore, we will ignore this technicality and pretend we are dealing with discrete time Markov chains with very small times between states.

8. The General Birth-Death Process  When the pop. = k, births and deaths happen as Poisson processes: birth rate k and death rate  k (  0 =0)  B(t,  t ) is the number of births in the period (t,t+  t )  D(t,  t ) is the number of deaths in the period (t,t+  t ) P{B(t,  t )= 0 | Pop. = k} = 1 - k  t +o (  t ) P{B(t,  t )= 1 | Pop. = k} = k  t +o (  t ) P{B(t,  t )> 1 | Pop. = k} = o (  t ) P{D(t,  t )= 0 | Pop. = k} = 1 -  k  t +o (  t ) P{D(t,  t )= 1 | Pop. = k} =  k  t +o (  t ) P{D(t,  t )> 1 | Pop. = k} = o (  t )

9. Differential Difference Equations  Define the probability that the pop. is k at time t as P k (t). Now, for k > 0 we have: Now, taking the limit as  t  0 These are known as differential difference equations

10. A Quick Aside – The Pure Birth Process  Consider process k = and  k =0 Which is the original Poisson process we started with (no surprise)!

11. The General Birth-Death Process as a Markov Chain 102 k 0 1 2 11 22 22 k-1 k kk  k+1... Note that we number the states from 0 so that the state number is the same as the population.

12. Equilibrium Probabilities  We are often interested in questions of the form: “What is the average size of the population?” or “What is the probability that the population is of size k at time t?”  We are therefore interested in the equilibrium probabilities.  Recall our balance equations:

13. Equilibrium Probabilities(2)  In the case of our Birth-Death process these are: rearrange to: compare with:

14. Solving the problem Rearrange (2): (1) (2) Substitute into (1) with k=1 We suspect (correctly) the following relation: (proving this is part of your coursework) Rearrange:

15. Completing the Birth-Death Process from other balance equation: Rearranging

16. Finally, the M/M/1 process  The M/M/1 queue is simply a birth death process with k = and  k = . Substituting into our previous equations we get: where  = /  is known as the utilisation factor for a stable system this is < 1 Therefore: From the geometric series:

17. The M/M/1 Process Solved  We now want to get the expected queue length: we use a familiar trick to get: From Little’s Theorem the average delay:

18. Average Queue Length in M/M/1 Utilisation E[N]