1. CTMCs & N/M/* Queues

2. Continuous Time Markov Chain (CTMC)
A CTMC is a continuous-time stochastic process {X(t), t  0} s.t.,  s,t  0 and  i, j, x(u)
P{X(t+s) = j | X(s) = i, X(u) = x(u), 0  u  s}
= P{X(t+s) = j | X(s) = i} – by Markov property
= P{X(t) = j | X(0) = i} = Pij(t)– by stationarity

3. What Is a CTMC? Some Definitions
Def. 1: A CTMC is a stochastic process, such that each time the process enters state i, the following holds
The amount of time spent in state i until the next transition is exponentially distributed (with mean νi)
When leaving state i, the process enters state j with probability pij, independent of the time spent in state i
Def. 2: Let Xj ~ Exp(νipij) represent the time to transition from state i to state j,  j≠i. Let τi = minj{Xj} be the time until the CTMC leaves state i for state m, where m = argminj{Xj}
The two definitions are equivalent

4. Solving CTMCs (From Geometric to Exponential Distributions)
E.g., M/M/1 queue: Birth-death CTMC
Poisson arrivals (exponential inter-arrival times)
P[one arrival in next dt] ≈ dt, P[more than one arrival in next dt] ≈ o(dt)
P[zero arrival in next dt] ≈ 1-dt
Exponential service times
P[one departure in next dt] ≈ dt, P[more than one departure in next dt] ≈ o(dt)
P[zero departure in next dt] ≈ 1-dt
Transition probability pij from i to j in next dt
pij = 0 for j > i+1 or j < i-1
pij = dt(1-dt) = dt, for j = i+1
pij = dt(1- dt) = dt, for j =i-1
As in discrete-time, we can write the balance equations
(+)i dt = i+1 dt +  i-1 dt for i  1, or i+1= i where  = /
Using ii = 1 gives i= (1-)i, i  0 1 2 λ μ n
n+1 S

5. Solving CTMCs General Balance Equations
Given an irreducible CTMC, assume that
 i’s s.t.  j, jνj = Σiiqij and Σii = 1
then the i’s are the CTMC’s limiting probabilities and the chain is ergodic
Rate of transition out of state j is equal to rate of transition into state j (νj = Σiqji  jΣiqji = Σiiqij)
Note: In the M/M/1 example, qii+1=, qii-1=, and νj=+ 

6. Time Reversibility
Similar property as for DTMCs: rate of transitions for state i to state j is the same as the rate of transition from state j to state i
πi qij = πj qji
Holds by default for birth-death chains
Check if it holds for other chains

7. Exploring The M/M/1 Queue
Average number of customers
Average time T in the system (Little’s law)
Average waiting time (time in system – service time)
Alternatively
Probability of more than k customers

8. Comparing Server Configurationsλ 1 2 K μ Kλ μ Kμ Kλ
Total arrival rate of K, total service capacity of K
Three configurations
K independent servers, with their own queue, and each allocated capacity , and handling one kth of the arrivals ()
One server of capacity K handling all arrivals (K) with a single queue
K servers, each of capacity, but with a shared queue handling all arrivals (K)
For Poisson arrivals and exponential service times, average number in the system is the same for 1) and 2), i.e., /(1 – ), and slightly larger for 3), i.e., /(1 – )PQ + k , where  = /
Average response times can, however, vary significantly, i.e., 2) is the best, followed by 3) and then 1)
T1 = 1/k( - ), T3 = 1/( - ) – k times bigger (!), and T2= T1  PQ + 1/ 

9. Finite Waiting Room System: M/M/1/K (All practical systems have limited storage space)
Same state transitions as M/M/1 up to state K
Only difference is in the normalization constant 1 2 λ μ
K-1 K

10. M/M/1/K – Blocking Probability
Arriving customers are blocked when they arrive to a full system (K customers)
System is full with probability K
Customers arrive according to a Poisson process, so they see a full system (are blocked) with probability K (remember PASTA)

11. Reconciling Apparent Differences
Recall our earlier application of Little’s law to packet transmissions over link
Arrival rate =, average transmission time =1/, load  =/
Applying Little’s law gave
 = 1x(1- 0)+0x 0= (1- 0)  0 = 1-
But from previous slide
However, arrival to server is not , and instead ’=(1-K)
So that

12. Multiple Servers The M/M/c/∞ Queue
State transitions are again a Markov chain
Transition rate of  from state i to state i+1
Transition rate of (i+1) from state i+1 to state i for 0  i  c-1, and c for i  c (all servers busy)
Recall that the minimum of 2 exponential r.v.’s with parameter 1 and 2 is exponential with parameter 1+2 1 2 λ μ c
c+1 2μ cμ

13. The M/M/c/∞ Queue State probabilities verify
Normalization condition gives

14. The M/M/c/∞ Queue - (2) Probability of queueing (all servers are busy)
Erlang C
Formula

15. The M/M/c/∞ Queue - (3) Average number in the queue NQ
Average waiting time W= NQ /
Average delay T= W+1/
Average number in the system N

16. Revisiting System Comparisons
Consider a 1 Gbps link under three configurations:
Single queue feeding single high-speed link
Single queue feeding ten 100Mbps links
Multiple (ten) queues each feeding a 100Mbps link
Assuming that packets arrive at a rate of 10 and that the service rate of the 1Gbps link is 10, what is the probability that the system is empty under each configuration?
Configuration 1 is an M/M/1/ queue, so the answer is π0 = 1-, where  = 10/10 = /
Configuration 2 is an M/M/10/ queue, so the answer is
Configuration 3 has 10 independent M/M/1/ queues that are each empty with probability 1-, so that they are all empty with probability π0 = (1-)10

17. Illustrating Our Three Configurations π0

18. The M/M/c/c System (No Queue) (“Easily” Generalizable to M/M/c/c+K)
c servers but no waiting room (blocking)
Same transition rates as M/M/c queue for i  c, and only difference is again in normalization 1 2 λ μ c 2μ cμ
and
Erlang B Formula

19. Properties of Erlang B Formula
Carried load (fraction of traffic not lost)
a’ = a(1-B(c,a))
Erlang B satisfies the following recurrence
Note: Erlang B also holds for general (non-exponential) service time distributions
so that
Trunking efficiency
(bigger systems are
more efficient!)

20. The M/M/ Queue
Poisson arrival, exponential service times, infinite number of servers
No queue as servers are always available
Simple extension of the M/M/c queue 1 2 λ μ c
c+1 2μ cμ
(c+1)μ
(c+2)μ
A Poisson
Distribution!