CTMCs & N/M/* Queues

Continuous Time Markov Chain (CTMC) A CTMC is a continuous-time stochastic process {X(t), t  0} s.t.,  s,t  0 and  i, j, x(u) P{X(t+s) = j | X(s) = i, X(u) = x(u), 0  u  s} = P{X(t+s) = j | X(s) = i} – by Markov property = P{X(t) = j | X(0) = i} = Pij(t)– by stationarity

What Is a CTMC? Some Definitions Def. 1: A CTMC is a stochastic process, such that each time the process enters state i, the following holds The amount of time spent in state i until the next transition is exponentially distributed (with mean νi) When leaving state i, the process enters state j with probability pij, independent of the time spent in state i Def. 2: Let Xj ~ Exp(νipij) represent the time to transition from state i to state j,  j≠i. Let τi = minj{Xj} be the time until the CTMC leaves state i for state m, where m = argminj{Xj} The two definitions are equivalent

Solving CTMCs (From Geometric to Exponential Distributions) E.g., M/M/1 queue: Birth-death CTMC Poisson arrivals (exponential inter-arrival times) P[one arrival in next dt] ≈ dt, P[more than one arrival in next dt] ≈ o(dt) P[zero arrival in next dt] ≈ 1-dt Exponential service times P[one departure in next dt] ≈ dt, P[more than one departure in next dt] ≈ o(dt) P[zero departure in next dt] ≈ 1-dt Transition probability pij from i to j in next dt pij = 0 for j > i+1 or j < i-1 pij = dt(1-dt) = dt, for j = i+1 pij = dt(1- dt) = dt, for j =i-1 As in discrete-time, we can write the balance equations (+)i dt = i+1 dt +  i-1 dt for i  1, or i+1= i where  = / Using ii = 1 gives i= (1-)i, i  0 1 2 λ μ n n+1 S

Solving CTMCs General Balance Equations Given an irreducible CTMC, assume that  i’s s.t.  j, jνj = Σiiqij and Σii = 1 then the i’s are the CTMC’s limiting probabilities and the chain is ergodic Rate of transition out of state j is equal to rate of transition into state j (νj = Σiqji  jΣiqji = Σiiqij) Note: In the M/M/1 example, qii+1=, qii-1=, and νj=+ 

Time Reversibility Similar property as for DTMCs: rate of transitions for state i to state j is the same as the rate of transition from state j to state i πi qij = πj qji Holds by default for birth-death chains Check if it holds for other chains

Exploring The M/M/1 Queue Average number of customers Average time T in the system (Little’s law) Average waiting time (time in system – service time) Alternatively Probability of more than k customers

Comparing Server Configurations λ 1 2 K μ Kλ μ Kμ Kλ Total arrival rate of K, total service capacity of K Three configurations K independent servers, with their own queue, and each allocated capacity , and handling one kth of the arrivals () One server of capacity K handling all arrivals (K) with a single queue K servers, each of capacity, but with a shared queue handling all arrivals (K) For Poisson arrivals and exponential service times, average number in the system is the same for 1) and 2), i.e., /(1 – ), and slightly larger for 3), i.e., /(1 – )PQ + k , where  = / Average response times can, however, vary significantly, i.e., 2) is the best, followed by 3) and then 1) T1 = 1/k( - ), T3 = 1/( - ) – k times bigger (!), and T2= T1  PQ + 1/ 

Finite Waiting Room System: M/M/1/K (All practical systems have limited storage space) Same state transitions as M/M/1 up to state K Only difference is in the normalization constant 1 2 λ μ K-1 K

M/M/1/K – Blocking Probability Arriving customers are blocked when they arrive to a full system (K customers) System is full with probability K Customers arrive according to a Poisson process, so they see a full system (are blocked) with probability K (remember PASTA)

Reconciling Apparent Differences Recall our earlier application of Little’s law to packet transmissions over link Arrival rate =, average transmission time =1/, load  =/ Applying Little’s law gave  = 1x(1- 0)+0x 0= (1- 0)  0 = 1- But from previous slide However, arrival to server is not , and instead ’=(1-K) So that

Multiple Servers The M/M/c/∞ Queue State transitions are again a Markov chain Transition rate of  from state i to state i+1 Transition rate of (i+1) from state i+1 to state i for 0  i  c-1, and c for i  c (all servers busy) Recall that the minimum of 2 exponential r.v.’s with parameter 1 and 2 is exponential with parameter 1+2 1 2 λ μ c c+1 2μ cμ

The M/M/c/∞ Queue State probabilities verify Normalization condition gives

The M/M/c/∞ Queue - (2) Probability of queueing (all servers are busy) Erlang C Formula

The M/M/c/∞ Queue - (3) Average number in the queue NQ Average waiting time W= NQ / Average delay T= W+1/ Average number in the system N

Revisiting System Comparisons Consider a 1 Gbps link under three configurations: Single queue feeding single high-speed link Single queue feeding ten 100Mbps links Multiple (ten) queues each feeding a 100Mbps link Assuming that packets arrive at a rate of 10 and that the service rate of the 1Gbps link is 10, what is the probability that the system is empty under each configuration? Configuration 1 is an M/M/1/ queue, so the answer is π0 = 1-, where  = 10/10 = / Configuration 2 is an M/M/10/ queue, so the answer is Configuration 3 has 10 independent M/M/1/ queues that are each empty with probability 1-, so that they are all empty with probability π0 = (1-)10

Illustrating Our Three Configurations  π0

The M/M/c/c System (No Queue) (“Easily” Generalizable to M/M/c/c+K) c servers but no waiting room (blocking) Same transition rates as M/M/c queue for i  c, and only difference is again in normalization 1 2 λ μ c 2μ cμ and Erlang B Formula

Properties of Erlang B Formula Carried load (fraction of traffic not lost) a’ = a(1-B(c,a)) Erlang B satisfies the following recurrence Note: Erlang B also holds for general (non-exponential) service time distributions so that Trunking efficiency (bigger systems are more efficient!)

The M/M/ Queue Poisson arrival, exponential service times, infinite number of servers No queue as servers are always available Simple extension of the M/M/c queue 1 2 λ μ c c+1 2μ cμ (c+1)μ (c+2)μ A Poisson Distribution!