1. CS352 - Introduction to Queuing Theory Rutgers University

2. CS352 Fall,20052 Queuing theory definitions (Bose) “the basic phenomenon of queueing arises whenever a shared facility needs to be accessed for service by a large number of jobs or customers.” (Wolff) “The primary tool for studying these problems [of congestions] is known as queueing theory.” (Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory." "Any system in which arrivals place demands upon a finite capacity resource may be termed a queueing system.” (Mathworld) “The study of the waiting times, lengths, and other properties of queues.” http://www2.uwindsor.ca/~hlynka/queue.html

3. CS352 Fall,20053 Applications of Queuing Theory Telecommunications Traffic control Determining the sequence of computer operations Predicting computer performance Health services (eg. control of hospital bed assignments) Airport traffic, airline ticket sales Layout of manufacturing systems. http://www2.uwindsor.ca/~hlynka/queue.html

4. CS352 Fall,20054 Example application of queuing theory In many retail stores and banks multiple line/multiple checkout system  a queuing system where customers wait for the next available cashier We can prove using queuing theory that : throughput improves increases when queues are used instead of separate lines http://www.andrews.edu/~calkins/math/webtexts/prod10.htm#QT

5. CS352 Fall,20055 Example application of queuing theory http://www.bsbpa.umkc.edu/classes/ashley/Chaptr14/sld006.htm

6. CS352 Fall,20056 Queuing theory for studying networks View network as collections of queues FIFO data-structures Queuing theory provides probabilistic analysis of these queues Examples: Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost

7. CS352 Fall,20057 Little’s Law Little’s Law: Mean number tasks in system = mean arrival rate x mean response time Observed before, Little was first to prove Applies to any system in equilibrium, as long as nothing in black box is creating or destroying tasks ArrivalsDepartures System

8. CS352 Fall,20058 Proving Little’s Law J = Shaded area = 9 Same in all cases! 1 2 3 4 5 6 7 8 Packet # Time 1 2 3 1 2 3 4 5 6 7 8 # in System 1 2 3 Time 1 2 3 Time in System Packet # 1 2 3 Arrivals Departures

9. CS352 Fall,20059 Definitions J: “Area” from previous slide N: Number of jobs (packets) T: Total time : Average arrival rate N/T W: Average time job is in the system = J/N L: Average number of jobs in the system = J/T

10. CS352 Fall,200510 1 2 3 4 5 6 7 8 # in System (L) 1 2 3 Proof: Method 1: Definition Time (T) 1 2 3 Time in System (W) Packet # (N) 1 2 3 =

11. CS352 Fall,200511 Proof: Method 2: Substitution Tautology

12. CS352 Fall,200512 Model Queuing System Server System Queuing System Queue Server Queuing System Use Queuing models to Describe the behavior of queuing systems Evaluate system performance

13. CS352 Fall,200513 Characteristics of queuing systems Arrival Process The distribution that determines how the tasks arrives in the system. Service Process The distribution that determines the task processing time Number of Servers Total number of servers available to process the tasks

14. CS352 Fall,200514 Kendall Notation 1/2/3(/4/5/6) Six parameters in shorthand First three typically used, unless specified 1. Arrival Distribution 2. Service Distribution 3. Number of servers 4. Total Capacity (infinite if not specified) 5. Population Size (infinite) 6. Service Discipline (FCFS/FIFO)

15. CS352 Fall,200515 Distributions M: stands for "Markovian", implying exponential distribution for service times or inter-arrival times. D: Deterministic (e.g. fixed constant) E k : Erlang with parameter k H k : Hyperexponential with param. k G: General (anything)

16. CS352 Fall,200516 Kendall Notation Examples M/M/1: Poisson arrivals and exponential service, 1 server, infinite capacity and population, FCFS (FIFO) the simplest ‘realistic’ queue M/M/m Same, but M servers G/G/3/20/1500/SPF General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First

17. CS352 Fall,200517 Poisson Process For a poisson process with average arrival rate, the probability of seeing n arrivals in time interval delta t

18. CS352 Fall,200518 Poisson process & exponential distribution Inter-arrival time t (time between arrivals) in a Poisson process follows exponential distribution with parameter

19. CS352 Fall,200519 Analysis of M/M/1 queue Given: : Arrival rate of jobs (packets on input link)  : Service rate of the server (output link) Solve: L: average number in queuing system L q average number in the queue W: average waiting time in whole system W q average waiting time in the queue

20. CS352 Fall,200520 M/M/1 queue model  WqWq W L LqLq

21. CS352 Fall,200521 Solving queuing systems 4 unknowns: L, L q W, W q Relationships: L= W L q = Wq (steady-state argument) W = W q + (1/  ) If we know any 1, can find the others Finding L is hard or easy depending on the type of system. In general:

22. CS352 Fall,200522 Analysis of M/M/1 queue Goal: A closed form expression of the probability of the number of jobs in the queue (P i ) given only and 

23. CS352 Fall,200523 Equilibrium conditions n+1nn-1    Define to be the probability of having n tasks in the system at time t

24. CS352 Fall,200524 Equilibrium conditions n+1nn-1   

25. CS352 Fall,200525 Solving for P 0 and P n Step 1 Step 2

26. CS352 Fall,200526 Solving for P 0 and P n Step 3 Step 4

27. CS352 Fall,200527 Solving for L

28. CS352 Fall,200528 Solving W, W q and L q

29. CS352 Fall,200529 Online M/M/1 animation http://www.dcs.ed.ac.uk/home/jeh/Simjava/qu eueing/mm1_q/mm1_q.html

30. CS352 Fall,200530 Response Time vs. Arrivals

31. CS352 Fall,200531 Stable Region linear region

32. CS352 Fall,200532 Example On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model, what is the probability of buffer overflow if the gateway had only 13 buffers. How many buffers are needed to keep packet loss below one packet per million?

33. CS352 Fall,200533 Example Measurement of a network gateway: mean arrival rate (l): 125 Packets/s mean response time (m): 2 ms Assuming exponential arrivals: What is the gateway’s utilization? What is the probability of n packets in the gateway? mean number of packets in the gateway? The number of buffers so P(overflow) is <10 -6 ?

34. CS352 Fall,200534 Example Arrival rate λ = Service rate μ = Gateway utilization ρ = λ/μ = Prob. of n packets in gateway = Mean number of packets in gateway =

35. CS352 Fall,200535 Example Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ/μ = 0.25 Prob. of n packets in gateway = Mean number of packets in gateway =

36. CS352 Fall,200536 Example Probability of buffer overflow: To limit the probability of loss to less than 10 - 6 :

37. CS352 Fall,200537 Example Probability of buffer overflow: = P(more than 13 packets in gateway) To limit the probability of loss to less than 10 -6 :

38. CS352 Fall,200538 Example Probability of buffer overflow: = P(more than 13 packets in gateway) = ρ 13 = 0.25 13 = 1.49x10 -8 = 15 packets per billion packets To limit the probability of loss to less than 10 -6 :

39. CS352 Fall,200539 Example Probability of buffer overflow: = P(more than 13 packets in gateway) = ρ 13 = 0.25 13 = 1.49x10 -8 = 15 packets per billion packets To limit the probability of loss to less than 10 -6 :

40. CS352 Fall,200540 Example To limit the probability of loss to less than 10 -6 : or

41. CS352 Fall,200541 Example To limit the probability of loss to less than 10 -6 : or = 9.96

42. CS352 Fall,200542 Empirical Example M/M/m system