1. CSE 550 Computer Network Design Dr. Mohammed H. Sqalli COE, KFUPM Spring 2012 (Term 112)

2. CSE-550-T112 Lecture Notes - 32 Outline Queuing Models Application to Networks Traffic Flow Analysis

3. Basic Components of a Queue 1. Arrival process 6. Service discipline 2. Service time distribution 4. Waiting positions 3. Number of servers 5. Customer population CSE-550-T112 3Lecture Notes - 3

4. Kendall Notation A/S/m/B/K/SD A: Arrival process S: Service time distribution m: Number of servers B: Number of buffers (system capacity) K: Population size SD: Service discipline CSE-550-T112 Lecture Notes - 34

5. CSE-550-T112 Lecture Notes - 35 Queuing Models - Notation The notation X/Y/N is used for queuing models X = distribution of the inter-arrival times Y = distribution of service times N = number of servers The most common distributions are: G = general independent arrivals or service times M = negative exponential distribution D = deterministic arrivals or fixed length service Example: M/M/1

6. CSE-550-T112 Lecture Notes - 36 Queuing Models - Single-server queues - M/G/1 model: The arrival rate is Poisson and the service time is general M/M/1 model: The standard deviation is equal to the mean, the service time distribution is exponential, i.e., service times are essentially random M/D/1 model: The standard deviation of service time is equal to zero, i.e., a constant service time The poorest performance is exhibited by the exponential service time (M/M/1), and the best by a constant service time (M/D/1) Usually, the exponential service time can be considered to be the worst case: An analysis based on this assumption will give conservative results

7. CSE-550-T112 Lecture Notes - 37 Queuing Models - Single-server queues - Coefficient of variation = σ T s / T s Zero: Constant service time (M/D/1) Example: all transmitted messages have the same length Ratio less than 1: Using M/M/1 model would give answers on the safe side: it will give queue sizes and times that are slightly larger than they should be Example: a data entry application for a particular form Ratio close to 1: This is a common occurrence and corresponds to exponential service time (M/M/1) Example: message sizes varying over the full range, shared LAN, and packet-switching networks Ratio greater than 1: Need to use the M/G/1 model and not rely on the M/M/1 model Example: a system that experiences many short messages, many long messages, and few in between

8. Key Variables 1 2 m Previous Arrival Arrival Begin Service End Service  ws r n nqnq nsns  Time CSE-550-T112 Lecture Notes - 38

9. Key Variables (cont)  Inter-arrival time = time between two successive arrivals.  Mean arrival rate = 1/E[  ] May be a function of the state of the system, e.g., number of jobs already in the system. s = Service time per job.  = Mean service rate per server = 1/E[s] Total service rate for m servers is m  n = Number of jobs in the system. This is also called queue length. Note: Queue length includes jobs currently receiving service as well as those waiting in the queue. CSE-550-T112 Lecture Notes - 39

10. Rules for All Queues Rules: The following apply to G/G/m queues: 1. Stability Condition: < m  Finite-population and the finite-buffer systems are always stable. 2. Number in System versus Number in Queue: n = n q + n s Notice that n, n q, and n s are random variables. E[n]=E[n q ]+E[n s ] CSE-550-T112 Lecture Notes - 310

11. Rules for All Queues (cont) 3. Number versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate  Mean response time 4. Similarly, Mean number of jobs in the queue = Arrival rate  Mean waiting time The above two Equations are known as Little's law. 5. Time in System versus Time in Queue r = w + s r, w, and s are random variables. E[r] = E[w] + E[s] CSE-550-T112 Lecture Notes - 311

12. Key Variables (cont) n q = Number of jobs waiting n s = Number of jobs receiving service r = Response time or the time in the system = time waiting + time receiving service w = Waiting time = Time between arrival and beginning of service CSE-550-T112 Lecture Notes - 312

13. Little's Law Mean number in the system = Arrival rate  Mean response time This relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing service. Named after Little (1961) Based on a black-box view of the system: In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions. Black Box Arrivals Departures CSE-550-T112 Lecture Notes - 313

14. Proof of Little's Law If T is large, arrivals = departures = N Arrival rate = Total arrivals/Total time= N/T Hashed areas = total time spent inside the system by all jobs = J Mean time in the system= J/N Mean Number in the system = J/T = = Arrival rate X Mean time in the system 12345678 1 2 3 4 Job number Arrival Departure Time 12345678 1 2 3 4 Number in System Time 123 1 2 3 4 Time in System Job number CSE-550-T112 Lecture Notes - 314

15. Application of Little's Law Applying to just the waiting facility of a service center: Mean number in the queue = Arrival rate  Mean waiting time Similarly, for those currently receiving the service, we have: Mean number in service = Arrival rate  Mean service time ArrivalsDepartures CSE-550-T112 Lecture Notes - 315

16. M/M/1 Queue M/M/1 queue is the most commonly used type of queues Used to model single processor systems or to model individual devices in a computer system Assumes that the interarrival times and the service times are exponentially distributed and there is only one server. No buffer or population size limitations and the service discipline is FCFS Need to know only the mean arrival rate and the mean service rate . State = number of jobs in the system 012j-1jj+1 …  … CSE-550-T112 Lecture Notes - 316

17. Results for M/M/1 Queue Birth-death processes with Probability of n jobs in the system: CSE-550-T112 Lecture Notes - 317

18. Results for M/M/1 Queue (Cont) The quantity /  is called traffic intensity and is usually denoted by the symbol . Thus: n is geometrically distributed. Utilization of the server = Probability of having one or more jobs in the system: CSE-550-T112 Lecture Notes - 318

19. Results for M/M/1 Queue (Cont) Mean number of jobs in the system: Variance of the number of jobs in the system: CSE-550-T112 Lecture Notes - 319

20. Results for M/M/1 Queue (Cont) Probability of n or more jobs in the system: Mean response time (using the Little's law): Mean number in the system = Arrival rate × Mean response time That is: CSE-550-T112 Lecture Notes - 320

21. Results for M/M/1 Queue (Cont) Mean number of jobs in the queue: Idle  there are no jobs in the system Busy period = The time interval between two successive idle intervals CSE-550-T112 Lecture Notes - 321

22. Example On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about two milliseconds to forward them. Using an M/M/1 model, analyze the gateway. What is the probability of buffer overflow if the gateway had only 13 buffers? How many buffers do we need to keep packet loss below one packet per million? Arrival rate = 125 pps Service rate  = 1/.002 = 500 pps Gateway Utilization  = /  = 0.25 Probability of n packets in the gateway = (1-  )  n = 0.75(0.25) n CSE-550-T112 Lecture Notes - 322

23. Example (Cont) Mean Number of packets in the gateway =  /(1-  ) = 0.25/0.75 = 0.33 Mean time spent in the gateway = (1/  )/(1-  )= (1/500)/(1-0.25) = 2.66 milliseconds Probability of buffer overflow P(more than 14 packets in the gateway) =  14 =  14  *  -9 ≈ 4 packets per billion packets To limit the probability of loss to less than 10 -6 : We need about ten buffers. CSE-550-T112 Lecture Notes - 323

24. Example (Cont) The last two results about buffer overflow are approximate. Strictly speaking, the gateway should actually be modeled as a finite buffer M/M/1/B queue. However, since the utilization is low and the number of buffers is far above the mean queue length, the results obtained are a close approximation. CSE-550-T112 Lecture Notes - 324

25. CSE-550-T112 Lecture Notes - 325 Queuing Models - Single-Server Queue - λ : average number of packets arriving per second [pps] Utilization, fraction of time the facility is busy: ρ = λT s Theoretical maximum input rate that can be handled by the system is: λ max = 1/T s Queues become very large near system saturation, growing without bound when ρ = 1 Practical considerations limit the input rate for a single server to 70-90% of the theoretical maximum Little's formula (general relationship) : E[n] = λT r and E[n q ] = λT w

26. CSE-550-T112 Lecture Notes - 326 Queuing Models - Multiserver Queue - Utilization: ρ = λT s /N Theoretical maximum input rate that can be handled by the system is: λ max = N/T s Traffic intensity: u = Nρ

27. CSE-550-T112 Lecture Notes - 327 Queuing Models - Multiple Single-server queues - Example of a Network of Queues Traffic Partitioning Traffic Merging Queues in Tandem

28. CSE-550-T112 Lecture Notes - 328 Queuing Models - Network of Queues - Jackson's theorem states that: In such a network of queues, each node is an independent queuing system, with a Poisson input determined by the principles of partitioning, merging, and tandem queuing Each node may be analyzed separately from the others using the M/M/1 or M/M/N model Results may be combined by ordinary statistical methods, e.g., mean delays at each node may be added to derive system delays

29. CSE-550-T112 Lecture Notes - 329 Application to a Packet-Switching Network Consider a packet-switching network: Consists of nodes interconnected by transmission links Each node acts as the interface for zero or more attached systems, each of which functions as a source and destination of traffic Each link is seen as a service station servicing packets

30. CSE-550-T112 Lecture Notes - 330 Inside a Router

31. CSE-550-T112 Lecture Notes - 331 Component Models Simplifications Packets (requests) arrive according to a Poisson process (exponential interarrival times) Infinite buffer size Independent queues (just add delays induced in the different queues encountered on the path)

32. CSE-550-T112 Lecture Notes - 332 Traffic Flow Analysis - Objective Estimate: Delay Utilization of resources (links) Traffic flow across a network depends on: Topology Routing Traffic workload (from all traffic sources) Desirable topology and routing are associated with: Low delays Reasonable link utilization (no bottlenecks)

33. CSE-550-T112 Lecture Notes - 333 Traffic Flow Analysis - Assumptions Topology is fixed and stable Links and routers are 100% reliable Processing time at the routers is negligible Capacity of all links is given (in bps) Traffic workload is given Г = [γ jk ] (in pps) Routing is given Average packet size is given

34. CSE-550-T112 Lecture Notes - 334 Analyzing Throughput The capacity of the network can also limit the number of connections/users it can handle for a particular type of service This is determined by finding out the narrowest available bandwidth in the path This is the network bottleneck The narrowest bandwidth can be a router, switch, or link

35. CSE-550-T112 Lecture Notes - 335 External Workload The external workload offered to the network is: Where: γ = total workload in packets per second γ jk = workload between source j and destination k N = total number of sources and destinations

36. CSE-550-T112 Lecture Notes - 336 Internal Workload The internal workload on link i is: λ i = Σ i Є  jk γ jk Where: γ jk = workload between source j and destination k  jk = path followed by packets to go from source j and destination k The total internal workload is: Where: λ = total load on all of the links in the network λ i = load on link i L = total number of links

37. CSE-550-T112 Lecture Notes - 337 Link Utilization Utilization of link i is: ρ i = λ i * T s i Service time for link i is: T s i = M / B i Where: M = Average packet length (in bits) B i = Data rate on the link (in bps) Average service rate : μ i = 1/T s i = B i / M ρ i = λ i / μ i = λ i * M / B i ρ b = max ( ρ i ) – Link b is the primary bottleneck Stability condition of a network is: ρ b < 1

38. CSE-550-T112 Lecture Notes - 338 Path Length and Packets Waiting Average length for all paths: Average number of packets waiting and being served for link i is: E[n i ] = λ i T ri Number of packets waiting and being served in the network can be expressed as (using Little's formula): γ T =

39. CSE-550-T112 Lecture Notes - 339 Link Delay Because we are assuming that each queue can be treated as an independent M/M/1 model, we have: The service time for link i is: T s i = M / B i,Then:

40. CSE-550-T112 Lecture Notes - 340 Network Delay Average delay experienced by a packet through the network: Putting all of the elements together, we get:

41. CSE-550-T112 Lecture Notes - 341 Applying M/M/1 Results to a Single Network Link Poisson packet arrivals with rate: λ = 2000 pps Fixed link capacity: C = 1.544 Mbps (T1 Carrier rate) We approximate the packet length distribution by an exponential with mean: L = 515 bits/packet Thus, the service time is exponential with mean: T s = L/C = 0.33 ms/packet i.e., packets are served at a rate of: μ = 1/T s = M / C = 3000 pps Using our formulas for an M/M/1 queue: ρ = λ/μ = λ*T s = 0.67 So, E[n] = ρ/(1- ρ) = 2 packets and: T r = E[n]/ λ = 1 ms

42. CSE-550-T112 Lecture Notes - 342 Exercise 1 The problem consists of 3 Routers A, B, C, and 6 Switches, a, b, c, d, e, and f Assume that the three Routers are connected according to a unidirectional ring topology (A-B-C-A) and that all links have the same capacity of 2 Mbps Assume that the Switches are connected as follows: (a, C), (b, C), (c, A), (d, A), (e, B), (f, B) The average packet size has been estimated equal to 2000 bits It has also been observed that the traffic generated by the various switches is Poissonian with rates as indicated in the following table showing the Inter- switches traffic in pps: Question: Find T, the average delay per packet abcdef a-2050103020 b -10204060 c5010-802010 d 2080-50 e30402050-100 f20601050100-

43. CSE-550-T112 Lecture Notes - 343

44. CSE-550-T112 Lecture Notes - 344 Animation of a Transmission Link Play with animation of a transmission link at http://poisson.ecse.rpi.edu/~vastola/pslinks/p erf/hing/mm1animate.html http://poisson.ecse.rpi.edu/~vastola/pslinks/p erf/hing/mm1animate.html

45. CSE-550-T112 Lecture Notes - 345 References The Art of Computer Systems Performance Analysis by Raj Jain, John Wiley, 1991. http://www.cse.wustl.edu/~jain/books/perfbo ok.htm William Stalling, “Queuing Analysis”, 2000 Dr. Khalid Salah (ICS, KFUPM), CSE 550 Lecture Slides, Term 032