1 Queuing Delay and Queuing Analysis

RECALL: Delays in Packet Switched (e.g. IP) Networks End-to-end delay (simplified) = End-to-end delay (simplified) = –(d prop + d trans + d queue + d proc ) … on each link Introduction 2 B A Where: Where:  Propagation delay (d prop ) = d/s (dependent on path)  Transmission delay (d trans ) = L/R (dependent on path)  Queuing delay (d queue ) = (dependent on load)  Processing delay (d proc ) = (minimal-insignificant/node)  Number of links (Q) = (dependent on path)

Queuing Analysis 3 Projected vs. Actual Response Time Why??

Queuing Analysis R: link bandwidth (bps) R: link bandwidth (bps) L: packet length (bits) L: packet length (bits) a: average packet arrival rate a: average packet arrival rate traffic intensity = La/R  La/R ~ 0: avg. queueing delay small  La/R -> 1: avg. queueing delay large  La/R > 1: more “work” arriving than can be serviced, average delay infinite! than can be serviced, average delay infinite! average queueing delay La/R ~ 0 Queueing delay (revisited) La/R -> 1 4

Queuing Analysis 5 Introduction- Motivation Address how to analyze changes in network workloads (i.e., a helpful tool to use) Address how to analyze changes in network workloads (i.e., a helpful tool to use) Analysis of system (network) load and performance characteristics Analysis of system (network) load and performance characteristics –response time –throughput Performance tradeoffs are often not intuitive Performance tradeoffs are often not intuitive Queuing theory, although mathematically complex, often makes analysis very straightforward Queuing theory, although mathematically complex, often makes analysis very straightforward

Queuing Analysis 6 Important Note Queuing theory is heavily dependent on basic probability theory (a pre- requisite for our graduate program) Queuing theory is heavily dependent on basic probability theory (a pre- requisite for our graduate program) If you need to refresh your knowledge in this area, please review the Stallings textbook, Chapter 7: Overview of Probability and Stochastice Processes. If you need to refresh your knowledge in this area, please review the Stallings textbook, Chapter 7: Overview of Probability and Stochastice Processes. I will not test you specifically on probability theory, but will reference it in coverage of the queuing topics addressed in this module. I will not test you specifically on probability theory, but will reference it in coverage of the queuing topics addressed in this module.

Queuing Analysis 7 Single-Server Queuing System Queuing System (Delay Box) Items Arriving (rate: (rate: ) (message, packet, cell) Items Lost Items Departing (rate: R)

Introduction 8 Router output port functions  buffering/queuing required when datagrams arrive from fabric faster than the transmission rate  scheduling discipline chooses among queued datagrams for transmission  sending discipline (servicing the queue) on the output link as determined by link protocol line termination link layer protocol (send) switch fabric datagram buffer(s) queueing Queue Queue server

Queuing Analysis 9 The Fundamental Task of Queuing Analysis Given: Arrival rate, Arrival rate, Service time, T s Service time, T s Number of servers, N Number of servers, N Determine: Items waiting, w Items waiting, w Waiting time, T w Waiting time, T w Items queued, r Items queued, r Residence time, T r Residence time, T r

Queuing Analysis 10 Parameters for Single-Server Queuing System Comments, assuming queue has infinite capacity: 1.At  = 1, server is working 100% of the time (saturated), so items are queued (delayed) until they can be served. Departures remain constant (for same L). 2.Traffic intensity, u = L /R. Note that T s = L/R, so: max = 1 / T s = 1 / (L/R) is the theoretical maximum arrival rate, and that L max /R = u = 1 at the theoretical maximum arrival rate

Queuing Analysis 11 Queuing Process - Example General Expression: T Rn+1 = T Sn+1 + MAX[0, D n – A n+1 ] Depth of the Queue

Queuing Analysis 12 General Characteristics of Network Queuing Models Item population Item population –generally assumed to be infinite therefore, arrival rate is persistent through time Queue size Queue size –infinite, therefore no loss –finite, more practical, but often immaterial Dispatching discipline Dispatching discipline –FIFO, typical –LIFO (when is this practical?) –Relative/Preferential, based on QoS

Queuing Analysis 13 Multiserver Queuing System Comments: 1.Assuming N identical servers, and  is the utilization of each server. 2.Then, N  is the utilization of the entire system, and the maximum utilization is N x 100%. 3.Therefore, the maximum supportable arrival rate that the system can handle is: max = N / T s = NR/L

Chapter 8 Overview of Queuing Analysis 14 Multiple Single-Server Queuing Systems

Queuing Analysis 15 Basic Queuing Relationships General Single Server Multiserver r = T r Little’s Formula  = T s  =  = w = T w Little’s Formula r = w +  u = T s =  N T r = T w + T s r = w + N  T s T sN

Queuing Analysis 16 Kendall’s notation Notation is X/Y/N, where: Notation is X/Y/N, where: X is distribution of interarrival times Y is distribution of service times N is the number of servers Common distributions Common distributions  G = general distribution if interarrival times or service times  GI = general distribution of interarrival time with the restriction that they are independent  M = negative exponential distribution of interarrival times (Poisson arrivals – p. 167) and service times  D = deterministic arrivals or fixed length service M/M/1? M/D/1?

Queuing Analysis 17 Important Formulas for Single- Server Queuing Systems Note Coefficient of variation: if  Ts = T s => exponential if  Ts = 0 => constant

Queuing Analysis 18 Mean Number of Items in System (r)- Single-Server Queuing  T s /T s = Coefficient of variation M/M/1 M/D/1

Queuing Analysis 19 Mean Residence Time – (T r ) Single-Server Queuing M/M/1 M/D/1

Queuing Analysis 20 Network Queue Performance: Key Fact The higher the variability in arrival rate at the router, relative to the service time on the output link(s), i.e.,  T s /T s, the poorer the performance of the router, especially at high rates of utilization.

Queuing Analysis 21 Multiple Server Queuing Systems Multiple Single- Server Queuing System Multiserver Queuing System

Queuing Analysis 22 Important Formulas for Multiserver Queuing Note: Useful only in M/M/N case, with equal service times at all N servers.

Queuing Analysis 23 Multiple Server Queuing Example (p. 203) Single server M/M/1 (2 nd Floor) Multiserver M/M/? (2 nd Floor) Multiple Single server M/M/1 (1 st Floor) M/M/1 (2 nd Floor) M/M/1 (3 rd Floor)

Queuing Analysis 24 MultiServer vs. Multiple Single- Server Queuing System Comparison (from example problem, pp ) Single server case (M/M/1): Single server utilization:  = 10 engineers x 0.5 hours each / 8 hour work day = 5/8 =.625 Average time waiting: T w =  T s / 1 -  = x 30 /.375 = 50 minutes Arrival rate: = 10 engineers per 8 hours = 10/480 = engineers/minute 90 th percentile waiting time: m T w (90) = T w /  x ln(10  ) = minutes Average number of engineers waiting: w = T w = x 50 = engineers

Queuing Analysis 25 Example: Router Queuing Internet … 9600bps = 5 packets/sec = 5 packets/sec L = 144 octets From data provided: T s = L/R = (144x8)/9600 =.12secT s = L/R = (144x8)/9600 =.12sec  = T s = 5 packets/sec x.12sec =.6  = T s = 5 packets/sec x.12sec =.6Determine: 1.T r = T s / (1-  ) =.12sec/.4 =.3 sec 2.r =  / (1-  ) =.6/.4 = 1.5 packets 3. m r (90) = - 1 = 3.5 packets 4.m r (95) = - 1 = 4.8 packets ln(1-.90) ln (.6) ln(1-.95) For 3 & 4, use: m r (y) = - 1 ln(1 – y/100) ln(1 – y/100) ln 

Queuing Analysis 26 Priorities in Queues – Two priority classes r

Chapter 8 Overview of Queuing Analysis 27 Priorities in Queues – Example Router queue services two packet sizes: Long = 800 octets Long = 800 octets Short = 80 octets Short = 80 octets Lengths exponentially distributed Lengths exponentially distributed Arrival rates are equal, 8packets/sec Arrival rates are equal, 8packets/sec Link transmission rate is 64Kbps Link transmission rate is 64Kbps Short packets are priority 1, Short packets are priority 1, Longer packets are priority 2 Longer packets are priority 2 From data above, calculate: T s 1 = L short /R = (80 x 8) / =.01 sec T s 2 = L long /R = (800 x 8) / =.1 sec  1 = T s 1 = 8 x 0.01 = 0.08  2 = T s 2 = 8 x 0.1 = 0.8  =  1 +  2 = 0.88 Find the average Queuing Delay (T r ) through the router: T r1 = T s1 + =.01 + = sec =.01 + = sec T r2 = T s2 + =.1 + = sec =.1 + = sec T r = T r1 + T r2 =.5 x x.833 = sec =.5 x x.833 = sec  1 T s 1 +  2 T s  1.08 x x T r 1 - T s          64Kbps TrTrTrTr 

Queuing Analysis 28 Network of Queues

Queuing Analysis 29 Elements of Queuing Networks

Queuing Analysis 30 Queuing Networks

Queuing Analysis 31 Jackson’s Theorem and Queuing Networks Assumptions: Assumptions: –the queuing network has m nodes, each providing exponential service –items arriving from outside the system at any node arrive with a Poisson rate –once served at a node, an item moves immediately to another with a fixed probability, or leaves the network Jackson’s Theorem states: Jackson’s Theorem states: –each node is an independent queuing system with Poisson inputs determined by partitioning, merging and tandem queuing principles –each node can be analyzed separately using the M/M/1 or M/M/N models –mean delays at each node can be added to determine mean system (network) delays

Queuing Analysis 32 Jackson’s Theorem - Application in Packet Switched Networks Packet Switched Network External load, offered to network:  =    jk  =    jkwhere:  = total workload in packets/sec  = total workload in packets/sec  jk = workload between source j  jk = workload between source j and destination k and destination k N = total number of (external) N = total number of (external) sources and destinations sources and destinations N N N N j=1 k=2 j=1 k=2 Internal load: =  i =  iwhere: = total on all links in network = total on all links in network i = load on link i i = load on link i L = total number of links L = total number of links L i=1 i=1 Note: Internal > offered load Internal > offered load Average length for all paths: Average length for all paths: E[number of links in path] = /  E[number of links in path] = /  Average number of item waiting Average number of item waiting and being served in link i: r i = i T ri and being served in link i: r i = i T ri Average delay of packets sent Average delay of packets sent through the network is: through the network is: T =  T =  where: M is average packet length and where: M is average packet length and R i is the data rate on link i R i is the data rate on link i 1 L i=1 i=1 M i R i - M i

Queuing Analysis 33 Estimating Model Parameters To enable queuing analysis using these models, we must estimate certain parameters: To enable queuing analysis using these models, we must estimate certain parameters: –Mean and standard deviation of arrival rate –Mean and standard deviation of service time (or, packet size) Typically, these estimates use sample measurements taken from an existing system Typically, these estimates use sample measurements taken from an existing system

Queuing Analysis 34 Sample Means for Exponential Distribution Sampling: The mean is generally the most important quantity to estimate: (  ) = X i Sample mean is itself a random variable Central Limit Theorem: the probability distribution tends to normal as sample size, N, increases for virtually all underlying distributions The mean and variance of X can be calculated as: E[  ]= E[X] =  Var[  ]=  2 x /N N  i = 1 1N1N