Queuing models Basic definitions, assumptions, and identities Operational laws Little’s law Queuing networks and Jackson’s theorem The importance of think time Non-linear results as saturation approaches Warnings References

What problem are we solving? Describe a system’s mean Throughput Response time Capacity Predict Changes in these quantities when system characteristics change See Stallings, Figure 1

Stallings, Figure 1

One queue arrivalsdepartures waitingserving = a job Server See Stallings, Figure 2

Stallings, Figure 2, Table 2

Basic assumptions One kind of job Inter-arrival times are independent of system state Service times are independent of system state No jobs lost because of buffer overflow Stability: λ < 1 / Ts In a network, no parallel processing of a given job visit ratios are independent of system state

Queuing definitions A / S / m / B / K / SD A = Inter-arrival time distribution S = Service time distribution m = Number of servers B = Number of buffers (system capacity) K = Population size SD = service discipline Usually specify just the first three: M/M/1

Usual assumptions A is often the Exponential distribution S is often Exponential or constant B is often infinite (all the buffer space you need) K is often infinite SD is often FCFS (first come, first served)

Exponential distribution Also known as “memoryless” Expected time to the next arrival is always the same, regardless of previous arrivals When the interarrival times are independent, identically-distributed, and the distribution is exponential, the arrival process is called a “Poisson” process

Poisson processes Popular because they are tractable to analyze You can merge several Poisson streams and get a Poisson stream You can split a Poisson stream and get Poisson streams Poisson arrivals to a single queue with exponential service times => departures are Poisson with same rate Same is true of departures from a M/M/m queue

Basic formulas: Stallings, Table 3b Assumptions Poisson arrivals No dispatching preference based on service times FIFO dispatching No items discarded from queue

Basic multi-server formulas

Expected response time For a single M/M/1 queue, expected residence (response) time is 1/(μ-λ), where μ is the server’s maximum output rate (1/Ts) λ is the mean arrival rate Example: Disk can process 100 accesses/sec Access requests arrive at 20/sec Expected response time is 1/(100-20) = sec/access

Example: near saturation 80% utilization Disk can do 100 accesses/sec Access requests arrive at 80/sec Expected response time: 1/(100-80) = 0.05 sec 90% utilization Expected response time: 1/(100-90) = 0.1 sec 95% utilization Expected response time: 1/(100-95) = 0.2 sec 99% utilization Expected response time: 1/(100-99) = 1 sec

Nearing saturation (M/M/1)

Operational law Little’s Law r= λ T r Example: T r = 0.3 sec average residence in the system λ = 10 transactions / sec r = 3 average transactions in the system

Queuing network Queue 1Queue 2 Queue 3

Jackson’s theorem Assuming Each node in the network provides an independent service Poisson arrivals Once served at a node, an item goes immediately to another node, or out of the system Then Each node is an independent queuing system Each node’s input is Poisson Mean delays at each node may be added to compute system delays

Queuing network example Servlet History data Order data T r = 50 msec T r = 60 msec T r = 80 msec λ = 12 jobs/sec P = 0.4 P = 0.6 Average system residence time= 0.4 * (50+60) * (50+80) (Jackson’s theorem)= 122 msec Average jobs in system= 12 * = (Little’s Law)

An example in a spreadsheet

Think time “think time” between transactions request a transaction Think time can have a huge effect on the arrival rate for a system.

Main Reference Queuing Analysis, William Stallings Cached copy on the resource page: nalysis.pdf