COMP60611 Fundamentals of Parallel and Distributed Systems Lecture 13 Queueing Theory: Part 1 John Gurd, Graham Riley Centre for Novel Computing School of Computer Science University of Manchester

Queueing Theory Mathematical study of waiting in queues (lines) Another model of concurrent activity Applicable to many situations Everyday situations, such as: Supermarket check-outs Bank tellers Airport landings Passport control Computer Science applications: Scheduling read/write accesses to disk Scheduling tasks on multiple servers 31/12/2018

Single-channel Queue 31/12/2018

Single Queue, Multiple Servers Arrival of Customers 31/12/2018

Multiple Queues, Multiple Servers Departure of Customers Arrival of Customers Customer free to join any queue 31/12/2018

Queueing Problems There are three basic elements to any queueing problem: The input process – the arrivals rate The queueing discipline – number of queues, etc. The service mechanism – the service rate 31/12/2018

Input Process Number of potential customers: Finite or infinite Number arriving simultaneously: One or many (with constant or varying batch size(s)) Intervals between arrivals: Constant, completely random, other statistical distribution Average rate of arrivals: Constant, varying with time, influenced by state of queue Outside influence: None, input is output of previous queue, network of queues Etc. 31/12/2018

Queueing Discipline Number of queues: Queue discipline: Etc. One, multiple (fixed number or varying numbers) Queue discipline: Served in order of arrival (FIFO – “fair” queue) Served in inverse order of arrival (LIFO – “unfair” queue) Served at random Served in priority order (based on “need”?) Changes from time-to-time Etc. 31/12/2018

Service Mechanism Number of service points: Number served: Etc. One, several (fixed number or varying numbers) Number served: One-at-a-time, in batches (with fixed or variable batch size(s)) Etc. 31/12/2018

Effectiveness of Queues For given assumptions about the three basic elements of a queueing problem (input, discipline, service), we can set up a mathematical model to predict the behaviour of the system, to answer questions such as: What is the average queue size? What is the average waiting time? What is the probability of waiting longer than a given time? Etc. Thereby we can compare different queuing scenarios and plan appropriate resources. 31/12/2018

Simple Example of Queue Behaviour Arrivals: let  be a constant that represents the average rate of arrivals. For random arrivals, in any short time interval (t,t+∂t), The probability that there is no arrival is Pr{no arrival} = 1 – ∂t + o(∂t), where lim(o(∂t)/ ∂t) = 0 as ∂t → 0 (i.e., o(∂t) is negligible compared with ∂t). The probability that there is one arrival is Pr{one arrival} = ∂t + o(∂t). The probability that there is more than one arrival is Pr{two or more arrivals} = o(∂t). N.B. The probability of an arrival (or not) in any interval (t,t+∂t) is independent of t and is independent of any other non-overlapping interval. Arrival (or non-arrival) in any given interval is thus not influenced in any way by what happened at earlier times. 31/12/2018

Poisson Arrivals The number of arrivals, N, in a finite interval (t,t+x) of length x is a random variable which has a Poisson distribution with parameter x. Pr{N=n} = probability of n arrivals in (t,t+x) = 31/12/2018

Poisson Arrivals Proof: Suppose that the interval x is divided into m equal subintervals of length x/m = ∂t, then: Pr{one arrival in any subinterval} = ∂t + o(∂t), Pr{no arrival in any subinterval} = 1 – ∂t + o(∂t). Hence: Pr{exactly n arrivals in (t,t+x)} = (One arrival in each of n subintervals, and no arrivals in the remaining (m – n) subintervals; note that m > n.) 31/12/2018

Poisson Arrivals Hence: Pr{exactly n arrivals in (t,t+x)} = Thus N has a Poisson distribution with parameter x. 31/12/2018

Inter-arrival Time The time that elapses between arrivals is called the inter-arrival time. Denoting this by the continuous random variable T, it can be shown that T has a negative exponential distribution: Pr{T>t} = Pr{no arrival in (0,t)} = Equivalently, Pr{T≤t} = Pr{arrival in (0,t)} = 1 – Pr{T>t} = 31/12/2018

Pr{no arrival in (0,t)} 0 2 4 6 8 10 t 31/12/2018

Pr{arrival in (0,t)} 0 2 4 6 8 10 t 31/12/2018

Equivalence The following three statements are equivalent: Pr{no arrivals in (t,t+∂t)} = 1 – ∂t + o(∂t) and Pr{one arrival in (t,t+∂t)} = ∂t + o(∂t). The number of arrivals in any interval of length x is a random variable N which has a Poisson distribution with parameter x. The inter-arrival time between successive arrivals is a random variable T which has a negative exponential distribution with parameter . We now turn our attention to the service mechanism. 31/12/2018

Service Mechanism We assume a single server and that the service time X is a random variable with a negative exponential distribution with parameter . Thus Pr{X>x} = Alternatively, the probability that a service that has been in progress for time x terminates in the interval (x,x+∂x) is ∂x + o(∂x). Or, the probability that a service that has been in progress for time x does not terminate in the interval (x,x+∂x) is 1 – ∂x + o(∂x). Alternatively, the number of customers N that can be serviced during the interval (t,t+x) is a random variable that has a Poisson distribution with parameter x and Pr{N=n} = 31/12/2018

Shorthand Notation for Queues Different kinds of queue are denoted by X/Y/Z, where X is the inter-arrival distribution, Y is the service time distribution, and Z is the number of servers. The permitted distributions are: M = negative exponential (Markovian) D = constant (Deterministic) G = General independent M/M/1 thus denotes a single server queue with random arrivals (negative exponential inter-arrival times) and random service completions (negative exponential service times); While D/M/2 denotes a two-server queue with regular arrivals (constant inter-arrival times) and random service completions (negative exponential service times). 31/12/2018