1. COMP60621 Designing for Parallelism
Lecture 7
Queueing Theory: Part 1
John Gurd, Graham Riley
Centre for Novel Computing
School of Computer Science
University of Manchester

2. Queueing Theory Mathematical study of waiting in queues (lines)
Another model of concurrent activity
Applicable to many situations
Everyday situations, such as:
Supermarket check-outs
Bank tellers
Airport landings
Passport control
Computer Science applications:
Scheduling read/write accesses to disk
Scheduling tasks on multiple servers
24 February 2019

3. Single Queue, Single Server
24 February 2019

4. Single Queue, Multiple Servers
Arrival of
Customers
24 February 2019

5. Multiple Queues, Multiple Servers
Departure of
Customers
Arrival of
Customers
Customer free to
join any queue
24 February 2019

6. Queueing Problems
There are three basic elements to any queueing problem:
The input process – the arrivals rate
The queueing discipline – number of queues, etc.
The service mechanism – the service rate
24 February 2019

7. Input Process Number of potential customers:
Finite or infinite
Number arriving simultaneously:
One or many (with constant or varying batch size(s))
Intervals between arrivals:
Constant, completely random, other statistical distribution
Average rate of arrivals:
Constant, varying with time, influenced by state of queue
Outside influence:
None, input is output of previous queue, network of queues
Etc.
24 February 2019

8. Queueing Discipline Number of queues: Queue discipline: Etc.
One, multiple (fixed number, varying numbers)
Queue discipline:
Served in order of arrival (FIFO – “fair” queue)
Served in inverse order of arrival (LIFO – “unfair” queue)
Served at random
Served in priority order (based on “need”?)
Changes from time-to-time
Etc.
24 February 2019

9. Service Mechanism Number of service points: Number served: Etc.
One, several (fixed number, varying numbers)
Number served:
One-at-a-time, in batches (with fixed or variable batch size(s))
Etc.
24 February 2019

10. Effectiveness of Queues
For given assumptions about the three basic elements of a queueing problem (input, discipline, service), we can set up a mathematical model to predict the behaviour of the system, to answer questions such as:
What is the average queue size?
What is the average waiting time?
What is the probability of waiting longer than a given time?
Etc.
Thereby we can compare different queuing scenarios and plan appropriate resources.
24 February 2019

11. Simple Example of Queue Behaviour
Arrivals: let  be a constant that represents the average rate of arrivals.
For random arrivals, in any short time interval (t,t+∂t),
The probability that there is no arrival is Pr{no arrival} = 1 – ∂t + o(∂t), where lim(o(∂t)/ ∂t) = 0 as ∂t → 0 (i.e., o(∂t) is negligible compared with ∂t).
The probability that there is one arrival is Pr{one arrival} = ∂t + o(∂t).
The probability that there is more than one arrival is Pr{two or more arrivals} = o(∂t).
N.B. The probability of an arrival (or not) in any interval (t,t+∂t) is independent of t and is independent of any other non-overlapping interval. Arrival (or non-arrival) in any given interval is thus not influenced in any way by what happened at earlier times.
24 February 2019

12. Poisson Arrivals
The number of arrivals, N, in a finite interval (t,t+x) of length x is a random variable which has a Poisson distribution with parameter x.
Pr{N=n} = probability of n arrivals in (t,t+x) =
24 February 2019

13. Poisson Arrivals
Proof: Suppose that the interval x is divided into m equal subintervals of length x/m = ∂t, then:
Pr{one arrival in any subinterval} = ∂t + o(∂t),
Pr{no arrival in any subinterval} = 1 – ∂t + o(∂t).
Hence:
Pr{exactly n arrivals in (t,t+x)} =
(One arrival in each of n subintervals, and no arrivals in the remaining (m – n) subintervals; note that m > n.)
24 February 2019

14. Poisson Arrivals Hence: Pr{exactly n arrivals in (t,t+x)} =
Thus N has a Poisson distribution with parameter x.
24 February 2019

15. Inter-arrival Time
The time that elapses between arrivals is called the inter-arrival time. Denoting this by the continuous random variable T, it can be shown that T has a negative exponential distribution:
Pr{T>t} = Pr{no arrival in (0,t)} =
Equivalently,
Pr{T≤t} = Pr{arrival in (0,t)}
= 1 – Pr{T>t} =
24 February 2019

16. Pr{no arrival in (0,t)} t
24 February 2019

17. Pr{arrival in (0,t)} t
24 February 2019

18. Equivalence The following three statements are equivalent:
Pr{no arrivals in (t,t+∂t)} = 1 – ∂t + o(∂t) and Pr{one arrival in (t,t+∂t)} = ∂t + o(∂t).
The number of arrivals in any interval of length x is a random variable N which has a Poisson distribution with parameter x.
The inter-arrival time between successive arrivals is a random variable T which has a negative exponential distribution with parameter .
We now turn our attention to the service mechanism.
24 February 2019

19. Service Mechanism
We assume a single server and that the service time X is a random variable with a negative exponential distribution with parameter . Thus Pr{X>x} =
Alternatively, the probability that a service that has been in progress for time x terminates in the interval (x,x+∂x) is ∂x + o(∂x). Or, the probability that a service that has been in progress for time x does not terminate in the interval (x,x+∂x) is 1 – ∂x + o(∂x).
Alternatively, the number of customers N that can be serviced during the interval (t,t+x) is a random variable that has a Poisson distribution with parameter x and Pr{N=n} =
24 February 2019

20. Shorthand Notation for Queues
Different kinds of queue are denoted by X/Y/Z, where X is the inter-arrival distribution, Y is the service time distribution, and Z is the number of servers.
The permitted distributions are:
M = negative exponential (Markovian)
D = constant (Deterministic)
G = General independent
M/M/1 thus denotes a single server queue with random arrivals (negative exponential inter-arrival times) and random service completions (negative exponential service times);
while D/M/2 denotes a two-server queue with regular arrivals (constant inter-arrival times) and random service completions (negative exponential service times).
24 February 2019