1. Lecture 2.7. Queuing Theory

2. Basic teletraffic concepts

3. 1 user making phone calls
TRAFFIC is a “stochastic process”
BUSY 1
IDLE 0
time
How to characterize this process?
statistical distribution of the “BUSY” period
statistical distribution of the “IDLE” period
statistical characterization of the process “memory”
E.g. at a given time, does the probability that a user starts a call result different depending on what happened in the past?

4. Traffic characterization suitable for traffic engineering
All equivalent (if stationary process)

5. Traffic Intensity: example
User makes in average 1 call every hour
Each call lasts in average 120 s
Traffic intensity =
120 sec / 3600 sec = 2 min / 60 min = 1/30
Probability that a user is busy
User busy 2 min out of 60 = 1/30
Dimensionless

6. Traffic generated by more than one users
Traffic intensity
(dimensionless, measured in Erlangs): U2 U3 U4
TOT

7. LOADING AND ITS CHARACTERISTICS Erlang (unit) The erlang (symbol E) as a dimensionless unit is used as a statistical measure of offered load. It is named after the Danish telephone engineer A.K.Erlang, the originator of traffic engineering and queueing theory. Traffic of one erlang refers to a single resource being in continuous use, or two channels being at fifty percent use each, and so on. For example, if an office had two telephone operators who are both busy all the time, that would represent two erlangs (2 E) of traffic, or a radio channel that is occupied for thirty minutes during an hour is said to carry 0.5 E of traffic.

8. Alternatively, an erlang may be regarded as a "use multiplier" per unit time, so 100% use is 1 E, 200% use is 2 E, and so on. For example, if total cell phone use in a given area per hour is 180 minutes, this represents 180/60 = 3 E. In general, if the mean arrival rate of new calls is λ per unit time and the mean call holding time is h, then the traffic in erlangs E is: E = λh

9. Example 1 5 users Each user makes an average of 3 calls per hour
Each call, in average, lasts for 4 minutes
Meaning: in average, there is 1 active call; but the actual number of active calls varies from 0 (no active user) to 5 (all users active), with given probability

10. Example 2 30 users Each user makes an average of 1 calls per hour
Each call, in average, lasts for 4 minutes
SOME NOTES:
In average, 2 active calls (intensity A);
Frequently, we find up to 4 or 5 calls;
Prob(n.calls>8) = 0.01%
More than 11 calls only once over 1M
TRAFFIC ENGINEERING: how many channels to reserve for these users!

11. A note on binomial coefficient computation

12. Infinite Users
Assume M users, generating an overall traffic intensity A
(i.e. each user generates traffic at intensity Ai =A/M).
We have just found that
Let Minfinity, while maintaining the same overall traffic intensity A

13. Poisson Distribution
Very good matching with Binomial (when M large with respect to A)
Much simpler to use than Binomial (no annoying queueing theory complications)

14. Limited number of channels
THE most important problem in circuit switching U1
The number of channels C is less than the number of users M (eventually infinite)
Some offered calls will be “blocked”
What is the blocking probability?
We have an expression for
P[k offered calls]
We must find an expression for
P[k accepted calls]
As: U2 X U3 X U4
TOT
No. carried calls versus t
No. offered calls versus t

15. Channel utilization probability
C channels available
Assumptions:
Poisson distribution (infin. users)
Blocked calls cleared
It can be proven (from Queueing theory) that:
(very simple result!)
Hence:

16. Blocking probability: Erlang-B
Fundamental formula for telephone networks planning
Ao=offered traffic in Erlangs
Efficient recursive computation available

17. NOTE: finite users Erlang-B obtained for the infinite users case
It is easy (from queueing theory) to obtain an explicit blocking formula for the finite users case:
ENGSET FORMULA:
Erlang-B can be re-obtained as limit case
Minfinity
Ai0
M·AiAo
Erlang-B is a very good approximation as long as:
A/M small (e.g. <0.2)
In any case, Erlang-B is a conservative formula
yields higher blocking probability
Good feature for planning

18. Capacity planning
Target: support users with a given Grade Of Service (GOS)
GOS expressed in terms of upper-bound for the blocking probability
GOS example: subscribers should find a line available in the 99% of the cases, i.e. they should be blocked in no more than 1% of the attempts
Given:
C channels
Offered load Ao
Target GOS Btarget
C obtained from numerical inversion of

19. Channel usage efficiency
Offered load (erl)
Carried load (erl)
C channels
Blocked traffic
Fundamental property: for same GOS, efficiency
increases as C grows!! (trunking gain)

20. Example GOS = 1% maximum blocking.
Resulting system dimensioning and efficiency:
40 erl
C >= 53
h = 74.9%
60 erl
C >= 75
h = 79.3%
80 erl
C >= 96
h = 82.6%
100 erl
C >= 117
h = 84.6%

21. Erlang B calculation - tables

22. Introduction to Queuing Theory

23. Queuing theory definitions
(Bose) “the basic phenomenon of queueing arises whenever a shared facility needs to be accessed for service by a large number of jobs or customers.”
(Wolff) “The primary tool for studying these problems [of congestions] is known as queueing theory.”
(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory." "Any system in which arrivals place demands upon a finite capacity resource may be termed a queueing system.”
(Mathworld) “The study of the waiting times, lengths, and other properties of queues.”

24. Applications of Queuing Theory
Telecommunications
Traffic control
Determining the sequence of computer operations
Predicting computer performance
Health services (eg. control of hospital bed assignments)
Airport traffic, airline ticket sales
Layout of manufacturing systems.

25. Example: application of queuing theory
In many retail stores and banks
multiple line/multiple checkout system  a queuing system where customers wait for the next available cashier
We can prove using queuing theory that : throughput improves increases when queues are used instead of separate lines

26. Example: application of queuing theory

27. Queuing theory for studying networks
View network as collections of queues
FIFO data-structures
Queuing theory provides probabilistic analysis of these queues
Examples:
Average length
Average waiting time
Probability queue is at a certain length
Probability a packet will be lost

28. Little’s Law Little’s Law: Mean number tasks in system =
Departures
Arrivals
Little’s Law: Mean number tasks in system =
= mean arrival rate x mean response time
Observed before, Little was first to prove
Applies to any system in equilibrium, as long as nothing in black box is creating or destroying tasks

30. Proving Little’s Law 1 2 3 4 5 6 7 8 # in System 1 2 3 Arrivals 1 2 3
Packet #
Departures
Time
1 2 3
Time in
System
Packet # 1 2 3
Time
J = Shaded area = 9
Same in all cases!

31. Definitions J: “Area” from previous slide N: Number of jobs (packets)
T: Total time
l: Average arrival rate
N/T
W: Average time job is in the system
= J/N
L: Average number of jobs in the system
= J/T

32. Proof: Method 1: Definition
1 2 3
Time in
System
(W)
Packet # (N) 1 2 3 1 2 3
# in
System
(L) =
Time (T)

33. Proof: Method 2: Substitution
Tautology

34. Model Queuing System Use Queuing models to
Describe the behavior of queuing systems
Evaluate system performance
Server System
Queuing System
Queue
Server

35. Characteristics of queuing systems
Arrival Process
The distribution that determines how the tasks arrives in the system.
Service Process
The distribution that determines the task processing time
Number of Servers
Total number of servers available to process the tasks

36. Kendall Notation 1/2/3(/4/5/6)
Six parameters in shorthand
First three typically used, unless specified
Arrival Distribution
Service Distribution
Number of servers
Total Capacity (infinite if not specified)
Population Size (infinite)
Service Discipline (FCFS/FIFO)

37. Distributions
M: stands for "Markovian", implying exponential distribution for service times or inter-arrival times.
D: Deterministic (e.g. fixed constant)
Ek: Erlang with parameter k
Hk: Hyperexponential with param. k
G: General (anything)

38. Kendall Notation Examples
M/M/1:
Poisson arrivals and exponential service, 1 server, infinite capacity and population, FCFS (FIFO)
the simplest ‘realistic’ queue
M/M/m
Same, but M servers
G/G/3/20/1500/SPF
General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First

39. Poisson Process
For a Poisson process with average arrival rate , the probability of seeing n arrivals in time interval delta t

40. Poisson process & exponential distribution
Inter-arrival time t (time between arrivals) in a Poisson process follows exponential distribution with parameter

41. Analysis of M/M/1 queue Given:
l: Arrival rate of jobs (packets on input link)
m: Service rate of the server (output link)
Solve:
L: average number in queuing system
Lq average number in the queue
W: average waiting time in whole system
Wq average waiting time in the queue

42. M/M/1 queue model l m Wq W L Lq

43. Solving queuing systems
4 unknowns: L, Lq W, Wq
Relationships:
L=lW
Lq=lWq (steady-state argument)
W = Wq + (1/m)
If we know any 1, can find the others
Finding L is hard or easy depending on the type of system. In general:

44. Analysis of M/M/1 queue
Goal: A closed form expression of the probability of the number of jobs in the queue (Pi) given only l and m

45. Equilibrium conditions
Define to be the probability of having n tasks in the system at time t

46. Equilibrium conditions

47. Solving for P0 and Pn
Step 1
Step 2

48. Solving for P0 and Pn
Step 3
Step 4

49. Solving for L

50. Solving W, Wq and Lq

51. Online M/M/1 animation

53. Response Time vs. Arrivals

54. Stable Region
linear region

55. Example
On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model, what is the probability of buffer overflow if the gateway had only 13 buffers. How many buffers are needed to keep packet loss below one packet per million?

56. Example Measurement of a network gateway:
mean arrival rate (l): 125 Packets/s
mean response time (m): 2 ms
Assuming exponential arrivals:
What is the gateway’s utilization?
What is the probability of n packets in the gateway?
mean number of packets in the gateway?
The number of buffers so P(overflow) is <10-6?

57. Example Arrival rate λ = Service rate μ =
Gateway utilization ρ = λ/μ =
Prob. of n packets in gateway =
Mean number of packets in gateway =

58. Example Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps
Gateway utilization ρ = λ/μ = 0.25
Prob. of n packets in gateway =
Mean number of packets in gateway =

59. Example Probability of buffer overflow:
To limit the probability of loss to less than 10-6:

60. Example
Probability of buffer overflow: = P(more than 13 packets in gateway)
To limit the probability of loss to less than 10-6:

61. Example
Probability of buffer overflow: = P(more than 13 packets in gateway) = ρ13 = = 1.49x = 15 packets per billion packets
To limit the probability of loss to less than 10-6:

62. Example
Probability of buffer overflow: = P(more than 13 packets in gateway) = ρ13 = = 1.49x = 15 packets per billion packets
To limit the probability of loss to less than 10-6:

63. Example
To limit the probability of loss to less than 10-6: or

64. Example
To limit the probability of loss to less than 10-6:
or = 9.96

65. 1. Some Queuing Terminology
To describe a queuing system, an input process and an output process must be specified.
Examples of input and output processes are:
Situation
Input Process
Output Process
Bank
Customers arrive at bank
Tellers serve the customers
Pizza parlor
Request for pizza delivery are received
Pizza parlor send out truck to deliver pizzas

66. The Input or Arrival Process
The input process is usually called the arrival process.
Arrivals are called customers.
We assume that no more than one arrival can occur at a given instant.
If more than one arrival can occur at a given instant, we say that bulk arrivals are allowed.
Models in which arrivals are drawn from a small population are called finite source models.
If a customer arrives but fails to enter the system, we say that the customer has balked.

67. The Output or Service Process
To describe the output process of a queuing system, we usually specify a probability distribution – the service time distribution – which governs a customer’s service time.
We study two arrangements of servers: servers in parallel and servers in series.
Servers are in parallel if all servers provide the same type of service and a customer needs only pass through one server to complete service.
Servers are in series if a customer must pass through several servers before completing service.

68. Queue Discipline
The queue discipline describes the method used to determine the order in which customers are served.
The most common queue discipline is the FCFS discipline (first come, first served), in which customers are served in the order of their arrival.
Under the LCFS discipline (last come, first served), the most recent arrivals are the first to enter service.
If the next customer to enter service is randomly chosen from those customers waiting for service it is referred to as the SIRO discipline (service in random order).

69. Finally we consider priority queuing disciplines.
A priority discipline classifies each arrival into one of several categories.
Each category is then given a priority level, and within each priority level, customers enter service on a FCFS basis.
Another factor that has an important effect on the behavior of a queuing system is the method that customers use to determine which line to join.

70. 2. Modeling Arrival and Service Processes
We define ti to be the time at which the ith customer arrives.
In modeling the arrival process we assume that the T’s are independent, continuous random variables described by the random variable A.
The assumption that each interarrival time is governed by the same random variable implies that the distribution of arrivals is independent of the time of day or the day of the week.
This is the assumption of stationary interarrival times.

71. Stationary interarrival times is often unrealistic, but we may often approximate reality by breaking the time of day into segments.
A negative interarrival time is impossible. This allows us to write
We define1/λ to be the mean or average interarrival time.

72. We define λ to be the arrival rate, which will have units of arrivals per hour.
An important question is how to choose A to reflect reality and still be computationally tractable.
The most common choice for A is the exponential distribution.
An exponential distribution with parameter λ has a density a(t) = λe-λt.
We can show that the average or mean interarrival time is given by

73. Using the fact that var A = E(A2) – E(A)2, we can show that
Lemma 1: If A has an exponential distribution, then for all nonnegative values of t and h,

74. A density function that satisfies the equation is said to have the no-memory property.
The no-memory property of the exponential distribution is important because it implies that if we want to know the probability distribution of the time until the next arrival, then it does not matter how long it has been since the last arrival.

75. Relations between Poisson Distribution and Exponential Distribution
If interarrival times are exponential, the probability distribution of the number of arrivals occurring in any time interval of length t is given by the following important theorem.
Theorem 1: Interarrival times are exponential with parameter λ if and only if the number of arrivals to occur in an interval of length t follows the Poisson distribution with parameter λt.

76. A discrete random variable N has a Poisson distribution with parameter λ if, for n=0,1,2,…,
What assumptions are required for interarrival times to be exponential? Consider the following two assumptions:
Arrivals defined on nonoverlapping time intervals are independent.
For small Δt, the probability of one arrival occurring between times t and t +Δt is λΔt+o(Δt) refers to any quantity satisfying

77. Theorem 2: If assumption 1 and 2 hold, then Nt follows a Poisson distribution with parameter λt, and interarrival times are exponential with parameter λ; that is, a(t) = λe-λt.
Theorem 2 states that if the arrival rate is stationary, if bulk arrives cannot occur, and if past arrivals do not affect future arrivals, then interarrival times will follow an exponential distribution with parameter λ, and the number of arrivals in any interval of length t is Poisson with parameter λt.

78. The Erlang Distribution
If interarrival times do not appear to be exponential they are often modeled by an Erlang distribution.
An Erlang distribution is a continuous random variable (call it T) whose density function f(t) is specified by two parameters: a rate parameter R and a shape parameter k (k must be a positive integer).
Given values of R and k, the Erlang density has the following probability density function:

79. Using integration by parts, we can show that if T is an Erlang distribution with rate parameter R and shape parameter k,
then
The Erlang can be viewed as the sum of independent and identically distributed exponential random variable with rate 1/

80. Using EXCEL to Compute Poisson and Exponential Probabilities
EXCEL contains functions that facilitate the computation of probabilities concerning the Poisson and Exponential random variable.
The syntax of the Poisson EXCEL function is as follows:
=POISSON(x,Mean,True) gives probability that a Poisson random variable with mean = Mean is less than or equal to x.
=POISSON(x,Mean,False) gives probability that a Poisson random variable with mean =Mean is equal to x.

81. The syntax of the EXCEL EXPONDIST function is as follows:
=EXPONDIST(x,Lambda,TRUE) gives the probability that an exponential random variable with parameter Lambda assumes a value less than or equal to x.
=EXPONDIST(x,Lambda,FALSE) gives the probability that an exponential random variable with parameter Lambda assumes a value less than or equal to x.

82. Modeling the Service Process
We assume that the service times of different customers are independent random variables and that each customer’s service time is governed by a random variable S having a density function s(t).
We let 1/µ be the mean service time for a customer.
The variable 1/µ will have units of hours per customer, so µ has units of customers per hour. For this reason, we call µ the service rate.
Unfortunately, actual service times may not be consistent with the no-memory property.

83. For this reason, we often assume that s(t) is an Erlang distribution with shape parameters k and rate parameter kµ.
In certain situations, interarrival or service times may be modeled as having zero variance; in this case, interarrival or service times are considered to be deterministic.
For example, if interarrival times are deterministic, then each interarrival time will be exactly 1/λ, and if service times are deterministic, each customer’s service time is exactly 1/µ.

84. The Kendall-Lee Notation for Queuing Systems
Standard notation used to describe many queuing systems.
The notation is used to describe a queuing system in which all arrivals wait in a single line until one of s identical parallel servers is free. Then the first customer in line enters service, and so on.
To describe such a queuing system, Kendall devised the following notation.
Each queuing system is described by six characters: 1/2/3/4/5/6

85. The first characteristic specifies the nature of the arrival process
The first characteristic specifies the nature of the arrival process. The following standard abbreviations are used:
M = Interarrival times are independent, identically distributed (iid) and exponentially distributed
D = Interarrival times are iid and deterministic
Ek = Interarrival times are iid Erlangs with shape parameter k.
GI = Interarrival times are iid and governed by some general distribution

86. The second characteristic specifies the nature of the service times:
M = Service times are iid and exponentially distributed
D = Service times are iid and deterministic
Ek = Service times are iid Erlangs with shape parameter k.
G = Service times are iid and governed by some general distribution

87. The third characteristic is the number of parallel servers.
The fourth characteristic describes the queue discipline:
FCFS = First come, first served
LCFS = Last come, first served
SIRO = Service in random order
GD = General queue discipline
The fifth characteristic specifies the maximum allowable number of customers in the system.
The sixth characteristic gives the size of the population from which customers are drawn.

88. In many important models 4/5/6 is GD/∞/∞
In many important models 4/5/6 is GD/∞/∞. If this is the case, then 4/5/6 is often omitted.
M/E2/8/FCFS/10/∞ might represent a health clinic with 8 doctors, exponential interarrival times, two-phase Erlang service times, a FCFS queue discipline, and a total capacity of 10 patients.

89. END