1. Queuing Theory
Simulation & Modeling

2. The Queuing Model Use Queuing models to
Queue
Server
Queuing System
Use Queuing models to
Describe the behavior of queuing systems
Evaluate system performance

3. Applications of Queuing Theory
Telecommunications
Computer Networks
Predicting computer performance
Health services (eg. control of hospital bed assignments)
Airport traffic, airline ticket sales
Layout of manufacturing systems.

4. Example application of queuing theory
In many stores and banks, we can find:
multiple line/multiple checkout system → a queuing system where customers wait for the next available cashier
We can prove using queuing theory that : throughput improves/increases when queues are used instead of separate lines

5. The Queuing Model A Queue System is characterized by
Server
Queuing System
A Queue System is characterized by
Queue (Buffer): with a finite or infinite size
The state of the system is described by the Queue Size
Server: with a given processing speed
Events: Arrival (birth) or Departure (death) with given rates

6. Example application of queuing theory

7. Open Queueing Networks
Open queueing network: external arrivals and departures
Number of jobs in the system varies with time
Throughput = arrival rate
Goal: To characterize the distribution of number of jobs in the system

8. Closed Queueing Networks
Closed queueing network: No external arrivals or departures
Total number of jobs in the system is constant
“OUT” is connected back to “IN”
Throughput = flow of jobs in the OUT-to-IN link
Number of jobs is given, determine the throughput

9. Queuing theory for studying networks
View network as collections of queues
FIFO data-structures
Queuing theory provides probabilistic analysis of these queues
Examples:
Average length
Average waiting time
Probability queue is at a certain length
Probability a packet will be lost

10. Basic Components of a Queue
2. Service time
distribution
1. Arrival
process
6. Service
discipline
5. Customer Population
4. Waiting
positions
3. Number of
servers
Example: students at a typical computer terminal room with a number of terminals. If all terminals are busy, the arriving students wait in a queue.

11. Characteristics of queuing systems
Arrival Process
The distribution that determines how the tasks arrives in the system.
Service Process
The distribution that determines the task processing time
Number of Servers
Total number of servers available to process the tasks
Storage capacity: are buffer finite or infinite?

12. Specification of Queueing Systems
Operating policies
Customer class differentiation
are all customers treated the same or do some have priority over others?
Scheduling/Queueing policies
which customer is served next
Admission policies
which/when customers are admitted

13. Service Disciplines First-Come-First-Served (FCFS)
Last-Come-First-Served (LCFS)
Last-Come-First-Served with Preempt and Resume (LCFS-PR)
Round-Robin (RR) with a fixed quantum.
Small Quantum  Processor Sharing (PS)
Infinite Server: (IS) = fixed delay
Shortest Processing Time first (SPT)
Shortest Remaining Processing Time first (SRPT)
Shortest Expected Processing Time first (SEPT)
Shortest Expected Remaining Processing Time first (SERPT).
Biggest-In-First-Served (BIFS)
Loudest-Voice-First-Served (LVFS)

14. 20 Buffers = 3 service + 17 waiting
Example
Three servers
20 Buffers = 3 service + 17 waiting
After 20, all arriving jobs are lost
Total of 1500 jobs that can be serviced
Service discipline is first-come-first-served

15. The Queuing Times
Queue
Server
Queuing System
Queuing Time
Service Time
Response Time (or Delay)

16. The Little’s Law
The long-term average number of customers in a stable system N, is equal to the long-term average arrival rate, λ, multiplied by the long-term average time a customer spends in the system, T.

17. Little's Law
Mean number in the system = Arrival rate  Mean response time
This relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing service
Based on a black-box view of the system:
In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions
Black Box
Arrivals
Departures

18. Little’s Law
a (t): the process that counts the number of arrivals up to time t.
d (t): the process that counts the number of departures up to time t.
N(t)= a(t)- d(t)
a(t)
d(t)
# Arrivals
Area J(t)
N(t)
Time t
Average arrival rate (up to t) λt= a(t)/t
Average time each customer spends in the system (Response Time) Tt= J(t)/a(t)
Average number in the system Nt= J(t)/t

19. Rules for All Queues (cont’d)
Number versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate  Mean response time
Similarly, Mean number of jobs in the queue = Arrival rate  Mean waiting time
This is known as Little's law.
Time in System versus Time in Queue r = w + s r, w, and s are random variables
E[r] = E[w] + E[s]

20. Example
The average queue length in the computer system was observed to be: 8.88, 3.19, and 1.40 jobs at the CPU, disk A, and disk B, respectively. What were the response times of these devices?
the device throughputs were determined to be:
The new information given in this example is:
Using Little's law, the device response times are:

21. Operational Quantities
Black Box
Quantities that can be directly measured during a finite observation period
T = Observation interval Ai = number of arrivals
Ci = number of completions Bi = busy time Bi

22. Utilization Law This is one of the operational laws
Operational laws are similar to the elementary laws of motion For example,
Notice that distance d, acceleration a, and time t are operational quantities. No need to consider them as expected values of random variables or to assume a distribution

23. Example
Consider a network gateway at which the packets arrive at a rate of 125 packets per second and the gateway takes an average of two milliseconds to forward them
Throughput Xi = Exit rate = Arrival rate = 125 packets/second
Service time Si = second
Utilization Ui= Xi Si = 125  = 0.25 = 25%
This result is valid for any arrival or service process. Even if inter-arrival times and service times to are not IID random variables with exponential distribution

24. Forced Flow Law
Relates the system throughput to individual device throughputs
In an open model, System throughput = # of jobs leaving the system per unit time
In a closed model, System throughput = # of jobs traversing OUT to IN link per unit time
If observation period T is such that Ai = Ci  Device satisfies the assumption of job flow balance
Each job makes Vi requests for i-th device in the system
If the job flow is balanced and C0 is # of jobs traversing the outside link => Ci – # of jobs visiting the i-th device: Ci = C0 Vi or Vi =Ci/C0 Vi is called visit ratio

25. Forced Flow Law (cont’d)
System throughput:

26. Forced Flow Law (cont’d)
Throughput of ith device:
In other words:
This is the forced flow law

27. Example
In a timesharing system, accounting log data produced the following profile for user programs
Each program requires five seconds of CPU time, makes 80 I/O requests to the disk A and 100 I/O requests to disk B
Average think-time of the users was 18 seconds
From the device specifications, it was determined that disk A takes 50 milliseconds to satisfy an I/O request and the disk B takes 30 milliseconds per request
With 17 active terminals, disk A throughput was observed to be I/O requests per second
We want to find the system throughput and device utilizations

28. Example (cont’d)
Since the jobs must visit the CPU before going to the disks or terminals, the CPU visit ratio is:

29. Example (cont’d) Using the forced flow law, the throughputs are:
Using the utilization law, the device utilizations are:

30. Example
A monitor on a disk server showed that the average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server?
Using Little's law:
Mean number in the disk server
= Arrival rate  Response time
= 100 (requests/second) (0.1 seconds)
= 10 requests