1. Job Fairness in Queue Scheduling:
Hanoch Levy
School of Computer Science
Tel Aviv University
Alternative name: “Satellite-linked Web Caches”
May 2005
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

2. To provide FAIRNESS in waiting/service
Why (ordered) Queues?
Why Fairness in Queues?
“Not Fair!!!”
“This is more Fair…”
To provide FAIRNESS in waiting/service
Queue = “A Fairness Management Facility”
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

3. Why Fairness in Queues? (2)
Fairness inherent/crucial part of queues
Recent studies, Rafaeli et. al. [2003] (experimental psychology):
Experiments on humans in queue scenarios
Fairness in queue is very important to people
Perhaps even more than delay itself!
The issue
Evaluate (metrics) Queue Fairness
Use on: System + scheduling policies
Aim: Applications
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

4. The Size vs. Seniority Dilemma
Mr. Short vs. Mrs. Long
Is it more fair to serve Short ahead of Long? By how much?
Mr. Short
Mrs. Long
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

5. Fairness of Job Scheduling in Queueing Systems, H. Levy
Outline
Queue Model
Job-Based systems, Flow-Based systems + applications
Prior work
“Requirements”
The performance issue: Delay vs. Service
The granularity level: How fine…
Dealing with stochastics
The physical entities: seniority, service requirement, resources
The Fairness Measures - Overview + properties:
Seniority based fairness
Service time based fairness
Resource allocation based fairness
Mixture based
Application perspective
References
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

6. Queue Model (single server)
Jobs
Service time
Arrivals
Departures
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

7. History: Queueing Theory and Fairness
Queueing theory: Decades of research
Delay of individual
Practical Applications: many & diverse
Fairness in queues:
Many importance statements:
Importance of fairness: Larson (1988), Palm (1953), Mann (1969), Whitt (1984), Rothkopf & Rech (1987)
Analysis: was Little, now growing(job fairness):
Morris & Wang (85)
Gordon (87)
Avi-Itzhak & Levy (96)
Bender, Chakrabarti . Muthukrishnan (98), Wierman & Harchol-Balter (03), Harchol-Balter et. al. (03)
Avi-Itzhak, Levy & Raz (05), Raz, Avi-Itzhak & Levy (04, 05a,b), Raz, Levy& Avi-Itzhak (04), Brosh, Levy & Avi-Itzhak (05)
Sandmann (05) (more is coming)
Exception: Flow Fairness
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

8. History: Queueing Theory and Fairness (2)
 We know only little about queue (job) fairness!
 More complex than measuring individual delay!
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

9. Designing a Metrics - Keep in mind:
To be used by:
Researchers
Designers / operators
Customers (appeal to)
Desired properties (“requirements”)
Fits application(s) [as many as possible]
Based on a “sound” intuitive basis
Fit widely accepted intuition in simple cases
Yields to analysis
Call center
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

10. The performance issues: Delay vs. Service
Job delay (waiting time, sojourn time)
Traditional queueing-theory measure
The major factor when service is guaranteed
What a wait…
Job (service) Completion
Have the job done
Less popular in queueing theory
Applies when service is not guaranteed
Ticket line for scarce tickets
My seat is gone…
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

11. Granularity of Fairness Evaluation
At what granularity level, should fairness be evaluated C1 C2 C0
Individual
Scenario
System (steady state)
All are important
Individual, scenario: build confidence (scale of reference) in metrics
System : to evaluate systems/policies
Note: All exist for individual waiting times
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

12. The Physical Factors (queues)
Seniority
(Arrival Time)
Size
(Service Requirement)
Resources
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

13. Size and Seniority preference principles (“requirements”)
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

14. Size and Seniority preference principles (“requirements”)
Seniority principle
Weak: All jobs same service times  if ai< aj then more fair to complete service of Ji before Jj
Strong: Ji and Jj same service times
Service-requirement principle
Weak: All jobs same arrival times  if si< sj then more fair to complete service of Ji before Jj
Strong: Ji and Jj same arrival times
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

15. How Scheduling policies meet the principles (are fair by principle)
FCFS
LCFS
ROS
(random)
Shortest job first
longest job first
Seniority + -
Service requirement
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

16. The Size vs. Seniority Dilemma
Mr. Short vs. Mrs. Long
Is it more fair to serve Short ahead of Long? By how much?
Mr. Short
Mrs. Long
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

17. Review of Measures (job based)
Seniority
(Arrival Time)
Seniority based
Service-requirement based
Size
(Service Requirement)
Resource based
Resources
Mixtures
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

18. Approach 1: Order (seniority) Fairness
(Arrival Time)
Gordon (87) – Ph.D MIT
Deals with skips and slips:
Performance measure of overtaking to quantify the level of “social justice”
B skips A (A experiences a slip): A arrives before B, B leaves before A
Considers several Markovian models
Analyses distribution of #slips, #skips
Measure: No explicit suggestion
Lets try: E[#skips ] - E [ # slips]
Problem: E[#skips ] - E [ # slips]=0
So perhaps: E[#skips ] B A
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

19. Approach 1: Order (seniority) Fairness: results
(Arrival Time)
Systems analyzed :
2 M/M/1 in parallel
2 M/M/1’s with one SJQ “smart” customer
Multi server system: M/M/m
Infinite server system: M/G/Inf
Analysis: distribution of SKIP and SLIP
Results:
AVG(#skips) = AVG (#slips)
Most systems: Dist(#skips) != Dist (#slips)
Smart customer: increases # skips, decreases # slips
M/U/Inf with symmetric service time dist: Dist(#skips) = Dist (#slips) [only system found] B A
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

20. Approach 1: Order (seniority) Fairness: How do I use it?
(Arrival Time)
Compute: Dist(#skips), Dist (#slips)
Analysis or simulation
How to use for fairness measure – open issue. B A
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

21. Measure 2: “service-time” Fairness
Size
(Service Requirement)
Bender, Chakrabarti . Muthukrishnan (1998), Harchol-Balter et. al. (2003)
Wierman & Harchol-Balter (2003):
Propose a Fairness Criterion
Emphasis on service requirement
Slowdown: for job of size x compute E[T(x)/x]
If the slowdown is lower then 1/(1-ρ) for all x - FAIR
Classification of a large variety of policies:
Always fair: fair for all dist. + loads
Always unfair: unfair for all dist. + loads
How my type is doing on average
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

22. Fairness of Job Scheduling in Queueing Systems, H. Levy
Size
(Service Req.)
Compute E[T(x)] (use queueing theory or simulation), divide by x and check rules
You end up with a criterion.
Can create a measure.
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

23. Measure 3: Resource Allocation Fairness (RAQFM)
Resources
Raz, Levy, Avi-Itzhak (04)
Aim at the dilemma between size and seniority
Focus on: fair share of resources
Ideal: At t, each customer deserves 1/N(t) of system resources (N(t)= # customers(t))
Compare warranted service with granted service
Warranted service (customer i)
Granted service:
Individual Discrimination:
System unfairness measure: Summary statistics (var) of
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

24. Resource allocation Fairness: Results
Resources
Adhering to principles:
STRONG Seniority service principles – YES
WEAK Service-time service principle – YES
STRONG Service-time service principle – NOT
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

25. Resource allocation Fairness: Results (2)
Resources
PS most fair
Reacts to both s.time and seniority
FCFS > LCFS (seniority dominant)
FCFS < PLCFS (s. time dominant)
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

26. Resource allocation Fairness: Results (3)
Resources
Multiple server more fair than single server with same combined rate
Single Queue more fair than Multi-Queue
Jockeying increases fairness
Jockeying from head more fair than jockeying from tail
Prioritizing short jobs is more fair in most cases, but not if the jobs are almost as large as the rest of the population
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

27. Resource Allocation Fairness: Applicability
Resources
Good for:
S. times and A. times arbitrary
Issue is: Waiting times
What a wait…
Applications:
Waiting lines where resources guaranteed
Call centers (non-scarce resources)
Web services
Supermarkets
Airport services
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

28. Resource Allocation Fairness: How Do I use it
Resources
Derive variance of discrimination at steady state
Use Queueing Theory + methodology developed
Good for variety of Markovian systems
Large systems need approximations (more research) Waiting lines where resources guaranteed
Use simulation
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

29. What fits? Go by the application
Service critical
Airline reservation, Call center, “Line for bread”
Order fairness
Delay sensitive, seniority-blind (and careless) customers
Computer systems (?)
“service time” fairness
Delay sensitive, variable service time
Call centers, Supermarkets, Banks, Computer systems
Resource allocation fairness
Seniority
(Arrival Time)
Size
(Service Requirement)
Resources
Mixture
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

30. How People Perceive Queue Fairness
Rafaeli et. al. (2005): Queue experiments on humans
FCFS is FAIR (seniority)
Violation of FCFS == unfair
Regardless if close to the person asked or away of it.
Size-issue – not addressed yet.
Multi-server:
Multi-Queue (MQ) less fair than single queue.
One queue in MQ is shorter  less fair.
If people PAY for the short Q (first-class)  Fair.
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

31. Fairness of Job Scheduling in Queueing Systems, H. Levy
Concluding remarks
Fairness in Queues is important
Measures must
Fit applications
Agree with one’s intuition / be consistent
Researcher, designer, customer
Yield to analysis
Research of subject in it’s early stages
Much more to study:
Scheduling policies
Weights
Multiple queues /servers
Complex structures
Relations between measures
Other measures…
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

32. Closing Words: Why Study Fairness in Queues (5 reasons)
The Fundamental: (“Q = fairness”)
Isn’t fairness why we have queues (for human services) in the first place?
The Scientific evidence: (Rafaeli et. al. [2003])
Fairness in queues important to people / perhaps > delay
The Inductive reasoning:
WFQ: 10’s of papers – fairness on O(1) microsecond jobs!!
The Challenge:
We know very little on Queue fairness
The problem is harder than Q delay!
The Practical: Relax your nerves…
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

33. Fairness of Job Scheduling in Queueing Systems, H. Levy
THANK YOU
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

34. Dealing with Stochastics: Actual measures vs. Expected values (2)
Harder to compute
Expected values:
May be misleading
“Like having salaries given in random (with their mean be the real salary)”.
How do they treat me now…
How do they treat short jobs on avg…
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

35. Dealing with Stochastics: The Black Box question
Customer does not see other customers
“Close approach”: don’t see (arrivals) don’t care.
Only size matters: Is my size class doing ok?
“Open approach”: Customer is still interested in Fairness even though he cannot check it directly
How do they treat me now…
How do they treat short jobs on avg…
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

36. Fairness of Job Scheduling in Queueing Systems, H. Levy
Measure 4: Mixture
Mixture
Sandmann (05)
Mixture: seniority and size
Accounts for 2 parameters:
#Overtaking (=#slips) [seniority]
#Large jobs [Size]
X is a large job for C if:
i) Upon C’s arrival X has no-less residual service than C.
ii) X departs before C.
Fairness measure:
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

37. Measure 4 - Mixture : results
Adhering to principles:
STRONG Seniority service principles – YES
STRONG Service-time service principle – YES
Difficulties: Asymmetry in sensitivity to size + seniority
E.g.: X arrives with 100 sec s.time, and waits an hour. Y arrives 1 hour later with 99 sec s.time. It is as fair to serve in both orders.
Same if 10 hours.
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

38. Measure 4 - Mixture : How do I use it
Analysis of skips – see Gordon (87) – Markovian – can do.
Analysis of Large – not provided yet (perhaps in the making).
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

39. Job-Based vs. Flow-Based +Applications (repeat)
Server Q
Jobs
Server Q
Jobs
FLOW BASED
Customer = Flow
Applications:
Networking:network level equipment:
Routers, gateways, load-balancers
JOB BASED
Customer = Job
Applications:
Networking: Application level equipment: Web server, file server
Supermarket, Bank, public office + alike
Call center
Computer system
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

40. A word on flow-based measures
Deal with flows (of packets)
Interested mainly in throughput
Literature:
Fair bandwidth allocation (network)
MinMax fairness (Jaffe (81))
Proportional Fairness (Kelly (97))
Fair Scheduling
Weighted Fair Queueing (WFQ)
Demers, Keshav and Shenker (1990), Greenberg and Madras (1992), Parekh (1992), Parekh and Gallager (1993), (1994), Golestani (1994), Rexford, Greenberg and Bonomi (1996), Bennet and Zhang (1996), others.
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

41. Measure 5: PS proximity WFQ/RFB literature
Scheduling fairness measures
Worst Case deviation from PS (extreme values)
Relative Fairness Bound (Golestani (94))
RFB:
Can we apply it to jobs (problematic):
Not “continuously active”
Try on L, S: assume FCFS (L arrives before): RFB=L
Try on L, S: assume LCFS (L arrives after): RFB=L
General M/G/1 (FCFS, LCFS): infinity!
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

42. Measure 5: PS proximity WFQ/RFB literature
Absolute Fairness Bound (AFB) (Greenberg and Madras (1992) and Keshav (1997))
Maximum (time) discrepancy between schedule and PS
Applying to jobs:
Try PS completion discrepancy of job
LCFS = FCFS = infinity!
Most non-PS based (non-WFQ): infinity! (SJF, LJF, SRPT..)
Good for very precise PS imitations
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

43. Measure 1: Order (seniority) Fairness
(Arrival Time)
Avi-Itzhak & Levy (96, 04)
Axioms (for G/D/1): what happens to unfairness measure when interchanging customers:
P1: Monotonicity in seniority difference of interchanged neighbors
P2: Reversibility of neighbor interchange
P3: Independence on position and time
P4: Fairness change is not affected by customers not interchanged
P4G: interchange of non-neighbors
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

44. Order Fairness: results
Seniority
(Arrival Time)
ai - Arrival time of customer i
Di - Waiting displacement of customer i
C > 0, arbitrary constant
Expected fairness per customer
Claim: Under FCFS:
FCFS most fair (LCFS least)
Thm : Let (W, W*) be the steady state waiting time under (policy, FIFO), then:
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

45. Order Fairness: results (2)
Seniority
(Arrival Time)
Adhering to principles:
Strong seniority service principle: YES
Service-time service principle: NOT
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

46. Order Fairness: Properties & Applicability
Seniority
(Arrival Time)
Good for:
S. times identical
S. times don’t matter
Issue is: Job completion
Applications:
Scarce-ticket lines
Some call-centers
My seat is gone...
FCFS is most fair (LCFS least)
Intuition & concepts:
Peoples’ strong belief in order fairness
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

47. Measure 1: Order (seniority) Fairness: How do I use it?
(Arrival Time)
Compute
Simulation
Compute Var[W] for steady state
Remark: range of variance
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

48. “Service-time” Fairness: Results
Size
(Service Req.)
Classification (not measure) of a large variety of policies
Any preemptive size based policy is always unfair (all loads & all service dists).
All non-size based non-preemptive policies are always unfair for service time dist defined on neighborhood of zero (short jobs discriminated).
Age based policies are always unfair
FCFS is always UNFAIR
LCFS (preemptive) +PS are always FAIR
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

49. “Service-time” Fairness: Results (2)
Size
(Service Req.)
Adhering to principles:
Seniority service principles – NOT
Service-time service principle –Open question
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

50. S. time Fairness: Properties & Applicability
Size
(Service Req.)
Good for:
A. times identical
A. times not known / don’t see the queue
Your size is always the same
Issue is: “wash” seniority by averaging.
Advantage: relatively simple analysis
Applications: Computer systems (?)
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

51. S. time Fairness: How do I use it
Size
(Service Req.)
Compute E[T(x)] (use queueing theory or simulation), divide by x and check rules
You end up with a criterion.
Can create a measure.
We are trying to use SRM to analyze the CSDS
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy

52. Fairness of Job Scheduling in Queueing Systems, H. Levy
11/20/2018
Fairness of Job Scheduling in Queueing Systems, H. Levy