Page 1 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Things I Thought I Knew about Queueing Theory, but was Wrong About (Part 2, Service Queues) Alan Scheller-Wolf Tepper School of Business Carnegie Mellon University Joint work with Ying Xu and Katia Sycara

Page 2 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 A Great Day As an advisor, one of the greatest things that can happen is the following: Student: “I can prove that XXX is true!” Advisor: “That can’t be right; everyone knows YYY is true.” Student: “I know, but I can prove it.” Advisor: “There must be a bug, show me.” Time passes... Advisor: “Huh, you are right, XXX is true!”

Page 3 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 My Goals Take you through “my great day” as an advisor Help you understand why everyone believed YYY Help you understand, mathematically, why XXX is true Help you see why, intuitively, XXX “has to be true” Discuss what general insights this gives

Page 4 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Quality Based Services (I) What are quality-based services? –Services in which longer service times generate greater value –Why greater value? For customer: –Better quality of service, e.g. diagnostic accuracy (e.g. Alizamir, de Vericourt and Sun, 2012; Wang et al. 2007, 2012) For firm: –Revenue generation (e.g. Ren and Zhou, 2008)

Page 5 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Quality Based Services (II) Examples of quality based services: –Health care –Call centers –Personal services –Education / Consulting –Others… OK, so why not just extend service times indefinitely? –Customers Don’t Like to Wait! (Duh!)

Page 6 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 The Central Problem Leisurely Service Short Wait SPEED-QUALITY TRADE-OFF PROVIDE VALUE WITHOUT TOO MUCH WAIT Lovejoy and Sethuraman (2000); task completion with deadlines

Page 7 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Providing Value Without Too Much Wait How? –Simple: Speed up when busy, slow down when not Dynamic (state dependent) control –Crabill (1972,1974) –Stidham and Weber (1989) –George and Harrison (2001) –Ata and Shneorson (2006) –Hopp et al. (2007)

Page 8 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 But What if a Dynamic Policy is Unrealistic? Why? –State information is expensive or difficult to get Heyman (1977), Harchol-Balter et al. (2003) –Service provision agreements may be customer- chosen or pre-sold Then we must use a static (state-independent) policy –This is a lower bound for the optimal dynamic policy performance

Page 9 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Let’s Take a Closer Look at the Static Case Why would you ever want to offer a variable service rate if you cannot use information on the state (congestion)? Maybe we have heterogeneous customers: Some value longer service more, others shorter waits –Mendelson and Whang (1990); M/M/1 pricing –Rao and Petersen (1998); M/M/1 pricing, class choice –Van Mieghem (2000); heavy traffic, Gc  –Gurvich and Whitt (2010); FQR routing for SL’s

Page 10 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 OK, What About Static Service and Homogeneous Customers? I could still offer variable service, but why? How? –I can’t use system information –I can’t take advantage of heterogeneous customer preferences And increasing variability in the system would increase delay –This is basic queueing, right, the PK formula???

Page 11 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 No! I will use a static policy For homogeneous customers I will randomly assign the arriving customers different grades, based on service rate –With probabilities and rates of my choosing And I will perform better than if I provide service of uniform rate How? –Let’s see…

Page 12 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Model Poisson BASE Service  I can assign arriving jobs different service rates      K But I must do so randomly, with probabilities p 1, p 2, …, p K I am in effect scaling the service distribution Each class (or grade) arrives Poisson( p k ) Single Server Queue

Page 13 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Service Discipline Assume non-preemptive policy r –Allowing preemption is, in general, a mess! So I have an M/G/1 queue with service discipline r

Page 14 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Performance Metrics Recall our Speed-Quality trade-off: Service Value increases with mean service time Delay cost increases with mean waiting time Assume both of these are linear for now Leisurely Service Short Wait

Page 15 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Objective Service Value is only a function of  and p Delay Cost is a function of  p, and r V(  p,r) =  u  k - h  k p k w k (  p,r)} pkpk kk

Page 16 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Results We first show that providing mixed service (K>1) is superior to providing pure service (K=1) How? –The key is r –We will take advantage of the variability we inject into the system –We will take advantage of the information we inject into the system with the variability or

Page 17 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 The Optimal Scheduling Rule We have a non-preemptive system Rule must be independent of state Rates are ordered:      …>  K Schedule according to SEPT for ANY  and p

Page 18 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 But Will This Really Work? I can use the heterogeneous rates to schedule short jobs before long ones But it is still hard to believe that it is beneficial to inject variability just so I can use SEPT!

Page 19 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Dominance Result (I) Let’s compare K=1 with K=2 Further, let’s assume What does this mean? p1p1 11 p2p2 22 += 1  Mean service time is the same in both systems

Page 20 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Dominance Result (II) Algebra and basic formulae yield: Can we make this < 1? Yes, if p1p1 11 p2p2 22 += E[w 2 ] E[w 1 ] (  1-  1 ) (1-p 1  ) 11 22 1 (1-  ) < When can we ensure this holds? Always! We pick the  ’s!

Page 21 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Dominance Result (III) But what about the service value? Since Service values are the same… So result holds even if not quality based service! p1p1 11 p2p2 22 += 1 

Page 22 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Dominance Result (IV) What does this condition do? Constrains difference in service rates (or variability to inject) As load increases, I have more freedom… Result extends to comparing k and k+1classes 11 22 1 (1-  ) < It is always beneficial to add another service class

Page 23 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Optimal Policy for a Given K Now assume we are unconstrained Lemma: Under optimal policy, p k w k = p k+1 w k+1 (we balance waiting time costs by class) And So, if we fix K and  all other parameters are determined! p k+1 pkpk  k+1 kk = = () 2  k+1 kk = () 2 (1-  ) 1/K

Page 24 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 How Do We Find Optimal  FOC: What do you notice  does NOT depend on?  (!) u(1-  )+CK{(1-  ) 1/K – (1-  ) -1/K }=0 h  2 E[X 2 ]  C =

Page 25 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 What does this mean? Since the server has the flexibility to set the service rates, it can always recover the optimal  no matter what  it faces. Similarly, if  k p k < 1, system can likewise adjust service rates and recover optimal value

Page 26 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Numerical Results: Benefit of Variability

Page 27 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Numerical Results: Relative Performance of Different Grades Shortest jobs: Most likely; Shortest wait; Smallest utility Longest jobs: Least likely; Longest wait; Largest utility

Page 28 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Extensions (I) We assumed linear service value and delay cost functions Results extend to convex value and linear cost –Convex service value just helps us There exist concave value and linear cost that satisfy –Concave service value works against us Non-linear cost makes waiting cost terms intractable.

Page 29 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Extensions (II) We assumed non-preemptive scheduling –If assume preempt-resume and exponential service, results extend. In general, if non-exponential service, optimal scheduling policy will depend on age of job in service, and could be arbitrarily complex (like TAGS)

Page 30 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Conclusions (I) For quality-based services it is beneficial to introduce variability in service times Even if: –You must make decisions statically –Customers are homogeneous –Customers do not care about quality It is always beneficial to add service grades

Page 31 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Conclusions (II) The optimal policy is insensitive to and whether or not you reject jobs Optimal parameters have a geometric structure –If parameters don’t have a geometric structure, then adjusting them can reduce delay, without adding any more classes (or information). Asymptotic benefit is about 5% at most

Page 32 Alan Scheller-Wolf Lunteren, The Netherlands January 16, 2013 Dank U! Any Questions?

Page 33 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Dank U! Any Questions?