Control for Stochastic Models via Diffusion Approximations Amy Ward, ANS Lecture Series 2008 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA A A A A A A 1.Control of High Volume Assemble-to-Order Systems Queue-length control via tracking policies 2.Control of a Many-Server Queueing Model Workload division

Fair Dynamic Routing in Large-Scale Heterogeneous-Server Systems Amy Ward Joint work with Mor Armony TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA A A A A A A ANS Lecture II 2008

The Inverted-V Model NKNK KK Calls arrive at rate (Poisson process). K server pools. Service times in pool k are exponential with rate  k N1N1 11 Experienced employees on average process requests faster than new hires. Gans and Shen (2007) …

The Problem Routing: When an incoming call arrives to an empty queue, which agent pool should take the call? The objective is to minimize steady-state wait time. NKNK KK N1N1 11 … No. The Slow-server Problem. 2 servers. It is sometimes necessary to keep customers waiting even when the slower server is idle in order not to starve the faster server. Threshold control is optimal. Lin and Kumar (1984). Is an exact analysis possible? General multi-heterogeneous server case is still open. (Vericourt and Zhou, 2006)

The Problem Routing: When an incoming call arrives to an empty queue, which agent pool should take the call? Armony (2005) shows routing to the fastest server first (FSF) asymptotically minimizes the steady- state wait time. BUT … asymptotically only the slow servers have any idle time. Is this fair? Do we care? NKNK KK N1N1 11 …

Fairness Call centers care! Employee burnout and turnover. Increased employee turnover leads to worse performance (Whitt 2006). Call Centers address fairness by routing to the server that has idled the longest (LISF). How does LISF perform? Do any other fair policies perform better? NKNK KK N1N1 11 …

The Fairness Problem Minimize E[Waiting Time] Subject to: E[# of idle servers of pool k] = f k E[Total # of idle servers] *All in steady-state NKNK KK N1N1 11 …

How to determine f? 2 classes. Expected waiting time is decreasing in the pool 1 idleness proportion f 1. So we would like to choose high f 1. Should we ensure all servers have the same utilization? Expected utilization of a pool k server Any fairness criterion that involves individual server utilization translates into a choice for f 1 The same effective processing rate for all servers:

The Fairness Problem Minimize E[Waiting Time] Subject to: E[# of idle servers of pool k] = f k E[Total # of idle servers] Solution Approach: 1.Solve approximating diffusion control problem. 2. Translate solution to original system. NKNK KK N1N1 11 …

Literature Review Conventional Heavy Traffic, Parallel Server Systems –Harrison (1998), Bell and Williams (2001) (2005) The Limit Regime –Halfin and Whitt (1981) The Inverted V Model –Armony (2005), Tezcan (2006), Atar (2007) Gurvich and Whitt (2007) Fairness literature in EE and CS –Deals with fairness towards flows/customers –Avi-Itzhak et al (2006), Weirman (2007) Fairness literature in human resources –Deals with the effect fairness on employee performancs –Cohen-Charash et al (2001), Colquitt et al (2001)

The Asymptotic Regime (under the assumption of work conservation) NKNK KK N1N1 11 …

Fairness Call centers care! Employee burnout and turnover. Increased employee turnover leads to worse performance (Whitt 2006). Call Centers address fairness by routing to the server that has idled the longest (LISF). How does LISF perform? Do any other fair policies perform better? NKNK KK N1N1 11 …

The Longest-Weighted-Idle Server First (LWISF) Policy LISF might not obtain the desired idleness constraint f k (LISF)= To fix this, we propose LWISF: LWISF routes to pool k if w k i k > w m i m, where i k is the idle time of the server that has been idle the longest in pool k f k = Proposition: LWISF asymptotically satisfies the idleness constraint BUT: Does it minimize E[Waiting time]? Can we do better?

The Asymptotic Behavior of LWISF LWISF is asymptotically equivalent to a preemptive policy that, at all times, balances the workload between server pools by requiring the fraction of idle servers in pool k is f k, where f 1 +f 2 + +f K =1. For the preemptive policy The proposition proof is due to Stone’s theorem. The asymptotic equivalence is due to Atar (2007). X has infinitesimal mean X has infinitesimal variance 2 .

The Asymptotic Equivalence The diffusion limit is identical. There is state-space collapse. Note that under the preemptive policy, there is state-space collapse for each. The state is the number in system. The LWISF policy maintains fixed ratios between the number of idle servers in each pool.

Fairness Call centers care! Employee burnout and turnover. Increased employee turnover leads to worse performance (Whitt 2006). Call Centers address fairness by routing to the server that has idled the longest (LISF). How does LISF perform? Do any other fair policies perform better? NKNK KK N1N1 11 …

The Fairness Problem Minimize E[Waiting Time] Subject to: E[# of idle servers of pool k] = f k E[Total # of idle servers] *All in steady-state NKNK KK N1N1 11 …

The Diffusion Control Problem Can you guess the solution?

Threshold Control

Solving the Diffusion Control Problem Step 1: Observe that Step 2: Hence an equivalent DCP is Step 3: We can now formulate the Lagrangian and solve.

Solving the Diffusion Control Problem Cont. What are the correct penalty parameters? The ones under which

Browne and Whitt (1995). How do we find the threshold levels?

Policy Translation: (2 classes) Use FSF. No servers idle. Number in system is x. Use SSF. Threshold level:

Use FSF excluding pool K (the fastest). Use FSF excluding pool K-1. Use FSF. No servers idle. Number in system is x. Policy Translation: (K classes)

Asymptotic Optimality (a.o.) Asymptotic Feasibility: Asymptotic Optimality: 1. ¼ is asymptotically feasible, and 2.If ¼ ’ is asymptotically feasible then Conjecture 1: The preemptive Threshold Policy is a.o. (ext. of Stone’s theorem; Atar, Budhjiraja and Ramanan (2007)) But what about the non-preemptive threshold policy? Showing a.o. would require a non-continuous form of s.s.c..

² -Asymptotic Optimality ( ² -a.o.) ² -Asymptotic Optimality: 1. ¼ is asymptotically feasible, and 2.If ¼ ’ is asymptotically feasible then The ² –Threshold Policy X Death rate slope ¹ 2 slope ¹ 1 L N

² -Threshold Policy: Construction 1. Construct upper and lower bound diffusion processes. 2.Construct a process whose drift is a convex combination of and 3. Choose  such that 4. The diffusion is  -optimal because 5. Hence for any  >0, there exists  (  ) such that

Policy Translation:  -Threshold Policy Use FSF. No servers idle. Number in system is x. Use SSF. Threshold level: Depends Adjustment:

Thm: The ² -Threshold Policy is ² -a.o. Proof: 1. Diffusion construction 2. Weak convergence (Stone) 3. State-space collapse (G&W) and Tightness and UI 4. Asymptotic lower bound (DCP solution) 5. Little’s Law

Asymptotic Performance (Predicted)  1 = 1,  2 = 2,  = 1,  = 1.5,  2 = 2  = 3 f(L) f1f1

Asymptotic Performance (Predicted)  1 = 1,  2 = 2,  = 1,  = 1.5,  2 = 2  = 3 f(L)

Asymptotic Performance (Simulation)  1 = 1,  2 = 2,  = 1,  = 1.5,  2 = 2  = 3, N 1 =300, N 2 =200, ¸ =674

Accuracy of Idleness Constraint  1 = 1,  2 = 2,  = 1,  = 1.5,  2 = 2  = 3, N 1 =300, N 2 =200, ¸ =674

Summary Formulation of the server fairness problem. Solution of the approximating diffusion control problem. Construction of threshold policy for the original system. ² -Threshold Policy ( ² TP) is ² –asymptotically optimal. TH outperforms LWISF.

Further Research Non-ldling assumption Multi-skill environment Server Compensation Schemes Acknowledgement: Rami Atar, Itay Gurvich, Tolga Tezcan & Assaf Zeevi