Energy-Aware Scheduling on Heterogeneous Processors Rhonda Righter, IEOR, UC Berkeley Joint work with Osman Akgun, IEOR, UC Berkeley Doug Down, McMaster University

Motivation Server farms and cloud computing networks Energy consumption and greenhouse gas emission have become major concerns Servers vary in their speeds and energy consumption How do we schedule our work to balance energy consumption and congestion?

Model Identical jobs with holding cost rate h m servers, exponential service times with service rate μi and server usage cost βi Decision: Assign a job to an available server or leave it idle to minimize total costs With/without reassignment (can jobs move?) With/without job arrivals (clearing system) Two servers/more than two servers

Prior Work – The slow server problem βi ≡ 0 – servers differ only in μi Weiss and Pinedo (1980), Agrawala et al. (1984), Lin and Kumar (1984), Walrand (1984), Kumar and Walrand (1986), Coffman et al. (1987), Viniotis and Ephremides (1988), Rosberg and Makowski (1990), Stockbridge (1991), Weber (1993), Xu (1994), Koole (1995), Luh and Viniotis (2002), de Vericourt and Zhou (2006)

Prior Work – The slow server problem Clearing system (no arrivals) with reassignment (jobs can move) Always use min{m,n} fastest servers (n = # jobs, m = # servers) Clearing system without reassignment Threshold policy is optimal: Use fastest available server j if n ≥ tj

Prior Work – The slow server problem Clearing system with reassignment Always use min{m,n} fastest servers Clearing system without reassignment Threshold policy: Use fastest j if n ≥ tj Poisson arrivals with reassignment Poisson arrivals without reassignment Two servers: Threshold policy is optimal More than 2 servers: Open

Our Results, with Usage Costs Clearing system with reassignment Threshold policy, but threshold and server preference order difficult to determine Clearing system without reassignment Threshold policy, server index = (h+βj)/μ j Poisson arrivals, rate λ, with reassignment Threshold policy, preference order depends on λ Poisson arrivals without reassignment 2 servers: Thresholds, λ-dependent preference More than 2 servers: Open

Our Results, with Usage Costs Clearing system with reassignment Threshold policy, but threshold and server preference order difficult to determine Clearing system without reassignment Threshold policy, server index = (h+βj)/μ j Poisson arrivals, rate λ, with reassignment Threshold policy, preference order depends on λ Poisson arrivals without reassignment 2 servers: Thresholds, λ-dependent preference More than 2 servers: Open

II. Clearing System, No Reassignment n identical jobs with holding cost rate h m servers with rate μj and usage cost βj Jobs cannot be removed from processors Theorem: Order servers in increasing order of There exist thresholds, 1 = t1 ≤ t2 ≤ … ≤ tm, such that it is optimal to assign a job to server j if – j is the lowest indexed available server and – n ≥ tj

II. Clearing, No Reassignment – Computation Theorem Details: Order servers so is increasing in j. Let There exist thresholds, 1 = t1 ≤ t2 ≤ … ≤ tm, such that it is optimal to assign a job to the lowest indexed server j if n ≥ tj where t1 = 1 and regardless of the states of less preferred (higher indexed) servers.

Proof Outline (for all cases I-IV) Define priorities for an individually optimal (IO) policy Characterize the IO policy That is, show the IO policy has the given threshold structure Show the IO policy is socially optimal (SO)

Proof Outline (for all cases I-IV) Define priorities for an individually optimal (IO) policy Assign arbitrary priorities to jobs in queue, available servers are offered to jobs in priority order Job i is offered servers rejected by jobs 1,…,i-1 Job i decides to accept or reject the server to minimize its own total expected holding and usage cost Characterize the IO policy Show the IO policy is socially optimal (SO)

2. Characterize IO Policy Lemma: If there is only one server j, and the server is idle, the expected cost for job i (the i’th job in the queue) is If the server is busy, it is Corollary 1: (when i=1) Server j is preferred to server j+1 under the IO policy if

2. Characterize IO Policy Corollary 2: Under the IO policy with m servers, - Server order: increasing order of - Threshold policy: Job i will accept the lowest indexed available server if it is offered to it and if where vi(j-1) is the expected cost for job i if it waits for one of the servers 1,…,j-1

2. IO policy – Computation Corollary 3: That is, given job i is willing to accept any of servers 1,…,j and not servers >j and servers 1,…, j are all busy, the expected cost for job i is the same as if it is the i+j’th job and there is a single super server with processing rate Mj and usage cost Bj. This cost is independent of the policies of the other jobs (except that they will not use servers >j)

2. IO policy – Computation Corollary 3: Corollary 3 and Corollary 2, which was that under IO, job i will accept the lowest indexed available server, j, if j is offered to i and if give us…

2. Thresholds for the IO policy Corollary 4: The IO policy is a threshold policy: If j is the lowest indexed available server then job tj is assigned to server j if n ≥ tj, where t1 = 1 (the most preferred server is always used), and for j = 2,…,m,

2. IO policy – “No recall” Corollary 5: Under the IO policy, rejected servers will never be used Under the IO policy, lower priority jobs have no effect on higher priority jobs  Job n has no externality

Proof Outline (SO is a threshold policy) Define priorities for an individually optimal (IO) policy ■ Show the IO policy is a threshold policy (we just did this) ■ Show the IO policy is SO (socially optimal)…

3. IO = SO – Proof Idea Backwards Induction: Deviating from IO for the first time step (with the lowest priority job, job n) and then following IO hurts the lowest priority job, job n, and has no effect on the other jobs because job n has no externality  Such a deviation cannot be SO

I. Clearing System With Reassignment Can reassign jobs to new servers at any time Now for the IO policy all jobs are labeled, and job i has preemptive priority over job i+1, and any job can leave a server at any time That is, at any time all servers are offered to all jobs in order according to their priorities IO is a threshold policy and IO = SO Both easy; again job n imposes no externality

I. Clearing System With Reassignment Can reassign jobs to new servers at any time IO is a threshold policy and IO = SO Determining server preference (and thresholds) is difficult No longer (h+βj)/μ j Server preference and thresholds are interdependent, and depend on all servers Can generate preferences and thresholds recursively, using a “super server” result

I. Clearing, Reassignment: Computation Let Π(j) be the j’th preferred server, i.e., the chosen server under the IO policy, if servers Π(1),…,Π(j-1) all busy, the other servers all idle, and given some server is chosen Let Πj = {Π(1),…,Π(j)}

I. Clearing, Reassignment: Computation Let Π(j) be the j’th preferred server Let Πj = {Π(1),…,Π(j)} Recursively compute

I vs II (With/Without Reassignment) If servers can be ordered so that and (faster = cheaper = smaller (h+β)/μ) then the thresholds can be related as follows: ti(no reassigment) = ti(reassignment) + ti(no reassigment, β=0) when ti(reassignment) > j

III. Arrivals and Reassignment Theorem: With reassignment and Poisson arrivals, rate λ: IO = SO, a threshold policy For IO policy: LCFP-P (Preemptive LCFP) Higher priority (e.g., arriving) jobs can preempt jobs in service and can move to other servers  lower priority jobs have no effect on higher priority jobs, i.e., Lowest priority job has no externality** IO policy is a threshold policy – easy IO = SO – same proof idea, using **

III. Arrivals and Reassignment General structure (threshold policy) easy Computation of the server preference order and thresholds is even more difficult So suppose only 2 servers…

III. Arrivals, Reassignment, 2 Servers If and then server 1 is always preferred If and then there exists λ0 such that server 1 is preferred iff λ < λ 0 The preferred server is always used The threshold for the other server is easily computed

III. Arrivals and Reassignment We use the IO LCFS-P policy to determine the thresholds, but For the SO policy, when a job is assigned, it can be any job (need not be LCFS-P) For example, FCFS with given tj’s is SO Or we could assign jobs to minimize moves: – Once a job is assigned to server j (when n ≥ tj) remove it from server j only if n drops below tj before the job finishes service

IV. Arrivals, No Reassignment, 2 Servers Theorem: Without reassignment and two servers, IO = SO, a threshold policy: The preferred server is always used Server 2 is used when n ≥ t Where the IO policy assumes LCFP-P Higher priority jobs in queue can preempt lower priority jobs in service, but cannot leave servers From a social perspective, jobs preempting other jobs have no effect (job swap) (indistinguishable jobs, exponential services)

IV. Arrivals, No Reassignment, 2 Servers Theorem: Without reassignment and two servers, IO = SO, a threshold policy: The preferred server is always used Server 2 is used when n ≥ t Where the IO policy assumes LCFP-P IO policy is a threshold policy – easy IO = SO Same idea: Deviating from IO for the first decision with job n hurts job n and has no effect on others

IV. Arrivals, No Reassignment, 2 Servers Theorem: Without reassignment and two servers, IO = SO, a threshold policy: The preferred server is always used Server 2 is used when n ≥ t Again, the SO policy can be implemented with arbitrary job assignments, given the threshold E.g., FCFS, without preemption

IV. Arrivals, No Reassignment, 2 Servers If and then server 1 is always preferred If and then there exists λ0 such that server 1 is preferred if λ < λ 0 In contrast to the reassignment case, we were unable to show that If and then server 2 is preferred if λ ≥ λ 0

IV. Arrivals, No Reassignment, >2 Servers Theorem: Without reassignment and more than two servers, IO ≠ SO Even with preemptive LCFP for the IO policy, The decision of whether or not to assign the lowest priority job in queue to a server can impose an externality on jobs with even lower priorities on less preferred servers The IO policy is still a threshold policy, but the threshold for a server depends on the relative priorities of jobs on less preferred servers

Summary Theorem: The IO policy is a threshold policy, and IO = SO, except for the case: Arrivals, no reassignment, and > 2 servers Determination of the server preference order and thresholds are complicated, but they can be obtained by considering the IO policy Showing that the IO policy is a threshold policy and that IO = SO gives an easy way to show the SO policy is a threshold policy