Exploring the Potential of CMP Core Count Management on Data Center Energy Savings Ozlem Bilgir * Margaret Martonosi * Qiang Wu * Princeton University Facebook, Inc.

Low Utilization Example: 1-week CPU utilization for Facebook servers CPU utilizations stay below 40% most of the time High Idle Power Idle power can be as much as 60% of max power Waste of power Data Center Inefficiencies CPU Utilization (%)

Previous Work on Data Center Power Management Server Consolidation [Pinheiro et al. 2003, Chen et al. 2008, etc.] Dynamic Voltage Frequency Scaling [Chen et al. 2005, Bertini et al. 2010, etc.] Power-gating in CMPs [Leverich et al. 2009, Madan et al. 2010, Kumar et al. 2009] Server reboots are not very desirable because of o Frequent code pushes, importance of robustness, high boot latencies, etc. DVFS leverage is decreasing because o Operating voltages are decreasing, prominence of leakage power Reducing core power with power-gating will be used in this work Abstraction layer between latency and core count

Our Envisioned System Front End Device N 1 N 2 N 3 N 4 Server 1Server 2Server 3Server 4 Assume 1 core in each server is ON tata Service times vary with exponential distribution tdtd Latency = t d – t a Latency = Queuing Time + Service Time Set of Research Questions: How many cores? Which cores? How are answers affected by boot time, bursts, etc.?

Our Approach: Multi-Server CMP Core Count Management Consolidate load to only a subset of cores in multi- server CMP systems Put the other cores in low power state Decide which cores to keep ON at a global level Constraint: Latency requirement is satisfied

Outline Motivation System and Research Overview Decision of ON Core Count Core Count Management Alternatives Performance and Power Models Methodology Results Conclusion

Decision of ON Core Count Look-up table based ON core count decision maker Gives total necessary number of ON cores for a given latency goal and an observed load rate ON core count of servers change at every period, T Created beforehand by observing obtained latency at different load rate and ON core count Core Count % Latency Goal 150msec 0% 4% 9% … 86% 200msec 1% 7% 13% … 97% 250msec 3% 9% 14% … 100% 10%

Core Count Management Alternatives Round-Robin Scheme Same Server Scheme Chip Turn On/Off Scheme Resource contention effect is low - Opportunity to dedicate empty servers to other applications Better performance under bursts

Power and Performance Models Power Performance Per-core service time is affected by contention C ratio is contention ratio

Outline Motivation System and Research Overview Decision of ON Core Count Core Count Management Alternatives Performance and Power Models Methodology Workload Parameters Results Conclusion

Real data from Facebook Stochastic Data Exponential dist. with stable means at 5%, 40% and 85% Workload Time

All Parameters ExplanationValue Server Count4 Core Count per Server4 Base Service Time100msec Service Time DistributionExponential Distribution Inter-arrival Time DistributionExponential Distribution Control Period10min Contention Ratio0%, 15% Idle Core Power40% 75% Latency Goal250msec, 150msec

Results: Effect of Load Rate on Energy Energy savings are greater at low load rates For 5% load, 80% savings For Facebook load, 35% savings At high load rates, most cores are ON, hence less savings SS and RR behave same because C ratio =0

Results: Effect of Load Rate on Latency All of them satisfy the latency goal Obtained latencies are much better than the latency goal With fewer cores, obtained latency would exceed latency goal

Results: Effect of Contention on Energy Energy consumption is affected by contention more in SS Increase in Same-Server is 30% Increase in Round-Robin is13%

Results: Effect of Latency Goal on Energy Tighter latency goal -> More cores ON At 150msec, SS energy consumption increase by 10% CTO energy consumption increases by 6%

Conclusion Using stochastic simulation and real workload, a range of core count management issues in CMPs explored Our CMP core count control mechanism connects high level information (latency) to low-level power management (core count) 35% core energy can be saved with Facebook workload

Exploring the Potential of CMP Core Count Management on Data Center Energy Savings Ozlem Bilgir * Margaret Martonosi * Qiang Wu * Princeton University Facebook, Inc.