Energy and heat-aware metrics for data centers Jaume Salom, Laura Sisó IREC - Catalonia Institute for Energy Research Ariel Oleksiak, Mateusz Jarus PSNC – Poznan Supercomputing and Networking Center Thomas Zilio IRIT – Institut de Recherche en Informatique de Toulouse 30 September

Introduction CoolEmAll will provide tools for planners and operators of DC to carry out flexible and fast simulations to improve energy efficiency and to reduce the carbon footprint associated Metrics suitable to quantify CoolEmAll improvement in energy efficiency 2 EuroEcoDC Workshop - Karlsruhe

Contents 1.Present status of DC metrics 2.Properties of CoolEmAll metrics 3.New metrics proposed 4.Description of experiment 5.Check Imbalance of Temperature 6.Further steps: other metrics to test 7.Conclusions 3 EuroEcoDC Workshop - Karlsruhe

Present status of DC metrics Metrics related to power for complete DC – PUE – Global KPI Metrics that consider energy reuse, carbon emissions or water use: – ERE,CUE,WUE Metrics to consider the power required in idle conditions – FVER Metrics for IT Components: – Power Usage, resource/Watt 4 EuroEcoDC Workshop - Karlsruhe

Properties of CoolEmAll metrics Focus on Energy not only on peak-power Focus on Temperature not only on Power Heat-aware metrics Focus on Useful Work on Applications not only on IT Consumption Selection of useful and consistent metrics to assess different granularity levels of a DC (CPU, rack, room) Holistic approach 5 EuroEcoDC Workshop - Karlsruhe

Properties of CoolEmAll metrics Granularity: – Node unit – Node group – Rack level – Room of a DC Focus on: – Resource usage – Energy – Heat-aware 6 EuroEcoDC Workshop - Karlsruhe

Metrics at node- group level Node-group cooling index Referred to the air inlet temperatures Recommended and allowed values by ASHRAE 7 New metrics proposed EuroEcoDC Workshop - Karlsruhe

Metrics at node- group level Node-group cooling index - meaning – CI NG,HI = 100% All intake temperatures ≤ max. recommended temperature. – CI NG,HI max. recommended temperature. – CI NG,LO = 100% All intake temperatures ≥ min. recommended temperature. – CI NG,LO < 100% At least one intake temperatures < min. recommended temperature. 8 New metrics proposed EuroEcoDC Workshop - Karlsruhe

Metrics at node- group, rack and DC level Imbalance of temperature of CPU – Im NG,temp =0 means all of nodes works at the same temperature 9 New metrics proposed EuroEcoDC Workshop - Karlsruhe

Description of experiment Prototype server RECS from Christmann Company RECS: high density multinode computer of 18 single server nodes withing one Rack Unit CPU: Intel Core i7-3615QE 2.30GHz, CPU Cache: 6144 KB, RAM: 16 GB Load OpenSSL Benchmark 10 EuroEcoDC Workshop - Karlsruhe

Description of experiment 6 configuration 11 EuroEcoDC Workshop - Karlsruhe 1. Idle2. Full 3. Left4. Right 5. Inlet6. Outlet

Check Imbalance of Temperature 12 EuroEcoDC Workshop - Karlsruhe Unexpected imbalance !

Check Imbalance of Temperature Analysis: – Failure of one fan at right side !  – Imbalance was higher when load was placed on right side instead of left side Metric recalculated assuming CPU temperature of the node with failed fan as average of other nodes with similar load 13 EuroEcoDC Workshop - Karlsruhe

Check Imbalance of Temperature 14 EuroEcoDC Workshop - Karlsruhe Balanced! “Inlet” configuration: temperature of loaded nodes affects temperature of idle nodes

Further steps: other metrics to test Idea: heat-aware + useful work + energy Other metrics that will be deeply analysed: 1.Relation Imbalance of temperature vs Temperature or Heat-Dissipated 2.Productivity (Useful work / Energy) 3.PUE Scalability 4.FVER 15 EuroEcoDC Workshop - Karlsruhe

Further steps: other metrics to test PUE Scalability 16 EuroEcoDC Workshop - Karlsruhe Source: The Green Grid. WP#49

Further steps: other metrics to test PUE Scalability 17 EuroEcoDC Workshop - Karlsruhe Source: The Green Grid. WP#49

Further steps: other metrics to test FVER – Fixed to variable energy ratio Source: BSC How much energy produces useful work and how much could be removed E_fixed  energy when useful work = 0 During flat operation DC can consume up to 80 % of peak power! 18 1st Review, , Brussels

Conclusions Imbalance of temperature permits to detect failure of IT equipment. Complementarity between Imbalance of temperatures and Node-Group-Cooling-Index Analysis of several metrics together: – Imbalance of Temperature – Power Usage (Power/Max Power rated) – Productivity (Useful work/Energy) – FVER – PUE Scalability will allow improve aware about cooling requirements and the possibility of reducing it. 19 EuroEcoDC Workshop - Karlsruhe

Conclusions Relation between – power, – cooling requirements, – resource-usage, and – workload management will be identified to disclose the appropriate strategies to improve the energy efficiency First results on tests of the first prototype have been collected. More experiments will be carried out to validate the proposed metrics 20 EuroEcoDC Workshop - Karlsruhe

Questions? Comments? EuroEcoDC Workshop - Karlsruhe 21