1. Dynamic Thermal Management in Charm++ Osman Sarood, Phil Miller, Esteban Meneses, Ehsan Totoni, Sanjay Kale Parallel Programming Lab (PPL) 1

2. Why care about energy? Data centers consume 2% of US Energy Budget in 2010[1] Cost $5.1 billion consumed 77 billion KWh Energy bill per annum[2]: – Sequoia: $4.47M – Blue waters: $5 M – K computer: $7.4M 2 1.Growth in data center electricity use 2005 to 2010, Jonathan Koomey 2.Based on 7 cent/Kwh

3. Why Cooling? Cooling accounts for 40-50% of total cost Average PUE ratio was 1.8 in 2011 Most data centers face hot spots responsible for lower temperatures in machine rooms Data center managers can save*: – 4% (7%) for every degree F (C) – 50% going from 68F(20C )to 80F(26.6C) *according to Mark Monroe of Sun Microsystem 3

4. Need for thermal management High core temperatures can increase: – Cooling energy consumption – (1 st part) – Failure rate since it doubles for every 10C increase in temperature – (2 nd part) – Machine machine energy consumption – (in progress) 4

5. 1 st Part: Reducing cooling energy consumption using thermal constraints 5

6. Core Temperatures 6 Running Wave2D on 128 cores: Temperature measurements after every second Average temperature goes from 32C to 52C Hottest core ends up 9C above the average *CRAC stands for Computer Room Air Conditioning

7. Core Temperatures Reducing cooling results: – increase of 6C in average temperature i.e. 58C – With reduced cooling (CRAC set-point 25.6) hottest core is 20C above average core temperature 7 Hotspot!

8. Dynamic Voltage and Frequency Scaling (DVFS) changing processor frequency/voltage to save power used cpufreq module from linux to change frequency/voltage pairs 8

9. Processor Timelines for 2 iterations of Wave2D w/o TempLDB Idle Time 9 Shows processor utilization during execution time (green & pink correspond to computations, white is idle time) One core can cause timing penalty/slowdown!

10. Charm++ to the rescue! Object-based over-decomposition – Helpful for refinement load balancing Migratable objects – Mandatory for our scheme to work – supports fault tolerance Time logging for all objects – Central to load balancing decisions Supports plugin load balancer Production-quality system used by many applications For more info, see http://charm.cs.illinois.edu/why/ 10

11. Temperature Aware Load Balancer Specify temperature range and sampling interval Runtime system periodically checks core temperatures Scale down/up frequency (by one level) if temperature exceeds/below maximum threshold at each decision time Transfer tasks from slow cores to faster ones See ‘‘Cool’ Load Balancing for HPC Data Centers’, IEEE Transactions on computers for details 11

12. Experimental Setup 128 cores (32 nodes), 10 different frequency levels (1.2GHz – 2.4GHz) Direct power measurement Dedicated CRAC Power estimation based on Applications: Jacobi2D, Mol3D, and Wave2D – Different power profiles Temperature range: 47C-49C 12

13. Average Core Temperatures in Check Avg. core temperature within 2 C range Can handle applications having different temperature gradients 13 CRAC set-point = 25.6C Temperature range: 47C-49C

14. Hotspot Avoidance 14 Maximum Difference without our scheme (w/o TempLDB) – Increases over time – Increases with CRAC set point Maximum Difference with our scheme (TempLDB) – Decreases with time – Insensitive to CRAC set point

15. Timing Penalty Decrease in cooling, increases: – Timing penalty – Advantage of our scheme 15

16. Processor Timelines for Wave2D w/o TempLB 16 TempLB

17. Machine Energy Consumption High base power coupled with timing penalty doesn’t allow machine energy savings. 17

18. Cooling Energy Consumption Our scheme saves up to 57% (better than w/o TempLDB) due to smaller timing penalty 18

19. Related Publications Osman Sarood, Laxmikant Kale, A `Cool’ Load Balancer for Parallel Applications, SC 11 Osman Sarood, Phil Miller, Ehsan Totoni, Laxmikant Kale, `Cool’ Load Balancing for High Performance Computing Data Centers, IEEE Transactions of Computers 19

20. 2 nd Part: Improving Reliability using Thermal Constraints 20

21. Temperature and Mean Time to Failure (MTBF) MTBF halves (failure rate doubles) for every 10C increase in temperature MTBF (M) can be modeled as where ‘A’ and ‘b’ are constants and ‘T’ is the temperature 21

22. Core temperatures and MTBF 22 Temperature histogram Wave2D on 128 cores (blue – cool cores, orange – hot cores)

23. Removing the hot spot 23 Remove hot spot: Avg. of hot cores = Avg. of cool cores Generate random temperature values for hot cores and calculate ‘M’

24. Constraining core temperature to lower values 24 Remove hot spot: Avg. of hot cores = Avg. of cool cores Generate random temperature values for all the cores and calculate ‘M’

25. Core temperature-MTBF relation 25

26. Core temperature-MTBF relation Experimental data 26

27. What did we learn? By constraining core temperatures one can select an MTBF (within a range) Execution time (slowdown) penalty associated with selecting the MTBF Each application can give rise to a different MTBF for the cluster due to temperature variations 27

28. Benefits? Is the decrease in MTBF good enough to reduce total execution time given the slowdown associated inusing DVFS? 28

29. Performance model 29 SymbolMeaning TTotal execution time WUseful work τCheck pointing period δCheck pointing time RRestart time μslowdown

30. Strategy Combine temperature control with fault tolerance Migrateable objects key for reducing DVFS associated slowdown 30

31. Improved temperature aware load balancing Communication friendly load balancing: – Instead of randomly picking a task to migrate to any overloaded processor, migrate a task that communicates the most with a given underloaded processor – Always try to converge to the initial mapping – Select ‘foreign’ task to migrate before ‘home’ tasks 31

32. Experimental setup Ran experiments on a 128-core cluster (no simulations) to see the prediction of the model Scaled down M: 4 hours Introduced random faults based on exponential distribution with a mean of ‘M’ Three applications – Jacobi2D: 5-point stencil application – Lulesh: Unstructured Lagrangian Explicit Shock Hydrodynamics application developed @ LLNL – Wave2D: finite difference for pressure propagation 32

33. Experimental setup Baseline for each application: – Run without temperature control – ‘M’ calculated using the actual temperature values we get without temperature constraint – `τ’ calculated using Daly’s formula: Temperature constrained experiment: – `M’ calculated using the max allowed temperature – `τ’ calculated using Daly’s formula 33

34. Reduction in execution time 34 Each experiment was longer than 1 hour having at least 40 faults Model closely matches the experiments

35. Savings in machine energy consumption 35 Actual measurements based on power meters installed in the PDUs

36. Prediction for larger machines 36 MTBF/socket: 20 years, checkpointing time (δ): 240 secs Improvement in MTBF

37. Future work Evaluating benefits of thermal control for message logging and parallel recovery Scheduling jobs for a data center under a fixed thermal and/or power budget Reducing frequency for least sensitive parts of code to reduce slowdown for TempLDB 37

38. Hot spot in Blue Waters? 38 63F 62F 69F 70F 69F 68F 65F 63F 64F 63F This shows a possibility of Inter-row hot spot. There might be Intra-row hot spots! Row 1 Row 7 The readings showing cold water temperature for each row

39. Acknowledgements We are thankful to Prof. Tarek Abdelzaher for using tarekc cluster 39

40. Questions 40

41. Optimum points for applications 41 Jacobi2D gets the max benefit Different optimum temperature thresholds along with maximum benefits

42. Reason for different optimum temperature thresholds 42 A move to the left (decrease in temperature threshold): Increases reliability (gain) Increases the slowdown due to temperature control (cost)

43. Prediction for larger machines 43 Proposed Exascale machine in ‘ExaScale Computing Study’ by Peter Kogge has an incredibly low Memory/FLOPS ratio Temperature threshold of 46C

44. Machine Energy Accounts for 50%-60% of total cost Earlier work: – A `Cool’ Balancer for Parallel Applications (SC11) concentrated on saving cooling energy – `Cool’ Load balancing for HPC Data Centers (IEEE Transactions on Computers Sept 2012) extended our work and its usefulness with MPI applications – limited machine energy savings Is it possible to reduce execution time penalty and machine energy while reducing cooling energy? 44

45. Execution Blocks for iterative applications Divide each iteration into Execution blocks (EBs) – different sections based on sensitivity to frequency – Manually done using HW performance counters Profile each EB for different frequency levels – Wall clock time (system clock) – Core power consumption (fast on- chip MSRs) 45

46. Execution Blocks (EBs) (NPB-IS) EB1 much more sensitive to frequency with the same power as EB2 EB2 wastes a lot of energy while running at max frequency! 46

47. EBTuner Profile each EB for all frequency values – Can be completed in milliseconds using energy MSRs of Sandy Bridge Periodic sampling of core temperatures – Temperature > Threshold Decrease frequency one notch for EB that results in minimum timing penalty – Temperature < Threshold Increase frequency one notch for EB that results in maximum time reduction 47

48. Timing penalty Increase in execution time compared to runs with no temperature control and all cores working at maximum possible frequency 48

49. Reduction in machine energy 49

50. Work in progress Extend the work to multiple nodes. – Use Charm++ since load balancing would be necessary – Solution for the incapability of present day chips to apply DVFS to individual cores 50

51. Timing Penalty/ Total Energy Savings Our scheme brings green line to red line – Moving left: saving total energy – Moving down: saving execution time penalty Slope: timing penalty (secs) per joule of energy saved 51 Normalization w.r.t all cores running at maximum frequency without temperature control