Power and Cooling of HPC Data Centers Requirements Roger A Panton Avetec Executive Director DICE

Background and Objective Avetec is under contract to evaluate power and cooling requirements for HPC data centers To survey power and cooling constraints and solutions, both current and future  From representative HPC data centers  Vendors including HPC systems, equipment and facilities Avetec contracted with IDC to conduct a survey:  Current power and cooling situation  Planning in place to address requirements  Forecasted solutions in the next three to five years

Survey Sample and Methodology Survey includes 41 respondents 28 HPC data centers 13 vendors of products and services Response rate was approximately 20% Respondents were from US, Europe and Asia HPC data centers were selected from the Top500 list Centers selected fell between number 50 and 250 Interviews were conducted by phone or in person Respondents had the option to complete the survey on their own

Initial General Findings HPC data centers’ averages Available floor space about 26,000 ft 2 Used floor space about 17,000 ft 2 Cooling capacity 22.7 million BTUs or 1,839 tons Annual power consumption MW HPC data centers costs Annual power cost is $2.9 million or $456 per KW Ten sites provided the percentage of their budget spent on power—average was 23% Two-thirds of the sites had budget for power and cooling upgrades  Average amount budgeted is $6.87 million

Initial Key Findings Study revealed wealth of information Key findings will be summarized in the following four areas: Current Situation Challenges and Expansion Constraints Current Approaches Future Solutions and Technologies

Current Situation Over 96% of the centers consider “green” design important Majority of sites expect power and cooling to impact future HPC center planning Majority of respondents have studied or implemented “greener” operations Most centers have used software models to analyze heat flow and/or power consumption Approximately half of the centers paid for power and cooling out of their budgets

Challenges and Expansion Constraints Majority of centers are starting to consider power and cooling efficiency equal to or more important than HPC computing performance Power and cooling issues are becoming the biggest barriers to expansion and upgrades HPC vendors are starting to see power and cooling as a brake on performance

Current Approaches Power and cooling costs are becoming a key factor in upgrade decisions Majority of centers have accomplished an air flow analysis to improve air cooling efficiency Use of chilled water for cooling is increasing The power and cooling issues are being discussed across the HPC community, i.e. data center, HPC systems vendors, and processor vendors

Future Solutions and Technologies Approximately two-thirds of centers plan to expand or build new data centers About half of the data centers have or are planning to distribute HPC resources Liquid computing is being considered as an alternative HPC centers and vendors differ sharply on the likelihood of any game-changing cooling technologies emerging in the next 3-5 years

Viewpoint Current status June 2008 marked a new milestone the first Pflop HPC system made the Top500 Top500 lists the sum of the top 500 systems:  1993: Sum of top 500 equaled 1 ½ Tflops  2004: Sum of top 500 equaled 1 Pflop  If current growth is maintained, sum will be 100 Pflops by 2012 The balance between facilities and infrastructure to accommodate new systems does not exist This imbalance leads to a policy question What should the supercomputer community response be to restore the balance?

Policy Discussion Should the community take a proactive position through collaborative discussions and then recommend a set of Public Policies? To start the discussions should: The Federal Government establish a timeframe and fund the following research areas:  Invest to maintain the current performance growth in HPC?  Invest in new cooling technologies to improve efficiencies?  Invest in low power higher performance processor(s)?  Invest in new material research for chips? HPC data centers need to become more accountable for power and cooling consumption

Join Us to Learn Other Findings Findings were extensive and cannot be fully covered today, such as: What alternative approaches are you exploring to meet future power & cooling requirements? Describe any new power & cooling solutions expect to implement in next 2-3 years. What special cooling methods do you expect to use in the future? Future HPC system acquisition power & cooling requirements. And more! Final study results will be unveiled at DICE Alliance 2009 on May 20 along with a panel discussion with some of the study participants

We Invite You to Attend DICE Alliance 2009 May Wittenberg University Barbara Kuss Science Center Springfield, Ohio Register at Want More Information?

Other DICE Alliance Topics Monday May 18 th 6:00-8:00 Opening Reception Tuesday May 19 th  Keynote: Be Careful What You Wish For (Jay Boisseau, PhD)  America’s Need for HPC (Retired, Congressman Dave Hobson)  Panel: Power & Cooling Efficiency in Data Centers (Ed Wahl, lead)  Panel: Public Policy in HPC (Charlie Hayes, lead) Wednesday May 20 th  Multicore Architecture: Panacea or Nightmare? (Earl Joseph, PhD)  Panel: The Integration of Scheduling HPC Resource Management & Data Lifecycle Management Tools (Jay Blair)  Panel: Power & Cooling Trends and Technology (Earl Joseph, PhD)