2. Petascale System Requirements for the Geosciences Richard Loft SCD Deputy Director for R&D

3. Petascale Collaboratory Reports l Charges: –Identify science drivers for petascale geoscience computing. (Bryan) –Determine technical feasibility and cost of creating such a system by 2010. (Loft) l Outcomes : –Collaboratory is in NSF Facility Plan as a facility “under study”. –Initiative to bring vendors and scientists together to discuss petascale geoscience computing

4. Petascale Geoscience l World class computational capability should be devoted to understanding the earth. –Earthquakes & Tsunamis –Severe weather - Katrina –Global warming –Land use policies l It doesn’t stop at the 1 PFLOPS. l Easy Part: Achieving ~ 1 PFLOPS peak –Nothing exceptional has to happen technologically to do it –Possible by 2008, first by 2010, multiple by 2012 –Highly parallel system - speed of single thread slowing. l Hard Part: Infrastructure. –Facilities to house them. –Efficient applications. –Systems and tools to help us understand the data they produce.

5. Geoscience really does need Petascale Computing (and Beyond)

6. Geocience Space Weather Turbulence Atmospheric Chemistry Climate Weather The Sun from the sun’s surface to the earth’s core…

8. Directly Resolving Weather Phenomena

9. Weather Research Forecast Model: Katrina Landfall: Aug 29, 2005 After W. Skamarock

10. WRF Model Performance 9.8% of Peak

11. Estimated Global NWP Requirements

12. Petascale Cat Herding Application Experts System Architects Software Architects Numericists geoscience Vendor Sales $ COTS CMOS Open Source Matlab ?

13. Data Driver l A Petascale system will produce 100 PB of geoscience data per year. l That’s 3.2 GB/s 7x24x365 l Hard: Mining that data torrent for science nuggets. l How do we drive petascale requirements for those DAV systems?

14. Taming the Torrent: Visualization Example l Key concepts 1.Integrate visualization into analysis process 2.interactively steer the analysis to subsets 3.Employ multiresolution data representation as a data reduction technique Size: Full 1/8 1/64 1/512

15. Facility Driver Uncertainty in fuel efficiency and power density leads to uncertainty petascale facility planning. You can’t turn a facility on a dime. Efficiency is good. How do we craft power-efficiency benchmarks?

16. Petascale TCO: Fuel Efficiency Top 20 systems Based on processor power rating only Blue Gene/L: How do we use this stuff?

17. Petascale Data Center Design Petascale estimate: 2-5 MW by 2010 Reliable Tier 2+ Redundant power/cooling systems Modular design Expandable. Mitigates risk. Floor Space (per module): 20,000 sq. ft. machine room 20,000 sq. ft. mechanical space Power (per module) 4 MW systems plus 4 MW mechanical Land: 13 Acres. A second order term in cost.

18. Application Driver: Collecting Geoscience Application Requirements How do we extract system requirements for systems that haven’t been built from applications that haven’t been written? No single set of requirements in fact exists. There are very few good models of application performance. Application experts intuition is often wrong. How should applications benchmarks be selected?

19. 1/10 Degree POP Ocean Model Credit: Frank Bryan, NCAR Agulhas Rings

20. POP X1 Performance 6x: System Architecture Does Matter! After Worley, et al.

21. POP Sensitivity to System Parameters

22. 1 Degree Barotropic Ocean Model Sensitivity to Latency and Bandwidth (Performance Model) On 4096 processors…

23. 3D pseudospectral code (2048^3) 1.6 GFLOPS sustained 6 µsec latency

24. Forget the Top 500 list! Rmax ranking vs application reality Are the HPC Challenge Benchmarks good metrics?

25. HPC Challenge: Flops Results

26. HPC Challenge: Bandwidth Results

27. HPC Challenge: Latency/STREAM Results

28. HPC Challenge: Local/Remote Balance

29. Discussion - Open Forum How do we drive petascale requirements for DAV systems? How do we extract system requirements for systems that haven’t been built from applications that haven’t been written? How should applications benchmarks be selected? Are the HPC Challenge Benchmarks good metrics? How do we craft power-efficiency benchmarks?

30. The End

31. Ready or not, the petascale is coming Proof by extrapolation…

32. Growth Rate Results - Top 10 Geoscience Centers on the Top500 List l Individual sites come and go. l Individual sites rise and fall. l Collectively, things are smoother. l Annual rate 1st ranked: l Annual rate 1st ranked: 1.81. l Annual mean growth rate: 1.84. l Annual rate 10th ranked: 1.90. l Currently –5 use IBM POWER processors –2 use vector systems –3 use other l Pack is slowly converging. l Use this rate to chart computing future… Log linear fit (mean)

33. March Towards the Petascale

34. Extrapolated Petascale System Power requirements … NCAR Facility

35. Additional Power and Storage Costs Wildcard factors: Katrina, Enron, etc.

36. Petascale Budget Breakdown (2007-2012)