1. MPI and OFA Divergent interests? Dan Caldwell, VP WW Channel Sales Scali, Inc

2. April 2008 OFA Presentation 2 The role of OFA in HPC  The traditional role of Open Fabrics focuses on cables that connect motherboards, switches, and storage in clusters  Growth of processors in HPC expected to be 30% in 2010 (per IDC), and number of cores per processor is 55% (estimate) and increasing  Therefore the bulk of the “HPC Interconnect Fabric” is moving from cables to the server motherboard and the various local buses

3. April 2008 OFA Presentation 3 MPI Performance in HPC systems  Scali has always been the performance MPI leader –http://www.supercomputingonline.com/article.php?sid= 15357http://www.supercomputingonline.com/article.php?sid= 15357  However, we suggest that OFA adopt a recommended performance measurement methodology –Productivity, or ‘jobs per day’, on a cluster –How do you know how well OFA is doing?  OFA must examine the performance within a multi-core node –A 16 core “Personal Supercomputer” will still run legacy MPI applications

4. April 2008 OFA Presentation 4 Single Node, 8 cores, osu_bw, 8 byte, Intel Xeon 3.00GHz (X5365)

5. April 2008 OFA Presentation 5 MPI enables other HPC functionality  Infiniband Trunking –Combining IB channels for greater throughput –Done for Sun / Tsukuba University in Japan  Suspend / Resume and Checkpoint – Restart –Generic functionality - not application specific –Demonstrated job migration with HPC4U in Brussels, February 11, 2008  And…. MPI based Power Management –Today – AMD / Barcelona only

6. April 2008 OFA Presentation 6 ”Waiting as fast as it can” (CPU spinning) – examples (AMD Barcelona Quad Core): Power Saving Potential Zone Considering 20 microseconds to change power states in an Quad core Opteron, latency above 10 milliseconds in an MPI collective can trigger a ”throttle down”. In a call with a 10 millisecond latency, 20 microseconds to throttle down plus 200 to re-establish full speed would use only 2.5% of the time of the call.

7. April 2008 OFA Presentation 7 Real World Initial Tests - SPEC MPI2007

8. April 2008 OFA Presentation 8 Conclusions  MPI and OFA need to co-exist, but we both need to expand our reach in HPC  Open Fabrics needs to embrace new functionality, performance metrics, and definitions of ‘fabric’  Power savings, user-model (core affinity policies), best practices in interconnect design, and emphasis on continued performance across the entire HPC system is critical to the relevancy of OFA.