1. MPI: Portable Parallel Programming for Scientific Computing William Gropp Rusty Lusk Debbie Swider Rajeev Thakur

2. Portable Parallel Programming Why Parallel? For performance: –Realtime needs (weather prediction) –Large memory (throughput) Why Portable? –Lifetime of code >> machine –Performance through algorithms

3. The Message Passing Model Parallel programs consist of cooperating processes, each with its own memory Processes send data to one another as messages Messages may have tags that may be used to sort messages Messages may be received in any order

4. What is MPI? Message-passing model Standard (specification) –Many implementations –MPICH most widely used Two phases: –MPI 1: Traditional message-passing –MPI 2: Remote memory, parallel I/O, and dynamic processes

5. Why was MPI needed? Software crisis in parallel computing –Each vendor provided their own, different interface –No “critical mass” of users Enabling libraries (code sharing) Turnkey parallel applications –CFD, pharmaceutical design, etc.

6. Quick Tour of MPI Point-to-point Collective Process groups and topology Profiling Other

7. Point-to-point Send/Receive Datatype –Basic for heterogeneity –Derived for non-contiguous Contexts –Message safety for libraries Buffering –Robustness and correctness A(10) B(20) MPI_Send( A, 10, MPI_DOUBLE, 1, …) MPI_Recv( B, 20, MPI_DOUBLE, 0, … )

8. Collective Process groups –Collections of cooperating processes –Hierarchical algorithms need nested collections Categories –Communication: Broadcast data –Computation: Global sum –Synchronization: Barrier

9. New MPI-2 Features Remote Memory Parallel I/O Dynamic Process Threads

10. MPI in the World Libraries –At ANL: PETSc, SUMMA3d, etc. –Others: Aztec, PESSL, ScaLAPACK, FFTW, etc. Applications –Electromagnetic Scattering –Groundwater pollution –Stellar atmospheres –Reactive flows –QCD –Navier-Stokes –Convective flows –Multi-physics in metal forming Papers

11. MPI Research at MCS Portable performance I/O Programming Environment New hardware –Network/Memory interconnect (VIA) New Programming Models –Threads –Remote memory

12. Learning More MPI Books –Using MPI –MPI: The Complete Reference –Parallel Programming with MPI Online Resources –http://www.mcs.anl.gov/mpihttp://www.mcs.anl.gov/mpi –MPI Forum: http://www.mpi-forum.orghttp://www.mpi-forum.org –Tutorials: http://www.mcs.anl.gov/mpi/learning.html http://www.mcs.anl.gov/mpi/learning.html