MPI: Portable Parallel Programming for Scientific Computing William Gropp Rusty Lusk Debbie Swider Rajeev Thakur

Portable Parallel Programming Why Parallel? For performance: –Realtime needs (weather prediction) –Large memory (throughput) Why Portable? –Lifetime of code >> machine –Performance through algorithms

The Message Passing Model Parallel programs consist of cooperating processes, each with its own memory Processes send data to one another as messages Messages may have tags that may be used to sort messages Messages may be received in any order

What is MPI? Message-passing model Standard (specification) –Many implementations –MPICH most widely used Two phases: –MPI 1: Traditional message-passing –MPI 2: Remote memory, parallel I/O, and dynamic processes

Why was MPI needed? Software crisis in parallel computing –Each vendor provided their own, different interface –No “critical mass” of users Enabling libraries (code sharing) Turnkey parallel applications –CFD, pharmaceutical design, etc.

Quick Tour of MPI Point-to-point Collective Process groups and topology Profiling Other

Point-to-point Send/Receive Datatype –Basic for heterogeneity –Derived for non-contiguous Contexts –Message safety for libraries Buffering –Robustness and correctness A(10) B(20) MPI_Send( A, 10, MPI_DOUBLE, 1, …) MPI_Recv( B, 20, MPI_DOUBLE, 0, … )

Collective Process groups –Collections of cooperating processes –Hierarchical algorithms need nested collections Categories –Communication: Broadcast data –Computation: Global sum –Synchronization: Barrier

New MPI-2 Features Remote Memory Parallel I/O Dynamic Process Threads

MPI in the World Libraries –At ANL: PETSc, SUMMA3d, etc. –Others: Aztec, PESSL, ScaLAPACK, FFTW, etc. Applications –Electromagnetic Scattering –Groundwater pollution –Stellar atmospheres –Reactive flows –QCD –Navier-Stokes –Convective flows –Multi-physics in metal forming Papers

MPI Research at MCS Portable performance I/O Programming Environment New hardware –Network/Memory interconnect (VIA) New Programming Models –Threads –Remote memory

Learning More MPI Books –Using MPI –MPI: The Complete Reference –Parallel Programming with MPI Online Resources – –MPI Forum: –Tutorials: