1. CS 584 Lecture 8
Assignment?

2. Message Passing Very close to Task-Channel model
Based on multi-processor
Set of independent processors
Connected via some communication net
All communication between processes is done via a message sent from one to the other

3. MPI Message Passing Interface Computation is made of:
One or more processors
Communicate by calling library routines
MIMD programming model
SPMD most common.

4. MPI Processes use point-to-point communication operations
Collective communication operations are also available.
Communication can be modularized by the use of communicators.
MPI_COMM_WORLD is the base.
Used to identify subsets of processors

5. MPI
Complex, but most problems can be solved using the 6 basic functions.
MPI_Init
MPI_Finalize
MPI_Comm_size
MPI_Comm_rank
MPI_Send
MPI_Recv

6. MPI Basics
Most all calls require a communicator handle as an argument.
MPI_COMM_WORLD
MPI_Init and MPI_Finalize
don’t require a communicator handle
used to begin and end and MPI program
MUST be called to begin and end

7. MPI Basics MPI_Comm_size MPI_Comm_rank
determines the number of processors in the communicator group
MPI_Comm_rank
determines the integer identifier assigned to the current process
zero based

8. MPI Basics #include <stdio.h> #include <mpi.h>
main(int argc, char *argv[]) {
int iproc, nproc;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &nproc);
MPI_Comm_rank(MPI_COMM_WORLD, &iproc);
printf("I am processor %d of %d\n", iproc, nproc);
MPI_Finalize(); }

9. MPI Communication MPI_Send MPI_Recv Sends an array of a given type
Requires a destination node, size, and type
MPI_Recv
Receives an array of a given type
Same requirements as MPI_Send
Extra parameter
MPI_Status variable.

10. MPI Basics Made for both FORTRAN and C Book attempts to show both
Standards for C
MPI_ prefix to all calls
First letter of function name is capitalized
Returns MPI_SUCCESS or error code
MPI_Status structure
MPI data types for each C type

11. Using MPI Based on rsh Path to compiler requires a .rhosts file
hostname login
Path to compiler
MPI_HOME /u2/faculty/snell/mpich
MPI_ARCH hpux
MPI_BIN MPI_HOME/lib/MPI_ARCH/ch_p4

12. Using MPI Write program Compile using mpicc in: Write process file
/u2/faculty/snell/mpich/lib/hpux/ch_p4
Write process file
host nprocs full_path_to_prog
0 for nprocs on first line
Run program
prog -p4pg process_file args

13. Example
2D finite differences code
Incomplete
In book on page 281

14. MPI
Message Passing programs are non-deterministic because of concurrency
Consider 2 processes sending messages to third
MPI does guarantee that 2 messages sent from a single process to another will arrive in order.
It is the programmer's responsibility to ensure computation determinism

15. MPI & Determinism Task-Channel model MPI Non-Determinism
One reader and one writer per channel
Separate channel for each communication
MPI
A Process may specify the source of the message
A Process may specify the type of message
Non-Determinism
MPI_ANY_SOURCE or MPI_ANY_TAG

16. Example for (n = 0; n < nproc/2; n++) {
MPI_Send(buff, BSIZE, MPI_FLOAT, rnbor, 1,
MPI_COMM_WORLD);
MPI_Recv(buff, BSIZE, MPI_FLOAT, MPI_ANY_SOURCE,
1, MPI_COMM_WORLD, &status);
/* Process the data */ }

17. Global Operations
Coordinated communication involving multiple processes.
Can be implemented by the programmer using sends and receives
For convenience, MPI provides a suite of collective communication functions.

18. Collective Communication
Barrier
Synchronize all processes
Broadcast
Gather
Gather data from all processes to one process
Scatter
Reduction
Global sums, products, etc.

19. Collective Communication

20. MPI_Reduce
MPI_Reduce(inbuf, outbuf, count, type, op, root, comm)

21. MPI_Reduce with MPI_SUM

22. MPI_Allreduce
MPI_Allreduce(inbuf, outbuf, count, type, op, root, comm)

23. Example Finite Differences code Notice the stages program 8.5 page 289
Signified by the global communication calls

24. Distribute
Problem Size
Distribute
Input data
Exchange
Boundary values
Find Max Error
Collect Results

25. Other MPI Features Asynchronous Communication Modularity
MPI_ISend
MPI_Wait and MPI_Test
MPI_Probe and MPI_Get_count
Modularity
Communicator creation routines
Derived Datatypes