1. MPI MPI = Message Passing Interface
Specification of message passing libraries for developers and users
Not a library by itself, but specifies what such a library should be
Specifies application programming interface (API) for such libraries
Many libraries implement such APIs on different platforms – MPI libraries
Goal: provide a standard for writing message passing programs
Portable, efficient, flexible
Language binding: C, C++, FORTRAN, Python programs

2. General MPI Programs #include <mpi.h>
main( int argc, char** argv ) {
MPI_Init( &argc, &argv );
/* main part of the program */ /*
Use MPI function call depend on your data partitioning and the parallelization architecture */
MPI_Finalize(); }

3. MPI Basics
MPI’s pre-defined constants, function prototypes, etc., are included in a header file. This file must be included in your code wherever MPI function calls appear (in “main” and in user subroutines/functions) :
#include “mpi.h” for C codes
#include “mpi++.h” * for C++ codes
include “mpif.h” for f77 and f9x codes
MPI_Init must be the first MPI function called
Terminates MPI by calling MPI_Finalize
These two functions must only be called once in user code.

4. MPI Basics
C is case-sensitive language. MPI function names always begin with “MPI_”, followed by specific name with leading character capitalized, e.g., MPI_Comm_rank. MPI pre-defined constant variables are expressed in upper case characters, e.g., MPI_COMM_WORLD.

5. Basic MPI Datatypes
MPI datatype C datatype MPI_CHAR signed char MPI_SIGNED_CHAR signed char MPI_UNSIGNED_CHAR unsigned char MPI_SHORT signed short MPI_UNSIGNED_SHORT unsigned short MPI_INT signed int MPI_UNSIGNED unsigned int MPI_LONG signed long MPI_UNSIGNED_LONG unsigned long MPI_FLOAT float MPI_DOUBLE double MPI_LONG_DOUBLE long double

6. MPI is Simple
Many parallel programs can be written using just these six functions.
MPI_Init
MPI_Finalize
MPI_Comm_size
MPI_Comm_rank
MPI_Send
MPI_Recv

7. Initialization Initialization: MPI_Init() initializes MPI environment
Must be called before any other MPI routine (so put it at the beginning of code)
Can be called only once; subsequent calls are erroneous.
MPI_Init(int *argc, char **argv)

8. Termination MPI_Finalize() cleans up MPI environment
Must be called before exits.
No other MPI routines can be called after this call, even MPI_Init()

9. Use Putty to Log in Penzias
Click Start, find PuTTy, and open it

10. Use Putty to Log in Penzias
Or download it from the following link: and open it.
1. Type the Host Name:
penzias.csi.cuny.edu
2. Click “Open”

11. Use Putty To Login in Penzias
Click Yes

12. Use Putty to Log in Penzias
1. Type your account, and press Enter
2. Type your password, and press Enter. Don’t press Backspace. The typed characters will NOT be shown.

13. Use Putty to Log in Penzias

14. Example 1 Compile
Type “cd example1”, you will enter the folder example1
Compile the MPI program by typing
“mpicc -o hello hello.c”, and you will have the executable file hello in the folder example1
Type “ls”, we can check the files and folders in the current folder example1

15. Example 1 Run Number of used processors
vi mpi.sh #!/bin/bash #BATCH --job-name slurm_mpi #SBATCH --nodes 2 #SBATCH --ntasks 4 #SBATCH --ntasks-per-node 16 #SBATCH --mem #SBATCH --partition partedu cd $SLURM_SUBMIT_DIR echo 0 > cap mpirun -np 4 ./hello
Number of used processors

16. Run a Job on HPC using Slurm
Common commands: sbatch: submit a job scancel: cancel a job squeue: display state of jobs