MPI MPI = Message Passing Interface Specification of message passing libraries for developers and users Not a library by itself, but specifies what such a library should be Specifies application programming interface (API) for such libraries Many libraries implement such APIs on different platforms – MPI libraries Goal: provide a standard for writing message passing programs Portable, efficient, flexible Language binding: C, C++, FORTRAN, Python programs

General MPI Programs #include <mpi.h> main( int argc, char** argv ) { MPI_Init( &argc, &argv ); /* main part of the program */ /* Use MPI function call depend on your data partitioning and the parallelization architecture */ MPI_Finalize(); }

MPI Basics MPI’s pre-defined constants, function prototypes, etc., are included in a header file. This file must be included in your code wherever MPI function calls appear (in “main” and in user subroutines/functions) : #include “mpi.h” for C codes #include “mpi++.h” * for C++ codes include “mpif.h” for f77 and f9x codes MPI_Init must be the first MPI function called Terminates MPI by calling MPI_Finalize These two functions must only be called once in user code.

MPI Basics C is case-sensitive language. MPI function names always begin with “MPI_”, followed by specific name with leading character capitalized, e.g., MPI_Comm_rank. MPI pre-defined constant variables are expressed in upper case characters, e.g., MPI_COMM_WORLD.

Basic MPI Datatypes MPI datatype C datatype MPI_CHAR signed char MPI_SIGNED_CHAR signed char MPI_UNSIGNED_CHAR unsigned char MPI_SHORT signed short MPI_UNSIGNED_SHORT unsigned short MPI_INT signed int MPI_UNSIGNED unsigned int MPI_LONG signed long MPI_UNSIGNED_LONG unsigned long MPI_FLOAT float MPI_DOUBLE double MPI_LONG_DOUBLE long double

MPI is Simple Many parallel programs can be written using just these six functions. MPI_Init MPI_Finalize MPI_Comm_size MPI_Comm_rank MPI_Send MPI_Recv

Initialization Initialization: MPI_Init() initializes MPI environment Must be called before any other MPI routine (so put it at the beginning of code) Can be called only once; subsequent calls are erroneous. MPI_Init(int *argc, char **argv)

Termination MPI_Finalize() cleans up MPI environment Must be called before exits. No other MPI routines can be called after this call, even MPI_Init()

Use Putty to Log in Penzias Click Start, find PuTTy, and open it

Use Putty to Log in Penzias Or download it from the following link: https://the.earth.li/~sgtatham/putty/latest/w32/putty.exe, and open it. 1. Type the Host Name: penzias.csi.cuny.edu 2. Click “Open”

Use Putty To Login in Penzias Click Yes

Use Putty to Log in Penzias 1. Type your account, and press Enter 2. Type your password, and press Enter. Don’t press Backspace. The typed characters will NOT be shown.

Use Putty to Log in Penzias

Example 1 Compile Type “cd example1”, you will enter the folder example1 Compile the MPI program by typing “mpicc -o hello hello.c”, and you will have the executable file hello in the folder example1 Type “ls”, we can check the files and folders in the current folder example1

Example 1 Run Number of used processors vi mpi.sh #!/bin/bash #BATCH --job-name slurm_mpi #SBATCH --nodes 2 #SBATCH --ntasks 4 #SBATCH --ntasks-per-node 16 #SBATCH --mem 20000 #SBATCH --partition partedu cd $SLURM_SUBMIT_DIR echo 0 > cap mpirun -np 4 ./hello Number of used processors

Run a Job on HPC using Slurm Common commands: sbatch: submit a job scancel: cancel a job squeue: display state of jobs