1. Advanced MPI Lab MPI I/O Exercises

2. 1 Getting Started Get a training account from the instructor Login (using ssh) to ranger.tacc.utexas.edu

3. 2 MPI I/O Exercises Objectives: Get familiar with MPI I/O Setting up an MPI I/O problem requires substantial overhead: –Set up communicators –Set up MPI datatypes for structures on disk and in memory –Set up fileviews. Lab time is finite, so we’ll start with an example, and just make some tweaks.

4. 3 I/O Exercises Copy the sample program to your directory: –~dstanzi/IOLab/ghost_rows.c –(FORTRAN version coming soon) Also, copy the sample job script: –~dstanzi/IOLab/ghost_job If you haven’t run jobs on Ranger before, you may want to peruse the user guide: –http://services.tacc.utexas.edu/index.php/ranger- user-guidehttp://services.tacc.utexas.edu/index.php/ranger- user-guide

5. 4 I/O Exercises This program operates on a 2x3 grid of 6 MPI tasks. –Each task has a 10x10 block of the final result array –Each local block is a 12x12 array, the data in the middle, with 1 ghost value all the way around the border. –The total 20x30 array is written on disk in a single “write_all()” command, grabbing the values from the middle of the 12x12 array, and skipping all the ghost values. –The trick is setting up *two* subarray datatypes, one for how the data exists on disk (filetype, no ghost rows), and one for how it exists in memory (memtype). The code illustrates best practice when your dataset is too large for the memory of any one node.

6. 5 I/O Exercises Once you’ve taken a look through the code, make the following modifications: –Change from one ghost value to two ghost values (8x8 data block). –Display the resulting file to check your answers –Modify the code to operate on a 2x4 grid of tasks instead of a 2x3 grid (change the total array size). Hint: You’ll need to change the MY_NSLOTS variable in the job script as well as the code –For the ambitious: Read the data back in to the 8 tasks. Rather than read in an 8x8 block, have each task read a 4x16 block (same total datasize, different distribution).

7. 6 Compiling and Running the Code This code will work best with mvapich2 and the intel compiler. Run these 3 commands before compiling to change your environment: login3$ module del mvapich login3$ module swap pgi intel login3$ module load mvapich2 You can compile using the command: mpicc ghost_rows.c Run using: qsub ghost_job