1. LA 4 CHAIN GISELA EPIKH School SPECFEM3D on Science Gateway

2. Why SPECFEM3D SPECFEM3D constrains MPI Job requirements SPECFEM3D JDL Infrastructure constrains SPECFEM3D in the SG progress

3. SPECFEM3D Simulates seismic wave propagation at the local or regional scale Is based on the spectral-element method (SEM)

4. SPECFEM3D needs to be compiled in each session Data files in:  $PATH/SPECFEM/in_data_files SPECFEM3D Compilation  $./configure FC=gfortran MPIFC=mpif90 – with-scotch-includedir=/usr/local/include  $ make all

5. SPECFEM3D is a MPI application Mesh generation  $ mpirun -np 2./xmeshfem3D Distributed database creation  $ mpirun -np 2./xgenerate_databases Running the solver  $ mpirun -np 2./xspecfem3D Output files in:  $PATH/SPECFEM3D/in_out_files/OUTPUT_F ILES

6. SPECFEM3D in a SE SPECFEM3D is stored in a SE: lcg-cr --vo prod.vo.eu-eela.eu -l lfn:/grid/prod.vo.eu- eela.eu/SPECFEM3D/SPECFEM3D_V2.0.1.tar.gz -d se.cecalc.ula.ve file://$HOME/SPECFEM3D/SPECFEM3D_V2.0.1.tar.gz lcg-cr --vo prod.vo.eu-eela.eu -l lfn:/grid/prod.vo.eu- eela.eu/SPECFEM3D/scotch_5.1.12b.tar.gz -d se.cecalc.ula.ve file://$HOME/SPECFEM3D/scotch_5.1.12b.tar.gz

7. JDL [ JobType = "normal"; NodeNumber = 2; Executable = "mpi-start-wrapper.sh"; Arguments = "xmeshfem3D xgenerate_databases xspecfem3D OPENMPI"; StdOutput = "SPECFEM3D_mpi-test.out"; StdError = "SPECFEM3D_mpi-test.err"; InputSandbox = {"mpi-start-wrapper.sh","mpi-hooks.sh","mpi-hooksNoCompile.sh","mpi-hooksSolver.sh","w_files.sh","w_files.tar.gz"}; OutputSandbox = {"SPECFEM3D_mpi-test.err","SPECFEM3D_mpi-test.out","output_files.tar.gz"}; Requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment) && Member("OPENMPI", other.GlueHostApplicationSoftwareRunTimeEnvironment) && other.GlueCEUniqueID == "cream01.cecalc.ula.ve:8443/cream-pbs-prod" ] ;

8. Mpi-start-wraper.sh 1/3 #!/bin/bash # Pull in the arguments. #Cambio JLGR #MY_EXECUTABLE=`pwd`/$1 MY_EXECUTABLE=$1 MPI_FLAVOR=$4 # Convert flavor to lowercase for passing to mpi-start. MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'` # Pull out the correct paths for the requested flavor. eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH` # Ensure the prefix is correctly set. Don't rely on the defaults. eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH export I2G_${MPI_FLAVOR}_PREFIX # Touch the executable. It exist must for the shared file system # check. # If it does not, then mpi-start may try to distribute the executable # when it shouldn't. touch $MY_EXECUTABLE # Setup for mpi-start. export I2G_MPI_APPLICATION=$MY_EXECUTABLE export I2G_MPI_APPLICATION_ARGS= export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh export I2G_MPI_POST_RUN_HOOK=../../mpi-hooks.sh # If these are set then you will get more debugging information. export I2G_MPI_START_VERBOSE=1 export I2G_MPI_START_DEBUG=0

9. Mpi-start-wraper.sh 2/3 # Invoke mpi-start. $I2G_MPI_START #turning off double compiling specfem3d and starting second stage (DATABASE)(jlgr 24/06/2012) # Setup for mpi-start. MY_EXECUTABLE=$2 touch $MY_EXECUTABLE export I2G_MPI_APPLICATION=$MY_EXECUTABLE #export I2G_MPI_APPLICATION_ARGS= #export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER export I2G_MPI_PRE_RUN_HOOK=mpi-hooksNoCompile.sh export I2G_MPI_POST_RUN_HOOK=../../mpi- hooksNoCompile.sh # Invoke mpi-start. $I2G_MPI_START #third stage (Solver)(jlgr 24/06/2012) # Setup for mpi-start. #MY_EXECUTABLE=$3 #touch $MY_EXECUTABLE # export I2G_MPI_APPLICATION=$MY_EXECUTABLE # export I2G_MPI_APPLICATION_ARGS= #export I2G_MPI_APPLICATION= #export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER #export I2G_MPI_PRE_RUN_HOOK=mpi-hooksSolver.sh #export I2G_MPI_POST_RUN_HOOK=../../mpi-hooksSolver.sh # Invoke mpi-start. #$I2G_MPI_START