1. Software Overview Environment, libraries, debuggers, programming tools and applications Jonathan Carter NUG Training 3 Oct 2005

2. 2 Overview Environment Libraries Debuggers Programming Tools Applications

3. 3 Environment Suse Linux Linux Networx HPC software stack Pathscale compilers PBS Pro Batch System GPFS HOME and SCRATCH file systems

4. 4 Environment (cont) Passwords –NIM password, use NIM to change Shells –Default shell is tcsh, to change use NIM interface Startup files –Read-only, use.ext if you want to add customizations –fixdots command will repair files

5. 5 Environment (cont) Access to libraries, tools and applications is controlled via modules Number of modules reduced compared with seaborg, since we only have 64-bit ABI and fewer software versions

6. 6 Libraries Math –ACML, fftw, NAG, parpack, scalapack, superlu, (imsl, petsc) I/O –hdf, hdf5, netcdf –MPI I/O available via MVAPICH Visualization –ncar

7. 7 Libraries (cont) ACML –Optimized for AMD processors –BLAS 1-3, FFT and LAPACK –“fast” math functions ( log is ~90 cycles instead of ~120) –“vector” math functions, e.g. vrda_log(n, x, y) is roughly twice as fast as log at n>20.

8. 8 Libraries (cont) Fortran I/O –Intel/AMD hardware is little-endian, Power is big-endian, so binary data is incompatible –pathf90 offers several features to write binary data in big-endian format

9. 9 Libraries (cont) pathf90 options –-byteswapio writes all data in format opposite to that of native processor –-conversion [native, little_endian, big_endian] –assign command FILENV=.assign export FILENV assign –N mips u:10

10. 10 Libraries (cont) Many libraries are linked at runtime, so LD_LIBRARY_PATH must include a path to each library linked against Modules environment takes care of this provided you load at compile time and at run time

11. 11 Debuggers Totalview and gdb available for serial applications module load totalview totalview progname [corefile] gdb progname [corefile]

12. 12 Debuggers (cont) Totalview and gdb are coming for parallel applications gdb opens xterm per processor module load totalview mpirun –tv –np procs progname mpirun –debug –np procs progname

13. 13 Debuggers (cont) Generating core files –Serial applications ulimit -S -c unlimited –Parallel applications, make shell script progname.sh and run this via mpirun (this problem will be resolved by PBS bugfix) #!/bin/bash ulimit -S -c unlimited exec progname

14. 14 Debuggers (cont) Useful Pathscale compiler options –Use –g flag to help debugging –Use –trapuv to set uninitialized (local, automatic, alloca() ; not Fortran allocatable) variables to NAN

15. 15 Debuggers (cont) Further information on Totalview –Etnus tutorial http://www.etnus.com/TotalView/started/getting_started.html http://www.etnus.com/TotalView/started/getting_started.html –LLNL tutorial –http://www.llnl.gov/computing/tutorials/totalview/

16. 16 Programming Tools AMD CodeAnalyst – suite of tools to optimize code for Opteron: coming soon ipm – mpi overhead and performance report papi – interface to hardware performance counters tau – suite of tools enabling performance analysis of application codes valgrind – multiple tools to check memory use, profile cache use, detect data race conditions in threaded applications

17. 17 Applications Chemistry & Materials Science –GAMESS, Gaussian, Gromacs, MOLPRO, NWChem, NAMD, VASP Math –maple, matlab, mathematica Visualization –idl, vmd Other –Subversion (client)