1. Case Study: PETSc ex19  Non-linear solver (snes)  2-D driven cavity code  uses velocity-velocity formulation  finite difference discretization on a structured grid  Instrumentation:  MPI  MPI wrapper interposition library  PDT (tau_instrumentor)  PETSc library sources  Application  Selective instrumentation (tau_reduce)

2. Configuring TAU  Profiling with wallclock time (on a quad PIII Linux machine)  % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit/ -useropt=-O2 –LINUXTIMERS  Tracing  % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit –useropt=-O2 –LINUXTIMERS  Profiling with PAPI  % configure –mpiinc=/usr/local/packages/mpich/include –mpilib=/usr/local/packages/mpich/lib –pdt=/usr/pkg/pdtoolkit/ -useropt=-O2 –papi=/usr/local/packages/papi  % setenv PAPI_EVENT PAPI_FP_INS  % setenv PAPI_EVENT PAPI_L1_DCM

3. Compiling: TAU Makefiles  Include TAU Stub Makefile ( /lib) in the user’s Makefile.  Variables:  TAU_CXXSpecify the C++ compiler used by TAU  TAU_CC, TAU_F90Specify the C, F90 compilers  TAU_DEFSDefines used by TAU. Add to CFLAGS  TAU_LDFLAGSLinker options. Add to LDFLAGS  TAU_INCLUDEHeader files include path. Add to CFLAGS  TAU_LIBSStatically linked TAU library. Add to LIBS  TAU_SHLIBSDynamically linked TAU library  TAU_MPI_LIBSTAU’s MPI wrapper library for C/C++  TAU_MPI_FLIBSTAU’s MPI wrapper library for F90  TAU_FORTRANLIBSMust be linked in with C++ linker for F90.  TAU_DISABLETAU’s dummy F90 stub library  Note: Not including TAU_DEFS in CFLAGS disables instrumentation in C/C++ programs (TAU_DISABLE for f90).

4. Profiling ex19 (wallclock time)

6. Tracing ex19

7. Callgraph Display for ex19

8. Using TAU with PAPI (PAPI_FP_INS)

9. Using TAU with PAPI (PAPI_L1_DCM)

10. Using TAU with PAPI (PAPI_FP_INS)

12. Support Acknowledgement  TAU and PDT support:  Department of Energy (DOE)  DOE 2000 ACTS contract  DOE MICS contract  DOE ASCI Level 3 (LANL, LLNL)  U. of Utah DOE ASCI Level 1 subcontract  DARPA  NSF National Young Investigator (NYI) award