1. Teuchos: Utilities for Developers & Users November 4 th, 1:15-4:30pm Roscoe Bartlett Mike Heroux Kris Kampshoff Kevin Long Paul Sexton Heidi Thornquist Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.

2. What is Teuchos (TEF-hos)?  Lightweight, portable utility package  What’s new?  First release in Trilinos 4.0  Templated BLAS/LAPACK interface  Templated serial dense matrix / vector  Timing classes  Templated parameter list  Numeric traits mechanisms ( ordinal / scalar )  Memory management: Reference-counted pointers Pre-allocated workspace  Exception handling  Command-line parsing  “Extended” utilities: XML parsing MPI communicators / traits Users Developers

3. Templated Parameter List  Design based on NOX parameter list  Adopted by Amesos, ML, Ifpack, AztecOO, …  Parameters can be numeric datatypes, pointers to vectors or functions, and other parameter lists. // Empty parameter list Teuchos::ParameterList My_List; // Setting parameters My_List.set("Max Iters", 1550); My_List.set("Tolerance", 1e-10); My_List.set("Solver", "GMRES"); // Create sublist Teuchos::ParameterList& Prec_List = My_List.sublist("Preconditioner"); // Setting parameters in sublist Prec_List.set("Type", "ILU"); Prec_List.set("Drop Tolerance", 1e-3);  Parameters can be input and retrieved with templated “set” and “get” methods, as well as helper functions.  Parameter lists can be queried for attributes. // Has a solver been chosen? bool solver_defined = My_List.isParameter(“Solver”); // Has a preconditioner been chosen? bool prec_defined = My_List.isSublist(“Preconditioner”); // Has a double-precision drop tolerance been chosen? (Helper function) bool drop_tol = Teuchos::isParameterType (Prec_List, “Drop Tolerance”); ReturnType foo_solver( Teuchos::ParameterList& solverPL,… ) { // Getting parameters int max_iters = solverPL.template get("Max Iters"); std::string solver = getParameter(solverPL, “Solver”) double tol = solverPL.get("Tolerance", 1e-8); // Get sublist ( same as making sublist ) Teuchos::ParameterList precPL = solverPL.sublist("Preconditioner"); … return Ok; }  Parameters can be input and retrieved with templated “set” and “get” methods, as well as helper functions.

4. Numeric Traits Mechanisms  Assumed: addition, subtraction, multiplication, and division  OrdinalTraits  zero & one  int & long int  ScalarTraits  zero, one, magnitude type, absolute value, conjugate, square root, random number generator, …  std::numeric_limits  float, double, complex, and complex  Required: none, a compile-time check is performed.  Arbitrary precision arithmetic Generic programming technique that uses templated interfaces to define the standard behavior of datatypes.

5. Teuchos Templated BLAS Example template ScalarTraits ::magnitudeType BLAS ::NRM2 const OrdinalType n, const ScalarType* x, const OrdinalType incx) const { OrdinalType izero = OrdinalTraits ::zero(); OrdinalType ione = OrdinalTraits ::one(); ScalarType result = ScalarTraits ::zero(); OrdinalType i, ix = izero; if ( n > izero ) { // Set the initial index. if (incx < izero) { ix = (-n+ione)*incx; } for(i = izero; i < n; i++) { result += ScalarTraits ::conjugate(x[ix]) * x[ix]; ix += incx; } return ScalarTraits ::squareroot(result); } } /* end NRM2 */ double DNRM2( const int n, const double* x, const int incx) { int i, ix = 0; double result 0.0; if ( n > 0 ) { // Set the initial index. if (incx < 0) { ix = (-n+1)*incx; } for(i = 0; i < n; ++i) { result += x[ix] * x[ix]; ix += incx; } result = std::sqrt(result); } return result; } /* end NRM2 */

6. Arbitrary Precision Libraries ARPREC & GNU MP  ARPREC (Bailey, et. al.)  uses arrays of 64-bit floating-point numbers to represent high- precision floating-point numbers.  provides support for datatypes with differing precision levels.  --enable-teuchos-arprec  GNU MP  uses fullwords, arbitrary precision mantissa, and limited precision exponent to represent high-precision floating-point numbers.  platform specific kernels written in assembly language perform common inner-loop functions.  provides support for datatypes with differing precision levels.  --enable-teuchos-gmp  What about communication of arbitrary precision variables?

7. Primitive Type Traits  A traits class for decomposing an object into an array of primitive objects  ARPREC: primitive type = double  GNU MP: primitive type = int, byte(?)  Can be used in communicating more complex datatypes  Essential for Tpetra, TSFCore, …  Platform dependent issues here...

8. Memory Management Teuchos::RefCountPtr  Reference-counted pointers  combat seg faults, memory leaks, headaches...  RefCountPtr Intro Paper: $(TRILINOS_HOME)/doc/RefCountPtr/RefCountPtrBeginnersGuideSAND.tex http://software.sandia.gov/Trilinos/RefCountPtrBeginnersGuideSAND.pdf RefCountPtr a_null_ptr, // Create a reference-counted NULL pointer. a_ptr2 = rcp(new A), // Initialize reference-counted pointers. a_ptr3 = rcp(new A); // "" A *ra_ptr2 = new A, // Initialize non-reference counted pointers. *ra_ptr3 = new A; // "" a_ptr2 = rcp(ra_ptr3); // Assign from a raw pointer (only do this once!) a_ptr3 = a_ptr1; // Assign one smart pointer to another. a_ptr2->f(); // Access a member of A using -> ra_ptr2->f(); // "" *a_ptr2 = *a_ptr3; // Dereference the objects and assign. *ra_ptr2 = *ra_ptr3; // "" // Get the raw C++ pointer. A* true_ptr = a_ptr1.get();

9. Memory Management Teuchos::Workspace  Pre-allocated workspace (Teuchos::WorkspaceStore)  Avoid calls to new/delete with arrays of un-initialized or default initialized objects as automatic variables on the stack (global).  Calls new/delete by default.  Very useful in codes/functions/methods that are commonly used and riddled with creation/destruction of arrays: int* index = new int[ num_vecs ]; #include “Teuchos_Workspace.hpp” int main( int argc, char* argc[] ) { size = 100; Teuchos::set_default_workspace_store( Teuchos::rcp(new Teuchos::WorkspaceStoreInitializeable(10*size))); Teuchos::WorkspaceStore *wss_(Teuchos::get_default_workspace_store().get()); Teuchos::Workspace b(wss_,size,false); Teuchos::Workspace index(wss_,size,false); return 0; }

10. Memory Management Teuchos::Workspace Memory block size = 100 Number of call loops = 500000 Timing raw new and delete for temporaries... time = 0.589 sec Timing std::vector for temporaries... time = 0.657 sec Timing std::valarray for temporaries... time = 0.627 sec Timing Teuchos::Workspace for temporaries... time = 0.268 sec *** Statistics for automatic array workspace: Number of megabytes of preallocated workspace = 0.001 Number of megabytes needed = 0.0008 Number of allocations using preallocated workspace = 500000 Number of dynamic allocations beyond preallocated workspace = 0 Relative time (lower is better): raw new/delete = 2.19776 std::vector = 2.45149 std::valarray = 2.33955 Teuchos::Workspace = 1

11. Exception Handling TEST_FOR_EXCEPTION( n > 100, std::out_of_range, "Error, n = " << n << is bad" ); /home/bob/project/src/my_source_file.cpp:225: n > 100: Error, n = 125 is bad  TEST_FOR_EXCEPTION macro:  Macro for throwing an exception with breakpointing: #define TEST_FOR_EXCEPTION( throw_exception_test, Exception, msg ) #define TEST_FOR_EXCEPT( throw_exception_test )  DEVELOPER’S NOTE:  If there is an error with this macro... Did you use “-DHAVE_CONFIG_H” in your makefile? Consider using “trilinos_make_macros.mak”

12. Teuchos::Command Line Processor  Basic command line parser for input from (argc,argv[])  Facilitates organization of configurable executables/examples  Creates help string for command line options  Command line options can be standard data types / enumerated lists  Let’s look at an example...

13. Teuchos::CommandLineProcessor Example #include “Teuchos_CommandLineProcessor.hpp” int main(int argc, char* argv[] ) { // Creating an empty command line processor looks like: Teuchos::CommandLineProcessor My_CLP; // Set an integer command line option. int NumIters = 1550; My_CLP.setOption("iterations", &NumIters, "Number of iterations"); // Set a boolean command line option. bool Precondition; My_CLP.setOption("precondition","no-precondition", &Precondition,"Preconditioning flag");... // Allow an unrecognized option to be ignored (warning is printed) My_CLP.recogniseAllOptions(true); // Throw and exception if an option found is not recognized or My_CLP.throwExceptions(false); Teuchos::CommandLineProcessor::EParseCommandLineReturn parseReturn= My_CLP.parse( argc, argv ); if (parseReturn != PARSE_SUCCESSFUL) return 1; return 0; } $./CLP_example.exe --help Usage:./CLP_example [options] options: --help Prints this help message --pause-for-debugging Pauses for user input to allow attaching a debugger --iterations int Number of iterations (default: --iterations=1550) --tolerance double Tolerance (default: --tolerance=1e-10) --solver string Linear solver (default: --solver="GMRES") --precondition bool Preconditioning flag --no-precondition (default: --no-precondition) --speed enum Speed of our solver Valid options: "slow", "medium", "fast" (default: --speed="medium")

14. Teuchos Developers Overview  Parameter list is easy to use and quickly being adopted by most Trilinos packages.  Templated tools libraries are essential for the development of templated solver libraries.  Numeric traits mechanism allows the use of arbitrary datatypes in templated code.  Templated BLAS/LAPACK wrappers and SDM allow easy integration of arbitrary precision for high-precision kernels.  Teuchos tools provide portable solutions for developers.  See Teuchos website for more info: http://software.sandia.gov/Trilinos/packages/teuchos