1. DOE/Office of Science/ASCR (Sandia National Laboratories)
PyTrilinos: A Python Interface to Parallel, Object-Oriented Solver Packages for Scientific Computing
Bill Spotz
DOE/Office of Science/ASCR
(Sandia National Laboratories)
SIAM Annual Meeting 2008
July 10, 2008

2. Computational Sciences at Sandia
Chemically reacting flows
Climate modeling
Combustion
Compressible flows
Computational biology
Electrical modeling
Heat transfer
Load balancing
Materials modeling
MEMS modeling
Mesh generation
Optimization and uncertainty quantification
Seismic imaging
Shock and multiphysics
Structural dynamics

3. The Trilinos Project
Provide a central repository for Sandia’s solver technology
Increase code-reuse
Organized on concept of “packages”
Minimize package interdependence
Maximize package interoperability
Provide a framework for SQE and SQA
Compliance with requirements
Nightly test harness
High degree of developer autonomy
Open source: GNU Lesser General Public License
Web site:
Next release: Version 9.0, September, 2008
Trilinos Users Group Meeting every November

4. The Trilinos Project Linear Algebra Services Tools & Utilities
Communicators, maps, vectors, matrices, graphs…
Linear Algebra
Services
Complex interface
to AztecOO
Direct, sparse
Tools &
Utilities
Java
Sparse kernels
Iterative, sparse
Reduction, transformation operators
Templated
Direct, dense
Iterative, sparse,
next generation
Extensions
Generic interface
Linear Solvers
Epetra
Jpetra
Tpetra
EpetraExt
Kokkos
RTOp
Nonlinear Solvers
Primary tools package
Teuchos
TriUtils
Galeri
Thyra
Isorropia
Moertel
Didasko
PyTrilinos
WebTrilinos
New_Package
Amesos
AztecOO
Belos
Komplex
Pliris
Stratimikos
Common testing tools
NOX
Matrix gallery
Common interface
Meta-Solvers
Partitioning & load balance
Eigensolvers
Anasazi
Rythmos
LOCA
MOOCHO
Mortar elements
Time-stepping algorithms
Trilinos examples
Continuation algorithms
Python Interface
Preconditioners
Constrained optimization
Web interface
Package template
Incomplete factorization
IFPACK ML
Claps
Meros
Multi-level
Domain decomposition
Segregated, block

5. The Trilinos Project
Recently, the scope of Trilinos has been expanded to include:
Meshing
Numerical methods
Finite element interface
Dynamic load balancing
Automatic differentiation
Probably others…

6. The Interoperability Problem
Ross has talked about the need for (and work on) a common abstract interface
Interfaces between Trilinos packages
Interfaces between Trilinos and external packages
We would like to extend this concept to the python wrappers
Interoperability with external python projects, such as SLIMpy
Many design decisions still to be made
Some early prototyping work has been done

7. PyTrilinos Linear Algebra Services Linear Solvers Preconditioners
Epetra (with extensive NumPy compatibility and integration)
EpetraExt (coloring algorithms and some I/O)
Linear Solvers
Amesos (LAPACK, KLU, UMFPACK, ScaLAPACK, SuperLU, SuperLUDist, DSCPACK, MUMPS)
AztecOO
Preconditioners
IFPACK ML
Nonlinear Solvers
NOX
Meta-Solvers
LOCA (python wrappers not yet caught up to recent redesigns)
Anasazi
Tools and Utilities
Teuchos (ParameterLists, communicators, traits and refcount pointers)
TriUtils
Galeri
Thyra (early development stage)

8. PyTrilinos Design
PyTrilinos is a “python package,” meaning a python module with sub-modules
Each “Trilinos package” translates into a PyTrilinos sub-module
Trilinos package namespaces also become sub-modules
PyTrilinos python interfaces match Trilinos C++ interfaces (within reason)
Generated by SWIG (simple wrapper interface generator)
Some classes made more “pythonic”

9. PyTrilinos Documentation
Trilinos documentation is handled by doxygen: special comments within code
Web pages updated twice daily
C++ doxygen comments are now automatically converted to python doc strings:
>>> from PyTrilinos import Epetra
>>> help(Epetra.Vector)
Returns standard Epetra_Vector documentation

10. PyTrilinos.Epetra Communicators Comm SerialComm MpiComm PyComm Maps
BlockMap
Map
LocalMap
Vectors
MultiVector
Vector
IntVector
SerialDense objects
SerialDenseOperator
SerialDenseMatrix
SerialDenseVector
SerialDenseSolver
IntSerialDenseMatrix
IntSerialDenseVector
Graphs
CrsGraph
Operators
Operator
RowMatrix
CrsMatrix

11. A Quick Detour… Python lists are not suitable for scientific computing
Flexible but inefficient
Heterogeneous data, noncontiguous memory
NumPy module provides needed functionality
Contiguous, homogeneous n-dimensional arrays
High-level interface
Part of SciPy
SciPy is a large, open source package for a wide variety of python interfaces to scientific software:
NetLib’s “greatest hits”

12. PyTrilinos.Epetra and NumPy
Array-like classes inherit from numpy.UserArray
MultiVector
Vector
IntVector
SerialDenseMatrix
SerialDenseVector
IntSerialDenseMatrix
IntSerialDenseVector
Methods throughout Epetra have arguments that accept or produce pointers to C arrays
Python input arguments accept python sequences
Python output arguments produce ndarrays

13. PyTrilinos.Teuchos Teuchos::ParameterList
Used by several Trilinos packages to set problem parameters
Maps string names to arbitrary-type values
Python implementation allows dictionary substitutions
The following conversions are supported:
Python
Dir
C / C++
bool 
int
float
double
string
std::string 
char *
dict 
ParameterList
wrapped ParameterList

14. PyTrilinos Demonstration
Governing equation:
Boundary conditions:
Exact solution:
CDS:
Oscillations:

15. Parallelism
Configure Trilinos with –enable-mpi, and PyTrilinos will also be mpi-enabled
Scripts are run in parallel in the standard way:
$ mpirun -np 4 python script.py …
When Epetra is imported, MPI_Init() is called automatically and MPI_Finialize() is registered with the atexit module.
Use the PyComm() function to obtain a communicator, and use a parallel programming paradigm:
Global and local IDs
Ghost nodes if necessary
Epetra Import and Export objects if needed

16. Conclusions Python lets developers focus on the problem
Memory management, garbage collection
Powerful, flexible containers
Clean, readable syntax
PyTrilinos provides access to powerful solver technologies
Rapid prototyping
Application development
Serial, parallel