TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National.

TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National Laboratories Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.

Outline Introduction of abstract numerical algorithms (ANAs) Background on Unified Modeling Language (UML) and “Design Patterns” Overview of TSF interfaces and software Discussion of different approaches for interoperability between Trilinos packages General recommendations and open questions related to TSF and Trilinos

Categories of Abstract Problems and Abstract Algorithms · Linear Problems: · Linear equations: · Eigen problems: · Nonlinear Problems: · Nonlinear equations: · Stability analysis: · Transient Nonlinear Problems: · DAEs/ODEs: · Optimization Problems: · Unconstrained: · Constrained: Trilinos Packages Belos Anasazi NOX LOCA Rhythms, TOX ??? MOOCHO

Common Environments for Scientific Computing Serial / SMP (symmetric multi-processor) All data stored in RAM in a single local process Code Data Processor Out of Core Data stored in file(s) (too big to fit in RAM) ProcessorDisk CodeData SPMD (Single program multiple data) Same code on each processor but different data Code Data(0) Proc 0 Code Data(1) Code Data(N-1) MPP (massively parallel processors) … Proc 1 Proc N-1

Introducing Abstract Numerical Algorithms An ANA is a numerical algorithm that can be expressed abstractly solely in terms of vectors, vector spaces, and linear operators (i.e. not with direct element access) Example Linear ANA (LANA) : Linear Conjugate Gradients scalar product defined by vector space vector-vector operations linear operator applications Scalar operations Types of operationsTypes of objects What is an abstract numerical algorithm (ANA)? Linear Conjugate Gradient Algorithm

Examples of Nonlinear Abstract Numerical Algorithms Class of Numerical ProblemExample ANA Nonlinear equations Newton’s method (e.g. NOX, MOOCHO) Initial Value DAE/ODE Backward Euler method (e.g. Rhythms, TOX???) Linear equations GMRES (e.g. Belos) Requires

Unified Modeling Language (UML) Good tutorial on object-orientation development process and short (about 200 pages) introduction to the UML. Unified Modeling Language · Graphical language for designing, documenting and communicating designs object-oriented software · Standardized by the Object Management Group (OMG) and backed by a large group of software companies · Allows the specification and description of object-oriented software at various levels of detail · Supported by many different CASE tools for · Designing OO software · Documenting OO software · Reverse engineering OO software · Creating skeleton code for OO software · Refactoring OO software

eFunc(in c: C) class BaseA { public: virtual ~BaseA() {} virtual int aFunc() const = 0; virtual C& c() = 0; … }; class DerivedB : public BaseA { public: … private: std::vector d_; }; class C { … }; class BaseD { … }; class DerivedE : public BaseE { public: void eFunc( const C& c ); … }; DerivedB aFunc(): int Overview of Basic UML Notation for Object Orientation (OO) BaseAC c DerivedE d 1…* BaseD Simple UML Class Diagram Compatible C++ declarations Inheritance Abstract class (italics) Concrete class (non-italics) Abstract member function (italics) Association Member function implementation (non-italics) Association Multiplicity Classes Relationshipss Dependency Abstract class and inheritance (italics)

Design Patterns : “Standard” Vocabulary for OO Some of the Design Patterns used in TSF · Creational Patterns · “Abstract Factory” · “Prototype” · Structural Patterns · “Adapter” · “Composite” · “Decorator” · “Proxy” · Behavior Patterns · “Strategy” · “Visitor” Design Patterns in UML Notation: http://exciton.cs.oberlin.edu/javaresources/DesignPatterns/default.htm Good tutorial on object-orientation and a catalog of 23 fundamental design patterns of object-oriented software “Common problems have common solutions!”

Software Componentization and TSF Interfaces 2) LAL : Linear Algebra Library (e.g. vectors, sparse matrices, sparse factorizations, preconditioners) ANA APP ANA/APP Iterface ANA Vector Interface ANA Linear Operator Interface 1) ANA : Abstract Numerical Algorithm (e.g. linear solvers, eigen solvers, nonlinear solvers, stability analysis, uncertainty quantification, transient solvers, optimization etc.) 3) APP : Application (the model: physics, discretization method etc.) Example Trilinos Packages: Belos (linear solvers) Anasazi (eigen solvers) NOX (nonlinear equations) Rhythms, TOX??? (ODEs,DAEs) MOOCHO (Optimization) … Example Trilinos Packages: Epetra/Tpetra (Mat,Vec) Ifpack, AztecOO, ML (Preconditioners) Meros (Preconditioners) Pliris (Interface to direct solvers) Amesos (Direct solvers) Komplex (Complex/Real forms) … Types of Software Components TSFCore ANA Interfaces to Linear Algebra TSFExtended APP to LAL Interfaces TSFExtended LAL to LAL Interfaces TSFCore::Nonlin ??? Examples: SIERRA NEVADA Xyce Sundance … LAL MatrixPreconditioner Vector ANA software allows for a much more abstract interface than APP or LAL software!

Different Platform Configurations for Running ANAs SPMD ANA LAL Proc 0 APP MPP ANA LAL Proc 1 APP ANA LAL Proc N-1 APP … Master/Slave ANA LAL Proc 0 (master) APP MPP LAL Proc 1 APP … Client/Server Master/Slave LAL Proc N-1 APP MPP Server LAL Proc 1 APP … LAL Proc N-1 APP LAL Proc 0 APP Client ANA If a code can not run in these modes it is not an ANA code!

Requirements for Abstract Numerical Algorithms and TSF An important consideration  Scientific computing is computationally expensive! Abstract Interfaces for Abstract Numerical Algorithms using TSF must: Be portable to all ASC (advanced scientific computing) computer platforms Provide for stable and accurate numerics Result in algorithms with near-optimal storage and runtime performance –Scientific computing is expensive! significant An important ideal  A customized hand-code algorithm in Fortran 77 should not provide significant improvements in storage requirements, speed or numerical stability! Abstract Interfaces for Abstract Numerical Algorithms using TSF should: Be minimal but complete (good object-oriented design principle) Support a variety of computing platforms and configurations –i.e. Serial/SMP, Out-of-Core, SPMD, master/slave and client/server Be (relatively) easy to provide new implementations (Subjective!!!) Not be too complicated (Subjective!!!) An important ideal  Object-oriented “overhead” should be constant and not increase as the problem size increases.

Fundamental TSFCore Linear Algebra Interfaces An operator knows its domain and range VectorSpaces A MultiVector has a collection of column vectors A MultiVector is a linear operator A Vector is a MultiVector and therefore is a linear operator A Vector knows its VectorSpace A linear operator can be applied to Vectors and MultiVectors A linear operator is a kind of operator Trilinos TSFCore website

Fundamental TSFCore Linear Algebra Interfaces VectorSpaces create Vectors and MultiVectors! Vector and MultiVector versions of apply(…) must agree with scalar product: Adjoints supported but are optional! Only one vector operation! Scalar product defined by vector space Creates domain spaces for MultiVector objects Interface that supports all element-wise vector reduction and transformation operations! VectorSpace defines compatibility of vectors and multi-vectors Creation of arbitrary subviews of columns

What’s the Big Deal about Vector-Vector Operations? Examples from OOQP (Gertz, Wright) Example from TRICE (Dennis, Heinkenschloss, Vicente) Example from IPOPT (Waechter) Currently in MOOCHO : > 40 vector operations! Many different and unusual vector operations are needed by interior point methods for optimization!

Vector Reduction/Transformation Operators Defined Reduction/Transformation Operators (RTOp) Defined z 1 i … z q i  op t ( i, v 1 i … v p i, z 1 i … z q i ) element-wise transformation   op r ( i, v 1 i … v p i, z 1 i … z q i ) element-wise reduction  2  op rr (  1,  2 ) reduction of intermediate reduction objects v 1 … v p  R n : p non-mutable (constant) input vectors z 1 … z q  R n : q mutable (non-constant) input/output vectors  : reduction target object (many be non-scalar (e.g. {y k,k}), or NULL) Key to Optimal Performance op t (…) and op r (…) applied to entire sets of subvectors (i = a…b) independently: z 1 a:b … z q a:b,   op( a, b, v 1 a:b … v p a:b, z 1 a:b … z q a:b,  ) Communication between sets of subvectors only for   NULL, op rr (  1,  2 )   2 RTOpT apply_op(in sv[1..p], inout sz[1…q], inout beta ) ROpDotProd apply_op(… ) TOpAssignVectors apply_op(… ) … Vector applyOp( in : RTOpT, apply_op(in v[1..p], inout z[1…q], inout beta) SerialVectorStd applyOp( … ) MPIVectorStd applyOp( … ) Software Implementation (see RTOp paper on Trilinos website “Publications”) “Visitor” design pattern (a.k.a. function forwarding) …

Technicalities about TSFCore Software · All interfaces are templated on Scalar type · Currently supported types: real (float, double), complex (std::complex, std::complex ), extended precision (GNU MP mpf_class) · Teuchos::ScalarTraits<> used for all math functions · Teuchos::RawMPITraits<> and Teuchos::PrimativeTypeTraits<> used for MPI · Smart reference-counted pointer class Teuchos::RefCountPtr<> used for all dynamic memory management · C++ exception handling mechanism used to throw exceptions derived from std::exception  TSFCore uses const everywhere religiously often resulting in const and non- const versions of many operations · Many operations have default implementations based on very few pure virtual methods · RTOpT operators (and wrapper functions) are provided for many common level-1 vector and multivector operations · Supported platforms (Includes all SIERRA and ICC platforms): Yey Teuchos! · Linux/g++: g++ 3.1 … 3.3.x, software.sandia.gov, CS LAN · DEC: Compaq C++ V6.5-014, stratus.sandia.gov, CS LAN · Linux/Intel: Intel C++ version 7.1, liberty.sandia.gov, ICC · Sun: Forte Developer 7 C++ 5.4 Patch 111715-14, sass3276a.sandia.gov, EngSci LAN · SGI: MIPSpro Compilers: Version 7.4.2m, sasg1099.sandia.gov, EngSci LAN · IBM: VisualAge C++ Professional for AIX, Version 7, saia028.sandia.gov, EngSci LAN (not yet) · Janus (ASC Red): ciCC = pgCC Rel 3.1-4i, sasn100.sandia.gov, EngSci LAN

TSFCore Interface C++ Specification (pure virtual functions) template class VectorSpace { public: virtual Index dim() const = 0; virtual bool isCompatible( const VectorSpace & vecSpc ) const = 0; virtual Teuchos::RefCountPtr > createMember() const = 0; … }; template class OpBase { public: virtual Teuchos::RefCountPtr > domain() const = 0; virtual Teuchos::RefCountPtr > range() const = 0; virtual bool opSupported(ETransp M_trans) const; }; template class LinearOp : virtual public OpBase { public: virtual void apply( const ETransp M_trans, const Vector &x,Vector *y,const Scalar alpha=1.0,const Scalar beta=0.0 ) const = 0; … }; template class MultiVector : virtual public LinearOp { public: virtual Teuchos::RefCountPtr > col(Index j) = 0; … }; template class Vector : virtual public MultiVector { public: virtual Teuchos::RefCountPtr > space() const = 0; virtual void applyOp( const RTOpPack::RTOpT &op, const size_t num_vecs,const Vector * vecs[], const size_t num_targ_vecs,Vector * targ_vecs[],RTOpPack::ReductTarget *reduct_obj,const Index first_ele,const Index sub_dim,const Index global_offset ) const = 0; … }; Shows only pure virtual functions on interfaces that must be implemented Default MultiVector implementation based on a Vector implementation Default Serial Vector, VectorSpace and VectorSpaceFactory implementation for domain space of MultiVectors for default MultiVectors Only two nontrivial functions to override if MultiVectors not needed! – LinearOp::apply(…) – Vector::applyOp(…) Providing an optimized MultiVector implementation requires overriding more functions However, most concrete subclasses should not derive from these base cases directly!

TSFCore Interface C++ Specification (explicit element access) template class MultiVector : virtual public LinearOp { public: … virtual void getSubMultiVector( const Range1D &rowRng, const Range1D &colRng,RTOpPack::SubMultiVectorT *sub_mv ) const; virtual void freeSubMultiVector( RTOpPack::SubMultiVectorT * sub_mv ) const; virtual void getSubMultiVector( const Range1D &rowRng, const Range1D &colRng,RTOpPack::MutableSubMultiVectorT *sub_mv ); virtual void commitSubMultiVector( RTOpPack::MutableSubMultiVectorT * sub_mv ); … }; template class Vector : virtual public MultiVector { public: … virtual void getSubVector( const Range1D& rng, RTOpPack::SubVectorT * sub_vec ) const; virtual void freeSubVector( RTOpPack::SubVectorT * sub_vec ) const; virtual void getSubVector( const Range1D& rng, RTOpPack::MutableSubVectorT * sub_vec ); virtual void commitSubVector( RTOpPack::MutableSubVectorT * sub_vec ); virtual void setSubVector( const RTOpPack::SparseSubVectorT & sub_vec ); … }; Explicit sub-views are requested and released in separate function calls (allows creation of temporary storage data with copies) All have default implementations in terms of advanced RTOp subclasses In general, explicit access to Vector and MultiVector elements is an extremely bad thing to do (i.e. client/server), however: Use cases where explicit element access is needed or is convenient –Debugging small problems (i.e. printing Vector or MultiVector elements) –Vectors and MultiVectors created from the domain space of a MultiVector (i.e. block GMRES) –Automatic interoperability of serial and MPI SPMD Vector and MultiVector objects.

Three Use Cases for Fundamental TSFCore Interfaces 1. TSFCore as interoperability layer for linear ANA objects (i.e. linear operators, vectors, multivectors, reduction/transformation operators) 2. Development of concrete TSFCore implementation subclasses 1. Support for decoupling application-defined scalar products form concrete vector spaces 2. Support for concrete implementations 1. Support and concrete subclasses for serial shared-memory platforms 2. Support and concrete subclasses for MPI SPMD distributed memory platforms 3. Concrete “adatper” subclasses for Epetra objects 3. Using TSFCore objects for the development of ANAs 1. C++ wrapper functions for prewritten RTOpT subclasses e.g. norm(x), assign(&y,x) … 2. Composite objects: 1. Product vectors, multi-vectors and vector spaces e.g. x = [ v1; v2; v3; … ; vn ]; 2. Composite linear operators 1. Additive linear operators: e.g. M = A + B + C + D; 2. Multiplicative linear operators: e.g. M = A * B * C * D; 3. Block linear linear operators: e.g. M = [ A, B ; C, D ]; 3. Handle classes for operator overloading (currently in TSFExtended) e.g. y = alpha*trans(A)*x + gamma*B*z + beta*y Trilinos TSFCore website

TSFCore Subclasses for Serial and MPI Platforms · Support base classes for serial vector space, vector and multi-vector subclasses · SerialVectorSpaceBase, SerialVectorBase, SerialMultiVectorBase · Provide complete implementation of all functions (utilizes level-3 BLAS from Teuchos) in terms of explicit element access provided by subclasses · Universal interoperability of Serial Vectors and MultiVectors:  All Vector and MultiVector objects that return VectorSpace objects with isInCore()==true and dim()==dim() are automatically compatible through explicit element sub-views! · Standard concrete implementations of serial vector space, vector and multi-vector subclasses · SerialVectorSpaceStd, SerialVectorStd, SerialMultiVectorStd · Provide general implementations that are very efficient default implementations · Note: Most serial applications only need to implement linear operators (e.g. derived from SerialLinearOpBase) · Support base classes for MPI SPMD vector space, vector and multi-vector subclasses · MPIVectorSpaceBase, MPIVectorBase, MPIMultiVectorBase · Provide complete implementation of all functions (utilizes level-3 BLAS from Teuchos) in terms of explicit element access provided by subclasses · Universal interoperability of MPI SPMD Vectors and MultiVectors:  All Vector and MultiVector objects that return VectorSpace objects that support the MPIVectorSpaceBase interface with same numbers of local elements are automatically compatible through explicit element sub-views! · Standard concrete implementations of MPI SPMD vector space, vector and multi-vector subclasses · MPIVectorSpaceStd, MPIVectorStd, MPIMultiVectorStd · Provide general implementations that are very efficient default implementations · Note: Most MPI applications only need to implement linear operators (e.g. derived from MPILinearOpBase)

TSF Composite ANA Linear Algebra Subclasses VectorSpace blocks 1…m ProductVectorSpace Vector blocks 1…m ProductVector > “Composite” subclasses allow a collection of objects to be manipulated as one object · Product vector spaces and product vectors: · Product vector spaces: · Product vectors: · Block linear operator: · Multiplicative (composed) linear operator: · Additive (summation) linear operator: blockRange blockDomain LinearOp BlockLinearOp blocks 1…* MultiplicativeLinearOp 1…* AdditiveLinearOp 1…*

Extending Functionality Beyond TSFCore Question: How to extend functionality beyond TSFCore interfaces? Examples of extended functionality · Explicit matrix transposition: i.e. A = A T · Preconditioner creation: i.e. A  P, where P  A -1 · LinearOp-LinearOp multiplication: i.e. C = A * B, where C is formed explicitly Possible approaches to extend functionality in an ANA · Extended mix-in interfaces inherited by concrete linear algebra subclasses · Explicit matrix transposition : e.g. TSFExtended::ExplicitlyTransposableOp · Preconditioner creation: e.g. TSFExtended::ILUFactorizableOp · LinearOp-LinearOp multiplication: e.g. through TSFExtended::RowAccessableOp (Not an ANA!) · “Strategy” interfaces and subclasses that are external from linear algebra subclasses · Example: Some ANA needs LinearOp-LinearOp multiplication C = A * B with A and B being Epetra_VbrMatrix objects with large element blocks (level-3 BLAS) and with output C being a Epetra_CrsMatrix (EpetraExt?)! TSFCore::LinearOp TSFCore::EpetraLinearOp Epetra_Operator Epetra_CrsMatrixEpetra_VbrMatrix LinearOpMultiplierStrategy EpetraVbrToCrsMultiplier SomeANA multiply( in A, in B, out C) Strategy interfaces offer the greater flexibility, better efficiency and preserve ANAs!

Dependencies between TSF-related Software “ANA to linear algebra” handles for operator overloading (e.g. y = A*x) (currently in TSFExtended) “ANA to linear algebra” composite subclasses (e.g. ProductVector) (currently in TSFCore and TSFExtended) “ANA to linear algebra” concrete implementation support subclasses (e.g. MPIVectorBase) (currently in TSFCore) Interfaces to steady-state nonlinear applications (currently in TSFCore::Nonlin, similar to NOX Group) Interfaces to iterative linear solvers (currently in Belos?) Interfaces to transient nonlinear applications (currently do not exist?) Various extended “strategy” interfaces (???) ??? “ANA to linear algebra” Interfaces (e.g. Vector, LinearOp) (currently in TSFCore) Extended APP-LAL and LAL-LAL interfaces to linear algebra objects (e.g. LoadableVector) (currently in TSFExtended) Extended APP-LAL and LAL-LAL handles (currently in TSFExtended) ???

Interfacing Packages to Implementations : Everyone for themselves! Package 1 Package Interface 1 Package Interface 1 / Implementation 1 adapter … Implementation 1 Implementation 2 Package Interface 1 / Implementation 2 adapter Package Interface 1 / Implementation N adapter Implementation N … Package 2 Package Interface 2 Package Interface 2 / Implementation 1 adapter Package Interface 2 / Implementation 2 adapter Package Interface 2 / Implementation N adapter Package M Package Interface M Package Interface M / Implementation 1 adpater Package Interface M / Implementation 2 adapter Package Interface M / Implementation N adapter ……… - Oh no, M * N adapter subclasses needed! - This is not a “scalable” approach! Package == ANA

Interfacing Packages to Implementations : Standard Interfaces (I) Package 1 Package Interface 1 … Implementation 1 Implementation 2 Implementation N … Package 2 Package Interface 2 Package M Package Interface M + Only M + N adapter subclasses needed! + This is a “scalable” approach! Std LANA Interface Std LANA Interface / Implementation 1 adapter Std LANA Interface / Implementation 2 adapter Std LANA Interface / Implementation N adapter Package Interface 1 / Std LANA Interface adapter Package Interface 2 / Std LANA Interface adapter Package Interface M / Std LANA Interface adapter …

Interfacing Packages to Implementations : Standard Interfaces (II) Package 1 … Implementation 1 Implementation 2 Implementation N … Package 2Package M + Only N adapter subclasses needed! + Less interface documentation, shared testing code Std LANA Interface Std LANA Interface / Implementation 1 adapter Std LANA Interface / Implementation 2 adapter Std LANA Interface / Implementation N adapter …

Interfacing Packages to Packages : Everyone for themselves! Package 1 Package Interface 1 … Package 2 Package Interface 2 Package Interface 2 / Package Interface 1 adapter Package 3 Package Interface 3 Package Interface 3 / Package Interface 2 adapter Package Interface 3 / Package Interface 1 adapter - Major assumption: “Package Interface 1” satisfies the requirements of “Package Interface 2”? Package M - Oh no, as many as 1 + 2 + 3 + … + M-1 = O(M 2 ) adapter subclasses needed if only one-way interactions! Concrete Examples: 1) NOX/LOCA interfaces had to be extended to support Belos interfaces 2) NOX/LOCA vector interface does not support MOOCHO vector interface Package Interface M … - This is not a “scalable” approach!

Interfacing Packages to Packages : Standard Interfaces (I) Package Interface 1 … Package 2 Package Interface 2 Implementation 1 Std LANA Interface Std LANA Interface / Implementation 1 adapter Package Interface 1 / Std LANA Interface adapter Implementation 2 Std LANA Interface / Implementation 2 adapter Extended LANA Interface 1 Package Interface 2 / Std LANA Interface adapter Package Interface 1 / Std LANA Interface adapter … + Only M adapters need + Each Package knows nothing of other packages - Result: Destroys ability to dynamic cast to “Extended LANA Inteface1” ! Package M - Major assumption!!! “Package Interface 1” supports functionality of “Std LANA Interface”? Concrete Example: Belos::BlockGmres using Epetra adapters defines a TSFCore::LinearOp subclass to be used as a preconditioner in another package that also uses Epetra adapters. Package 1

Concrete Example Problem : Belos, Epetra and TSFCore TSFCore::LinearOp TSFCore::MultiVector TSFCore::VectorSpace Belos::Operator TSFCore::EpetraLinearOp TSFCore::MPIVectorSpaceBase Belos::TSFCoreOperator Belos::TSFCoreMultiVec TSFCore::MPIMultiVectorBase Belos::BlockGmres > The problem: TSFCore::BelosLinearOp::range() returns a TSFCore::BelosVectorSpace object The TSFCore::BelosVectorSpace object creates a TSFCore::BelosMultiVector object MPIMultiVectorBase::applyOp( … ) is called on EpetraMultiVector object passing in the TSFCore::BelosMultiVector object TSFCore::BelosMultiVector::range() is called which returns a TSFCore::BelosVectorSpace object The dynamic cast from TSFCore::BelosVectorSpace to TSFCore::MPIVectorSpaceBase fails  TSFCore::BelosMultiVector TSFCore::BelosVectorSpace range, domain The scenario: Create a MultiVector object use the range space of a TSFCore::BelosLinearOp object and assign it to a MultiVector object created from a TSFCore::EpetraVectorSpace. range, domain TSFCore::EpetraVectorSpace TSFCore::EpetraMultiVector > Belos::MultiVec TSFCore::BelosLinearOp The Delema: What to use for range and domain spaces? Define TSFCore::VectorSpace only in terms of Belos interfaces!

Concrete Example Solution : Belos, Epetra and TSFCore TSFCore::LinearOp TSFCore::MultiVector TSFCore::VectorSpace Belos::Operator TSFCore::EpetraLinearOp TSFCore::MPIVectorSpaceBase Belos::TSFCoreOperator Belos::TSFCoreMultiVec TSFCore::MPIMultiVectorBase Belos::BlockGmres The solution: TSFCore::BelosLinearOp::range() returns a TSFCore::EpetraVectorSpace object The TSFCore::EpetraVectorSpace object creates a TSFCore::EpetraMultiVector object MPIMultiVectorBase::applyOp( … ) is called on EpetraMultiVector object passing in the TSFCore::EpetraMultiVector object TSFCore::EpetraMultiVector::range() is called which returns a TSFCore::EpetraVectorSpace object The dynamic cast from TSFCore::EpetraVectorSpace to TSFCore::MPIVectorSpaceBase succeeds The scenario: Create a MultiVector object use the range space of a TSFCore::BelosLinearOp object and assign it to a MultiVector object created from a TSFCore::EpetraVectorSpace. range, domain TSFCore::EpetraVectorSpace TSFCore::EpetraMultiVector > Belos::MultiVec The Delema: What to use for range and domain spaces? Define TSFCore::VectorSpace in terms of underlying TSFCore::VectorSpace! TSFCore::BelosLinearOp

Interfacing Packages to Packages : Standard Interfaces (II) Package Interface 1 … Package 2 Package Interface 2 Implementation 1 Std LANA Interface Std LANA Interface / Implementation 1 adapter Package Interface 1 / Std LANA Interface adapter Implementation 2 Std LANA Interface / Implementation 2 adapter Extended LANA Interface 1 Package Interface 2 / Std LANA Interface adapter … + Fewer new adapters needed + Dynamic casting of most objects maintained Package Interface 1 / Std LANA Interface adapter The Recommended Practice! Package 1

Pros and Cons of Standard Interfaces Reasons to adopt “standard” interfaces “Automatic” interoperability only comes through “standard” interfaces Only way to guarantee interoperability is even possible (without revisions) One set of documentation Common set of unit tests for common interface objects Reasons not to adopt “standard” interfaces Greater package autonomy / fewer dependencies Fewer layers of indirection within a single package Less sophisticated object-oriented concepts within a single package

Recommendations for TSF and Trilinos Goal: Adopt “standard interfaces” but minimize dependencies Minimize “standard” interface code needed for interoperability Encourage use of “standard” interfaces but allow packages to keep their own interfaces as local APIs All ASC/internal uses of Trilinos packages should adhere to “standard interfaces” Encourage academic and external collaborators to use “standard interfaces” but offer local package interfaces as an alternative Partition “standard” interface code as much as possible to minimize dependencies: Linear ANA interfaces (currently in TSFCore) SPMD Matrix/Vector loading and accessing (currently in TSFExtended) Interfaces to linear solvers (being developed in Belos) Interfaces to steady-state nonlinear problems (currently in TSFCore::Nonlin) Interfaces to transient non-linear problems (not yet developed) Separate “convenience” code from “standard interface” code Composite LANA objects (i.e. product vectors) (currently in TSFCore) Composite APP/LAL objects (currently in TSFCore and TSFExtended) Handle classes and operator overloading (currently in TSFExtended)

Open Questions related to TSF How do we make it “easiest” to develop concrete TSF interface implementations of vector spaces, vectors, multi-vectors and linear operators? Is it reasonable to expect beginner C++ programmers to debug though all TSF code? Analogy: Should beginning Microsoft Windows users be able to debug though Windows code when something goes wrong? Example: STL is good but difficult to impossible to debug through With that said, is the code in TSFCore and TSFExtended too complicated for semi-experienced C++ developers to debug through? What type of documentation for TSF is required beyond what is already available? How should the code in TSF be organized in to packages and managed in packages? Should all linear algebra interfaces be put into a single package? Should handles for operator overloading automatically include matrix/vector loading and extraction interfaces? Should certain types of code be allowed to be disabled at configure time?

The End Let the Discussion Begin!

Extra Slides

TSFCore Support for APP-specific Scalar Products Linear operators must “obey” application-specific scalar product · Specifically, for the linear operator (i.e. LinearOp, MultiVector etc.): · the following adjoint relation must hold: Goal of TSFCore support subclasses => Separate the definition of application-specific scalar product from data-structure and computing platform concrete implementations of vector spaces, vectors, multi-vectors and linear operators as much as possible. VectorSpace LinearOp ScalarProd EuclideanScalarProdLinearOpScalarProd rangeScalarProdVecSpc, domainScalarProdVecSpc ScalarProdVectorSpaceBase EuclideanLinearOpBase op AppSpecificScalarProd … ConcreteMultiVector … ConcreteVectorSpace …

Different Approaches for Developing Concrete Subclasses Approach 1: Define “simpler” interfaces and include standard “adapter” subclasses Example: NOX::Abstract::Group, NOX::Epetra::Group and NOX::Epetra::Interface Approach 2: Define templated adapter subclasses with “simpler” traits classes Example: fei::Vector, snl_fei::Vector<> and snl_fei::VectorTraits<> Approach 3: Define support base classes that override “general” functions in terms of “simpler” pure virtual functions which adapter subclasses define Currently used by TSFCore => Example: TSFCore::SerialVectorBase, TSFCore::MPIVectorBase Advantages Provides tailored “simple” set of functions to override Subclasses have full control to override every virtual function, not just “simpler” functions Disadvantages Subclass developers will see the “general” functions even though they should, in most cases, ignore them General Vector … Serial Vector Implementation 1 General Serial Vector Base Serial Vector Implementation 2 MPI Vector Implementation 1 MPI Vector Implementation 2 General MPI Vector Base Serial Vector Implementation 1 Adapter Serial Vector Implementation 2 Adapter MPI Vector Implementation 1 Adapter MPI Vector Implementation 2 Adapter … … … …

Different Approaches for Developing Concrete Subclasses Approach 1: Define “simpler” interfaces and include standard “adapter” subclasses Example: NOX::Abstract::Group, NOX::Epetra::Group and NOX::Epetra::Interface Advantages Provides tailored “simple” interfaces for subclass development for specific categories of implementations Subclass developers never even see “general” interface Disadvantages Only operations in “simple” interface can be overridden, not in “general” interface. General Vector … Serial Vector Implementation 1 General Serial Vector Adapter Serial Vector Implementation 2 MPI Vector Implementation 1 MPI Vector Implementation 2 General MPI Vector Adapter Simple Serial Vector Base Simple MPI Vector Base Serial Vector Implementation 1 Adapter Serial Vector Implementation 2 Adapter MPI Vector Implementation 1 Adapter MPI Vector Implementation 2 Adapter … … … …

Different Approaches for Developing Concrete Subclasses Approach 2: Define templated adapter subclasses with “simpler” traits classes Example: fei::Vector, snl_fei::Vector<> and snl_fei::VectorTraits<> Advantages Provides tailored “simple” traits interfaces and does not require writing subclasses Less subclasses than Approach 1 Subclass developers never even see “general” interface Disadvantages Only operations in “simple” traits interface can be overridden, not in “general” interface Templated traits may or may not be more complicated than using general inheritance Traits can not maintain state for a particular object while general subclasses can General Vector Serial Vector Implementation 1 General Serial Vector Adapter Serial Vector Implementation 2 MPI Vector Implementation 1 MPI Vector Implementation 2 General MPI Vector Adapter … … … ConcreteType Create traits class specializations for all concrete implementations Instantiate templated adapters on the concrete implementation types Uses SerialVectorTraits<> Uses MPIVectorTraits<>

TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National.

Similar presentations

Presentation on theme: "TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National.

Similar presentations

Presentation on theme: "TSF and Abstract Interfaces in Trilinos TSFCore, TSFExtended … Roscoe A. Bartlett Department 9211: Optimization and Uncertainty Estimation Sandia National."— Presentation transcript:

Similar presentations

About project

Feedback