Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL Eric Phipps Andy Salinger, Roger Pawlowski 9233 – Computational Sciences Trilinos User Group Meeting November 3, 2004 Solving Large-Scale Continuation and Bifurcation Problems with LOCA

Why Do We Need Stability Analysis Algorithms for Large-Scale Applications? Nonlinear systems exhibit instabilities, e.g.: Multiple steady-states Ignition Buckling Onset of Oscillations Phase Transitions These phenomena must be understood in order to perform computational design and optimization. Established stability/bifurcation analysis libraries exist: AUTO (Doedel) CONTENT (Kuznetsov) MATCONT (Govaerts) We need algorithms, software, and experience to impact ASCI- and SciDAC-sized applications. Stability/bifurcation analysis provides qualitative information about time evolution of nonlinear systems by computing families of steady-state solutions. LOCA: Library of Continuation Algorithms

Outline What LOCA is/does Structural mechanics examples in Salinas –Snap-through buckling of an arch –Euler buckling of a beam LOCA software and configure options Overview of LOCA’s design and implementation –Bordering algorithms –Super groups/vectors Summary of LOCA’s current capabilities New capabilities since last TUG –Multi-parameter continuation –Householder arclength continuation –Modified turning point bordering algorithm

LOCA: Library of Continuation Algorithms LOCA provides: Parameter Continuation: Tracks a family of steady state solutions with parameter Linear Stability Analysis: Calculates leading eigenvalues via Anasazi (Thornquist, Lehoucq) Bifurcation Tracking: Locates neutral stability point (x,p) and tracks as a function of a second parameter Application code provides: Nonlinear steady-state residual and Jacobian fill: Newton-like linear solves: Second parameter, 1 1 3

Pseudo Arc-length Continuation Solves for Solution and Parameter Simultaneously

Codimension 1 Bifurcations Turning Point Pitchfork Hopf Combustion Buckling of an Arch Buckling of a Beam Pattern formation Cell differentiation (morphogenesis) Vortex Shedding Predator-Prey models Flutter

Snap-through Buckling of a Symmetric Arch Unstressed state of beam is flat Negative gravity used to bend beam into an arch Ends are hinged Beam is loaded in center 100 Salinas beam elements Continuation parameter is center load

Snap-through Buckling of a Symmetric Arch Change in stability at the turning point

Snap-through Buckling of a Symmetric Arch Tracking the turning point in a second parameter 1 solution 3 solutions 1 solution Locus of turning points Pseudo arc-length continuation on turning point equations Bending moment as continuation parameter Solving for load

Euler Buckling of a 3D Beam 1x1x50 solid aluminum beam 4x4x200 Salinas hex8 elements Ends are hinged, right end constrained to move along x-axis Continuation parameter is horizontal load at right end Buckles at load = 3770, beam theory predicts 3290.

LOCA v2.0 Complete rewrite of LOCA v1.0 (C-library) around NOX in C++ Trilinos package inside NOX subdirectory: –Trilinos/packages/nox/src-loca Completely dependent on NOX (i.e., you can’t build LOCA without NOX) Leverages NOX interface to application code Uses design of NOX to implement continuation and bifurcation tracking in a generic way All LOCA SQA tools are combined with NOX –Autoconf/automake –Documentation –Bugzilla, Bonsai –Mail lists

Summary of Relevant Configuration Options Top-level option to build LOCA in Trilinos: –--enable-loca (Default is on) Most configuration options mirror NOX: –--enable-loca-lapack – Enable LOCA LAPACK support (automatically enabled if NOX LAPACK support is enabled) –--enable-loca-epetra – Enable LOCA Epetra support (automatically enabled if NOX Epetra support is enabled) –--enable-loca-lapack-examples – Build LOCA LAPACK examples (automatically enabled if NOX LAPACK examples are enabled) –--enable-loca-epetra-examples – Build LOCA Epetra examples (automatically enabled if NOX Epetra examples are enabled) Other options –--with-loca-anasazi – Build LOCA-Anasazi interface (for automated eigen-analysis during continuation run) –--with-loca-mf – Build LOCA interface to MF (multi-parameter continuation) Other Trilinos options that must be in place –--enable-teuchos, --enable-teuchos-complex, --enable-anasazi (if LOCA Anasazi support is enabled)

LOCA Designed for Easy Linking to Existing Newton-based Applications Algorithmic choices for LOCA: Must work with iterative (approximate) linear solvers on distributed memory machines Non-Invasive Implementation (e.g. matrix blind) Should avoid or limit:  Requiring more derivatives  Changing sparsity pattern of matrix  Increasing memory requirements LOCA targets existing codes that are: Steady-State, Nonlinear Newton’s Method Large-Scale, Parallel

Bordering Algorithms Meet these Requirements Full Newton Algorithm Bordering Algorithm Pseudo Arc-length Continuation

Bordering Algorithms Meet these Requirements … but 4 solves of per Newton Iteration are used to drive singular! Turning Point Bifurcation Full Newton Algorithm Bordering Algorithm

Abstraction of Continuation Process Given initial guess, step size –Solve nonlinear equations to find 1 st point on curve –while !stop Compute predictor Compute predicted point Solve continuation equations for using as initial guess If successful –Postprocess (e.g., compute eigenvalues, output data) –Increase step size Else –Decrease step size –Restore previous solution End if If or or –stop = true –End while LOCA Stepper Predictor modules Step size modules NOX + continuation/ bifurcation groups

NOX implements various methods for solving Code to evaluate is encapsulated in a Group. NOX solver methods are generic, and implemented in terms of group/vector abstract interfaces: NOX solvers will work with any group/vector that implements these interfaces. NOX Nonlinear Solver (Kolda, Pawlowski, Hooper, Shadid) GroupVector computeF()dot() computeJacobian()scale() computeNewton()norm() applyJacobianInverse()update()

Idea: Given a vector to store and a group representing the equations, build an extended (“super”) group representing, e.g., pseudo arc-length continuation equations: and a super vector to store the solution component and parameter component. Super groups/vectors are generic: All abstract group/vector methods for super groups/vectors implemented in terms of methods of the underlying groups/vectors. Super groups are NOX groups: Extended nonlinear equations solved by most NOX solvers Super Vectors and Super Groups

Continuation Groups LOCA::Continuation::ExtendedGroup LOCA::Continuation::NaturalGroup LOCA::Continuation::ArclengthGroup LOCA::Continuation::AbstractGroup setParam() getParam() operator = () computeDfDp() computeEigenvalues() printSolution() Mandatory Default implementation available Optional NOX::Abstract::Group Concrete group

LOCA::Continuation::ArclengthGroup::applyJacobianInverse(const NOX::Abstract::Vector& input, NOX::Abstract::Vector& result) const { const LOCA::Continuation::ExtendedVector& con_input = dynamic_cast (input); LOCA::Continuation::ExtendedVector& con_result = dynamic_cast (result); const NOX::Abstract::Vector& input_x = con_input.getXVec(); double input_p = con_input.getParam(); NOX::Abstract::Vector& result_x = con_result.getXVec(); double& result_p = con_result.getParam(); NOX::Abstract::Vector* b = input_x.clone(NOX::ShapeCopy); underlyingGroupPtr->applyJacobianInverse(input_x, result_x); underlyingGroupPtr->applyJacobianInverse(*dfdpVecPtr, *b); result_p = (predictorVecPtr->getXVec().dot(result_x) – input_p) / (predictorVecPtr->getXVec().dot(*b) – predictorVecPtr->getParam()); result_x.update(-result_p, *b, 1.0); delete b; } Arc-length Group applyJacobianInverse()

Turning Point, Pitchfork Groups LOCA::Continuation::AbstractGroup LOCA::Bifurcation::TPBord::ExtendedGroup LOCA::Bifurcation::TPBord::AbstractGroup computeDJnDp() computeDJnDxa() applySingularJacobianInverse() NOX::Abstract::Group Concrete group LOCA::Bifurcation::PitchforkBord::ExtendedGroup LOCA::Continuation::AbstractGroup

Can overload many additional methods if better techniques are available –block solves –singular matrix solves –estimating derivatives: Interfacing Application Codes to LOCA GroupVectorRequired by setX()dot()NOX computeF()scale() computeJacobian()norm() applyJacobianInverse()update() setParam()Continuation Turning point Pitchfork computeMassMatrix()Shift-invert computeShiftedMatrix()Cayley applyShiftedMatrixInverse() computeComplex()Hopf applyComplexInverse()

LOCA’s Current Capabilities (New since last TUG in red) Single parameter continuation –Natural –Pseudo Arc-length –Householder arc-length Multi-parameter continuation Bifurcations –Turning point –Modified turning point –Pitchfork –Hopf Predictors –Constant (i.e., Euler) –Tangent –Secant –Random –Restart Step size control –Constant –Adaptive Status tests for bifurcations Natural & artificial homotopy Computing eigenvalues with Anasazi –Jacobian inverse –Shift-Invert –Cayley Native support for –LAPACK –Epetra

Multi-Parameter Continuation Multi-parameter continuation supplied through Multifario (MF) code (Mike Henderson, IBM) –General purpose code for covering an implicitly defined manifold Generic link through LOCA –LOCA stepper wraps MF driver –LOCA’s continuation groups implement MF’s interface –Uses new NOX multi-vector support MF library in Trilinos3PL Two examples in LOCA repository: –Chan (LAPACK), –Tcubed (Epetra) Resulting data files best visualized with OpenDX (

Multi-Parameter Continuation Example: Chan Problem

Householder Pseudo Arc-Length Continuation (New) Idea of Homer Walker*: Solve Q is given by a Householder transformation: Advantage – Nearly twice as fast Eliminates 2 nd solve of bordering method Applying Q only requires dot product + saxpy No change in preconditioning Disadvantage – Non-generic Currently only have Epetra implementation Belos implementation coming soon *H.F Walker, SIAM J. Sci. Comput., 1999 Newton solve for pseudo arc-length continuation:

Improving the Turning Point Bordering Algorithm Bordering Algorithm for Newton Updates Turning Point Equations blow up in the direction of as However, don’t. Idea: restrict to be orthogonal to and adjust bordering algorithm appropriately, e.g., solve where

Solve where Then Modified Turning Point Bordering Algorithm

3D Rayleigh-Benard Problem in 5x5x1 box (208K unknowns, 16 processors) Salsa application code Aztec GMRES solver Ifpack RILU preconditioner RILU fill factor: 2 RILU overlap: 2 Krylov space: 500 F = Turning point residual

Where We’re Going From Here Improve robustness/user interface –More step size control algorithms for homotopy problems –Continued work on improved bifurcation tracking algorithms –Tests for automatic bifurcation location –Automatic branch switching??? Incorporate –High order predictors –Constraint enforcement Complete transition to a multivector-based implementation Finish off the Belos and Epetra group implementations Implement LOCA-TSF adaptor Tests