1. Adaptive grid refinement

2. Adaptivity in Diffpack Error estimatorError estimator Adaptive refinementAdaptive refinement A hierarchy of unstructured gridsA hierarchy of unstructured grids Solution method: Multigrid, often optimal complexitySolution method: Multigrid, often optimal complexity As an add-on Diffpack libraryAs an add-on Diffpack library

3. Parallel Computing in Diffpack

4. Two parallelization approaches Flexible, user-friendly, extensibleFlexible, user-friendly, extensible Linear-algebra-level parallelizationLinear-algebra-level parallelization –An add-on library of parallel matrix/vector operations –Rapid transition from sequential simulator to parallel simulator Simulator-parallel approachSimulator-parallel approach –High-level parallelization of multilevel methods –Extensive reuse of original sequential simulator in subdomain solves –A generic implementation framework

5. Grid partition Basis for computation work decompositionBasis for computation work decomposition General treatment of unstructured gridsGeneral treatment of unstructured grids –Arbitrary number of procs determined at run-time –Non-overlapping partition –Controllable addition of overlap (if desired) –Good load balancing - high parallel efficiency

6. Partition example

7. Linear-algebra-level parallelization Easy to useEasy to use –access to all existing Diffpack iterative methods, preconditioners and convergence monitors –need only to add a few lines of new code –arbitrary choice of number of procs at run-time Parallel matrix/vector operationsParallel matrix/vector operations –inner-product of two vectors –matrix-vector product –preconditioning - block contribution from subgrids

8. Example one Highly unstructured grid Highly unstructured grid Discontinuity in the coefficient Discontinuity in the coefficient

9. Measurements for example one 130,561 degrees of freedom 130,561 degrees of freedom Overlapping subgrids Overlapping subgrids BiCGStab + (block) RILU prec. BiCGStab + (block) RILU prec.

10. An observation “The hope among early domain decomposition workers was that one could write a simple controlling program which would call the old PDE software directly to perform the subdomain solves. This turned out to be unrealistic because most PDE packages are too rigid and inflexible.” “The hope among early domain decomposition workers was that one could write a simple controlling program which would call the old PDE software directly to perform the subdomain solves. This turned out to be unrealistic because most PDE packages are too rigid and inflexible.” - Smith, Bjørstad and Gropp - Smith, Bjørstad and Gropp The remedy: Correct use of object-oriented programming techniques. Correct use of object-oriented programming techniques.

11. Approach 2: simulator-parallel Parallelization of multilevel methodsParallelization of multilevel methods One subdomain is assigned with a sequential simulatorOne subdomain is assigned with a sequential simulator A generic implementation frameworkA generic implementation framework Systematic and flexibleSystematic and flexible –O-O programming enables extensive code reuse –Easy to incorporate multilevel algorithm modification –Different grid types, local solution methods etc. on different subdomains –DD on non-matching subgrids

12. Example two Laplace equation

13. Measurements for example two BiCGStab - global iterative solution method BiCGStab - global iterative solution method Parallel DD as preconditioner Parallel DD as preconditioner MG as subdomain solver MG as subdomain solver Overlapping subgrids Overlapping subgrids

14. Measurements for example two BiCGStab - global iterative solution method BiCGStab - global iterative solution method One parallel MG iteration as preconditioner One parallel MG iteration as preconditioner Overlapping subgrids Overlapping subgrids

15. Scalability Measurements obtained on 16 processors

16. Example three

17. Measurements for example three BiCGStab - global iterative solution method BiCGStab - global iterative solution method One parallel MG iteration as preconditioner One parallel MG iteration as preconditioner Overlapping subgrids Overlapping subgrids

18. 2-phase porous media flow

19. PEQ: SEQ: Multigrid V-cycle in subdomain solves

20. Nonlinear water waves Fully nonlinear 3D water waves Primary unknowns:

21. Parallel simulation of water waves

22. Parallel efficiency Fixed number of subdomains M =16.Fixed number of subdomains M =16. Subdomain grids from partition of a global 41x41x41 grid.Subdomain grids from partition of a global 41x41x41 grid. Simulation over 32 time steps.Simulation over 32 time steps. DD as preconditioner of CG for the Laplace eq.DD as preconditioner of CG for the Laplace eq. Multigrid V-cycle as subdomain solver.Multigrid V-cycle as subdomain solver.

23. Overall efficiency Number of subdomains equal to number of processorsNumber of subdomains equal to number of processors *For P=2 parallel BiCGStab is used.