1. Xing Cai University of Oslo
Numerical Simulation of 3D Fully Nonlinear Waters Waves on Parallel Computers
Xing Cai
University of Oslo

2. Outline of the Talk Mathematical model Numerical scheme (sequential)
Parallelization strategy (domain decomposition)
Object-oriented implementation
Numerical experiment
PARA'98

3. Mathematical Model Fully nonlinear 3D water waves Primary unknowns:
PARA'98

4. Numerical Scheme Physical domain: Transformation: (a fixed domain)
PARA'98

5. Numerical Scheme Operator splitting At each time level: PARA'98
FDM for updating free surface conditions
FEM solution of an elliptic boundary value problem in
PARA'98

6. Preconditioning Computational cost
Elliptic boundary value problem - most CPU intensive
Resulting system of linear equations
Preconditiong
Computational cost
PARA'98
N- number of unknowns

7. The Question Starting point: an o-o water wave simulator
(built in Diffpack: C++ environment for scientific computing)
How to do the parallelization?
Different approaches on different levels:
Automatic parallelization
Parallelization on the low matrix-vector level
Parallelization on the level of simulators
PARA'98

8. Parallelization Strategy
Domain Decomposition
Divide and conquer
Solution of the original large problem through iteratively solving many smaller subproblems -- solution method or preconditioner
Flexible -- localized treatment of irregular geometries, singularities etc
Very efficient numerical methods -- even on sequential computers
Suitable for coarse grained parallelization
PARA'98

9. Overlapping Domain Decomposition
Alternating Schwarz method for two subdomains
Example: solving an elliptic boundary value problem in
A sequence of approximations
where
PARA'98

10. Additive Schwarz Method
Numerical Foundation
Additive Schwarz Method
Subproblems are of the same form as the original large problem, with possibly different boundary conditions on artificial boundaries.
Subproblems can be solved in parallel.
PARA'98

11. Convergence of the Solution
Example:
Solving the Poisson
problem on the unit
square
PARA'98

12. Coarse Grid Correction
Numerical Foundation
Coarse Grid Correction
Important for good DD convergence
Run on each processor, shared with subdomain simulators on the same processor
PARA'98

13. Some Observations Parallel Computing Domain Decomposition
efficiency relies on the parallelization
Domain Decomposition
suits well for parallel computing
a good parallelization strategy
Object-Oriented Programming Technique
flexible and efficient sequential simulators
can be used in subdomain solves -- main ingredient of DD
PARA'98

14. A simulator-parallel model
New Programming Model
A simulator-parallel model
Each processor hosts an arbitrary number of subdomains
balance between numerical efficiency and load balancing
One subdomain is assigned a sequential simulator
Flexibility -- different types of grids, linear system solvers, preconditioners, convergence monitors etc. are allowed for different subproblems
Domain decomposition on the level of subdomain simulators!
PARA'98

15. Simulator-Parallel Reuse of existing sequential simulators
Data distribution is implied
No need for global data
Needs additional functionalities for exchanging nodal values inside the overlapping region
Needs some global administration
PARA'98

16. A Generic Programming Framework
An add-on library (SPMD model)
Use of object-oriented programming technique
Flexibility and portability
Simplified parallelization process for end-user
PARA'98

17. The Administrator Parameter Interface
solution method or preconditioner, max iterations, stopping criterion etc
DD algorithm Interface
access to predifined numerical algorithm e.g. CG
Operation Interface (standard codes & UDC)
access to subdomain simulators, matrix-vector product, inner product etc
PARA'98

18. The Subdomain Simulator
Subdomain Simulator -- a generic representation
C++ class hierarchy
Interface of generic member functions
PARA'98

19. Adaptation of Sequential Simulator
Class SubdomainSimulator - generic representation of a sequential simulator.
Class SubdomainFEMSolver - generic representation of a sequential simulator using FEM.
A new sequential wave simulator that fits in the framework is
readily extended from the
existing sequential simulator,
also being a subclass of
SubdomainFEMSolver.
SubdomainSimulator
PARA'98
SubdomainFEMSolver
WaveSimulator
NewWSimulator

20. Algorithmic efficiency
Performance
Algorithmic efficiency
efficiency of original sequential simulator(s)
efficiency of domain decomposition method
Parallel efficiency
communication overhead (low)
coarse grid correction overhead (normally low)
synchronization overhead
load balancing
subproblem size
work on subdomain solves
PARA'98

21. Parallel Simulation of Waves

22. Parallel Efficiency Fixed number of subdomains M=16.
Subdomain grids from partition of a global 41x41x41 grid.
Simulation over 32 time steps.
DD as preconditioner of CG for the Laplace eq.
Multigrid V-cycle as subdomain solver.
PARA'98

23. Overall Efficiency Number of subdomains equal to number of processors
PARA'98
*For P=2 parallel BiCGStab is used.

24. Summary Efficient solution of elliptic boundary value problems
Parallelization based on DD
Introduction of a simulator-parallel model
A generic framework for implementation
PARA'98