1. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Miriam Mehl
Ionel Muntean, Tobias Neckel, Tobias Weinzierl
Computer Science
TU München
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

2. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Why Cartesian Grids?
numerical efficiency
+ (adaptivity, multigrid)
hardware efficiency
??? (not automat.)
flexibility
+ (adaptivity, marker and cell)
accuracy
physical correctness
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

3. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Numerical Efficiency
hierarchically structured Cartesian grids
arbitrarily local adaptivity
full approximation schemes
efficient multigrid methods
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

4. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Hardware Efficiency
½ -1
cell-oriented operator evaluation
constant difference stencils
no neighbour relations
low storage
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

5. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Hardware Efficiency
Peano curve
processing order of grid cells
time locality of data access
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

6. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Hardware Efficiency
stacks as data structures
spatial locality of data access
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

7. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Hardware Efficiency
cache-misses 110% of minimum
runtime 5 times DiMe (regular grid)
3D Poisson
sphere, adaptive
23,118,848 dofs
# processes
Speedup 4
3.73 8
6.85 16
12.93
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

8. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Flexibility
geometric adaptivity
Eulerian approach (marker-and-cell)
complicated changing geometries
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

9. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Accuracy
geometric adaptivity
cutting-cell methods
hierarchical operator generation
second order accuracy in geometry
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

10. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Physical Correctness
Verstappen, 2001:
symmetry requirements
energy and momentum conservation
FEM:
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

11. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Physical Correctness
FEM-basis: u-v-coupled, piecewise linear
correct velocity interpolation
dynamical adaptivity, coupling surface
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

12. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Why Cartesian Grids?
numerical efficiency
+ (adaptivity, multigrid)
hardware efficiency
+ (space-filling curves, stacks)
flexibility
+ (adaptivity, marker and cell)
accuracy
+ (adapt., cutting-cell, hier. op. Gen.)
physical correctness
+ (divergence preserv. FE basis)
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

13. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Numerical Results
Regular grid code F3F:
symmetry preserving FV discretisation
fully parallel
full 3D functionality
platforms up to now:
HLRB2 (SGI Altix 4700)
TU München Infinicluster (128 CPU Opteron)
Universität Stutgart Mozart (128 CPU Xeon cluster)
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

14. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Numerical Results
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

15. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Numerical Results
Adaptive grid code Peano:
2D Navier-Stokes
parallel Poisson
platforms up to now:
HLRB2 (SGI Altix 4700)
TU München Infinicluster (128 CPU Opteron)
PC cluster
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

16. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Numerical Results
Free channel flow: Re=1111
in preparation to DNS
boundary layer: adaptively refined
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München

17. Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Conclusion + Outlook
appropriateness of our approach
concept for adaptive grids: Navier-Stokes
Cartesian grids: applications
next steps:
fully functional 3D parallel adaptive NS-solver
refinement criteria for turbulent boundary layers
runtime optimisation on supercomputers
Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids
Computer Science, TU München