1. A parallel scientific software for heterogeneous hydrogeoloy
Conference on Parallel CFD
Antalya, Turkey
May 2007

2. Damien Tromeur-Dervout
Jocelyne Erhel INRIA Rennes
Jean-Raynald de Dreuzy Geosciences Rennes
Anthony Beaudoin
LMPG, Le Havre
Etienne Bresciani
INRIA Rennes
Damien Tromeur-Dervout
CDCSP, Lyon
Partly funded by Grid’5000
french project

3. Physical context: groundwater flow
Tracer evolution during one year (Made, Mississippi)
Heterogeneous permeability
Injection of tracer
Dispersion
Flow
From Barlebo et al. (2004)
Flow governed by the heterogeneous permeability
Solute transport by advection and dispersion

4. Physical context: groundwater flow
Porous geological media
fractured geological media
Spatial heterogeneity
Lack of observations
Stochastic models of flow and solute transport
-random velocity field
-random solute transfer time and dispersivity
3D Discrete Fracture Network
Flow in highly heterogeneous porous medium

5. Numerical modelling strategy
natural
system
Characterization of heterogeneity
Model validation
Head
Simulation of flow
and solute transport
Numerical
Stochastic
models
Physical
model
Simulation
results

6. Hydrolab scientific software
Object-oriented and modular with C++
Parallel algorithms with MPI
Efficient numerical libraries
free software

7. Hydrolab platform: physical models
Porous
Media
Physical equations
Fracture
Networks
Fractured
Porous

8. Permeability field in porous media
Simple 2D or 3D geometry
stochastic permeability field
finitely or infinitely correlated
finitely correlated medium
Multifractal
D2=1.7
Multifractal
D2=1.4

9. Well test Interpretation
Generalized flux equation
10 h
100 h
D=1
dw=2
D=2
dw=2
Cristalline aquifer of
Ploemeur (Brittany, France)
D=1.5
dw=2.8

10. Macro-dispersion analysis
Longitudinal dispersion
Transversal dispersion

11. Natural Fractured Media) l (
0.1 1 10
100 -7 -6 -5 -4 -3 -2 -1 2 n
probability density n ( l )~ l
-2.7
Fracture length l
Fractures exist at any scale with no correlation
Fracture length is a parameter of heterogeneity
Site of Hornelen, Norway

12. Discrete Fracture Networks with impervious matrix
Stochastic computational domain
length distribution has a great impact : power law n(l)=l-a
3 types of networks based on the moments of length distribution
Existing mean
No variation
2 < a < 3
Existing mean
Existing variation
No third moment
3 < a < 4
Existing mean
Existing variation
Existing third moment
a > 4

13. Output of simulations in 2D fracture networks : upscaling

14. Flow equations: Darcy law and mass conservation
Physical equations
Flow equations: Darcy law and mass conservation
Transport equations: advection and dispersion
Saturated medium: one water phase
Constant density: no saltwater
Constant porosity and constant viscosity
Linear equations
Steady-state flow or transient flow
Inert transport: no coupling with chemistry
No coupling between flow and transport
No coupling with heat equations
No coupling with mechanical equations
Classical boundary conditions
Classical initial conditions

15. Hydrolab platform: numerical methods
PDE solvers
EDO/DAE solvers
Linear solvers
Particle tracker Multilevel methods
Monte-Carlo method
UQ methods
Parametric simulations

16. Steady-state flow equations
Darcy law and mass conservation
Boundary conditions
2D porous medium
3D fracture network
Nul flux
Given head
Nul flux
Given Head
Given Head
Nul flux

17. Uncertainty Quantification methods
Probabilistic framework
Given statistics of the input data,
compute statistics of the random solution
stochastic permeability field K
stochastic network Ω
n(l)=l-a
stochastic flow equations
stochastic velocity field

18. Monte-Carlo simulations
For j=1,…,M
sample permeability field Kj
sample network Ωj
compute vj

19. Spatial discretization
2D heterogeneous porous medium
Finite volume and regular grid
3D Discrete Fracture Network
Mixed Finite Elements and non structured grid

20. Meshing a 3D fracture network
Direct mesh : poor quality or unfeasible
Projection of the fracture network: feasible and good quality

21. Mesh and flux computation in 3D fracture networks

22. Parallel algorithms
Domain decomposition

23. Parallel computing facilities
©INRIA/Photo Jim Wallace
Clusters at Irisa
Grid’5000
project
Numerical model
Funded by
French
Government
and Brittany
council
16,8 millions of unknowns in 100 seconds
with 16 processors

24. Discrete flow numerical model
Linear system Ax=b
b: boundary conditions and source term
A is a sparse matrix : NZ coefficients
Matrix-Vector product : O(NZ) opérations
Direct linear solvers: fill-in in Cholesy factor
Regular 2D mesh : N=n2 and NZ=5N
Regular 3D mesh : N= n3 and NZ=7N
Fracture Network :
N and NZ depend on the geometry
N = 8181
Intersections and 7 fractures
Parallel sparse linear solvers

25. 2D heterogeneous porous medium memory size and CPU time with PSPASES
Sparse direct linear solvers
2D heterogeneous porous medium
memory size and CPU time with PSPASES
Theory : NZ(L) = O(N logN)
Theory : Time = O(N1.5)
variance = 1, number of processors = 2

26. 2D heterogeneous porous medium CPU time with HYPRE/AMG
Sparse iterative linear solvers
2D heterogeneous porous medium
CPU time with HYPRE/AMG
variance = 1, number of processors = 4
residual=10-8
Linear complexity of BoomerAMG

27. Flow computation in 2D porous medium
Finitely correlated permeability field
Impact of permeability variance
matrix order N = 106
matrix order N =
PSPASES and BoomerAMG independent of variance
BoomerAMG faster than PSPASES with 4 processors

28. parallel sparse linear solvers
2D heterogeneous porous medium
Direct and multigrid solvers
Parallel CPU time
matrix order N = 106
matrix order N =
variance = 9

29. Advection-dispersion equation Boundary conditions Initial condition
Solute transport
Nul flux and  C/ n=0
Advection-dispersion equation
Boundary conditions
Initial condition
injection
Fixed head and  C/ n=0
Fixed head and C=0
Nul flux and  C/ n = 0

30. Solute transport : particle tracker
Homogeneous molecular diffusion Dm
Stochastic differential equation
First-order explicit scheme
injection
Parallel particle tracker
Bunch of independent particles
Subdomain decomposition
Many independent particles
Bilinear interpolation for V

31. Solute transport in 2D heterogeneous porous media
Permeability
Particles
Horizontal velocity
Vertical velocity
=1 and Pe=1
=3 and Pe=100

32. particle tracker : convergence analysis
Pure advection
Advection-diffusion with Pe=100
Longitudinal
dispersion
Transversal
dispersion
Np = 1000 is a good trade-off between efficiency and convergence

33. particle tracker : impact of diffusion and heterogeneity
Grid size 8192*8192 with 32 processors
More efficient in pure advection case
Slightly more efficient with moderately heterogeneous media

34. particle tracker : parallel performances
Grid size 4096*4096
Good speed-up in any configuration

35. Current work and perspectives
Iterative linear solvers for 3D fracture networks
3D heterogeneous porous media
Subdomain method with Aitken-Schwarz acceleration
Transient flow in 2D and 3D porous media
Grid computing and parametric simulations
Future work
Porous fractured media with rock
Site modeling
UQ methods