1. An efficient parallel particle tracker For advection-diffusion simulations In heterogeneous porous media Euro-Par 2007 IRISA - Rennes August 2007

2. Jocelyne Erhel INRIA Rennes Jean-Raynald de Dreuzy Geosciences Rennes Anthony Beaudoin LMPG, Le Havre Partly funded by Grid’5000 french project

3. From Barlebo et al. (2004) Dispersion Flow Injection of tracer Tracer evolution during one year (Made, Mississippi) Heterogeneous permeability Physical context: groundwater flow Flow governed by the heterogeneous permeability Solute transport by advection and dispersion

4. Physical context: groundwater flow Spatial heterogeneity Stochastic models of flow and solute transport -random velocity field -random solute transfer time and dispersivity Lack of observationsPorous geological media Solute dispersionHorizontal velocity Permeability field

5. Longitudinal dispersion Transversal dispersion Macro-dispersion analysis

6. HYDROLAB: parallel software for hydrogeoloy Numerical methods Physical models Porous Media Solvers PDE solvers ODE solvers Linear solvers Particle tracker Utilitaries Input / Output Visualization Results database Parameters database Parallel and grid tools Geometry PARALLEL-BASED SCIENTIFIC PLATFORM HYDROLAB Open source libraries Boost, FFTW, CGal, MPI, Hypre, Sundials, OpenGL, Xerces-C UQ methods Monte-Carlo Fracture Networks Fractured- Porous Media Object-oriented and modular with C++ Parallel algorithms with MPI Efficient numerical libraries

7. Parallel computing facilities ©INRIA/Photo Jim Wallace Numerical model Clusters at Irisa Grid’5000 project Funded by French Government and Brittany council

8. Solute transport Fixed head and C=0 Fixed head and  C/  n=0 Nul flux and  C/  n = 0 injection Advection-dispersion equations Boundary conditions Initial condition Flow equations

9. injection Solute transport : particle tracker Many independent particles Bilinear interpolation for V Homogeneous molecular diffusion Stochastic differential equation First-order explicit scheme D=D m I, U=mean(V), Pe= U / D m

10. particle tracker : convergence analysis N p = 2000 is a good trade-off between efficiency and convergence Longitudinal dispersion Transversal dispersion Pure advectionAdvection-diffusion with Pe=100

11. Not very sensitive to heterogeneity More efficient in pure advection case Linear complexity with the number of cells in each direction particle tracker : impact of diffusion and heterogeneity

12. Parallel particle tracker injection Domain decomposition Velocity V already distributed in parallel flow computations Each processor tracks the particles inside its subdomain

13. One processor owns one subdomain If there are particles at interfaces then Track them inside the subdomain Until they reach an interface The particles arriving at the right boundary go out of the domain Send particles (non blocking) To all neighbors (empty buffer if no particle) Recv particles (blocking) From all neighbors (empty buffer if no particle)

14. safeguards When is it finished ? No deadlock ? No starvation ? Buffers emptied before a new send ? Local counter Np Global counter N Global Sum of all local counters of particles The End when N=0 No Deadlock and no starvation

15. Workload Injection by packets of independent particles Injection at time 0 in only a few processors Load balancing ? Performances ?

16. Injection by packets Pipelined Packets: balanced workload

17. Scalability issues Good speed-up and good scalability

18. Large scale computations Grid size 4096*4096 Good speed-up in any configuration

19. Current work and perspectives Current work 3D heterogeneous porous media Grid computing and Monte-Carlo simulations Future work Dispersion tensor Reactive transport UQ methods