Robust Grid-based environment for large scale lattice-Boltzmann simulations Maddalena Venturoli Matt Harvey Peter Coveney Giovanni Giupponi Jonathan Chin Robin Pinning Andrew Porter Steven Pickles AHM 2005
Scientific motivation: flow in porous media Study multiphase fluid flow in porous media using realistic geometries from XMT data resolution of 4.9 m full data set ~ voxels Large scale simulations
Fluid represented by one particle distribution function on a lattice, discrete velocities Time evolution: advection and collision. Conserves mass and momentum: recovers Navier-Stokes equation Multiphase: local (nearest neighbour) force to take into account interactions with other components (coupling constant) Shan-Chen model Lattice-Boltzmann method Color coding
Advantages Easy to implement and “embarrassingly” parallel: only local interactions Scales as N (number of lattice sites) Complex boundary conditions can be treated by simple rules Fluid/fluid interfaces emerge from the dynamic binary/ternary (surfactants) miscible/immiscible fluids linear parallel performance up to 1024 CPUs (gold star at HPCx) Grid enabled for computational steering and visualization Checkpointing facility, platform independent format Lattice-Boltzmann code: LB3D (CCS-UCL) (performance steering and job migration)
Parameter space exploration Two-phase fluid: coupling constant (miscibility/surface tension) Mixture composition Rock wettability Flow rate Monitor evolution of interfaces as function of time Computational steering and visualization increasing rock wettability increasing water concentration
RealityGrid steering library Real-time interaction with running applications Relatively easy to implement in existing codes Dynamic attach/detach Pause/Resume/Stop Parameter steer/monitor: Qt-steerer, Web portal (EPCC), PDA Checkpointing Job migration/restart (checkpoint tree)
RealityGrid infrastructure
Creating a robust Grid environment: a user perspective Portability issues Security and certification Globus related issues Implementation of a local Registry at UCL UKLight connectivity Visualization (security issues) Co-allocation of resources Now easier to bring new users “up-and-running” (ex.LAMMPS)
Bentheimer: permeability vs system size
Multiphase flow generalized Darcy’s law J = flux, X = force, S = saturation k = relative permeability binary immiscible mixture oil/water colour label red/blue rock (3d) is water wet
Results: relative permeabilities LB simulationsExperimental data SPEJ, run on NCSA
Drainage data set, simulations run on 256 procs at HPCx
Contact: