E. WES BETHEL (LBNL), CHRIS JOHNSON (UTAH), KEN JOY (UC DAVIS), SEAN AHERN (ORNL), VALERIO PASCUCCI (LLNL), JONATHAN COHEN (LLNL), MARK DUCHAINEAU (LLNL), BERND HAMANN (UC DAVIS), CHARLES HANSEN (UTAH), DAN LANEY (LLNL), PETER LINDSTROM (LLNL), JEREMY MEREDITH (ORNL), GEORGE OSTROUCHOV (ORNL), STEVEN PARKER (UTAH), CLAUDIO SILVA (UTAH), XAVIER TRICOCHE (UTAH), ALLEN SANDERSON (UTAH), HANK CHILDS (LLNL) On Integral Curves in AMR Simulations

Introduction Simulation domains span vast spatial scales Not possible to adapt the mesh to finest region using rectilinear grid Unstructured grids impose considerable memory overhead and require lookup structure for cell location Simulation domains span vast spatial scales Not possible to adapt the mesh to finest region using rectilinear grid Unstructured grids impose considerable memory overhead and require lookup structure for cell location

AMR combines adaptivity of unstructured grids with implicit connectivity of regular grids Customer: Applied Partial Differential Equation Center (APDEC) at LBNL –Simulation of complex flow fields using AMR technique AMR combines adaptivity of unstructured grids with implicit connectivity of regular grids Customer: Applied Partial Differential Equation Center (APDEC) at LBNL –Simulation of complex flow fields using AMR technique Adaptive Mesh Refinement

AMR data sets Multiple refinement levels Several domains represented as rectilinear grids Data in finer levels replace data in coarser levels Cell-centered data Multiple refinement levels Several domains represented as rectilinear grids Data in finer levels replace data in coarser levels Cell-centered data

Integral curves Integral curves for visualization of streamlines, streaklines, pathlines etc. Essential visualization tool providing easy understanding of flow data Integral curves for visualization of streamlines, streaklines, pathlines etc. Essential visualization tool providing easy understanding of flow data

Integral curves Numerical integration involves: 1.Selecting an initial point 2.Locating the cell containing the point 3.Interpolating the vector field and calculating the next point Numerical integration involves: 1.Selecting an initial point 2.Locating the cell containing the point 3.Interpolating the vector field and calculating the next point pkpk p k+1 p k+2 p k+3 p k+4

Goal Integral curve computation Considering AMR hierarchy Process the curve integration in each domain separately (parallel processing) Integral curve computation Considering AMR hierarchy Process the curve integration in each domain separately (parallel processing) Disregarding level hierarchyConsidering level hierarchy

Proper handling of cell-centered data Use dual-mesh representation to interpolate the vector field Use dual-mesh representation to interpolate the vector field

Proper handling of cell-centered data “Gaps” between domainsDual grid using “ghost” cells Use additional “ghost” cells (resulting from simulation or computed using stored information)

Algorithm Start in domain in finest possible level Build dual mesh Advance integration step If step inside nested domains (finer level) Intersect with the bounding box of the finer domain Restart the algorithm inside the finer domain If outside domain Intersect with the domain bounding box Restart in the next domain Start in domain in finest possible level Build dual mesh Advance integration step If step inside nested domains (finer level) Intersect with the bounding box of the finer domain Restart the algorithm inside the finer domain If outside domain Intersect with the domain bounding box Restart in the next domain

Implementation Implementation in VisIt –AMR as first class data type AMR Data organization: – Nesting structure Information about finer level domains nested in the coarse domain – Neighbor structure Information about the neighbor domains in the same refinement level – Ghost information Additional “outer” cells around a domain Refined cells Implementation in VisIt –AMR as first class data type AMR Data organization: – Nesting structure Information about finer level domains nested in the coarse domain – Neighbor structure Information about the neighbor domains in the same refinement level – Ghost information Additional “outer” cells around a domain Refined cells

Solar system simulation Interaction of the solar wind with the interstellar medium Computational region about 1000 AU Some structures 0.01 AU To fine to be modeled without AMR AMR Mesh –five refinement levels –20037 domains Interaction of the solar wind with the interstellar medium Computational region about 1000 AU Some structures 0.01 AU To fine to be modeled without AMR AMR Mesh –five refinement levels –20037 domains

Interplanetary magnetic field lines Parker spiral Past the termination shock

Interstellar magnetic field lines Wrapping the Heliopause

More examples Argon bubble Vortex cores

Mapped Grids Locally rectangular computational grid Mapped to physical space via mapping function applied to each grid node Locally rectangular computational grid Mapped to physical space via mapping function applied to each grid node

Mapped Grids Data: Fusion simulation example – Magnetic/Velocity field – Mapping field Visualization: –Compute field lines in the block-structured domain –Map the result to physical space Data: Fusion simulation example – Magnetic/Velocity field – Mapping field Visualization: –Compute field lines in the block-structured domain –Map the result to physical space

Current / Future Work Embedded Boundary / Material interfaces – represented as level set – represented as volume fraction – MIR challenges: Preserve volume fraction Continuous representation Global vs. local methods Embedded Boundary / Material interfaces – represented as level set – represented as volume fraction – MIR challenges: Preserve volume fraction Continuous representation Global vs. local methods

Thank you for attention!