Recent results with Goddard AMR codes Dae-Il (Dale) Choi NASA/Goddard, USRA Collaborators J. Centrella, J. Baker, J. van Meter, D. Fiske, B. Imbiriba (NASA/Goddard)

Slides:



Advertisements
Similar presentations
School of something FACULTY OF OTHER School of Computing An Adaptive Numerical Method for Multi- Scale Problems Arising in Phase-field Modelling Peter.
Advertisements

Numerical Relativity & Gravitational waves I.Introduction II.Status III.Latest results IV.Summary M. Shibata (U. Tokyo)
Maximal Slicing of Schwarzschild or The One-Body-Problem of Numerical Relativity Bernd Reimann Mexico City, 8/12/2003 ICN, UNAM AEI.
07 Nov 2003Gravitational Wave Phenomenology Workshop1 Numerical Relativity Deirdre ShoemakerCornell University  The role of numerical relativity in gravitational-wave.
Current status of numerical relativity Gravitational waves from coalescing compact binaries Masaru Shibata (Yukawa Institute, Kyoto University)
Capability and Validity of GRAstro_AMR Mew-Bing Wan E. Evans, S. Iyer, E. Schnetter, W.-M. Suen, J. Tao, R. Wolfmeyer, H.-M. Zhang, Phys. Rev. D 71 (2005)
Toward Binary Black Hole Simulations in Numerical Relativity Frans Pretorius California Institute of Technology BIRS Workshop on Numerical Relativity Banff,
Parallelizing stencil computations Based on slides from David Culler, Jim Demmel, Bob Lucas, Horst Simon, Kathy Yelick, et al., UCB CS267.
Improved constrained scheme for the Einstein equations: an approach to the uniqueness issue in collaboration with Pablo Cerdá-Durán Harald Dimmelmeier.
An efficient parallel particle tracker For advection-diffusion simulations In heterogeneous porous media Euro-Par 2007 IRISA - Rennes August 2007.
Gravitational Collapse in Axisymmetry Collaborators: Matthew Choptuik, CIAR/UBC Eric Hircshmann, BYU Steve Liebling, LIU APS Meeting Albuquerque, New Mexico.
ASCI/Alliances Center for Astrophysical Thermonuclear Flashes Simulating Self-Gravitating Flows with FLASH P. M. Ricker, K. Olson, and F. X. Timmes Motivation:
Black hole binary mergers: recent developments in numerical relativity First-term presentation By Nikos Fanidakis Supervisors: Carlton Baugh, Shaun Cole,
Mesh refinement methods in ROMS Laurent Debreu INRIA, Grenoble, France In collaboration with Patrick Marchesiello and Pierrick Penven (IRD, Brest, France)
Parallel Mesh Refinement with Optimal Load Balancing Jean-Francois Remacle, Joseph E. Flaherty and Mark. S. Shephard Scientific Computation Research Center.
Gravitational Physics Personnel:C. R. Evans B. Brill T. Garrett M. Peppers ResearchSources of Gravitational Radiation Interests:Numerical Relativity &
High Performance Computing 1 Parallelization Strategies and Load Balancing Some material borrowed from lectures of J. Demmel, UC Berkeley.
1 Binary Black Holes, Gravitational Waves, & Numerical Relativity Joan Centrella NASA/GSFC Bernard’s Cosmic Stories June 2006 Valencia, Spain.
Spin, Charge, and Topology in low dimensions BIRS, Banff, July 29 - August 3, 2006.
New Coupled Models of Emerging Magnetic Flux in Active Regions W. P. Abbett, S. A. Ledvina, and G.H. Fisher.
LIGO-G Z Merger phase simulations of binary- systems: status Waveforms – today BH-BH BH-NS NS-NS Main difficulties/unknowns Outlook (for some.
Stratified Magnetohydrodynamics Accelerated Using GPUs:SMAUG.
The Astrophysics of Gravitational Wave Sources Conference Summary: Ground-Based Detectors ( Hz) Kimberly New, LANL.
Sharp Interface Tracking in Rotating Microflows of Solvent Extraction Hyunkyung Lim, Valmor de Almeida, and James Glimm OAK RIDGE NATIONAL LABORATORY &
Improved pipelining and domain decomposition in QuickPIC Chengkun Huang (UCLA/LANL) and members of FACET collaboration SciDAC COMPASS all hands meeting.
A New Code for Axisymmetric Numerical Relativity Eric Hircshmann, BYU Steve Liebling, LIU Frans Pretorius, UBC Matthew Choptuik CIAR/UBC Black Holes III.
Objective of numerical relativity is to develop simulation code and relating computing tools to solve problems of general relativity and relativistic astrophysics.
“Einstein Gravity in Higher Dimensions”, Jerusalem, Feb., 2007.
“Models of Gravity in Higher Dimensions”, Bremen, Aug , 2008.
1 Massive Black Hole Mergers: As Sources and Simulations John Baker Gravitational Astrophysics Laboratory NASA/GSFC CGWP Sources & Simulations February.
Albert-Einstein-Institut Black Hole Initial Data for Evolution Distorted Black Holes: “Brill Wave plus Black Hole” (NCSA model)
1+log slicing in gravitational collapse. Ingredients of successful binary black hole simulations Pretorius Generalized harmonic coordinates Excision ____________________________.
1 Calculating Gravitational Wave Signatures from Black Hole Binary Coalescence Joan Centrella Laboratory for High Energy Astrophysics NASA/GSFC The Astrophysics.
1 GWDAW11 Dec 19, 2006 John Baker LISA source modeling and data analysis at Goddard John Baker – NASA-GSFC K. Arnaud, J. Centrella, R. Fahey, B. Kelly,
Computational Relativity The largest field of enquiry historically has been the field of exact solutions of Einstein's field equations. The Einstein field.
Black holes sourced by a massless scalar KSM2105, FRANKFURT July, 21th 2015 M. Cadoni, University of Cagliari We construct asymptotically flat black hole.
1 Building Bridges: CGWA Inauguration 15 December 2003 Lazarus Approach to Binary Black Hole Modeling John Baker Laboratory for High Energy Astrophysics.
Numerical Relativity is still Relativity ERE Salamanca 2008 Palma Group Alic, Dana · Bona, Carles · Bona-Casas, Carles.
Boson Star collisions in GR ERE 2006 Palma de Mallorca, 6 September 2006 Carlos Palenzuela, I.Olabarrieta, L.Lehner & S.Liebling.
Cactus Workshop - NCSA Sep 27 - Oct Cactus For Relativistic Collaborations Ed Seidel Albert Einstein Institute
Phase transition induced collapse of Neutron stars Kim, Hee Il Astronomy Program, SNU 13th Haengdang Symposium, 11/30/2007.
Black Hole Universe Yoo, Chulmoon ( YITP) Hiroyuki Abe (Osaka City Univ.) Ken-ichi Nakao (Osaka City Univ.) Yohsuke Takamori (Osaka City Univ.) Note that.
Domain Decomposition in High-Level Parallelizaton of PDE codes Xing Cai University of Oslo.
Cosmological Heavy Ion Collisions: Colliding Neutron Stars and Black Holes Chang-Hwan Lee Current Status of Numerical Approaches.
S.Klimenko, December 16, 2007, GWDAW12, Boston, LIGO-G Z Coherent burst searches for gravitational waves from compact binary objects S.Klimenko,
Computational General Relativistic Astrophysics at Wash U: What are we doing lately? Nov., 2006 Numerical Relativity Group Washington University.
Intermediate mass black hole and numerical relativity Zhoujian Cao Institute of Applied Mathematics, AMSS
Applying Numerical Relativity and EOB to Black Hole Binary Observation Sean McWilliams NASA Goddard Space Flight Center University of Maryland Collaborators:
PARAMESH: A PARALLEL, ADAPTIVE GRID TOOL FOR THE SPACE SCIENCES Kevin Olson (NASA/GSFC and GEST/Univ. of MD, Baltimore) Presented, AISRP PI Meeting April,
Astrophysics to be learned from observations of intermediate mass black hole in-spiral events Alberto Vecchio Making Waves with Intermediate Mass Black.
TR&D 2: NUMERICAL TOOLS FOR MODELING IN CELL BIOLOGY Software development: Jim Schaff Fei Gao Frank Morgan Math & Physics: Boris Slepchenko Diana Resasco.
Black Hole Universe -BH in an expanding box- Yoo, Chulmoon ( YITP) Hiroyuki Abe (Osaka City Univ.) Ken-ichi Nakao (Osaka City Univ.) Yohsuke Takamori (Osaka.
PTA and GW detection --- Lecture K. J. Lee ( 李柯伽 ) Max-Planck Institute for Radio astronomy Aug
Cactus Workshop - NCSA Sep 27 - Oct Scott H. Hawley*, Matthew W. Choptuik*  *University of Texas at Austin *  University of British Columbia
Gauge conditions for black hole spacetimes Miguel Alcubierre ICN-UNAM, Mexico.
LISA double BHs Dynamics in gaseous nuclear disk.
BLACK HOLES. BH in GR and in QG BH formation Trapped surfaces WORMHOLES TIME MACHINES Cross-sections and signatures of BH/WH production at the LHC I-st.
Numerical Relativity in Cosmology - my personal perspective - Yoo, Chulmoon ( Nagoya U. ) with Hirotada Okawa ( Lisbon, IST ) New Perspectives on Cosmology.
1 Merging Black Holes and Gravitational Waves Joan Centrella Chief, Gravitational Astrophysics Laboratory NASA Goddard Space Flight Center Observational.
MWRM, Nov 16-19, 2006, Washington University, Saint Louis Area Invariance of Apparent Horizons under Arbitrary Lorentz Boosts Sarp Akcay Center for Relativity.
Geometrically motivated, hyperbolic gauge conditions for Numerical Relativity Carlos Palenzuela Luque 15 December
Port AMSS-NCKU code to GPU Zhoujian Cao Academy of Mathematics and System Science, CAS Cowork with Zhihui Du, Steven Brandt, Frank Loeffler and Quan Yang.
Xing Cai University of Oslo
Manuel Tiglio Hearne Institute for Theoretical Physics
On Behalf of the LIGO Scientific Collaboration and VIRGO
Search for gravitational waves from binary black hole mergers:
Black Hole Binaries Dynamically Formed in Globular Clusters
Low Order Methods for Simulation of Turbulence in Complex Geometries
Apply discontinous Galerkin method to Einstein equations
Presentation transcript:

Recent results with Goddard AMR codes Dae-Il (Dale) Choi NASA/Goddard, USRA Collaborators J. Centrella, J. Baker, J. van Meter, D. Fiske, B. Imbiriba (NASA/Goddard) J. D. Brown and L. Lowe (NCSU) Supported by NASA ATP PSU NR Lunch, APR 29, 2004

Outline Codes/Features Codes/Features Summary of past and present works Summary of past and present works Brill waves Brill waves Binary black holes Binary black holes Future Future

Codes “Hahndol” (= “One-Stone”= “Ein-stein” in Korean) “Hahndol” (= “One-Stone”= “Ein-stein” in Korean) Vacuum Evolution code: 3+1 BSSN. Vacuum Evolution code: 3+1 BSSN. Free evolution (BSSN gauges imposed). Free evolution (BSSN gauges imposed). FMR, AMR and Parallel (Paramesh): scalability good. FMR, AMR and Parallel (Paramesh): scalability good. Puncture BHs, Waves. Puncture BHs, Waves. AMRMG_3D (NCSU) AMRMG_3D (NCSU) Elliptic solver: Multi-grid. Elliptic solver: Multi-grid. Parallel, AMR support based on Paramesh. Parallel, AMR support based on Paramesh. Initial data generator: Brill wave, 2BH, single distorted BH. Initial data generator: Brill wave, 2BH, single distorted BH.

Key Features: Simplicity Simple grid structure: A hierarchy of the logically cartesian grid blocks with identical structure. Considerably simplified data structure is known at compile time. Simple grid structure: A hierarchy of the logically cartesian grid blocks with identical structure. Considerably simplified data structure is known at compile time. Simple tree structure: Grid blocks are managed by simple tree structure which tracks the spatial relationships between blocks. Simple tree structure: Grid blocks are managed by simple tree structure which tracks the spatial relationships between blocks. Mesh refinement is “ block-based ” and uses “ bisection ” method (De Zeeuw & Powell, 1993) Mesh refinement is “ block-based ” and uses “ bisection ” method (De Zeeuw & Powell, 1993) Block is a basic “ unit ” for mesh-refinement and domain decomposition. Block is a basic “ unit ” for mesh-refinement and domain decomposition. No clusterer needed. No clusterer needed. Simple communication patterns: Blocks are distributed amongst available processors in ways which maximize block locality and minimize inter- processor communications. Simple communication patterns: Blocks are distributed amongst available processors in ways which maximize block locality and minimize inter- processor communications.  May be crucial for parallel implementation

Mesh Refinement Works Mesh Refinement Works Fixed Mesh Refinement (FMR) Fixed Mesh Refinement (FMR) Study of refinement interface conditions with linear waves [JCP 193, 398 (2004) (physics/ )] Study of refinement interface conditions with linear waves [JCP 193, 398 (2004) (physics/ )] Key Ideas: Quadratic interpolation combined with “flux” matching guarantees 2 nd order convergence and minimizes interface noises. Key Ideas: Quadratic interpolation combined with “flux” matching guarantees 2 nd order convergence and minimizes interface noises. Single puncture BH: thorough convergence study with 8 levels of FMR [gr-qc/ ]. Single puncture BH: thorough convergence study with 8 levels of FMR [gr-qc/ ]. Key results: OB at ~100M, high resolution (h ~ 1/64) near puncture. Key results: OB at ~100M, high resolution (h ~ 1/64) near puncture. Already very helpful in 2BH simulations [work in progress]. Already very helpful in 2BH simulations [work in progress]. Distorted BH [work in progress, D. Fiske]. Distorted BH [work in progress, D. Fiske]. Adaptive Mesh Refinement (AMR) Adaptive Mesh Refinement (AMR) Weak GW simulations [PRD 62, (2000)] Weak GW simulations [PRD 62, (2000)] 2-level; fine grid tracking the waves. 2-level; fine grid tracking the waves. Brill wave simulations [work in progress] Brill wave simulations [work in progress] Zooming into critical regime. Zooming into critical regime.

Brill Waves Initial Data: Time symmetric (axi-symmetric) Brill wave solution. Initial Data: Time symmetric (axi-symmetric) Brill wave solution. B.C.: Octant + Sommerfeld outgoing except. B.C.: Octant + Sommerfeld outgoing except. First order shock avoidance slicing [M. Alcubierre, CQG 20, 607 (2003)],. First order shock avoidance slicing [M. Alcubierre, CQG 20, 607 (2003)],. AMR interface conditions: 2 nd order interpolation followed by “flux” matching  matching function and first derivatives of function. AMR interface conditions: 2 nd order interpolation followed by “flux” matching  matching function and first derivatives of function. Adaptive regridding based on the first derivatives of variables. Adaptive regridding based on the first derivatives of variables. Physics: find the critical parameter, A*, and study the critical phenomena (& later, extend to non-axisymmetry). Physics: find the critical parameter, A*, and study the critical phenomena (& later, extend to non-axisymmetry). Previous estimate of the critical parameter: 4.7 < A*< 5.0 [M. Alcubierre, et al, PRD 61, (2000), Use 128^3 grids ]. Previous estimate of the critical parameter: 4.7 < A*< 5.0 [M. Alcubierre, et al, PRD 61, (2000), Use 128^3 grids ]. Hahndol: Zooming into critical regime: current estimation  4.80 < A* < Hahndol: Zooming into critical regime: current estimation  4.80 < A* < 4.85.

Brill Wave: Preliminary results Dispersal for A < 4.8 Dispersal for A < 4.8 Lapse collapses for A > 4.85 Lapse collapses for A > < A* < 4.85: results are sensitive to various parameters such as location of outer boundary and resolution. 4.8 < A* < 4.85: results are sensitive to various parameters such as location of outer boundary and resolution.

Brill Wave: Preliminary Results A = 4.84 A = x 64 x 64 base grid (h~0.125) 64 x 64 x 64 base grid (h~0.125) 3 additional levels  finest resolution = (effective resolution of 512 x 512 x 512 unigrid) 3 additional levels  finest resolution = (effective resolution of 512 x 512 x 512 unigrid) Snapshots for lapse (on Z=0 plane) Snapshots for lapse (on Z=0 plane) Working on to find AH to confirm BH formation. Working on to find AH to confirm BH formation. Caution: Inadequate resolution may give completely wrong outcome! Caution: Inadequate resolution may give completely wrong outcome! Run with only 2 additional levels results in dispersal (finest resolution = ) Run with only 2 additional levels results in dispersal (finest resolution = ) Further study is under way. Further study is under way.

Binary Black Hole Simulation (Head-on collision) Initial Data (time = 0) Initial Data (time = 0) Simple cases can be done by hand: two equal mass non-spinning black holes with zero initial velocity. Simple cases can be done by hand: two equal mass non-spinning black holes with zero initial velocity. Spatial metric on 3d spacelike hypersurface, Spatial metric on 3d spacelike hypersurface, Evolution (time > 0) Evolution (time > 0) Lapse condition (1+log) Lapse condition (1+log) Shift condition (Hyperbolic driver) Shift condition (Hyperbolic driver) Mesh Refinement Mesh Refinement Source region: scale ~ M, put more grid points. Source region: scale ~ M, put more grid points. Wavezone: scale ~ ( )M, put less grid points. Wavezone: scale ~ ( )M, put less grid points. Boundary of computational domain: ~ a few hundred M. Boundary of computational domain: ~ a few hundred M.

Binary Black Hole Simulations (Mesh Structure) Mesh Refinement allows one to put outer boundary as far as possible. Efficient distribution of grid points: more near black holes.

BBH Head-on collision Initial separation = 5M, M=2, Two event horizons initially separated. Initial separation = 5M, M=2, Two event horizons initially separated. Mesh refinement calculations. (OB at 120M) Mesh refinement calculations. (OB at 120M) g xx on Z=0 plane. g xx on Z=0 plane. Gauge wave followed by physical wave. Gauge wave followed by physical wave.

BBH Head-on collision Coordinate conditions [g tx, g tt ]. Coordinate conditions [g tx, g tt ]. Two black hole merges into a single black hole. Two black hole merges into a single black hole. Gauge wave comes out first. Gauge wave comes out first. Assume profile of a single black hole after merger. Assume profile of a single black hole after merger.

Future Attacking both “astrophysics” and “physics” problems. Attacking both “astrophysics” and “physics” problems. Astrophysics: orbiting black hole binaries, distorted black holes  gravitational wave astrophysics. Astrophysics: orbiting black hole binaries, distorted black holes  gravitational wave astrophysics. Physics: Brill wave, etc. Physics: Brill wave, etc. Analysis tools for mesh refinement Analysis tools for mesh refinement Horizon finders Horizon finders Invariants, GW extraction Invariants, GW extraction Focus on LISA source modeling: GW extraction for black holes binaries  Data analysis. Focus on LISA source modeling: GW extraction for black holes binaries  Data analysis.