1. San Diego Supercomputer Center, UCSD
Simulating the Cosmic History of Baryons Discoveries Using Advanced Computing Michael L. Norman, Physics Dept., UC San Diego
validation
San Diego Supercomputer Center, UCSD
Keck Observatory, HI
SciDAC M. L. Norman

2. Cosmic History of Baryons
gravitational instability
phase transitions
Recombination: matter & radiation decouple
t~380,000 yr precision era (CMB)
surveys
Baryogensis: GUT phase transition
t~10(-12) s speculative
Nucleosynthesis: formation of light nuclei
t~1-100 s precision era (BBNS)
Structure Formation:
50 Myr < t < 14 Gyr synthesis era
linear perturbation theory
nonlinear simulations
SciDAC M. L. Norman

3. We are here
SciDAC M. L. Norman

4. Cosmological N-body Simulation
A. Evrard and the Virgo Consortium
SciDAC M. L. Norman

5. SciDAC M. L. Norman

6. Multiscale Challenge
Multiscale Challenge
SciDAC M. L. Norman

7. Grand Challenges in Computational Hydrodynamic Cosmology
Formation and evolution of stellar systems on all scales and epochs
Chemical enrichment and reionization of intergalactic medium
Formation of massive black holes and nature of the quasar phenomenon
Cosmological constraints on nature of dark matter and dark energy
SciDAC M. L. Norman

8. Outline Cosmology’s Standard Model Universe in a Box
History of Baryons: Discoveries using Advanced Computing
Exciting Opportunities Ahead
Cosmological limits on dark matter mass
Measuring dark energy equation of state
SciDAC M. L. Norman

9. Cosmology’s Standard Model
Concordance model
H0=72+/-7 km/s/Mpc
expansion rate accelerating (q0<0)
flat universe (k=0)
dominated by dark matter and dark energy
baryons minor constituent
Perlmutter (2003), Physics Today
SciDAC M. L. Norman

10. Evidence for an Accelerating Universe
S. Perlmutter, Physics Today (2003)
SciDAC M. L. Norman

11. Cosmic Microwave Background Temperature Fluctuations 380,000 yr ABB
NASA WMAP
DT/T ~ Dr/r ~ 10-4
SciDAC M. L. Norman

12. CMB Angular Power Spectrum
SciDAC M. L. Norman

13. Mass-Energy Budget of the Universe (WMAP+SNe+XRC)WL
SciDAC M. L. Norman

14. Universe in a Box

15. The Universe is an IVP suitable for computation
Globally, the universe evolves according to the Friedmann equation
cosmological constant
Hubble parameter
mass-energy
density
spacetime
curvature
scale factor a(t)
SciDAC M. L. Norman

16. The Universe is an IVP... Locally*, its contents obey:
Newton’s laws of gravitational N-body dynamics for stars and cold dark matter
Euler or MHD equations for baryonic gas/plasma
Atomic and molecular processes important for radiative cooling of gas and condensation to form stars and galaxies
Radiative transfer equation for photons
 Numerical astrophysics on a cosmic scale
(*scales << horizon scale ~ ct)
SciDAC M. L. Norman

17. baryonic universe radiative transfer radiation background
self-shielding
photo-ionization
photo-heating
photo-evaporation
ionizing
flux
absorption
infall
galaxies
IGM
feedback
(energy, metals)
SF-recipe
multi-species
hydrodynamics
Three actors in the evolution of the baryonic universe:
galaxies, IGM, and radiation backrounds
the baryons are animated by the gravity of the dark matter whose clustering has been well studied using N-body simulations
over the past 10 years, my group has been building codes and code modules to handle the complexities of the baryonic/photonic component
each actor is coupled to the others through two-way interfaces (describe)
N-body dynamics
cosmic expansion
self-gravity
dark matter
dynamics
SciDAC M. L. Norman

18. Cold Dark Matter Dominant mass constituent: Wcdm~0.23
Only interacts gravitationally with ordinary matter (baryons)
Candidates: WIMPs or axions
Collisionless dynamics governed by Vlasov-Poisson equation
Solved numerically using fast N-body methods
SciDAC M. L. Norman

19. Gridding the Universe But what about initial conditions?
Transformation to comoving coordinates x=r/a(t)
Triply-periodic boundary conditions
But what about initial conditions?
a(t1)
a(t2)
a(t3)
SciDAC M. L. Norman

20. Matter Power Spectrum P(k)
Concordance
model
SciDAC M. L. Norman

21. Gravitational Instability: Origin of Cosmic Structure
very small fluctuations r A C
<r> x B
gravity amplifies fluctuations r A C
<r> x B
SciDAC M. L. Norman

22. Formation of the Cosmic Web:
Sky Dome Rendering for DomeFest 2005
Michael Norman, Brian O’Shea, UCSD
Donna Cox, Robert Patterson, Stuart Levy, UIUC
Steve Cutchin, Amit Chourasia, SDSC

23. Technical Details Simulation (Enzo) Data Volume rendering
Dark matter, gravity, multispecies gas dynamics, photo-ionization and, radiative cooling
1 billion cells, 1 billion particles
512 cpu, NCSA TeraGrid cluster
Data
512x512x512 arrays of density
2000 timesteps
1 Terabyte of data
Volume rendering
SDSC IBM DataStar
SciDAC M. L. Norman

24. SciDAC M. L. Norman

25. Structured Adaptive Mesh Refinement (Berger and Colella 1989)
SciDAC M. L. Norman

26. Cosmological Adaptive Mesh Refinement (Bryan & Norman 1997)
Spatial dynamic range unlimited in principle
Today:
L/D = 104 in statistical volumes
L/D =1010 single objects of interest
Petascale:
L/D =106 in statistical volumes
SciDAC M. L. Norman

27.
SciDAC M. L. Norman

28. Enzo Implementation Details
Multi-scale in space and time
Arbitrary # levels of refinement
Arbitrary # grids per level
Portable, MPI-parallel, C++/C/F77 hybrid
Nonlocal dynamic load balancing
Ported to IA64, SGI Altix, IBM SP, BG/L, your mother’s Linux cluster, …..
SciDAC M. L. Norman

29. SciDAC M. L. Norman

30. Galaxy Formation and Large Scale Structure
Technical details
2563 base grid
>32,000 grid 8 levels of refinement
110,000 cpu-hrs on 128 cpu Origin2000
0.5 TB of data
Run at NCSA in 1999
Science credit: M. Norman, G. Bryan, B. O’Shea
Image credit: D. Cox et al.

31. Computational Discoveries using Advanced Computing
First baryonic
condensations
SciDAC M. L. Norman

32. “Bottom-Up” Galaxy Formation
Lacey & Cole (1993)
large galaxies form from mergers of smaller galaxies
where does this begin?
What are the first objects to form?
SciDAC M. L. Norman

33. First objects: a well-posed problem
Initial conditions specified: Wi, P(k)
Macroscopic dynamics understood
Microphysics of primordial gas known
Have 3D solution-adaptive algorithms
Have adequate computer power
February 2003
SciDAC M. L. Norman

34. Formation of First Stars Adaptive Mesh Refinement Simulation Abel, Bryan & Norman (2001)
1 x
10 x
100 x
1000 x
Cosmic scales
107 x
106 x
105 x
104 x
Solar system scales
SciDAC M. L. Norman

35. Birth and Death of the First Star in the Universe
Science credit: T. Abel, G. Bryan, M. Norman
Movie credit: R. Kaehler & T. Abel
SciDAC M. L. Norman

36. Impact of the first stars
the first stars in the universe began forming around 50 million years after the big bang
they were exceptionally massive and bright, bringing an earlier end to the cosmic “dark ages” than previously thought
when they exploded as supernovae they seeded the universe with heavy elements essential for planets and life
they kick-started the cosmogonic sequence which eventually formed galaxies, clusters and superclusters
SciDAC M. L. Norman

37. Computational Discoveries using Advanced Computing
structure of
intergalactic
medium
SciDAC M. L. Norman

38. The Intergalactic Medium
Source: M. Murphy
SciDAC M. L. Norman

39. Structure of the IGM N=10243 L=54 Mpc/h quasar
Earth
N=10243
L=54 Mpc/h
Simulated HI absorption spectrum
Baryon Overdensity, z=3
SciDAC M. L. Norman

40. Matter Power Spectrum P(k)
LCDM
SciDAC M. L. Norman

41. Computational Discoveries using Advanced Computing
whereabouts
of missing
baryons
SciDAC M. L. Norman

42. Missing Baryons at z=0
Galaxies in local universe account for only 10% of baryons we know exist due to three independent measurements, which all agree to 2s
Big bang nucleosynthesis
CMB anisotropies
IGM absorption at high redshift
Where are the baryons now?
SciDAC M. L. Norman

43. Whereabouts of the missing baryons: Warm-Hot intergalactic gas
warm-hot gas
“galaxies”
Cen & Ostriker (1998)
N=5123
SciDAC M. L. Norman

44. Exciting Opportunities Ahead (require Terascale and beyond)
Predicting properties of first galaxies
Understanding quasar-galaxy connection
Self-consistent simulation of the reionization era
Cosmological limits on dark matter mass
Measuring the dark energy equation of state
SciDAC M. L. Norman

45. Effect of DM particle mass on first objects: critical threshold
25 keV
10 keV
O’Shea & Norman (2005)
SciDAC M. L. Norman

46. Measuring Dark Energy EOS
Principal goal of NASA/DOE JDEM mission
Approach: precision measurements of expansion history of the universe using Type Ia SN standardizable candles
Complimentary approach: redshift distribution of galaxy clusters
SciDAC M. L. Norman

47. Lightcone Simulation (A. Evrard et al. 2003)
ct (Gyr) -1 -2 -3 -4 -5
Evrard et al.
Single, P3M
L/D=104
Dark matter only
Our plan
Multiple, 5123 AMR
Optimal tiling of lightcone
L/D=105
Dark matter + gas
SciDAC M. L. Norman

48. A software facility for physical cosmology
Cosmic Simulator
A software facility for physical cosmology
A new collaboration between LLNL and UCSD
Scientific data management focus
Simulations: LLNL Thunder, BG/L
Data management: SDSC SRB
Public UCSD
Science driver:
LSST (Large Synoptic Survey Telescope)
SciDAC M. L. Norman