1. Module on Computational Astrophysics Jim Stone Department of Astrophysical Sciences 125 Peyton Hall : ph. 258-3815: jmstone@princeton.edu jmstone@princeton.edu www.astro.princeton.edu/~jstone Lecture 1: Introduction to astrophysics, mathematics, and methods Lecture 2: Optimization, parallelization, modern methods Lecture 3: Particle-mesh methods Lecture 4: Particle-based hydro methods, future directions

2. Andromeda – 2 million light years away Do we really need to compute force from every star for distant objects?

3. Solving the force problem with software -- tree codes Distance = 25 times size

4. Organize particles into a tree. In Barnes-Hut algorithm, use a quadtree in 2D

5. In 3D, Barnes-Hut uses an octree

6. If angle subtended by the particles contained in any node of tree is smaller than some criterion, then treat all particles as one. Results in an Nlog(N) algorithm.

7. Alternative to Barnes-Hut is KD tree. KD tree is binary - extremely efficient Requires N to be power of 2 N nodes = 2N-1

8. Parallelizing tree code. – Equal particles  equal work. Solution: Assign costs to particles based on the work they do – Work unknown and changes with time-steps Insight : System evolves slowly Solution: Count work per particle, and use as cost for next time-step. Best strategy is to distribute particles across processors. That way, work of computing forces and integration is distributed across procs. Challenge is load balancing

9. A Partitioning Approach: ORB Orthogonal Recursive Bisection: – Recursively bisect space into subspaces with equal work Work is associated with bodies, as before – Continue until one partition per processor High overhead for large no. of processors

10. Another Approach: Costzones Insight: Tree already contains an encoding of spatial locality. Costzones is low-overhead and very easy to program

11. Methods so far… Time integration second-order leap frog variable time steps importance of keeping time symmetry higher-order (Hermite) schemes Force evaluation direct summation (PP) tree codes

12. Running a billion particles. Even with GRAPE, direct summation is limited to N ~ 10 6 Tree codes can reach 10 7-8 particles (with very efficient parallelization) Even more efficient methods are required to go beyond 10 9 particles. Particle-Mesh Methods

13. Particle-mesh methods. Calculate the force on a particle from the gravitational potential  of all the particles, rather than using Newton’s Law of gravity to evaluate inter-particle forces directly  is a scalar function of x given by Poisson equation Where  is the mass density

14. Steps in a PM code. 1.Assign particles to mesh to compute  2.Solve Poisson equation with appropriate BCs 3.Compute force on particles 4.Integrate motion Illustrate concepts with a 1D algorithm.

15. 1. Assigning particles to grid Discretize space into Ng zones in 1D i-1 i i+1 Simplest method is to assign entire mass of particle to zone which contains it = Nearest Grid Point scheme Results in a very course distribution of  i.

16. Better method is particle-in-cell (PIC) Treat particles as having finite size, assume some distribution for the mass of the particle within this size. e.g., use top-hat distribution for particle mass: Then distribute mass of particles amongst zones according to overlap Can adopt more complex shapes for mass-distribution of particles, e.g. Gaussian, etc. x  m/H H i-1 i i+1

17. 2. Solving Poisson equation       i-1  i  i+1  Ng+1       i-1  i  i+1  Ng+1 In 1D, Poisson equation is: elliptic PDE Discretize  on a mesh at cell-centers using Ng grid points Then centered FDE for Poisson equation is: For i=1, need  0 ; for i=Ng need  Ng+1 ; -- Boundary Conditions

18. Getting boundary conditions       i-1  i  i+1  Ng+1       i-1  i  i+1  Ng+1 Periodic boundary conditions are most common:  0 =  Ng ;  Ng+1 =  1 Otherwise, standard to use a multipole expansion. Often, the first term (monopole) is good enough (if boundaries are far from most of the mass)

19. The FDE for Poisson equation Can be written as a set of coupled linear equations, N g eqns. Results in a tri-diagonal matrix. Very efficient algorithms are possible for “tri-di” systems (Hockney & Eastwood p185). Forward elimination/Back substitution allows system to be solved in only 4N g operations.

20. 3. Computing Force on particles       i-1  i  i+1  Ng+1       i-1  i  i+1  Ng+1 Once potential is know, force computed from Note forces are located at cell edges: Some sort of spatial interpolation is needed to get force at location of particles. Order of interpolation should be consistent with order of function used to distribute mass in Step 1. F i-3/2 F i-1/2 F i+1/2 F i+3/2

21. Extension to 3D. 1.For particle assignment, mass smoothing function must be sphere 2.For boundary conditions, use spherical multipole expansion 3.Solving Poisson equation: Discretize in 3D as

22. FDE in 3D becomes: Note this can again be written as a set of coupled linear equations N x N y N z equations in all. Very large sparse banded matrix of size N x N y N z by N x N y N z Even a very small problem (20 3 ) leads to large matrix 8000 x 8000 “Typical” size grid (100 3 ) leads to 10 6 x 10 6 matrix Clearly need very efficient methods to solve matrix. Same issues apply to any method for solving elliptic PDEs.

23. Methods to solve matrix 1.Relaxation schemes. Guess solution, then “relax”. e.g. Multigrid 2.Sparse banded solvers, e.g. Conjugate Gradient methods 3.Fourier methods - very powerful, but require periodic BCs

24. Adaptive PM Using a fixed grid to solve the Poisson equation is not efficient in regions with structure on small length scales. Instead, adaptively embed fine grids in dense regions. Grid spacing in each fine level factor of 2 smaller than next coarsest level. Dynamic range is then 2 Nlevels-1. For elliptic systems, solution on fine mesh alters solution on coarse mesh globally -- must solve on all levels simultaneously