Daniel Blackburn Load Balancing in Distributed N-Body Simulations

N-Body Simulations A simulation of a dynamic system of particles under the interaction of a distance mediated force For instance, a simulation of the stars within a cluster or galaxy under the force of gravity

Barnes-Hut Algorithm N-Body simulations can be computed by direct integration  For each particle, calculate the interaction with every other particle  Running time is O (n 2 ) There are many efficient algorithms for N-Body Simulations Barnes-Hut algorithm is based on treating groups of distant particles as a single entity  Running time is O (n log n)

Barnes-Hut Cont. An oct-tree is constructed to contain the particles  Each node corresponds to a segment of simulation space  The root contains the entire simulation space  The children of each node subdivide the space of the node into 8 equally sized cubic segments  The nodes of any layer are non-overlapping  Each leaf holds 1 particle  Each non-leaf stores the mass and center of gravity of all the particles stored by its children Forces are calculated between a particle and a tree node  Let L be the length of the node and D be the distance between the node's center of gravity and the particle.  If L/D < 1 then calculate the force the node exerts on the particle  Otherwise, compute the interaction between the particle and the 8 children of the node

Interactions for one particle

Distributed Barnes-Hut Naïve N-Body Simulations do not require load balancing  Equal computation required for every particle But in Barnes-Hut  Particles in high density areas require more computations than particles in low density areas.  Nearby particles are treated as individuals, but far away particles are calculated as groups  Particles move during the simulation so a good partitioning at the start may become a poor partitioning by the end of the simulation

Data-Shipping vs. Function Shipping Each process constructs a local tree for the particles it controls Interactions must be computed for particles which reside on other processes Two approaches  Data shipping: each process requests enough of the tree of every other process to compute interactions between its particles and remote process.  Function shipping: A list of particles is sent to every other process to compute  Hybrid: Processes share some information about their trees, and use function shipping otherwise

Static Partitioning, Static Assignment The simulation space is broken into k * N equally sized segments Each process is statically assigned k pieces and is responsible for all particles within its segments Particles may transition between processes as they move Relies on distributed segments to overcome load imbalance Gives up some locality to achieve balance

Static Partitioning, Dynamic Assignment The simulation space is broken into k * N equally sized segments A load is calculated for each segment based on the number of calculations done in the last step Each process is dynamically assigned contiguous pieces and is responsible for all particles within its segments Uses a Morton ordering or Z curve to maximize adjacent segments that are physically contiguous Improves locality and load balancing over static assignment, but at increased cost

Dynamic Partitioning, Dynamic Assignment Processes coordinate to construct a combined tree Each node contains the load experienced during the last step Each process does a walk through the tree claiming nodes up to its share of the total load When a process has claimed its share of the load, it signals the next process where it left off and the next process begins its walk from that point

K-Means Clustering Simulation space is divided into N clusters using a K- Means clustering algorithm Ensures that close particles are assigned to the same process regardless of where they are in simulation space Centroids and cluster assignments are recomputed each step The distance function for each centroid is scaled based on the load of the cluster during the last step