1. Distributed Load Balancing for Parallel Agent-based Simulations Biagio Cosenza*, Gennaro Cordasco, Rosario De Chiara, Vittorio Scarano ISISLab, Dipartimento di Informatica Università degli Studi di Salerno, Italy

2. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Outline Agent-based Simulations – Reynolds’ model Agents partitioning Load balancing with two processors Generalization to multiple workers Results

3. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Agent-based Simulations Simulate the actions and interactions of autonomous individual agents with a view to assessing their effects on the system as a whole – Investigated since the 1980s – Early works take inspiration from particles [R83] Applications – Biology, economics, sociology … Why parallel? – Huge number of agents – Complex behaviours Birth Social interactions Aging Death Fly around

4. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Alignment Cohesion Separation Reynolds’ Model Alignment Cohesion Separation Alignment: steer towards the average heading of local flock-mates Cohesion: steer to move toward the average position of local flock-mates Separation: steer to avoid crowding local flock- mates

5. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Parallelization of Agent-based Simulation Space Partitioning – Distributed (not centralized) – Even distribution (load balancing) Target architecture is a cluster of workstations

6. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Parallelization of Agent-based Simulation The simulation carried out on 4 workers

7. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Agent-based Distribution sparse distributiondense distribution

8. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Load balancing with two processors

9. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Two Processors Schema Assumptions – the measure of the computational load of each worker is linear in the number of agents – agents are uniformly distributed along the dimension where the splitting occurs

10. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Two-procs Schema Region wide load balancing (Dynamic1) Assumption: agents uniformly distributed slice of space containing a single agent, under the uniformly distributed agents assumption Remarks: s r (s l ) is the number of agents in S r (S l ) number of agents to be moved to balance the load between the left and right processor

11. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza The Oscillation Problem Agents are usually more dense on the boundary – The oscillation is an effect of the assumption of uniformity of distribution, that underestimates the amount of agents approaching the boundary When a large flock of agents rapidly moves between the two workers – the boundary is moved too aggressively in the attempt of balancing the flock movement and it oscillates

12. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Two-procs Schema Mitigated region wide load balancing (Dynamic2) Remarks: s r (s l ) is the number of agents in S r (S l ) Introducing a mitigation factor k slice of space containing a single agent, under the uniformly distributed agents assumption number of agents to be moved to balance the load between the left and right processor mitigation factor (we used k=0.5)

13. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Two-procs Schema Restricted assumption load balancing (Dynamic 3) Remarks: ε is the radius of agents’ AOI s r (s l ) is the number of agents in S r (S l ) e r (e l ) is the number of agents in E r (E l ) Relaxed assumption of uniformity of the distribution of the agents to the spaces E l and E r, instead of S l and S r Limit the per-step movement of the boundary to ε

14. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Generalization to Multiple Workers We apply the 2-workers schema to couples of neighbors workers, on alternate steps

15. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Test Settings Hardware – IBM HS21 Cluster available at ENEA CRESCO Project, Portici (Italy) – Each node: 2 Xeon QuadCore Clovertown E5345 at 2.33 GHz, 16 GByte RAM – Infiniband network Software Implementation – OpenSteer, C++, Intel compiler, Message passing (MPI) The set of tests we performed is the result of the Cartesian product: {1,000, 10,000, 100,000} × number of agents {8, 16, 32, 64} × number of processors {static, dynamic1, dynamic2, dynamic3} load balancing strategy

16. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Results Distribution of agents per worker Each color represents a different worker. x-axis indicates the simulation step (1, 000 simulation steps are depicted) while th y-axis represents the distribution of agents (a) (100,000; 8; static) (d) (100, 000; 8; dynamic3) (b) (100, 000; 8; dynamic1) (c) (100, 000; 8; dynamic2) (a) (d) (b) (c)

17. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Results Scalability A comparison between (100,000;*; static) (dotted line) and (100,000;*; dynamic3) (continuous line) number of procs number of steps per second

18. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Agents’ Partitioning Demo Video

19. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Conclusion Agent-based Simulation and Parallel Computing – Well suited – Load balancing is challenging We introduced a new load balancing schema – Lightweight – Distributed – Good scalability – Up to 1,000,000 agents Next steps – 1,000,000,000 – Complex behaviours

20. PDP 2011 - Ayia Napa (Cyprus) - February 9-11, 2010Biagio Cosenza Thanks for your attention Questions? http://www.isislab.it/projects/DistrSteer/