2. DataSys Laboratory Dr. Ioan Raicu Juan Carlos Hernández Munuera, MS 2011 Hui Jin, Tonglin Li Paper submission: –Ke Wang, Ioan Raicu. “SimMatrix: Exploring Many-Task Computing through Simulations at Exascales”, under review at ACM HPDC 2012 SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales2

3. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales3

4. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales4

5. 5

6. Pat Helland, Microsoft, The Irresistible Forces Meet the Movable Objects, November 9 th, 2007 Today (2011): Multicore Computing –O(10) cores commodity architectures –O(100) cores proprietary architectures –O(1000) GPU hardware threads Near future (~2018): Manycore Computing –~1000 cores/threads commodity architectures 6SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales

7. Top500 Performance Development, http://top500.org/static/lists/2011/11/TOP500_201111_Poster.pdf 7 Today (2012): 10 Petaflop Computing –O(100K) nodes (100X in the last 10 years) –O(1M) cores (1000X in the last 10 years) Near future (~2018): Exaflop Computing –~1M nodes (10X) –~1B processor-cores/threads (1000X)

8. Concurrency –Parallel programmability Resilience –MTTF decreases, MPI suffers I/O and Memory –Minimizing data movement Heterogeneity –Accelerators, GPUs, MIC Energy –20MW limitation 8SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales

9. Bridge the gap between HPC and HTC Applied in clusters, grids, and supercomputers Loosely coupled apps with HPC orientations Many activities coupled by file system ops Many resources over short time periods SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales9

10. Falkon –Fast and Lightweight Task Execution Framework –http://datasys.cs.iit.edu/proje cts/Falkon/index.htmlhttp://datasys.cs.iit.edu/proje cts/Falkon/index.html Swift –Parallel Programming System –http://www.ci.uchicago.edu/s wift/index.phphttp://www.ci.uchicago.edu/s wift/index.php SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales10

11. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales11

12. Address major exascale computing challenges: –Concurrency –Resilience –I/O and Memory –Heterogeneity Explore techniques to enable MTC at exascales Design, Analyze, and Implement a distributed data- aware execution fabric (MATRIX) supporting HPC/MTC workloads Integrate MATRIX with parallel programming systems (e.g. Swift, Charm++, MapReduce) and with the FusionFS distributed file system Prove that MTC applications can scale to exascales SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales12

13. Explore techniques to enable MTC to scale to exascales –Design, Analyze, and Implement a discrete-event simulator (SimMatrix) enabling the study of MATRIX at extremely large scales (e.g. exascales) –Identified work stealing as a viable technique to achieve load balance at exascales –Provide evidence that work stealing is scalable by identifying optimal parameters affecting the performance of work stealing SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales13

14. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales14

15. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales Efficiently manage the distributed computing power of workstations, servers, and supercomputers in order to maximize job throughput and system utilization. –Load balancing is critical Different scheduling strategies –Centralized scheduling hinders the scalability –Hierarchical scheduling has long job turnaround time –Distributed scheduling is a promising approach at exascales Work Stealing – a distributed scheduling strategy –Starved processors steal tasks from overloaded ones –Various parameters affect performance: Number of tasks to steal Number of neighbors Static or Dynamic random neighbors 15

16. Client Submit tasks Client Arbitrary Node Figure 1: Simulation architectures; the left part is the centralized one with a single dispatcher connecting all nodes, the right part is the homogeneous distributed topology with each node having the same number of cores and neighbors SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales16 Dispatcher

17. Continuous time simulations –Abandoned the idea of creating a separate thread per simulated node: we found that on our 48-core system with 256GB of memory, we were limited to 32K threads Discrete event simulations –The only viable approach (today) to explore scheduling techniques at exascales (millions of nodes and billions of cores) –Created a unique object per simulated node, and converted any behavior to an event SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales17

18. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales18

19. Figure 2: Event State Transition Diagram SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales19 All events are inserted to the queue, sorted based on the occurrence time ascending Handle the first event, advance the simulation time and update the event queue Implemented as red-black tree based “TreeSet” in Java, which ensures Θ(log ) time for insert & remove

20. Node load information –Nested hash maps provides extremely fast performance at large scales Dynamic Task Submission –Aims to reduce the memory foot-print Dynamic Poll interval –Exponential backoff to reduce the number of messages and increase speed of simulation SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales20

21. SimMatrix is developed in JAVA –Sun 64-bit JDK version 1.6.0_22 –1500 lines of code –Code accessible at: http://datasys.cs.iit.edu/projects/SimMatrix/index.html SimMatrix has no other dependencies SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales21

22. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales22

23. Fusion system: –fusion.cs.iit.edu –48 AMD Opteron cores at 1.93GHz –256GB RAM –64-bit Linux kernel 2.6.31.5 –Sun 64-bit JDK version 1.6.0_22 SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales23

24. Throughput –Number of tasks finished per second. Calculated as total-number-of- tasks/simulation-time. Efficiency –The ratio between the ideal simulation time of completing a given workload and the real simulation time. The ideal simulation time is calculated by taking the average task execution time multiplied by the number of tasks per core. Load Balancing –We adopted the coefficient variance of the number of tasks finished by each node as a measure the load balancing. The smaller the coefficient variance, the better the load balancing is. It is calculated as the standard- deviation/average in terms of number of tasks finished by each node. Scalability –Total number of tasks, number of nodes, and number of cores supported. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales24

25. Synthetic workloads: –Uniform distributions with different average task lengths, such as 10s (ave_10), 100s (ave_100), 1000s (ave_1000), 5000s (ave_5000), 10000s (ave_10000), and 100000s (ave_100000); also all tasks of 1 sec each (all_1) Realistic application workloads: –General MTC workload from 2008-2009 trace of 173M tasks; average task length 64±486s (mtc_64), using Gamma Distribution SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales25

26. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales Validate SimMatrix against the state-of-the-art MTC systems (e.g. Falkon), to ensure that the simulator can accurately predict the performance of current petascale systems. 26

27. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales27 Fine grained workloads: 2%  99.3% efficiency increase Coarse grained workloads: 99%  99.999% efficiency increase

28. Memory consumption <13 KB/task <200 GB CPU Time <90 us/task <260 hours SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales28

29. Efficiency 90%+ Co-variance <0.06 Load imbalance of <600 tasks from 10K tasks per node SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales29

30. 30 Stealing half of neighbor’s work is best strategy!

31. 31 Requires linear number of neighbors for good performance!

32. 32 An increasing number of neighbors are needed for 90%+ efficiency, with the largest scales requiring square root neighbors (e.g. 1K neighbors from 1M nodes!

33. 33 The same optimal parameters achieve 90%+ efficiency across many different workloads!

34. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales34 Centralized scheduling has severe bottleneck, especially for workload with fine granularity. Distributed scheduling has great scalability, for workload with coarse granularity, there is no obvious upper bound

35. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales35 Good Load Balancing Square Root Dynamic Neighbors Starvation Square Root Static Neighbors Good Load Balancing Quarter Static Neighbors Starvation 2 Static Neighbors

36. SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales36 Steady state utilization is ~100% at exascales

37. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales37

38. Real Job Scheduling Systems: –Condor (University of Wisconsin), Bradley et al, 2012 –PBS (NASA Ames), Corbatto et al, 2012 –LSF Batch (Platform Computing of Toronto), 2011 –Falkon (University of Chicago), Raicu et al, SC07 Job Scheduling System Simulators: –simJava (University of Edinburgh), Wheeler et al, 2004 –GridSim (University of Melbourne, Australia), Buyya et al, 2010 Load Balancing: –Neighborhood averaging scheme, Sinha et al, 1993 –Charm++ (UIUC), Zheng et al, 2011 Scalable Work Stealing –Dinan et al, SC09 –Blumofe et al, Scheduling multithreaded computations by work stealing, 1994 SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales38

39. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales39

40. Designed, Analyzed, and Implemented a discrete-event simulator (SimMatrix) enabling the study of MTC workloads at exascales Identified work stealing as a viable technique to achieve load balance at exascales Provided evidence that work stealing is scalable by finding optimal parameters affecting the performance of work stealing –Number of tasks to steal is half –Dynamic random neighbors strategy is required –There must be a squared root number of neighbors SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales40

41. Introduction & Motivation Long-Term Aims and Contributions SimMatrix Architecture Implementation Evaluation Related Work Contributions Future Work & Conclusion SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales41

42. Explore work stealing for manycore processors with 1000 cores Enhancing the network topology model to allow complex networks Insight from SimMatrix will be used to develop MATRIX, a distributed task execution fabric –MATRIX will employ work stealing for distributed load balancing –MATRIX will be integrated with other projects, such as Swift (a data-flow parallel programming systems) and FusionFS(a distributed file systems) SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales42

43. Exascale systems bring great opportunities in unraveling of significant scientific mysteries There are significant challenges to achieve exascales, such as concurrency, resilience, I/O and memory, heterogeneity, and energy MTC requires a highly scalable and distributed task/job management system at large scales –Distributed scheduling is likely an efficient way to achieve load balancing, leading to high job throughput and system utilization Work stealing is a scalable method to achieve load balance at exascales given the optimal parameters SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales43

44. More information: –http://datasys.cs.iit.edu/~kewang/http://datasys.cs.iit.edu/~kewang/ –http://datasys.cs.iit.edu/projects/SimMatrix/http://datasys.cs.iit.edu/projects/SimMatrix/ Contact: –kwang22@hawk.iit.edukwang22@hawk.iit.edu Questions? SimMatrix: SIMulator for MAny-Task computing execution fabRIc at eXascales44