1. June 28, 2015 1 Resource and Test Management in Grids Rapid Prototyping in e-Science VL-e Workshop, Amsterdam, NL Dick Epema, Catalin Dumitrescu, Hashim Mohamed, Alexandru Iosup, Ozan Sonmez Parallel and Distributed Systems Group Delft University of Technology

2. June 28, 2015 2 Outline A Brief Introduction to Grid Computing Koala: Processor and Data Co-Allocation in Grids  The Co-Allocation Problem in Grids  The Koala Design  Koala and the DAS Community  The Future of Koala GrenchMark: Analyzing, Testing, and Comparing Grids  Grid Performance Evaluation Issues  The GrenchMark Architecture  Experience with GrenchMark Take home message

3. June 28, 2015 3 A Brief Introduction to Grid Computing Typical grid environment e.g., the DAS Applications [!] Resources Compute (Clusters) Storage (Dedicated) Network Virtual Organizations, Projects (e.g., VL-e), Groups, Users Grids vs. (traditional) parallel production environments Dynamic Heterogeneous Very large-scale (world) No central administration → Most problems are NP-hard, need experimental validation

4. June 28, 2015 4 Outline A Brief Introduction to Grid Computing Koala: Processor and Data Co-Allocation in Grids  The Co-Allocation Problem in Grids  The Koala Design  Koala and the DAS Community  The Future of Koala GrenchMark: Analyzing, Testing, and Comparing Grids  Grid Performance Evaluation Issues  The GrenchMark Architecture  Experience with GrenchMark Take home message

5. June 28, 2015 5 The Co-allocation Problem in Grids (1) Motivation Co-allocation = the simultaneous allocation of resources in multiple clusters to single applications which consist of multiple components Reasons Use more resources than available at single cluster at given time Create a specific virtual environment (e.g., visualization cluster, geographically spread data) Achieve reliability through replication on multiple clusters Avoid resource contention on the same site (e.g., batches)

6. June 28, 2015 6 The Co-allocation Problem in Grids (2) Overall Example global queue LS local queues with local schedulers local jobs global job KOALA clusters LS load sharing co-allocation Source: Dick Epema

7. June 28, 2015 7 The Co-allocation Problem in Grids (3) Details: Processors and Data Co-Alloc. Jobs have access to processors and data from many sites Files stored at different file sites, replicas may exist Scheduler decides on job component placement at execution sites Jobs can be of high or low priority Source: Hashim Mohamed

8. June 28, 2015 8 The Co-allocation Problem in Grids (4) Details: Co-Allocated Job Types fixed jobs Job component size and placement fixed by user non-fixed jobs Job component size fixed by user, placement by scheduler decision semi-fixed jobs Job component size and placement by scheduler decision / fixed by user flexible jobs Job component size and placement by scheduler decision

9. June 28, 2015 9 The Koala Design Selection Placing job components Control Transfer executable and input files Instantiation Claiming resources selected for each job component Run Submit, then monitor job execution (fault-tolerance) Source: Hashim Mohamed

10. June 28, 2015 10 The Koala Selection Step Many Placement Policies Originally supported co-allocation policies: Worst-Fit: balance job components across sites Close-to-Files: take into account the locations of input files to minimize transfer times (Flexible) Cluster Minimization: mitigate inter-cluster communication; can also split the job automatically But, different application types require different ways of component placement So: Modular structure with pluggable policies Take into account internal structure of applications

11. June 28, 2015 11 The Koala Selection Step HOCs: Exploiting Application Structure Higher-Order Components: Pre-packaged software components with generic patterns of parallel behavior Patterns: master-worker, pipelines, wavefront Benefits: Facilitates parallel programming in grids Enables user-transparent scheduling in grids Most important additional middleware: Translation layer that builds a performance model from the HOC patterns and the user-supplied application parameters Supported by KOALA (with Univ. of Münster) Initial results: up to 50% reduction in runtimes

12. June 28, 2015 12 Problem: How to support many application types, each with specific (and difficult) requirements? Solution: runners (=interface modules) Currently supported: Any type of single-component job MPI/DUROC jobs Ibis jobs HOC applications API for extensions: write your own! The Koala Instantiation Step The Runners runner

13. June 28, 2015 13 Koala and the DAS Community Extensive experience gathered while assessing various co-allocation policies: over 25,000 completed jobs! Koala has been released on the DAS in Sep 2005 [ www.st.ewi.tudelft.nl/koala/ ] www.st.ewi.tudelft.nl/koala/ Hands-on Tutorials (last in Spring 2006) Documentation (web-site) Papers IEEE Cluster’04, Dagstuhl FGG’04, EGC’05, IEEE CCGrid’05, IEEE Cluster’06, etc. Koala helps you get results: IEEE CCGrid’06, others submitted

14. June 28, 2015 14 The Future of Koala Support for more applications types, e.g., Workflows, Parameter sweep applications Scheduling your application? Communication-aware and application-aware scheduling policies: Take into account the communication pattern of applications when co-allocating Also schedule bandwidth (in DAS3) Support heterogeneity DAS3 DAS2 + DAS3 DAS3 + Grid’5000 + RoGRID Peer-to-peer structure instead of hierarchical grid scheduler

15. June 28, 2015 15 Outline A Brief Introduction to Grid Computing Koala: Processor and Data Co-Allocation in Grids  The Co-Allocation Problem in Grids  The Koala Design  Koala and the DAS Community  The Future of Koala GrenchMark: Analyzing, Testing, and Comparing Grids  Grid Performance Evaluation Issues  The GrenchMark Architecture  GrenchMark and the DAS Community Take home message

16. June 28, 2015 16 Grid Performance Evaluation Current Practice Performance Indicators Define my own metrics, or use U and AWT/ART, or both Workload Structure Run my own workload; Mostly all users are created equal assumption (unrealistic) Do not make comparisons (incompatible workloads) No repeatability of results (e.g., background load) Need a common performance evaluation framework for Grid: GrenchMark

17. June 28, 2015 17 GrenchMark: a Framework for Analyzing, Testing, and Comparing grids What’s in a name? grid benchmark → working towards a generic tool for the whole community: help standardizing the testing procedures, but benchmarks are too early; we use synthetic grid workloads instead What’s it about? A systematic approach to analyzing, testing, and comparing grid settings, based on synthetic workloads A set of metrics and workload units for analyzing grid settings [JSSPP’06] A set of representative grid applications Both real and synthetic Easy-to-use tools to create synthetic grid workloads Flexible, extensible framework

18. June 28, 2015 18 GrenchMark Overview: Easy to Generate and Run Synthetic Workloads

19. June 28, 2015 19 … but More Complicated Than You Think Workload structure User-defined and statistical models Dynamic jobs arrival Burstiness and self-similarity Feedback, background load Machine usage assumptions Users, VOs Metrics A(W) Run/Wait/Resp. Time Efficiency, MakeSpan Failure rate [!] (Grid) notions Co-allocation, interactive jobs, malleable, moldable, … Measurement methods Long workloads Saturated / non-saturated system Start-up, production, and cool-down scenarios Scaling workload to system Applications Synthetic Real Workload definition language Base language layer Extended language layer Other Can use the same workload for both simulations and real environments

20. June 28, 2015 20 GrenchMark and the DAS community Generic Performance Evaluation [IEEE CCGrid’06] Grid System Analysis Performance testing, What-if analysis Functionality Testing in Grid Environments System functionality testing, Periodic testing Comparing Grid Settings Single site vs. co-allocated jobs Releasing the Koala Grid Scheduler on the DAS 5,000+ jobs successfully run (in all workloads); Functionality tests for 3 different job submission modules GrenchMark has been released in Nov 2005 [ grenchmark.st.ewi.tudelft.nl ] grenchmark.st.ewi.tudelft.nl

21. June 28, 2015 21 GrenchMark: Iterative Research Roadmap Open- GrenchMark Community Effort JSSPP’06 Simple functional system A.Iosup, J.Maassen, R.V.van Nieuwpoort, D.H.J.Epema, Synthetic Grid Workloads with Ibis, KOALA, and GrenchMark, CoreGRID IW, Nov 2005. University of Dortmund Complex extensible system A.Iosup, D.H.J.Epema, GrenchMark: A Framework for Analyzing, Testing, and Comparing Grids, IEEE CCGrid'06, May 2006.

22. June 28, 2015 22 PDS Group/TU Delft - resource and test management in Grid systems Koala: Processor and Data Co-Allocation in Grids [ www.st.ewi.tudelft.nl/koala/ ] - Grid scheduling with co-allocation and fault-tolerance - many placement policies available - extensible runners system - easy-to-use, flexible - tutorials, on-line documentation, papers www.st.ewi.tudelft.nl/koala/ GrenchMark: Analyzing, Testing, and Comparing Grids [ grenchmark.st.ewi.tudelft.nl ] - generic tool for the whole community - generates diverse grid workloads - easy-to-use, flexible, portable, extensible, … grenchmark.st.ewi.tudelft.nl Take home message

23. June 28, 2015 23 Thank you! Questions? Remarks? Observations? All welcome! grenchmark.st.ewi.tudelft.nl/ www.st.ewi.tudelft.nl/koala