Decoupled Resource Selection and Application Scheduling with Virtual Grids R, Zhang, A. Chien, A. Mandal, C. Koelbel, H. Casanova, J. Chou, K. Kennedy, R. Huang

Motivation Application scheduling algorithms can be unscalable (albeit polynomial) and thus unusable in large-scale environment One reason for unscalability is that they perform implicit resource selection. Over the past years, Grid infrastructures have been deployed at larger and larger scales, with envisioned deployments comprising tens of thousands of resources. Therefore, scheduling algorithm scalability is a critical problem.

Three Hypotheses One can achieve better scalability by decoupling resource selection from scheduling (aka “decoupled” algorithms). One can achieve similar performance as the non-decoupled approach (aka “one step” algorithms) by selecting resources judiciously. If the application is communication intensive, one can achieve better performance by structuring resources into “close” (in terms of connectivity) groups.

Experimental Design Case Study: Workflow Applications (DAGs) Using Anirban’s scheduler as a starting point Define and generate the experimental environment including universe of compute and network resources and DAGs representing different applications. Three scheduling approaches Improve the scheduler’s implementation so that it can handle large-scale environments (over 36k nodes). Equip the scheduler with the capability to sort and select resources, and schedules applications within pre-selected resources in a decoupled fashion. Query vgES, using vgDL, to get VGs with different structures and try scheduling applications within those VGs. Conduct experiments, compare the three approaches

Experimental Environment We use simulation Environment Application model Representative DAGS from EMAN and Montage A few simple parameters varied, e.g., width, comp/comm ratio Network model We use BRITE to generate network topology but also two random sets that follow normal distributions Resource model We use Yang-Suk’s synthetic cluster generator Assumptions Performance model is accurate and network measurements are also accurate There is no other load on the nodes we use. Binding is instantaneous and always successful The time to obtain resource information is negligible

One-step Approach Run a polynomial-time scheduling algorithm over all resources Objective: minimize application turnaround time (scheduling time + makespan) We measure the scheduling time and we compute the makespan Scheduling algorithms Greedy, Anirban’s min-min, min-max, sufferage heuristic

Decoupled Approach Perform resource selection Random selection (out of 36K resources, pick X at random) Guided selection (out of 36K resources, I pick the X fastest in terms of clockrate) vgDL specification and selected resources returned as a VG Run the one-step algorithms over the selected resources Measure the time for selection and the time to compute the schedule, and compute the schedule length

Experimental Methodology (without vgES) BRITE DML file Cluster Generator DML parser file random selection non-random selection Scheduler Alg #1 Alg #2 Alg #3 ...

Experimental Methodology (with vgES) BRITE DML file vgES DB Cluster Generator DML Wrapper Agent vgFAB vgDL spec Scheduler Alg #1 Alg #2 Alg #3 ... VG

Three Questions What is the gain in scalability? How does one create a “good” vgDL spec? What is the change in the total schedule length?

Scalability

One-step vs Pre-selection

What kind of VG to ask for? VG Structure The overall structure is LooseBAG of TightBag of Nodes. The argument is it guarantees the desired subset of resources that are both fast and close together. Type of resources We have a rough performance model for certain processors and prefer nodes with higher clock speed What if there isn’t a performance model or the performance is not good enough? Number of resources Simplest estimation is based on the DAG width More experiments will help create more precise models

vgdl query VG = LooseBagOf ( tb ) [1:500] [LooseBag.Nodes == 379] { tb = TightBagOf ( node ) [1:500] [Rank= Nodes] node = [ (Processor == OPTERON) || (Processor== ITANIUM ) ] Rank=Clock }

VG Performance

VG Performance

Future work Communication / Computation Ratio Generate VGDL from DAG The current approach is just sum all the computation time on each nodes and communication time between them Both the experiment result and analysis shows that it is not good enough to reveal the property of the DAG Generate VGDL from DAG The structure of the VGDL largely depends on the comm/comp ratio of the DAG. It’s a trade off between better or closer resources. More experiments are needed to determine the right approach. Experiments on real resources The Pegasus/vgES integration Schedule on the test bed Consider the cost model Schedule based on Cluster (hybrid model) Fault Tolerance(detect failure and reschedule after it happens)