Scientific Workflow Scheduling in Computational Grids Report: Wei-Cheng Lee 8th Grid Computing Conference IEEE 2007 – Planning, Reservation, and Data/Network-Awareness

Abstract The GRACCE scheduler applies advanced scheduling techniques, such as resource negotiation and reservation, data/network-aware scheduling and performance prediction in the resource allocation and execution planning process. To evaluate the scheduler, we have set up an experimental environment that models a computational grid in those aspects relevant to workflow scheduling. The results show the average performance improvement, using the GRACCE scheduler, is about 20% under high resource loads.

Outline WORKFLOW SCHEDULING IN COMPUTATIONAL GRIDS 2 Simulation and Performance Results 4 INTRODUCTION 31 RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING 33 Conclusions and Further Work 35

INTRODUCTION The very important issue in executing a scientific workflow in computational grids is how to map and schedule workflow tasks onto multiple distributed resources and handle task dependencies in a timely manner to deliver users’ expected performance. DAGMan Taverna Karajan Triana They require users to arrange resources for workflow tasks in advance and manually specify resource details. ASKALON Pegasus Gridbus The grid scheduling hierarchy is not taken into account, thus making assumptions that are too optimistic in computational grid environments.

INTRODUCTION GRACCE scheduler applies advanced scheduling techniques : Resource negotiation and reservation, Data/network-aware scheduling Performance prediction in the resource allocation Execution planning process The most of the performance improvement are accomplished through the reduction of the queue waiting time of workflow tasks on the resource’s local schedulers. (GRid Application Coordination, Collaboration and Execution)

WORKFLOW SCHEDULING IN COMPUTATIONAL GRIDS A workflow scheduler is an application-level scheduler whose goal is to improve the application performance, i.e. to complete the workflow execution as fast as possible.

A. Performance Analysis of Workflow Execution 1) Task Execution Time 2) Data Transfer Time 3) Queue Waiting Time B. Workflow Scheduling 1) Resource Allocation Strategies 2) Workflow-Orchestrated Co-Allocations 3) Network/Data-Aware Scheduling P C t1 t2 domain1 domain2 WORKFLOW SCHEDULING IN COMPUTATIONAL GRIDS Resource load The number of CPUs requested by the job The jobs currently running and queued in the local scheduler Just-in-time allocation Look-ahead allocation

C. GRACCE Scheduling Framework WORKFLOW SCHEDULING IN COMPUTATIONAL GRIDS

RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING The GRACCE Scheduler is a data/network-aware workflow scheduler : A. Resource Allocation for A Workflow Task : 1) Resource Discovery and Evaluation 2) Resource Negotiation and Reservation

RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING The resource negotiation and reservation process is thus a two-phase handshaking between the Allocator and the resource owner. Allocator Local Scheduler ResvRequest ResvResponse ResvAccept ResvConfirm This process can be represented by the following two formulas:

RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING B. Workflow Execution Planning To plan the workflow execution, two important parameters for each task are required : 1. The (estimated) execution time of the task on the candidate resource. 2. The network bandwidth between the target resources for its parent tasks and the candidate resource for the task. Using a Predictor to represent the prediction operations as follows:

RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING

RESOURCE ALLOCATION AND WORKFLOW EXECUTION PLANNING

SIMULATION AND PERFORMANCE EVALUATION A. The Simulation Environment 1) Simulation of Grid Resources and Local Schedulers

SIMULATION AND PERFORMANCE EVALUATION 2) Simulation of Job Execution A job execution is simulated using a timer thread: when the thread starts, the job starts; when it times out, the job completes. 3) Random Job Generator The job generator is able to maintain the average resource load at a specific value between 0.0 and 1.0. If the current resource load is less than the expected load, the job generator creates and submits jobs.

SIMULATION AND PERFORMANCE EVALUATION B. Performance Evaluation 1) Performance Evaluation of a 7-Task Workflow 2) Performance Evaluation of a 20-Task Workflow 3) Collect the execution times of 60 workflows that were generated by a random workflow generator.

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 7-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 20-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) Performance Evaluation of a 20-Task Workflow SIMULATION AND PERFORMANCE EVALUATION

) SIMULATION AND PERFORMANCE EVALUATION

SIMULATION AND PERFORMANCE EVALUATION

Wei-Cheng Lee