A scheduling component for e-Science Central Anirudh Agarwal Jacek Cała

Introduction. – Cloud-based workflow management system for data analytics. – Workflows composed of blocks which can be written in Java, R, Octave, JavaScript, Gnuplot, recently also bash. – Portable system – workflows can run on a laptop, cluster, private or public clouds. EUBrazil Cloud Connect – to create an intercontinental, federated infrastructure for the scientific use. – combined effort between Brazil and several EU countries. – 3 user applications to demonstrate potential of the EUBCC infrastructure: Leishmania Virtual Laboratory, Heart Simulation, Biodiversity and climate change 2

EUBrazil Cloud Connect AAI Opportunistic Cloud HPC COMPSs PMES CSGRID e-SC PDAS fogbow Private Cloud mc2 Users Execution & Provisioning Services Infrastructure Providers COMPSse-SC API Programming Frameworks & Services Data Providers IMVMRC LSF OCCICDMI BES x509 oAuth2 OVF VOMS OGE 3

EUBrazil Cloud Connect AAI Opportunistic Cloud HPC COMPSs PMES CSGRID e-SC PDAS fogbow Private Cloud mc2 Users Execution & Provisioning Services Infrastructure Providers COMPSs e-SC API Programming Frameworks & Services Data Providers IMVMRC LSF OCCICDMI BES x509 oAuth2 OVF VOMS OGE 4

e-Science Central workflow execution model Workflows are constructed from a number of interacting blocks. Each workflow invocation is deployed onto one engine as a single job. Each engine can process one or more workflows at a time. Workflows can be composite -- can submit sub-workflow invocations allowing for parallelism. 5

Advantages of the current model Simple management: – single pool of engines, – the pool can grow and shrink according to needs, – engines can be of different speed. Good scalability: – very little overheads. 6

Limitations of the current model To simple for more sophisticated needs: – heterogeneous workflows/blocks, – heterogeneous hardware infrastructure. No control over invocation dispatch policy: – no priorities – e.g. admin == user, – no fairness – single user can block the system submitting 1000s of invocations, – invocation messages may be consumed in an unfavourable manner. Invocation messages which are once moved to the JMS queue cannot be re-allocated. 7

Selected scheduler requirements To run workflows based on their hard and soft requirements and static and dynamic infrastructure capabilities: – support for heterogeneous workflows and resources <= federated resources, – data-aware scheduling, – user-defined scheduling policies. To allow system to adapt in size dynamically (cloud bursting, opportunistic resources). To allow users to specify the priority for the workflows. Improve the use of resources available: – offer users/administrators some optimisation strategies. 8

Our focus in EUBCC To run workflows based on their hard and soft requirements and static and dynamic infrastructure capabilities: – support for heterogeneous workflows and resources <= federated resources, – data-aware scheduling, – user-defined scheduling policies. To allow system to adapt in size dynamically (cloud bursting, opportunistic resources). To allow users to specify the priority for the workflows. Improve the use of resources available: – offer users/administrators some optimisation strategies. 9

Proposed solution Add a scheduling component (as a pluggable module) between the e-SC server and engines. Make use of the Performance Monitor which gathers information about the system. Have a one-one JMS queue for each engine (pool?). Based on a Scheduling Policy choose the best engine to send the workflow to. Make sure the pending workflows can be rescheduled when all the execution threads are busy. 10

Proposed Solution (cont.) 11

Progress so far… 12

DEMO 1 13

Progress so far (cont.) Current scheduling policy based on CPU load – not effective – just as a PoC. More advanced queue management – able to dynamically attach a new engine to a “scheluder” queue, – able to grow the queue pool if needed. Able to save workflow invocations in the scheduler when all the engine execution threads are exhausted – currently assuming there is 1 execution thread per engine. 14

Current problems and issues Simple CPU load policy. Engine vs engine pool per queue. Impact of the delay between engine --> PM --> scheduler. Missing event-based communication between the engine and server. 15

DEMO 2 16

Delay problem E-SC serverScheduler Performance Monitor Engine JMS Queue Start Workflow Check for Jobs Get Information from PM Send job to correct engine Update PM Update Server about job status 5 sec delay Gets wrong engine information from PM because of 5 second delay Wrong engine maybe selected or wrong task maybe assigned 17

Expected issues and problems For more sophisticated policies: – Lack of input information about the task and its inputs and outputs: hardware/software requirements and capabilities, absence of time completion for the task rules out many scheduling policies, data locality can play important role. Support for cloud bursting e.g. interaction with an Infrastructure Manager Support for simulation – e.g. integration with WorkflowSim 18

DISCUSSION 19