QoS-based Scheduling of e-Research Application Workflows on Global Grids Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia Gridbus Sponsors

2 GRIDS Melbourne Youngest and one of the rapidly growing research labs in our School/University: Founded in 2002 Houses: Research Fellows/PostDocs Research Programmers PhD candidates Honours/Masters students Funding National and International organizations Australian Research Council & DEST Many industries (Sun, StorageTek, Microsoft, IBM, Microsoft) University-wide collaboration: Faculties of Science, Engineering, and Medicine Many national and international collaborations. Academics Industries Software: Widely in academic and industrial users. Publication: My research team produces over 20% of our Dept’s research output. EducationR & D + Community Services: e.g., IEEE TC for Scalable Computing

3 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

4 Power Grid Inspiration: Seamlessly delivering electricity as a utility to users

5 (5) Computing Grid: Delivering IT services as the 5 th utility after water, gas, electricity, and telephone eScience eBusiness eGovernment eHealth Multilingual eEducation …

6 Grid-like Vision In 1969, Leonard Kleinrock, one of the chief scientists of the original ARPA project which seeded the Internet, wrote: "As of now, computer networks are still in their infancy, but as they grow up and become sophisticated, we will probably see the spread of "computer utilities", which, like present electric and telephone utilities, will service individual homes and offices across the country“ Despite major advances in hardware and software systems over the past 35 years, we are yet to realize this vision. How far are we still from delivering computing as a utility?

7 Computing and Communication Technologies Evolution: ! * Sputnik * ARPANET * * Ethernet * TCP/IP * IETF * Internet Era* WWW Era * Mosaic * XML *PC Clusters *Crays *MPPs *Mainframes * HTML * W3C *P2P *Grids *XEROX PARC worm COMPUTING Communication * Web Services * Minicomputers *PCs *WS Clusters *PDAs *Workstations *HTC 2010 * e-Science * Computing as Utility * e-Business * SocialNet Control Centralised Decentralised

8 What is Grid? (It means different things to different people) IBM On Demand Computing Microsoft.NET Oracle 10g Sun N1 – Sun Grid Engine HP Adaptive Enterprise Amazon Electric Cloud Services United Devices and related companies: Harvesting Unused Desktop resources

9 What is Grid? [Buyya et. al.] A type of parallel and distributed system that enables the sharing, exchange, selection, & aggregation of geographically distributed “autonomous” resources: Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc; Software – e.g., ASPs renting expensive special purpose applications on demand; Catalogued data and databases – e.g. transparent access to human genome database; Special devices/instruments – e.g., radio telescope – searching for life in galaxy. People/collaborators. depending on their availability, capability, cost, and user QoS requirements. Wide area

10 How does Grids look like? A Bird Eye View of a Global Grid Grid Resource Broker Resource Broker Application Grid Information Service Grid Resource Broker database R2R2 R3R3 RNRN R1R1 R4R4 R5R5 R6R6 Grid Information Service

11 Classes of Grid Services / Types of Grids Computational Services – CPU cycles Pooling computing power: TeraGrid, AusGrid, ChinaGrid, IndiaGrid, UK Grid,… Data Services Collaborative data sharing generated by instruments, sensors, persons: LHC Grid, Napster Application Services Access to remote software/libraries and license management—NetSolve Interaction Services eLearning, Virtual Tables, Group Communication (Access Grid), Gaming Knowledge Services The way knowledge is acquired, processed and managed—data mining. Utility Computing Services Towards a market-based Grid computing: Leasing and delivering Grid services as ICT utilities. Computational Grid Data Grid ASP Grid Interaction Grid Knowledge Grid Utility Grid infrastructure Users

12 How Are Grids Used? High-performance computing Collaborative data-sharing Collaborative design Drug discovery Financial modeling Data center automation High-energy physics Life sciences E-Business E-Science Natural language processing & Data Mining Utility computing

13 e-Science Environment: Supporting Collaborative Science Distributed instruments Distributed computation Distributed data Peers sharing ideas and collaborative interpretation of data/results Remote Visualization Data & Compute Service Cyberinfrastructure E-Scientist

14 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

15 Grid Challenges Security Resource Allocation & Scheduling Data locality Network Management System Management Resource Discovery Uniform Access Computational Economy Application Construction

16 Some Grid Initiatives Worldwide Australia Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch Brazil OurGrid, EasyGrid LNCC-Grid + many others China ChinaGrid – Education CNGrid - application Europe UK eScience EU Grids.. and many more... India Garuda  Japan NAGERI Korea... N*Grid Singapore NGP USA Globus GridSec AccessGrid TeraGrid Cyberinfrasture and many more... Industry Initiatives IBM On Demand Computing HP Adaptive Computing Sun N1 Microsoft -.NET Oracle 10g Infosys – Enterprise Grid Satyam – Business Grid StorageTek –Grid.. and many more Public Forums Global Grid Forum Australian Grid Forum Conferences: CCGrid Grid HPDC E-Science billion – 3 yrs 1 billion – 5 yrs 450million – 5 yrs 486million – 5 yrs 1.3 billion (Rs) 27 million 2? billion 120million – 5 yrs

17 Open-Source Grid Middleware Projects

18 Driving Theme: Community Grids vs. Utility Grids Type Feature Community GridsUtility Grids User QoSBest effortContract/SLA Service Pricing Not considered / free access Usage, QoS level, Market supply and demand Example Middleware Globus, Condor, OMII, Unicore Nimrod-G, Gridbus, & many inspired efforts

19 The Gridbus Melbourne: Enable Leasing of ICT Services on Demand WWG Pushes Grid computing into mainstream computing Gridbus

20

21 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

22 What do Grid players want? Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by utilizing distributed resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute resources for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity Strategy: maximise return on investment

23 What do Grid players require? They need tools and technologies that help them in value expression, value translation, and value enforcement. Grid Service Consumers (GSCs): How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I map jobs to resources to meet my QoS needs? How do I manage Grid dynamics and get my work done? … Grid Service Providers (GSPs) How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …

24 Principle 1: Service Oriented Architecture (SOA) A SOA is a contractual architecture for offering and consuming software as services. There are four entities that make up an SOA service provider, service registry, and service consumer (also known as service requestor). The functions or tasks that the service provider offers, along with other functional and technical information required for consumption, are defined in the service definition or contract. provider registry consumer contract

25 Principle 2: Market-Oriented (Grid) Computing- (a) Sustained Resourced Sharing and (b) Effective Management of Shared Resources Grid Economy

26 Market-based Systems = Self-managed and Self-regulated systems. Complexity present in Grid systems is similar to one present in human economies.

27 Grid Node N Service-Oriented Grid Architecture Grid Service Consumer Programming Environments Grid Resource Broker Grid Service Providers Grid Explorer Schedule Advisor Trade Manager Job Control Agent Deployment Agent Trade Server Resource Allocation Resource Reservation R1R1 Misc. services Information Service R2R2 RmRm … Pricing Algorithms Accounting Grid Node1 … Core Middleware Services … … Health Monitor Grid Market Services JobExec Info ? Secure Trading QoS Storage Sign-on Grid Bank Applications Data Catalogue

28 Gridbus and Complementary Technologies – realizing Utility Grid AIX Solaris WindowsLinux.NET Grid Fabric Software Grid Applications Core Grid Middleware User-Level Middleware Grid Bank Grid Exchange & Federation JVM Grid Brokers: X-Parameter Sweep Lang. Gridbus Data Broker MPI CondorSGETomcatPBS Alchemi Workflow IRIXOSF1 Mac Libra GlobusUnicore … … Grid Market Directory PDBCDB Worldwide Grid Grid Fabric Hardware … … PortalsScienceCommerceEngineering … … Collaboratories … … Workflow Engine Grid Storage Economy Grid Economy NorduGridXGrid ExcellGrid Nimrod-G Gridscape

29 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

30 Workflow-based Applications Workflow applications Scientific and engineering domains (e.g., biology, astronomy, chemistry) Task execution is based on their control and data dependencies. (Protein annotation workflow: London e-Science Centre)

31 Workflow for VR-Based Respiratory Treatment Planning System Surface Extraction (SJTU) Grid GenerationExperiment (CSIRO) MRI Scans VR Visualization CFD SimulationComparison Virtual Treatment

32 Driving Theme: Community Grids vs. Utility Grids Type Feature Community GridsUtility Grids User QoSBest effortContract/SLA Service Pricing Not considered / free access Usage, QoS level, Market supply and demand Example Workflow Systems Triana, MyGrid, Askalon, DAGMan, Pegasus, GrADS Kepler Gridbus Grid Workflow Engine

33 Workflow Scheduling Scheduling on Community Grids Minimize the execution time based on best effort (ignores factors such as monetary cost of resource access and various users’ QoS satisfaction levels.) Scheduling on Utility Grids Focuses on mapping workflow tasks on services to satisfy users’ QoS constraints (e.g. deadline, the quality of produced data). Supports negotiation and establishment of SLA as a contract between users and providers Optimize performance under most important QoS constraints imposed by users. Minimize execution cost while meeting a specified deadline. Minimize execution time while meeting a specified budget. Support SLA-based allocation of resources so that multiple competing demands from users can be managed with the aim of enhancing providers profit.

34 Cost-based Workflow Scheduling Objective Function Minimize the execution cost and yet meet the time constraints imposed by users. …… 4, , 20 1, 200 Time, price task ……

35 Workflow Management Systems Support composition, deployment, and execution management of workflow applications: Workflow language Graphical environment for workflow composition and monitor Grid middleware integration Data management Fault-tolerance QoS-based SLA negotiation Scheduling...

36 Grid Workflow Application Modeling & Definition Tools Grid Workflow Specification and Verification Grid Workflow Management System Resource Info Service Application Catalogue Build Time Run Time Workflow Design & Monitoring Workflow Execution Control & Monitoring Interaction with Grid resources Interaction with VO Info services QoS-based Workflow Scheduling Fault Management Data Management Data Catalogue Virtual Organization R2Rn …… SLA-based Resource Allocation System (Plug-in for Existing Local Resource Managers) Negotiation Services Execution Monitor E-Researchers/Users feedback Core Grid Services Synchrotron Data source Global Grid

37 Architecture GSP Workflow Planning Workflow Execution Workflow Management System Grid Service Grid Market Directory marketplace Service Discovery Advance Reservation ServiceRequest(SLA) contract violation ReservationRequest(SLA) Workflow Scheduling GSP: Grid Service Provider Feedback SLA: Service Level Agreement Workflow Specification Performance Estimator QoS Monitor Executor QoSRequest

38 Methodology Discover available services and estimate execution time for every task. Group workflow tasks into task partitions. Distribute users’ overall deadline into every task partition. Query available time slots, generate optimized schedule for each task partition and make advance reservations. Start workflow execution and reschedule when the initial schedule is violated at run-time.

39 Predicting Execution Time Reservation-enabled Utility Services Resource services Provide proportions of hardware resources (e.g. computing processors, network bandwidth, storage, memory) as a service for remote client access. Simulation, analytical modeling, empirical and historical data. Application services Allow remote clients to use their specialized applications. Provide estimated service times based on the metadata of user’s service requests.

40 Workflow Task Partitioning Simple task Synchronization task T6T6 T7T7 T 14 T5T5 T 10 T8T8 T2T2 T9T9 T3T3 T4T4 T 11 T 12 T 13 T1T1 T6T6 T7T7 T 14 T5T5 T 10 T8T8 T2T2 T9T9 T3T3 T4T4 T 11 T 12 T 13 Branch Before partitioning.After partitioning. T1T1

41 Deadline Assignment/Distribution P1. Any assigned sub-deadline must be greater than or equal to the minimum processing time of the corresponding task partition. P2. The overall deadline is divided over task partitions in proportion to their minimum processing time. P3. The cumulative sub-deadline of any independent path between two synchronization tasks must be same. P4. The cumulative sub-deadline of any path from entry task to exit task is equal to the overall deadline. 350 (43) (152) (217) (284) (350) (53) (187) (120) (187) (269) (350) (269) (253) 350

42 Planning Generates an optimized schedule for advanced reservation and run-time execution. Solve the problem based on divide-and-conquer. Generate a optimized schedule for each partition based on its assigned sub-deadline. A local optimized schedule minimizes execution cost while meeting its assigned sub-deadline. A optimized schedule constructed by local schedules. Task partition optimization Synchronization Task Scheduling Branch Task Scheduling 350

43 Task Partition Scheduling Synchronization task scheduling Only one task. Solution: select the cheapest service that can process the task and transfer data within the assigned sub-deadline. Branch task scheduling One simple task in a branch. Multiple tasks in a branch. Model a branch as a Markov Decision Process (MDP) T1T1 T2T2 T3T3

44 Experiments Different Workflow Structures Pipeline ParallelHybrid structure (fMRI’s neuroscience workflow) (Protein annotation workflow: London e-Science Centre)

45 (Simulation) Experiments MI (million instructions) represents length of tasks MIPS (Million Instructions per Second) represents the processing capability of services. Service type represents different types of services. 15 types of services, each supported by 10 different service providers with different processing capability. Service ID Processing Time (sec) Cost (G$) Bandwidth (Mbps) Cost/sec (G$/sec) Table I. Service speed and corresponding price for executing a task. Table II. Transmission bandwidth and corresponding price.

46 Experiments Compared heuristics Greedy cost sorts services by their prices. assigns as many tasks as possible to cheapest services without exceeding the deadline. Deadline-level divides workflow tasks into levels based on their depth in the workflow graph. assigns sub-deadlines to each task level equally.

47 Results

48 Results

49 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

50 Utility-driven Cluster RMS Architecture for GSPs

51 Economy-based Admission Control & Resource Allocation Uses the pricing function to compute cost for satisfying the QoS of a job as a means for admission control Regulate submission of workload into the cluster to prevent overloading Provide incentives Deadline -- $ Execution Time -- $ Cluster Workload -- $ Cost acts as a mean of feedback for user to respond to

52 Impact of Penalty Function on Utility

53 Normalised Comparison of FCFS, Libra & Libra+$

54 Impact of Increasing Dynamic Pricing Factor on GSP Profitability

55 Agenda Introduction Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Market-based Management and Gridbus Software Stack Grid Workflows and QoS Scheduling Architecture, Design and Implementation Performance Evaluation: Simulation based workflows SLA-based Resource Allocation Utility based allocation, pricing, performance results Summary and Conclusion

56 Summary and Conclusion Grids exploit synergies that result from cooperation of autonomous entities: Resource sharing, dynamic provisioning, and aggregation at global level  Great Science and Great Business! Grids have emerged as enabler for Cyberinfrastructure that powers e-Science and e-Business applications. SOA + Market-based Grid Management = Utility Grids Grids allow users to dynamically lease Grid services at runtime based on their quality, cost, availability, and users QoS requirements. Delivering ICT services as computing utilities. QoS Scheduling of Workflows and SLA-based resource allocation enables ability of Grids to serve as IT backbone for delivering utility computing services.

57 Thanks for your attention! We Welcome Cooperation in Research and Development! eScience2007.org

Backup MDP etc.

59 Markov Decision Process (MDP) Effective for solving sequential decision problems. A MDP model contains: A set of possible system states A set of possible actions A real valued reward (penalty) function A transition of each action’s effects in each state

60 MDP Model States A state consists of current execution task, ready time and current location. Actions An action in the MDP is to allocate a time slot on a service to a task. t : input data transmission time plus the processing time of the service. c: transmission cost plus the service cost.

61 Immediate penalty obtained from taking action a in state s and transitioning to state s’. Expected penalty The sum of immediate penalties from current state to a terminal state. The optimal action for state s is: MDP Model =, otherwise, sub-deadline Expected penalty

62 Implementation Value iteration is a standard dynamic programming algorithm compute a new value function for each state based on the current value of its next state. value iteration proceeds in an iterative fashion and can converge to the optimal solution quickly. record a number of candidate solutions while finding the optimal time slot.

63 Rescheduling Re-adjust sub-deadline and re- compute optimal schedules for unexecuted task partitions. Reschedule minimum number of tasks.