A Prediction-based Real-time Scheduling Advisor Peter A. Dinda Carnegie Mellon University

2 Outline Real-time scheduling advisor model and interface Prediction-based implementation Randomized evaluation using load trace playback

3 The Problem Solved by the Real-time Scheduling Advisor At time t now, the application gives you a task with compute requirements t nom, a deadline t now +t nom (1+slack), a confidence level c, and a list of hosts in a shared, unreserved distributed computing environment. The application can run the task on any of the hosts. Choose a host from the list such that the task, if run on that host, will meet the deadline with probability c or better, if possible.

4 Model Task model Compute-bound Initiated by user actions (interactive applications) Arrive aperiodically Do not overlap Must be started immediately (t now ) Application model Knows task’s compute requirements (t nom ) Knows appropriate slack for task –deadline = t now + (1+slack)t nom Can run task on one of a set of hosts Real-time scheduling advisor recommends the most appropriate host

5 RTSA Interface int RTAdviseTask(RTSchedulingAdvisorRequest &req, RTSchedulingAdvisorResponse &resp); struct RTSchedulingAdvisorRequest { double tnom; double slack; double conf; Host hosts[]; } struct RTSchedulingAdvisorResponse { double tnom; double slack; double conf; Host host; RunningTimePredictionResponse runningtime; } Deadline: t now + t nom (1+slack) Hosts to choose from Required certainty of meeting deadline Most appropriate host Confidence interval for running time on host

6 Prediction-based Implementation

7 Anchoring this talk Studied statistical properties of host load signals Found appropriate predictive models for host load signals Developed RPS toolkit for building fast, low overhead resource prediction systems Developed load trace playback technique for reconstructing load Built host load prediction system This talk: description and evaluation of the real-time scheduling advisor Assume this works (later talk)

8 Scheduling Strategies Prediction-based (MEAN, LAST, AR(16)) Operation –Acquire running time predictions for each host –Select host at random from those where confidence interval is below deadline –If none exist, choose host with lowest expected running time Return host and running time prediction MEASURE Return host with current lowest measured load No running time prediction RANDOM Return random host No running time prediction

9 Performance Metrics Fraction of deadlines met “Will the deadline be met?” Depends on (at least) strategy, slack, and resource availability Fraction of deadlines met when possible “If strategy claims deadline will be met, will the deadline be met? Should depend only on strategy Application can try other t nom, slack Number of possible hosts “How much randomness is introduced?” Helps to avoid disastrous advisor synchronization

10 Methodology Recreate “scenario” (load on a set of hosts) on manchester testbed using load trace playback Schedule and run randomized tasks random arrival times (5 to 15 seconds apart) t nom randomly selected from 0.1 to 10 secs Slack randomly selected from 0 to 2 Randomly selected strategy Data-mine results

11 4LS Scenario Four PSC alpha cluster hosts axp0 (interactive), axp4, axp5, axp10 (batch) high load, high variability Traces start Tuesday, August 12, ,000 tasks run in 36 hours

12 Terminology I will Use Scheduling feasibility How likely it is that a host exists on which deadline can be met Increases with slack, decreases with t nom Also depend on variation among the hosts Predictor sensitivity How likely that the deadline will be missed due to a bad prediction Low when scheduling feasibility is high or low Highest near critical slack Critical slack Slack at which scheduling feasibility is 50%

13 Overview of Results AR(16) prediction-based strategy is superior Fraction of deadlines met at least as good as MEASURE, and much improved at critical slack Fraction of deadlines met when possible higher than all competitors and most independent of slack and nominal time Introduces similar randomness as other prediction-based strategies Performance metrics depend slack, nominal time

14 Fraction of Deadlines Met Versus Slack

15 Fraction of Deadlines Met Versus t nom

16 Fraction of Deadlines Met Versus t nom (near critical slack)

17 Fraction of Deadlines Met When Possible Versus Slack

18 Fraction of Deadlines Met When Possible Versus t nom

19 Fraction of Deadlines Met When Possible Versus t nom (Near Critical Slack)

20 Number of Possible Hosts Versus Slack

21 Number of Possible Hosts Versus t nom

22 Number of Possible Hosts Versus t nom (Near Critical Slack)

23 Conclusions MEASURE greatly increases chance of meeting deadlines compared to RANDOM AR(16) increases that chance with miniscule additional overhead Especially near critical slack and for short tasks In addition, AR(16) can tell the application, with high accuracy, whether the deadline will be met before the task is run Gives the application opportunity to negotiate AR(16) introduces appropriate randomness into their choices, reducing chance of conflict AR(16) Prediction-based Real-time Scheduling Advisor is a useful tool