1. An Adaptive Multi-Objective Scheduling Selection Framework For Continuous Query Processing Timothy M. Sutherland Bradford Pielech Yali Zhu Luping Ding and Elke Rundensteiner Worcester Polytechnic Institute Worcester, MA, USA A Presentation @ IDEAS, Montreal, Canada, July 27, 2005

2. Continuous Query (CQ) Processing Register Continuous Queries Stream Query Engine Stream Query Engine Streaming Data Streaming Result May have different QoS Requirements. May have time- varying rates and data distribution. Available resources for executing each operator may vary over time. Run-time adaptations are required for stream query engine.

3. Runtime Adaptation Techniques Operator Scheduling Query Optimization Distribution Load Shedding Others

4. Operator Scheduling for CQ Operator Scheduling  Determines the order to execute operators  Allocates resources to query operators Existing Scheduling Algorithms  Round Robin (RR)  First In First Out (FIFO)  Most Tuples In Queue (MTIQ)  Chain [BBM03]  Others 3 1 4 stream A stream B scheduler 2 1, 2, 3, 4

5. Properties of Existing Scheduling Algorithms Uni-Objective  Designed for a single performance objective Increase throughput Reduce memory Reduce tuple delay Fixed Objective  Cannot change objective during query run May be insufficient for CQ processing

6. Performance Requirements in CQ May be multi-objective  Example Run time-critical queries on memory-limited machine Two performance goals: less result delay and less memory May vary over time  Example System resource availability may change during query run Under light workload: faster throughput Under heavy workload: less memory Existing scheduling algorithms  Not designed for multiple changing objectives  As a result, each has its strengths and weaknesses

7. Scheduling Example: FIFO FIFO Start at leaf and process the newest tuple until completion. Time : End User σ = 1 T = 0.75 σ =.1 T = 0.25 σ = 0.9 T = 1 Stream 3 1 2 0 0 0 0 3 1 2 1 0 0 0 0 3 1 2 1 0 0 0.9 1 3 1 2 1 0 0.09 0 1.25 3 1 2 2 0.09 0 0 2 3 1 2 2 0 0.9 3 3 1 2 2 0.09 0 3.25 3 1 2 3 0.18 0 0 4 FIFO’s queue size grows quickly. FIFO has fast first outputs

8. Scheduling Example: MTIQ MTIQ Schedule the operator with the largest input queue. Time : End User σ = 1 T = 0.75 σ =.1 T = 0.25 σ = 0.9 T = 1 Stream 3 2 1 0 0 0 0 3 1 2 0 1 0 0 0 3 1 2 1 1 0 0 0.9 3 1 2 2 1 0 0 1.8 3 1 2 1 0 0.1 0.8 2.25 3 1 2 1 0 0.1 1.7 3.25 3 1 2 1 0 0.2 0.7 3.5 3 1 2 1 0 0.2 1.6 4.5 3 1 2 1 0 0.3 0.6 4.75 3 1 2 1 0 0.3 1.5 5.75 3 1 2 2 0 0.4 0.5 6 MTIQ’s queue size grows at a slower rate Tuples remain queued for longer time

9. Performance Comparison

10. Summary of Problem What does CQ need:  Runtime Adaptation  Multiple prioritized optimization goals  Dynamically changing optimization goals Our solution -- AMoS  AMOS  Adaptive Multi-Objective Scheduling Selection Framework

11. Outline Introduction and Motivation The AMoS Framework Experimental Evaluation Conclusion

12. General Idea of AMoS Framework Meta-scheduler  User specify multiple performance objectives  Rank scheduling algorithms based on performances  Dynamically select the current best algorithm Design logic  Simple, low overhead and effective Algorithm Evaluator Algorithm Selector Selecting Strategy Scheduling Algorithm Library Statistics Performance Requirements Decision Request Scheduler Decision

13. Specifying Performance Objectives Metric: Any statistic calculated by the system. Quantifier: Maximize or Minimize Weight: The relative weight / importance of this metric  The sum of all weights is exactly 1. MetricQuantifierWeight Output RateMaximize0.60 MemoryMinimize0.25 DelayMinimize0.15

14. Adapting Scheduler Step 1: Scoring Statistics Periodically Step 2: Scoring Scheduling Algorithms Step 3: Selecting Scheduling Algorithm

15. Step 1: Scoring Statistics MetricQuantifierWeight Output RateMaximize0.60 MemoryMinimize0.25 DelayMinimize0.15 Output rateMemoryDelay MTIQ0.100.120.46 FIFO0.20-0.31-0.16 Chain0.15-0.500.21 Stats Score Matrix Performance Objectives Update stats scores of the current scheduling algorithm A current  Z i_new -- score of stat i for A current  μ i H, max i H, min i H -- mean, max and min history value of stat i.  μ i C -- most recent collected stat i.  decay -- decay factor in (0, 0.5). Exponentially decay out-of-date data. Z i_new ∈ (-1, 1). quantifierZ i_new meaning Maximize> 0Perform above average Maximize< 0Perform below average Minimize> 0Perform below average Minimize< 0Perform above average Output rateMemoryDelay MTIQ FIFO Chain

16. Step 2: Scoring Scheduling Algorithms Output rateMemoryDelay MTIQ0.100.120.46 FIFO0.20-0.31-0.16 Chain0.15-0.500.21 Output rateMemoryDelayScore MTIQ0.10 0.120.46 0.96 FIFO0.20-0.31-0.161.22 Chain0.17-0.50-0.211.65 MetricQuantifierWeight Output RateMaximize0.60 MemoryMinimize0.25 DelayMinimize0.15 Performance Objectives Stats Score Matrix Update score of scheduling algorithm A.  Score A -- score for A.  z i – score of stat i for A.  q i – -1 for minimize, 1 for maximize  w i – Weight in table of objectives.  Add 1 to shift from [-1, 1] to [0, 2]. Score A ∈ (0, 2). quantifierZiZi meaningqiziqizi effect on Score A Maximize> 0above average> 0Increase score Maximize< 0below average< 0Decrease score Minimize> 0below average< 0Decrease score Minimize< 0above average> 0Increase score

17. Issues In Scheduler Selection Process The framework need to learn each algorithm  Solution: All algorithms are initially run for once Algorithm did poorly earlier may be good now  Solution: Periodically explore other algorithms  Reason for adopting the Roulette Wheel Strategy

18. Step 3: Selecting Next Algorithm Roulette Wheel [MIT99] Chooses next algorithm with a probability equivalent to its score Favors the better scoring algorithms, but will still pick others. Well performed ones have better chances Others also have chances to be explored Lightweight so overhead is very low Proven to be effective in experimental study

19. Overall Flow of the Adapting Process Periodically Scoring statistics Initially run all algorithms once Rank candidate algorithms Select next algorithm to run Input: performance objectives & candidate scheduling algorithms Repeat until query is done change requested? change requested? Y N

20. Summary of the AMoS Framework Light-weight  Use runtime statistics collected by the system  Ranking formula are simple yet effective Self learning  No apriori information needed  Learn behaviors of scheduling algorithms on the fly Easily extendable  Add more scheduling algorithms  Add more performance objectives  Add more selecting strategies

21. Outline Introduction and Motivation The AMoS Framework Experimental Evaluation Conclusion

22. Experimental Setup Evaluated in CAPE system [ cape04 ]  A prototype continuous query system Query plans  Consists of join and select operators Input streams  Simulated with Poisson arrival pattern Performance objectives  Different number of objectives  Different weight of objectives

23. One Performance Objective

24. Two Performance Objectives 50% focus on output rate, 50% focus on tuple delay

25. Two Performance Objectives (cont.) 70% focus on tuple delay, 30% focus on output rate 30% focus on tuple delay, 70% focus on output rate

26. Three Performance Objectives Equal focus (33%) on output rate, memory and tuple delay

27. Conclusions Identified the lack of support for multi-objective adaptation  Existing approaches only focus on single objective  Cannot change objective during query run Proposed a novel scheduling framework:  Allows applications to control performance objectives  Alters scheduling algorithm based on run-time performances  Independent of scheduling algorithms or performance objectives AMoS strategy shows very promising experimental results.  Developed and evaluated in the CAPE system  W/ single objective, performs as well as the best algorithm  W/ multiple objectives, overall better than any algorithm

28. Thank You! For more information, please visit: davis.wpi.edu/~dsrg