Preference-Aware Query and Update Scheduling in Web-databases Huiming Qu Department of Computer Science University of Pittsburgh Joint work with Prof. Alex Labrinidis

Huiming Qu, University of 12/18/ Applications  Applications Online stock info. Weather, news… Inventory monitoring Network monitoring  Web-DB workloads Real-only queries Write-only updates Intensive and/or bursty  We focus on scheduling queries and updates to enhance Web- Database performance WWW Browser WWW Server Web-database Data Warehouse Queries Updates

Huiming Qu, University of 12/18/ Why scheduling?  Users care about Timeliness and Staleness  Impact of scheduling A simple test  FIFO  FIFO-UH (Update High)  FIFO-QH (Query High) 2PL-HP [Abbott92] No best for both dimensions  Combining performance metrics Set constraint on one metric and optimize another [Kang04] Construct a single metric based on weighted aggregation [Abadi05]

Huiming Qu, University of 12/18/ Outline  Introduction  Quality Contracts Framework for unifying timeliness and staleness metrics through user preferences  QUTS Query Update Time Share Scheduling according to Quality Contract framework  Experiments  Related work  Conclusions

Huiming Qu, University of 12/18/ Quality Contracts (QC)  Captures user preference Between quality metrics Among different queries  Quality Contract Convert performance on individual metric to the “worth” to users

Huiming Qu, University of 12/18/ Quality Contract (QC) – cont.  Each Query is attached with a QC monotonically non- increasing positive QC functions QC  Optimization goal system profit, P  More details in [Labrinidis06] [Qu07] QoS function QoD function Step QC QoS function QoD function Linear QC

Huiming Qu, University of 12/18/ Outline  Introduction  Quality Contracts  QUTS: Query Update Time Share Scheduling  Experiments  Related work  Conclusions

Huiming Qu, University of 12/18/ Profit guided scheduling  Considers both value and constraint Scheduling according to utility function in real time systems VRD [haritsa93] Value/constraint inflated priorities for queries and updates valueconstraint Queriesqos max rt max Updatesqod max uu max OK Incomparable V RD

Huiming Qu, University of 12/18/ Baseline algorithms  Single priority queue FIFO w/ incomparable priorities, queries and updates cannot be put together  Dual priority queue [adelberg95] Separate update queue & query queue Prioritize queries and updates correspondingly UH (Update High) QH (Query High)  Eliminate hard-wired preference for either QoS or QoD! Profit-guided resource allocation (which queue to execute)

Huiming Qu, University of 12/18/ QUTS (Query Update Time Share)  Dual priority queue with dynamic concatenation  Two-level scheduling scheme (meta-scheduling) Q1: how much CPU should be allocated to queries? Q2: how to deploy CPU allocation?

Huiming Qu, University of 12/18/  Query CPU percentage ρ (0 ≤ ρ ≤ 1)  Approximate system profit P using ρ Q1 - how much CPU to assign?  Optimal solution to maximize P QoS max profit QoS QoD Query share QoD max profit Update shareQuery share  Quadratic function with linear constraints

Huiming Qu, University of 12/18/ Q2 - how to deploy CPU allocation?  Periodically compute ρ and alleviate random variation  With in a single adaptation period, the percentage of CPU assigned to queries should be ρ. time

Huiming Qu, University of 12/18/ Q2 - how to deploy CPU allocation?  Adaptation period ω is divided to N atom time  For each, the system refreshes the system state according to ρ Query High: Update High:  Length of atom time Too small → too many interruptions Too large → miss opportunities time … ρ = 0.5

Huiming Qu, University of 12/18/ Outline  Introduction  Quality Contracts  QUTS: Query Update Time Share Scheduling  Experiments  Related work  Conclusions

Huiming Qu, University of 12/18/ Experiment Setup  Real stock web site traces on April 24, 2000 # queries: # updates: # stocks: 4602 Query execution time5~9ms Update execution time1~5ms Default atom time10ms Default adaptation time1000ms AlgorithmsFIFOUHQHQUTS Meta SchedulingFIFOUpdate highQuery highProfit guided Query prioritiesFIFOVRD (Value over Relative Deadline) Update prioritiesFIFOVRD (Value over Relative Deadline)  Scheduling algorithms compared through simulation

Huiming Qu, University of 12/18/ Experiment Design 1. Performance Compare QUTS with FIFO, UH, and QH under the entire spectrum of QC; 2. Adaptability Evaluate the adaptability of QUTS to changing workloads (QCs); 3. Sensitivity Evaluate the sensitivity of QUTS to its two parameters (ω and ).

Huiming Qu, University of 12/18/ Performance – QC setup  QC  Performance metric Profit Percentage QOD max %0.10.2…0.9 QOS max %0.90.8…0.1 qod max $10 ~ $19$20 ~ $29…$90 ~ $99 qos max $90 ~ $99$80 ~ $89…$10 ~ $19 rt max 50ms ~ 100ms uu max 1 9 sets of QC groups

Huiming Qu, University of 12/18/ Performance – All QC groups Linear QCs Step QCs QUTS gives near maximal profit from both QoS and QoD. Step QCs and linear QCs follow the same trend.

Huiming Qu, University of 12/18/ Performance – Various QC groups FIFO QH UH QUTS QUTS consistently achieves near maximal profit under the whole spectrum of QC

Huiming Qu, University of 12/18/ QUTS Adaptability  Time is divided into four segments.  In odd segments, each QC has qod max = 5 x qos max  In even segments, each QC has qos max = 5 x qod max QUTS adapts to QC changes fast!

Huiming Qu, University of 12/18/ QUTS Sensitivity  Adaptation Period ω Performance varies little for a wide range of ω  Atom Time Better when ≥ transaction execution time ( = 10ms) (ω = 1000ms)

Huiming Qu, University of 12/18/ Outline  Introduction  Quality Contracts  QUTS: Query Update Time Share Scheduling  Experiments  Related work  Conclusions

Huiming Qu, University of 12/18/ Related work  Web-databases [Adelberg et al. 1995] [Challenger et al. 2000] [Luo et al. 2002] [Datta et al. 2002] [Labrinidis et al. 2004] [Qu et al. 2006] [Labrinidis et al. 2006] [Xu et al. 2006] …  Real time databases [Abbott et al., 1988] [Sha et al., 1991] [Haritsa et al., 1993] [Ramamritham et al., 1994] [Burns et al., 2000] …  Stream Processing [Carney et al., 2002] [Das et al., 2003] [Babcock et al., 2004] [Sharaf et al., 2005] [Abadi et al., 2005] …  Economic Models [Ferguson et al., 1996] [Stonebraker et al., 1996] …

Huiming Qu, University of 12/18/ Conclusions  We proposed a Quality Contract framework to capture user preferences in web-database systems, and a meta schedule scheme, QUTS, to dynamically concatenate query queue and update queue according to user queries.  We showed with extensive simulation study on real data traces that QUTS outperforms the baseline algorithms in the whole spectrum of Quality Contract setup, QUTS adapts fast to the changing workloads, and QUTS has little sensitivity to its two parameters.

Huiming Qu, University of 12/18/ Future work  Quality Contract generation strategies Synthetic workload Provide suggestions to users to select QCs I/O scheduling for disk-based Web databases

Huiming Qu, University of 12/18/ Thanks!  Comments & Questions? Huiming Qu

Huiming Qu, University of 12/18/ UNIT System (User-ceNtrIc Trans- action Management)  Web-databases Dual priority queue  Updates > queries  EDF for queries  FIFO for updates 2PL-HP  UNIT: load control Load Balancing Controller Query Admission Control Update Frequency Modulation USM Load Balancing Controller Queries Data Reject Failure Deadline Missed Failure Success Data Stale Failure UNIT +/- queries Admission Control Frequency Modulation Updates Statistics +/- updates

Huiming Qu, University of 12/18/ Sensitivity (atom time)  Profit drops slowly while atom time increases.  Extra versions need to be maintained increase while atom time decreases. Sweet spot

Huiming Qu, University of 12/18/ Workload characteristics

Huiming Qu, University of 12/18/ Questions  Skewed queries  Atom time -> number of tran.  Why restarts  Network delay comparable to database delay when facing high loads of updates Scalability point of view  Plot the correlation on time (U vs. Q)