Time Sensitive Computation of Aggregate Functions over Distributed Imprecise Data Qi Han, Matthew Ba Nguyen Sandy Irani, Nalini Venkatasubramanian Distributed Systems Middleware Group http://www.ics.uci.edu/~dsm School of Information and Computer Science University of California-Irvine

Motivation Real time applications often make decisions based on timely results aggregated over distributed data Example: real-time fault localization and problem diagnosis How many nodes are overloaded? (count) What is the total latency along the path N1N2    Nk? (sum) What is the bottleneck (minimum link bandwidth) along a path N1N2    Nk? (min) Recent advances in communication, mobile computing and embedded systems have enabled a variety of real-time distributed applications such as stock information exchange, environmental sensing etc. These applications often make decisions based on timely results aggregated over distributed data from varying source. For example, as distributed systems and networks continue to grow in size and complexity, real-time fault localization becomes more challenging. It is crucial to build an active real time information gathering system that can ask the right question at the right time.

timeliness, accuracy and cost-effectiveness Challenges Continuous stream of fast changing source data Diverse user requirements in terms of data accuracy and service timeliness Effective utilization of underlying computation, communication and storage resources competing goals of timeliness, accuracy and cost-effectiveness Providing a real-time information architecture poses several challenges. Firstly, information sources provide a continuous stream of data that can dynamically vary over time. The information may need to be captured and stored rapidly and accurately. Secondly, users requiring access to this data present diverse requirements in terms of accuracy of the data and timeliness of the service. Thirdly, the collection should be done in an unobtrusive manner. Therefore, we are faced with the competing goals of timeliness, accuracy and cost. Fortunately, many applications are willing to tolerate information imprecision and bounded delivery latency. We would like to exploit these accuracy and latency margins to ensure that most applications receive information at the desired level of accuracy and timeliness while minimizing resource consumption.

This paper… Previous research This paper addresses Tradeoff between accuracy and cost storing data in ranges Tradeoffs between accuracy, cost and timeliness (single data item) – RTSS’2003 This paper addresses Tradeoffs between accuracy, cost and timeliness for aggregate functions (count, sum, min) Probing part of the sources might be sufficient How the server selects an appropriate subset of sources to probe so that the overall probing cost is minimized without violating accuracy and timeliness requirements of aggregate queries Given the data intensive nature of the system, a database is a must for the system to function efficiently. Range-based approach… Directly applying the approaches to distributed real time applications will not be effective. Since queries will not only have accuracy constraints, but also time constraints which specify the latest time by which the results of an aggregate query are expected to be available. This paper complements previous work in RTSS 2003 by addressing how the server… Specifically,

Problem Characterization s1: E1 [L1,U1] s2: E2 [L2,U2] si: Ei [Li,Ui] Sn: En [Ln,Un] …... server c1/d1 c2/d2 ci/di cn/dn Database [L1,U1],[L2,U2] … [Ln,Un] Queries: f(s1,s2,…,si…,sn), A, D A: accuracy constraint D: time constraint Example: min(s1,s2…,sn), 3, 1 minute System model: Source model: Query model: T_f varies (depending on whether it is parallel or sequential probing) After probing l=u=e Cost and latency in probing each source vary A: the actual value differs from the returned answer by at most A source 1 source 2 source i source n

Time-sensitive Computation of Aggregate Functions Compute the function based on stored values in the database If the answer meets accuracy constraint: done Otherwise: select probing set Only consider SD: the subset of sources whose diD Batch selection The entire set of sources to probe is selected before the probes actually occur Iterative selection The source is probed one at a time In order to achieve the goal of minimizing the probing cost under time and accuracy constraints of user queries, we compute the function based on stored approximations in the database. If the answer does not satisfy the accuracy constraints of the user request, we decide on a set of sources to probe for exact values in order to improve the answer precision. Two basic approaches to probing set selection can be applied. In batch selection, the precision constraint must be guaranteed for any possible precise values for the sources in the probing set; iterative selection is an online approach, the function is evaluated every time after a source is probed and stopped when the answer is precise enough. The answer gradually refines to be more precise over time. In this case, the goal is to shrink the answer as fast as possible.

Batch Selection of Source Probing Set For Count (Batch_COUNT) Problem: calculate the number of source values that fall inside the range r=[l,u]: fcount=|{si|si[l,u]}| l r u Inside s1 s2 Outside s3 s4 s5 Uncertain s6 Algorithm: if |U|>A: we must probe |U|-A sources to determine the function within the desired accuracy If |USD||U|-A: order the sources in |USD| according to increasing cost and probe the first |U|-A sources in this ordering o/w: cannot determine the the function to within the desired accuracy We can divide the sources into three sets: Inside, outside and uncertain.

Batch Selection of Source Probing Set for Sum (BATCH_SUM) If we compute the function based on the stored intervals in the database, then the smallest possible sum occurs when all values are the lower bounds, and the largest possible sum occurs when all values are the upper bounds.

Batch Selection of Source Probing Set for Min (BATCH_MIN) Example:

Issues in Computing min In the case of computing count or sum Know in advance exactly the benefit of probing any particular source, thus can decide in advance which sources to probe count: probing any source decreases the uncertainty by 1 sum: probing source si decreases the uncertainty by ui-li In the case of computing min The number of probes required may vary depending on the values of the sources Example: s1=[0,5], s2=[1,6], A=1 If probe s1 and e1=2, then min=[1,2], done If probe s1 and e1=5, then min=[1,5], so s2 must be probed

Iterative Selection of Source Probing Set for Min BATCH_MIN is a worst case analysis which assumes that the values returned always maximize the remaining uncertainty. Now we consider an average-case approach in which we assume that the value of each source is distributed uniformly over its range.

Performance Evaluation Baseline policies compared with GREEDY: probe all LAZY: probe none RANDOM Performance metrics Cost Accuracy ratio (a/A): measures how close the answer interval a matches the accuracy constraint A Latency ratio (d/D): measure how close the time d spent answering the query matches the time constraint D Accuracy satisfaction ratio: the percentage of queries with their accuracy constraints met Deadline satisfaction ratio: the percentage of queries with their time constraints met When a/A<=1, the accuracy requirement is met; the smaller, the more accurate.

Basic Performance Results for Computing Count Not surprisingly, GREEDY achieves the best answer accuracy at the price of highest cost and latency. In contrast, LAZY provides the most coarse answer instantly by not probing any sources. BATCH exhibits similar answer accuracy and latency to RANDOM with slightly lower probing cost. However, more queries meet their deadlines by using BATCH. This is because BATCH gives higher priority to time constraints than accuracy constraints, I.e., the best possible answer (in terms of accuracy) is provided only if the deadline is met.

Basic Performance Results for Computing Min Comparing to RANDOM, the deadline satisfaction ratio of BATCH is much higher than RANDOM, since it does not probe those sources whose probing latency is higher than D. Iterative selection probes sources sequentially, the latency of a query is the sum of each probing latency. Given the same D the number of sources to be probed is decreased than BATCH, which leads to higher accuracy ratio (less accurate result) and lower accuracy satisfaction ratio.

Performance of Computing Count under varying accuracy constraints When A is small, only probing sources in S_D cannot provide a satisfactorily accurate answer, that is why the first part of the curve is horizontal; When A increases, fewer probings can provide accurate answer.

Performance of Computing Count under varying time constraints The several turning points in the curve of the probing cost matches exactly the several stages of the algorithms. We only probe S_D. Smaller deadline leads to small S_D, therefore probing cost increases as deadline increases. At the same time, the accuracy ratio decreases but still larger than 1. When the deadline reaches a point where we can select a subset of S-D to probe, the probing cost is decreased since we select those with smaller probing costs. When the deadline increases further, no more improvement will be obtained since the same subset of sources will be probed to meet the accuracy and time constraints.

Conclusions The worst case analysis (batch selection algorithms) provides a bound on the cost to satisfy queries regardless of the exact values of sources More sophisticated models such as Gaussian distribution can be used to capture the change of source values Also interesting to conduct competitive analysis of algorithms for answering aggregate queries Worst case analysis assumes that the values returned always maximize the remaining uncertainty