1. By: Gang Zhou Computer Science Department University of Virginia 1 Medians and Beyond: New Aggregation Techniques for Sensor Networks CS851 Seminar Presentation

2. CS851 2005 Gang Zhou 2 Outline  Motivations, State of Art, Contributions  The Q-Digest Scheme  Queries on Q-Digest  Experimental Evaluation  Conclusions Be prepared! I have questions for you! Be prepared! I have questions for you!

3. CS851 2005 Gang Zhou 3 Motivations  Trade Computation for Communication  Transmitting one bit over radio is at least three orders of magnitude more expensive in terms of energy consumption than executing a single instruction  Support Aggregation Queries  Need aggregated answer, not a single raw reading  Quantile query  Nth  value  Reverse quantile query  Value  Nth  Consensus query  Most frequent?  Histogram

4. CS851 2005 Gang Zhou 4 State of Art  TinyDB project in Berkeley & Cougar project in Cornell  Pros:  Energy efficient in-network data aggregation  Work very well in singleton sensor values  MIN, MAX, AVERAGE, SUM, COUNT  Cons:  Do not deal with complex aggregate measures  Median, Quantile, Reverse Quantile, Consensus  [Zhao et. al. 2003]  Algorithms for constructing summaries like MAX, AVG  Focus more on network monitoring and maintenance  [Przydatek et. al. 2003]  Secure aggregation

5. CS851 2005 Gang Zhou 5 Contributions  Propose Q-Digest for Approximated Aggregation  Provide Strict Theoretical Guarantees on the Approximation Quality of the Queries in Terms of the Message Size  Evaluate the performance of Q-Digest in Simulation

6. CS851 2005 Gang Zhou 6 Roadmap  Motivations, State of Art, Contributions  The Q-Digest Scheme  Queries on Q-Digest  Experimental Evaluation  Conclusions and Discussions

7. CS851 2005 Gang Zhou 7 Properties of Q-Digest  Each node v in tree T is a bucket;  Whose range [v.min, v.max] defines the position and width of the bucket;  Has counter count(v);  Given the compression parameter K, a node v is in q-digest iff it satisfies:  (1) If not a leaf, no high count;  (2) If not the root, a node and its children should not have low count;  A q-digest is a set of buckets of different sizes and their associated counts;

8. CS851 2005 Gang Zhou 8 Building a Q-Digest  Going bottom up to check whether any node violates digest property (2)  If yes, delete itself and its sibling, and merge to its parent;  Key feature of q-digest: Detailed information concerning data values which occur frequently are preserved in the digest, while less frequently occurring values are lumped into larger buckets resulting in information loss.

9. CS851 2005 Gang Zhou 9 Merging Q-Digest  Parent node merge Q1(n1,K) and Q2(n2,K) from children How about merging Q1(n1,k1) and Q2(n2,K2)?  Each node has different communication ability  Each node has different power level  Powerful node can have bigger K while less powerful node can have smaller K value. Can we still get the same accuracy? Is that feasible?

10. CS851 2005 Gang Zhou 10 Space Complexity and Error Bound (1/4) What dos it mean 3K? 3K bites? The root node does not satisfy property (2).?? 3K means 3K pairs

11. CS851 2005 Gang Zhou 11 Space Complexity and Error Bound (2/4) What about the leaf node, which does not satisfy property (1)? It doesn’t matter, because a leaf node is not the ancestor of any node.

12. CS851 2005 Gang Zhou 12 Space Complexity and Error Bound (3/4)

13. CS851 2005 Gang Zhou 13 Space Complexity and Error Bound (4/4)

14. CS851 2005 Gang Zhou 14 Representation of a Q-Digest  Now to transmit the q-digest we send a set of tuple of the following form which requires a total of bits for each tuple.

15. CS851 2005 Gang Zhou 15 Roadmap  Motivations, State of Art, Contributions  The Q-Digest Scheme  Queries on Q-Digest  Experimental Evaluation  Conclusions and Discussions

16. CS851 2005 Gang Zhou 16 Quantile Query(1/3)  Quantile query:  Given a fraction 0<q<1, find the value whose rank in sorted sequence of the n values is qn.  Answer the query:  Sort nodes in q-digest in increasing v.max; breaking ties by putting smaller ranges first;  Scan the sorted list and add the counts of nodes;  For some node v, the sum becomes more than qn, and the v.max is reported as the estimate of the quantile;

17. CS851 2005 Gang Zhou 17 Quantile Query(2/3)  The confidence factor  Why need this?  is the worst case error estimation, which only occurs for a very pathological input case  What is it?  Confidence factor is defined as: (maximum weight of any path from root to leaf in Q)/n

18. CS851 2005 Gang Zhou 18 Confidence Factor Example  N=15, k=5, =8 1 1 5 7 3 3 3 3 1 1 5 7 3 3 3 3 (maximum weight of any path from root to leaf in Q)/n = 7/15 < < = 3 * log8 / 3K = 3*3/3*5 = 9/15 = 3 * log8 / 3K = 3*3/3*5 = 9/15

19. CS851 2005 Gang Zhou 19 Roadmap  Motivations, State of Art, Contributions  The Q-Digest Scheme  Queries on Q-Digest  Experimental Evaluation  Conclusions and Discussions

20. CS851 2005 Gang Zhou 20 Performance Evaluation  Settings  Routing tree  Breadth first search tree  Sensor field  1000 x 1000 area with 1000 sensor nodes  2000 x 2000 area with 4000 sensor nodes  Sensor value  Random  Correlated :  United States Geological Survey  Compare with List scheme:  List: Report all (value, count) back to base station; no in-network aggregation;

21. CS851 2005 Gang Zhou 21 Error and Message Size  160 bytes message size can get 5% error  400 bytes message size can get 2% error

22. CS851 2005 Gang Zhou 22 Total Data Transmission  Q-digest transmit less data than list  Random input needs more transmission than correlated data

23. CS851 2005 Gang Zhou 23 Residual Power  For every byte transmitted, one unit of 40000 unit of power is depleted. (How about reception?)  In List, 0.02% nodes have residual power fraction less than ½. (???)

24. CS851 2005 Gang Zhou 24 Conclusions  Propose Q-Digest for Approximated Aggregation  Provide Strict Theoretical Guarantees on the Approximation Quality of the Queries in Terms of the Message Size  Evaluate the performance of Q-Digest in Simulation

25. CS851 2005 Gang Zhou 25 Thank you!