1 Rethinking Data Management for Storage-centric Sensor Networks Yanlei Diao, Deepak Ganesan, Gaurav Mathur, and Prashant Shenoy CIDR 2007 Proceedings of the Third Biennial Conference on Innovative Data Systems Research (CIDR), Asilomar, CA, January 2007.

2 STONES STONES Project STONES  STO rage for N etworked E mbedded S ystems  Energy-efficient Storage for Sensors  Sensor Database  PRESTO, TSAR, Capsule, …etc

3 Papers PRESTO: Feedback-Driven Data Management in Sensor Networks ACM/USENIX NSDI 2006 TSAR: A Two Tier Sensor Storage Architecture Using Interval Skip Graphs ACM Sensys 2005 Capsule: An Energy-Optimized Object Storage System for Memory-Constrained Sensor Devices ACM Sensys 2006 Proxy Cache Sensor Nodes

4 PRESTO Introduction PRESTO Proxy PRESTO Sensor precision(query) > confidence interval x(t+1)-x(t) > worst-case deviation

5 TSAR Introduction

6 Capsule Introduction

7 Outline Introduction StonesDB Architecture Local Database Distributed Data Management Current Status and Conclusions References

8 Introduction Data Management in Sensor Networks Live Data Management  Real-time queries  Only small window of data is important  Event detection and notification  Push-down Filters, AQP, …etc Archival Data Management  Database outside the sensor networks  View sensor networks as database  Analysis of past events, Historical trends

9 Introduction Example: Smart Home and Smart Biz Live Data Management? Archival Data Management?

10 Introduction Centralized Archival Data Management Internet DBMS Database Management System User Query Data Access Sensors with high data rate?! camera, acoustic, vibration… lossless aggregation… Low data rate, High query rate 22.1 ˚C 21.5 ˚C 21.8 ˚C ?

11 Introduction Storage-centric Archival Data Management Internet User Query Data Access Local Storage Flash Memory acoustic image Push query to sensors!  limited capabilities  flash memory efficiency High data rate, Low query rate ?

12 Introduction Sensor Node Hardware Today Mica2 mote  6 MHz Processor  4 KB RAM  128 KB FLASH iMote2  13 – 416 MHz Processor  32 MB RAM  32 MB FLASH

13 Introduction Technology Trends Communication Storage Energy cost of storage compared to that of communication Generation of Sensor Platforms Energy Cost (per byte)

14 Design Goals Exploit local flash memory  Cheap, energy-efficient flash memory Exploit resources-rich proxies  cache data and split query plans Support a rich set of queries  SQL-type queries  data mining and similarity search queries Support heterogeneity  configurable to heterogeneous sensor platforms

15 StonesDB Architecture Two-tier Sensor Networks Local Database Distributed Data Management Layer user specified confidence bound

16 StonesDB Architecture System Operations Image Retrieval … Proxy Cache of Image Summaries 找出沒有洋蔥頭臉部表 情的圖片...

17 StonesDB Architecture System Operations Image Retrieval 找出沒有洋蔥頭臉部表 情的圖片... Query Engine Partitioned Access Methods … Sensor Local Storage

18 Local Database Architecture of Local Database Stream Index Summary

19 Local Database Costs and Benefits of Access Methods Cost for B+ tree insertion  Cost for sequential scan  H-level B+ tree cost for page read/write readings per page Sequential scan is 340 times more energy efficiency! When depth of B+ tree is 2… Scan is better when data is not accessed very frequently. Lazy index construction!

20 Local Database Partitioned Access Methods temporal segments B+ TreeR Tree Write-Once Indexing!

21 Local Database Summarization and Aging All available storage gets filled…  When to drop these summaries?  How to drop these summaries?  Graceful query quality degradation. local storage capacity Resolution 4 Resolution 1Resolution 2 Resolution 3 Multi-resolution Summarization: Local Storage Allocation

22 Local Database Summarization and Aging High query accuracy Low compactness Low query accuracy High compactness How long should a summary be stored in the network?

23 Local Database Summarization and Aging Query Accuracy Time Quality Difference present past 95% 50% user-desired quality degradation system-provided step function Objective: minimize the worst case quality difference

24 Distributed Data Management The Problems Proxy Cache of Image Summaries What summaries to cache? What resolution of summaries? How should a query plan be split? I want the data of …

25 Distributed Data Management Querying the Proxy Cache Internet User Query Statistical models? Low-resolution data? Metadata of images? Response Gateway, Proxy Sensor Nodes summaries from sensors to the proxy queries from the proxy to sensors query execution at the sensors results back to the proxy min

26 Distributed Data Management Querying the Sensor Tier Gateway, Proxy Sensor Nodes Cache miss… Not meet accuracy requirement? User Query How to split the query plan? Use Query Processing Engine…

27 Distributed Data Management Querying the Sensor Tier Sensor Nodes Gateway, Proxy Number of cars over past half hour? Proxy Cache of Image Summaries Partially process the query at the proxy!

28 Distributed Data Management Querying the Sensor Tier Sensor Nodes Gateway, Proxy Average temperature between PM 1:00 - 2: Proxy Cache of Data Summaries Refine the result at the sensor node! … 19.2 ˚C 19.5 ˚C 20.2 ˚C 21.1 ˚C 20.0 ˚C 18.6 ˚C average temperature every two hours

29 Current Status and Conclusions Implemented Capsule  Flash-based object store  Energy-efficient data structure (lists, arrays, trees)  Currently extended with summarization, aging, and partitioned indexing. Other related systems  TSAR: separate data from metadata  PRESTO: implement a proxy cache No running system for StonesDB architecture.

30 References The STONES Project STONES  CAPSULE  PRESTO  The Wireless Sensor Networks Group at UMASS 