Presentation is loading. Please wait.

Presentation is loading. Please wait.

Streaming Sensor Data Fjord / Sensor Proxy Multiquery Eddy

Similar presentations


Presentation on theme: "Streaming Sensor Data Fjord / Sensor Proxy Multiquery Eddy"— Presentation transcript:

1 Streaming Sensor Data Fjord / Sensor Proxy Multiquery Eddy
Architecture for combining streaming data with static data sources Streaming (Windowed) Operators Multiquery Eddy Share memory and processing between stream queries TinyOS and Telegraph Enhancements to TOS for Sensor Data Processing

2 Fjords Query-plan like data structure for combining streaming (push) and traditional (pull) data sources. Operators assume non-blocking queue interface between each other. Queues implement push vs. pull Pull from A to B : Suspend A, schedule B until it produces data. A cannot go forward until B produces data. Push from B to A : A polls, scheduler thread invokes B until it produces data. A can process other inputs while waiting for B. Supports parallelism between operators via queues, state machines, and OS (e.g. NIC buffers, DMA) in operator transparent way.

3 Fjords (Continued) Key Insight: Stream-based systems need to operate on traditional (pull-based) sources too! Example: Combine traffic streams with web-based accident reports to correlate accidents with impact on freeway conditions. Existing streaming solutions cannot do this!

4 Fjord Operator Example
Example: Zipper Join (similar to band-join, Dewitt et. al. VLDB '91). Operator agnostic with respect to data-flow direction on input and output.

5 Sensor Proxy Energy-sensitive database operator
Buffer sensor tuples and route to multiple user queries to hide query load from sensors Push aggregation operators into sensors to reduce communications load Dynamically adjust sample rate based on user demand Push results into Fjords so that other operators don’t block waiting on slow or dead sensors

6 Multiquery Eddy Observation: Queries over streams apply only to now.
Old and new queries always looking at same point in data set Idea: only allocate on copy of each tuple; route that copy to all user queries Second idea: combine operators over the same data stream to increase efficiency

7 Multiquery Eddy (Example and Performance)
Tuple Throughput vs Number of Queries

8 TinyOS and Telegraph Goal: Enhance TinyOS and TinyOS sensors with infrastructure to participate in Telegraph queries Establish a consistent catalog and sampling interface across all TinyOS sensors Enable selection predicate / aggregation push- down into sensors Partially distribute query plans across sensor networks


Download ppt "Streaming Sensor Data Fjord / Sensor Proxy Multiquery Eddy"

Similar presentations


Ads by Google