Download presentation
Presentation is loading. Please wait.
Published byEleanor Willis Modified over 5 years ago
1
Streaming Sensor Data Fjord / Sensor Proxy Multiquery Eddy
Architecture for combining streaming data with static data sources Streaming (Windowed) Operators Multiquery Eddy Share memory and processing between stream queries TinyOS and Telegraph Enhancements to TOS for Sensor Data Processing
2
Fjords Query-plan like data structure for combining streaming (push) and traditional (pull) data sources. Operators assume non-blocking queue interface between each other. Queues implement push vs. pull Pull from A to B : Suspend A, schedule B until it produces data. A cannot go forward until B produces data. Push from B to A : A polls, scheduler thread invokes B until it produces data. A can process other inputs while waiting for B. Supports parallelism between operators via queues, state machines, and OS (e.g. NIC buffers, DMA) in operator transparent way.
3
Fjords (Continued) Key Insight: Stream-based systems need to operate on traditional (pull-based) sources too! Example: Combine traffic streams with web-based accident reports to correlate accidents with impact on freeway conditions. Existing streaming solutions cannot do this!
4
Fjord Operator Example
Example: Zipper Join (similar to band-join, Dewitt et. al. VLDB '91). Operator agnostic with respect to data-flow direction on input and output.
5
Sensor Proxy Energy-sensitive database operator
Buffer sensor tuples and route to multiple user queries to hide query load from sensors Push aggregation operators into sensors to reduce communications load Dynamically adjust sample rate based on user demand Push results into Fjords so that other operators dont block waiting on slow or dead sensors
6
Multiquery Eddy Observation: Queries over streams apply only to now.
Old and new queries always looking at same point in data set Idea: only allocate on copy of each tuple; route that copy to all user queries Second idea: combine operators over the same data stream to increase efficiency
7
Multiquery Eddy (Example and Performance)
Tuple Throughput vs Number of Queries
8
TinyOS and Telegraph Goal: Enhance TinyOS and TinyOS sensors with infrastructure to participate in Telegraph queries Establish a consistent catalog and sampling interface across all TinyOS sensors Enable selection predicate / aggregation push- down into sensors Partially distribute query plans across sensor networks
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.