Presentation is loading. Please wait.

Presentation is loading. Please wait.

3/14/2005 CS851: Data Services for Advanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash.

Published byKellie Rice Modified over 9 years ago

Similar presentations

Presentation on theme: "3/14/2005 CS851: Data Services for Advanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash."— Presentation transcript:

1 3/14/2005 CS851: Data Services for Advanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash Agarwalla, JunSuk Shin, Phillip Hutto, Arnab Paul, and Umakishore Ramachandran Georgia Institute of Technology Presented by Lei Fang, Mar 14, 2005

2 3/14/2005 2 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

3 3/14/2005 3 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

4 3/14/2005 4 CS851: Data Services for Advanced System Applications Future Wireless Sensor Networks More powerful hardware Resource-demand applications Streaming media Surveillance Image-based tracking interactive vision Hierarchal data fusion

5 3/14/2005 5 CS851: Data Services for Advanced System Applications Future Wireless Sensor Networks Hierarchal data fusion

6 3/14/2005 6 CS851: Data Services for Advanced System Applications Data Fusion Challenges Why is developing fusion applications challenging? Time sensitive Synchronization of the data from multiple streams Distributed computing Distributed fusion Limited power In-network aggregation Power-aware routing Dynamic relocation of fusion functions

7 3/14/2005 7 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

8 3/14/2005 8 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

9 3/14/2005 9 CS851: Data Services for Advanced System Applications DFuse Fusion API: Help to develop distributed sensor fusion applications Fusion API: Help to develop distributed sensor fusion applications Fusion Placement: Support automatic management of fusion point placement and migration Fusion Placement: Support automatic management of fusion point placement and migration

10 3/14/2005 10 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

11 3/14/2005 11 CS851: Data Services for Advanced System Applications DFuse Architecture - Input Target applications Target applications  Hierarchical fusion functions on time sequenced data items Input Input  Task Graph Vertex: Vertex: source, sink, or immediate fusion points Assume data sources are known at query time   Fusion Function Transform a correlated set of inputs to a “fused” output item May contract or expand the streaming data Each fusion point performs one fusion function

12 3/14/2005 12 CS851: Data Services for Advanced System Applications DFuse Architecture - Components Data Fusion Module Data Fusion Module  Implement fusion API:  Implement fusion API: allows any synthesis operation to on stream of data to be specified as a fusion function Fusion Placement Module Fusion Placement Module   Determine the best placement of fusion functions

13 3/14/2005 13 CS851: Data Services for Advanced System Applications DFuse Architecture

14 3/14/2005 14 CS851: Data Services for Advanced System Applications DFuse Architecture – Working principles Fusion Module provides fusion API which Fusion Module provides fusion API which allows any synthesis operation to on stream of data to be specified as a fusion function.   The user inputs the task graph and fusion functions to DFuse at some designated node (root node) The placement module at the root node maps the fusion functions onto the physical network   Subject to some cost function   Periodically re-evaluate the health of fusion point nodes and their neighbors, and moves the fusion functions

15 3/14/2005 15 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

16 3/14/2005 16 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

17 3/14/2005 17 CS851: Data Services for Advanced System Applications DFuse – Fusion Module Fusion API Capabilities Fusion API Capabilities  Structure Management Fusion Channel – Abstracts inputs and encapsulates a general fusion function (filter, collage, …) Fusion Channel – Abstracts inputs and encapsulates a general fusion function (filter, collage, …) provide data buffering and synchronization facilities provide data buffering and synchronization facilities E.g. items are fused and accessed by timestamp E.g. items are fused and accessed by timestamp

18 3/14/2005 18 CS851: Data Services for Advanced System Applications DFuse – Fusion Module Fusion API Fusion API  Correlation control Specify and collect correlation sets (related input items supplied to the fusion function) Specify and collect correlation sets (related input items supplied to the fusion function)  Computation management Handle the specification, application and migration of fusion functions Handle the specification, application and migration of fusion functions  Memory management Caching, prefetching, and buffer management Caching, prefetching, and buffer management

19 3/14/2005 19 CS851: Data Services for Advanced System Applications DFuse – Fusion Module Fusion API Fusion API  Failure/latency handling Allow partial fusion Allow partial fusion Accept a variable number of input items Accept a variable number of input items  Status and feedback handling Allow interaction between fusion functions and data sources Allow interaction between fusion functions and data sources Supply status information via “status” register Supply status information via “status” register Support a command set via “command” register Support a command set via “command” register

20 3/14/2005 20 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

21 3/14/2005 21 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

22 3/14/2005 22 CS851: Data Services for Advanced System Applications Placement Module – Role Assignment Distributed role assignment algorithms Distributed role assignment algorithms  Map a fusion function to a network node  Three roles: end point, relay, fusion point

23 3/14/2005 23 CS851: Data Services for Advanced System Applications Placement Module – Role Assignment Role Assignment Heuristic Role Assignment Heuristic   Phase 1: Naive role assignment Start at root node Algorithm: Naive Tree Building Upon receiving the application task graph: if (I can host the root fusion function) { accept the root fusion function; for each input { delegate subtree to “richest” neighbors; } else delegate entire tree to the “richest” neighbor;

24 3/14/2005 24 CS851: Data Services for Advanced System Applications Placement Module – Role Assignment Role Assignment Heuristic Role Assignment Heuristic   Phase 2: Optimization (Role Transfer) Improve overall health of the overlay network Health is valued by cost function (power, data transfer rate) Each fusion node periodically broadcasts its health If a neighboring node has better health, it informs the fusion node The fusion node transfer fusion task to the neighbor with best health   Phase 3: Maintenance (application is running)

25 3/14/2005 25 CS851: Data Services for Advanced System Applications Placement Module – Sample Cost Functions The lower the cost function value, the better the node health! Minimize transmission cost 1 (MT1): Minimize power variance (MPV): C MPV ( k ) = 1/power ( k ) Minimize Transmission cost vs. power (MTP): C MTP ( k, f ) = C MT1 ( k, f ) * C MPV ( k ) Minimize transmission cost 2 (MT2): C MT2 ( k, f ) = ( power ( k ) > threshold ? (C MT1 ( k, f ) : Infinity ) How to transfer the role ?

26 3/14/2005 26 CS851: Data Services for Advanced System Applications Placement Module – Local Minima Two simple metrics   Simple hop-count   Fusion data expansion or contraction information Two types of optimization   Linear: All inputs to a fusion node are coming from one relay node   Triangular: Multiple paths for inputs to reach a fusion point

27 3/14/2005 27 CS851: Data Services for Advanced System Applications Placement Module –Optimization Linear Optimization 1000 1500 SourceRelay Fusion Point (data contraction) Sink 1000 1500 SourceFusion Point Relay Sink 1000 1500 SourceFusion Point (data expansion) Relay Sink 1000 1500 SourceRelay Fusion Point Sink

28 3/14/2005 28 CS851: Data Services for Advanced System Applications Placement Module –Optimization Triangular Optimization 1000 1500 fp1 Relay fp3 (data contraction) fp4 fp2 1000 1500 fp1 fp3 fp2

29 3/14/2005 29 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

30 3/14/2005 30 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

31 3/14/2005 31 CS851: Data Services for Advanced System Applications DFuse – Implementation Built on top of Stampede   Stampede provides distributed threads, timestamps, registers with synchronization, automatic garbage collection, communication channels, etc.   All buffers implemented as Stampede channels   Map the fusion attribute to the timestamp of channel   Status and command registers implemented with Stampede registers Decouple the fusion and placement module   Interface: Built in communication channel and protocol

32 3/14/2005 32 CS851: Data Services for Advanced System Applications D-Stampede Distributed programming system for Stream-based ubiquitous computing Key Features Index and correlate different data streams temporally. Perform automatic distributed garbage collection of unnecessary stream data. Provide parallelism Support distributed heterogeneous components. Dynamic application

33 3/14/2005 33 CS851: Data Services for Advanced System Applications D-Stampede Hardware model Skiff camera Data Aggregators Sensors Actuators Unix / Linux / NT cluster Channels / queues Sensor Fusion

34 3/14/2005 34 CS851: Data Services for Advanced System Applications D-Stampede Computational model-Thread/Channel Graph Computational model-Thread/Channel Graph thread Channel thread Channel i_conn o_conn many to many connections time sequenced data correlation of streams put(ts, item) get(ts, item) consume(ts)

35 3/14/2005 35 CS851: Data Services for Advanced System Applications D-Stampede Implementation Implementation

36 3/14/2005 36 CS851: Data Services for Advanced System Applications D-Stampede Programming with D-Stampede Programming with D-Stampede

37 3/14/2005 37 CS851: Data Services for Advanced System Applications Implementation – Fusion Module Built on top of Stampede Functions:   createFC/destroyFC   attachFC/detachFC: connect/disconnect application thread with specified fusion channel   getFCItem/putFCItem: request/put fused data items

38 3/14/2005 38 CS851: Data Services for Advanced System Applications Implementation – Placement Module Event-based simulation   Input: Application task graph, network topology   Assumes ideal MAC and perfect routing knowledge   Simulates the fusion placement heuristic Naïve tree building Optimization Maintenance

39 3/14/2005 39 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

40 3/14/2005 40 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

41 3/14/2005 41 CS851: Data Services for Advanced System Applications Evaluation – Fusion API cost

42 3/14/2005 42 CS851: Data Services for Advanced System Applications Evaluation – FP migration cost

43 3/14/2005 43 CS851: Data Services for Advanced System Applications Evaluation – Placement Algorithm Implement the tracker application with fusion API and deploy it on the iPAQ farm Implement the tracker application with fusion API and deploy it on the iPAQ farm The location of filter and collage determined by placement module movePC API: role transfer Data Sources

44 3/14/2005 44 CS851: Data Services for Advanced System Applications Evaluation – Simulation Placement module adjusts power level based on number of the amount of data sent and   Based on ORiNOCO 802.11b PC card specs   Only consider network communication costs Optimization phase: runs for 2s   Re-evaluate health every 100ms During maintenance phase, role transfer evaluated every 50s

45 3/14/2005 45 CS851: Data Services for Advanced System Applications Evaluation – Traffic and Runtime

46 3/14/2005 46 CS851: Data Services for Advanced System Applications Evaluation – Traffic and Runtime

47 3/14/2005 47 CS851: Data Services for Advanced System Applications Evaluation – Power Variance

48 3/14/2005 48 CS851: Data Services for Advanced System Applications Evaluation – Runtime/Energy/Role Transfers

49 3/14/2005 49 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

50 3/14/2005 50 CS851: Data Services for Advanced System Applications Outlines Motivation Motivation DFuse DFuse  Architecture  Fusion Module  Placement Module  Implementation  Evaluation Conclusion Conclusion

51 3/14/2005 51 CS851: Data Services for Advanced System Applications Conclusions Future sensor networks will support data fusion applications DFuse provides fusion API to ease the development such applications Distributed fusion placement increases application runtime by dynamic role assignment

52 3/14/2005 52 CS851: Data Services for Advanced System Applications Critiques Inflexible: DFuse can’t be implemented in current sensor network Assume data sources are known before Stationary sources and sinks Complicated, resource demand Assume every two node in the network is reachable Migrate a particular fusion channel is a local decision and only immediate neighbor nodes are queried to see if their heath for hosting the fusion channel would be better. Insufficient evaluation: Only consider network communication cost when evaluate power consumption Only consider network communication cost when evaluate power consumption Implemented on iPAQ farm, but most of the work via event-based simulation on desktop

53 3/14/2005 53 CS851: Data Services for Advanced System Applications Related work --- Data Fusion Interface TAG (Tiny Aggregation): TAG (Tiny Aggregation): P Provides a simple SQL-like declarative language for expressing aggregation queries over streaming sensor data. Intelligently distributes and executes aggregation queries in the sensor network in a time and power-efficient manner. Example: Difference between Tag query and SQL query Output of a Tag query is a stream of values Record :. Each group is time-stamped and the readings used to compute an aggregate record all belong to the same time interval, or epoch. SELECT AVG(volume),room FROM sensors WHERE floor = 6 GROUP BY room HAVING AVG(volume) > threshold EPOCH DURATION 30s SELECT (agg(expr), attrs) FROM sensors WHERE {selPreds} GROUP BY {attrs} HAVING {havingPreds} EPOCH DURATION i

54 3/14/2005 54 CS851: Data Services for Advanced System Applications Related work --- Data Fusion Interface TAG (Tiny Aggregation): TAG (Tiny Aggregation):  Structure of Data aggregation Three pieces: – Merging function f – Initializer i – Evaluator e General form: = f(, ) –,, are multi-valued partial state record – Example: f- for Average Each partial state record consists of f(, ) = i- How to initiate a state record for a single sensor value Eg. i(x) = :Average over a sensor value of x e- Compute the actual value of aggregation e( )  Pros: Proven to give more accurate results then centralized approach In-network aggregation: Save power In-network aggregation: Save power Simple Simple  Cons: Static mapping fusions roles Static mapping fusions roles

55 3/14/2005 55 CS851: Data Services for Advanced System Applications Related work --- Dynamic Role Assignment Generic Role Assignment for WSN  Architecture

56 3/14/2005 56 CS851: Data Services for Advanced System Applications Related work --- Dynamic Role Assignment Generic Role Assignment for WSN  Example: In-network aggregation Rules/Algorithm

57 3/14/2005 57 CS851: Data Services for Advanced System Applications

Download ppt "3/14/2005 CS851: Data Services for Advanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash."

Similar presentations

Network II.5 simulator ..

Network II.5 simulator ..
1 A Real-Time Communication Framework for Wireless Sensor-Actuator Networks Edith C.H. Ngai 1, Michael R. Lyu 1, and Jiangchuan Liu 2 1 Department of Computer.

1 A Real-Time Communication Framework for Wireless Sensor-Actuator Networks Edith C.H. Ngai 1, Michael R. Lyu 1, and Jiangchuan Liu 2 1 Department of Computer.
University of South Australia Distributed Reconfiguration Avishek Chakraborty, David Kearney, Mark Jasiunas.

University of South Australia Distributed Reconfiguration Avishek Chakraborty, David Kearney, Mark Jasiunas.
Transmission Power Control in Wireless Sensor Networks CS577 Project by Andrew Keating 1.

Transmission Power Control in Wireless Sensor Networks CS577 Project by Andrew Keating 1.
Decentralized Reactive Clustering in Sensor Networks Yingyue Xu April 26, 2015.

Decentralized Reactive Clustering in Sensor Networks Yingyue Xu April 26, 2015.
1 Message Oriented Middleware and Hierarchical Routing Protocols Smita Singhaniya Sowmya Marianallur Dhanasekaran Madan Puthige.

1 Message Oriented Middleware and Hierarchical Routing Protocols Smita Singhaniya Sowmya Marianallur Dhanasekaran Madan Puthige.
Presented By- Sayandeep Mitra 13000110043 7 TH SEMESTER Sensor Networks(CS 704D) Assignment.

Presented By- Sayandeep Mitra TH SEMESTER Sensor Networks(CS 704D) Assignment.
한국기술교육대학교 컴퓨터 공학 김홍연 TinyDB : An Acquisitional Query Processing System for Sensor Networks. - Samuel R. Madden, Michael J. Franklin, Joseph M. Hellerstein,

한국기술교육대학교 컴퓨터 공학 김홍연 TinyDB : An Acquisitional Query Processing System for Sensor Networks. - Samuel R. Madden, Michael J. Franklin, Joseph M. Hellerstein,
1 Sensor Network Databases Ref: Wireless sensor networks---An information processing approach Feng Zhao and Leonidas Guibas (chapter 6)

1 Sensor Network Databases Ref: Wireless sensor networks---An information processing approach Feng Zhao and Leonidas Guibas (chapter 6)
Stampede A Cluster Programming Middleware for Interactive Stream-oriented Applications Umakishore Ramachandran, Rishiyur Nikhil, James Matthew Rehg, Yavor.

Stampede A Cluster Programming Middleware for Interactive Stream-oriented Applications Umakishore Ramachandran, Rishiyur Nikhil, James Matthew Rehg, Yavor.
1 Prediction-based Strategies for Energy Saving in Object Tracking Sensor Networks Yingqi Xu, Wang-Chien Lee Proceedings of the 2004 IEEE International.

1 Prediction-based Strategies for Energy Saving in Object Tracking Sensor Networks Yingqi Xu, Wang-Chien Lee Proceedings of the 2004 IEEE International.
A Stratified Approach for Supporting High Throughput Event Processing Applications July 2009 Geetika T. LakshmananYuri G. RabinovichOpher Etzion IBM T.

A Stratified Approach for Supporting High Throughput Event Processing Applications July 2009 Geetika T. LakshmananYuri G. RabinovichOpher Etzion IBM T.
DNA Research Group 1 CountTorrent: Ubiquitous Access to Query Aggregates in Dynamic and Mobile Sensor Networks Abhinav Kamra, Vishal Misra and Dan Rubenstein.

DNA Research Group 1 CountTorrent: Ubiquitous Access to Query Aggregates in Dynamic and Mobile Sensor Networks Abhinav Kamra, Vishal Misra and Dan Rubenstein.
The Cougar Approach to In-Network Query Processing in Sensor Networks By Yong Yao and Johannes Gehrke Cornell University Presented by Penelope Brooks.

The Cougar Approach to In-Network Query Processing in Sensor Networks By Yong Yao and Johannes Gehrke Cornell University Presented by Penelope Brooks.
A Survey of Wireless Sensor Network Data Collection Schemes by Brett Wilson.

A Survey of Wireless Sensor Network Data Collection Schemes by Brett Wilson.
A Hierarchical Energy-Efficient Framework for Data Aggregation in Wireless Sensor Networks IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 55, NO. 3, MAY.

A Hierarchical Energy-Efficient Framework for Data Aggregation in Wireless Sensor Networks IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 55, NO. 3, MAY.
Probabilistic Data Aggregation Ling Huang, Ben Zhao, Anthony Joseph Sahara Retreat January, 2004.

Probabilistic Data Aggregation Ling Huang, Ben Zhao, Anthony Joseph Sahara Retreat January, 2004.
Matching Data Dissemination Algorithms to Application Requirements John Heidermann, Fabio Silva, Deborah Estrin Presented by Cuong Le (CPSC538A)

Matching Data Dissemination Algorithms to Application Requirements John Heidermann, Fabio Silva, Deborah Estrin Presented by Cuong Le (CPSC538A)
1 Introduction to Load Balancing: l Definition of Distributed systems. Collection of independent loosely coupled computing resources. l Load Balancing.

1 Introduction to Load Balancing: l Definition of Distributed systems. Collection of independent loosely coupled computing resources. l Load Balancing.
Aggregate Query Processing in Ad-Hoc Sensor Networks Yong Yao Database lunch, Apr. 15th.

Aggregate Query Processing in Ad-Hoc Sensor Networks Yong Yao Database lunch, Apr. 15th.

Similar presentations

Ads by Google