1. Advanced media-oriented systems research: Ubiquitous capture, access, and interpretation n Faculty involved with RI-related projects l Kishore Ramachandran, Mustaque Ahamad, Karsten Schwan, Richard Fujimoto, Ken Mackenzie, Sudha Yalamanchili, Irfan Essa, Jim Rehg, Gregory Abowd, Yannis Smaragdakis, Santosh Pande, Calton Pu, Ling Liu,... n Federal funding l NSF RI, NSF ITR, DARPA, DOE n State funding l Yamacraw, GT Broadband Institute n Industry funding (equipment and personnel) l HP, Intel, Microsoft, IBM

2. MediaBroker and DFuse: Powerful support for emerging applications Kishore Ramachandran Jim Rehg, Phil Hutto, Ken Mackenzie, Irfan Essa College of Computing, Georgia Tech Kath Knobe, Jamey Hicks, Raj Kumar HP Labs Students : Rajnish Kumar, Matt Wolenetz, Ilya Bagrak, Martin Modahl, Bikash Agarwalla, Junsuk Shin, Arnab Paul, Sameer Adhikari, Nissim Harel, Hasnain Mandviwala, Yavor Angelov, David Hilley

3. www.amonline.net.au/invertebrates/mal/gallery/octopus.htm

4. n Distributed system components l sensors, actuators, embedded processors, PDAs, laptops, clusters… “OCTOPUS” DIAGRAM head / arms / tentacles Skiff camera Data Aggregators Sensors Actuators Unix / Linux / NT cluster Sensor Fusion

5. Killer App? n Application context l distributed sensors with varying capabilities l control loop involving sensors, actuators l rapid response time at computational perception speeds

6. n Distributed Collaboration n Emergency Response l Collaborative search and rescue l Evacuation management n “Aware” Environments Key Requirements n Data distribution and Infrastructure adaptation Emerging Applications

7. Application Characteristics n Physically distributed heterogeneous devices n Interfacing and integrating with the physical environment n Diverse stream types (low to high BW) n Diverse computation, communication and power capabilities (from embedded sensors to clusters) n Stream fusion/transformation, with loadable code n Resource scarcities n Dynamic join/leave of application components Two projects: n MediaBroker n DFuse

8. Abstractions…abstractions n Goals: l Temporal guarantees, efficient streams, distributed synchronization, fusion support l Simplify application development n Key result: Fusion Channels l built on top of D-Stampede [ICDCS 2002] A Fusion Channel (a ‘Virtual Sensor’) Inputs (sensors or other fusion channels) Consumers (actuators or other fusion channels) F()...

9. The MediaBroker n An architecture for Data Distribution l Type management, Scalability, Stream “sharing” n A sample scenario for distributed collaboration l Physically distributed participants in a classroom setting l Interaction via a variety of devices l System takes care of any needed data transformations and distribution MediaBroker Infrastructure MediaBroker Infrastructure

10. MediaBroker Infrastructure ( ) D-Stampede audio video sonar Re-publish fusion sharing derivation bio applications virtual sensors registry discovery  Fusion Channels and Virtual Sensors  Resource Discovery and Sharing  Sensor-Provided Attributes  Sensor-Derived Attributes  Scalability  Access Control and Resource Scheduling  Security

11. MediaBroker and Typed Streams n Experimental “type-lattice” registry, discovery, and resolution structure Fusion Channel Performs text-to-speech Fusion Channel Outputs A/V as MPEG, Incorporates text as caption Today’s Lectur e:

12. DFuse [ACM SenSys 2003] n An architecture for Infrastructure Adaptation n A sample scenario for an aware environment l Field trip for a class! l Deployed power-constrained sensors l Dynamic wireless network consisting of the students’ PDAs l In-network stream filtering and aggregation

13. DFuse Fundamentals n Fusion Module: Deploys task graph on sensor network l Comprehensive API for fusion and migration l Low-overhead n Placement Module: Employs a self-stabilizing algorithm to place fusion points in the network l Energy and application aware cost functions l Localized decisions Sink (Display) Cameras Filter Collage Sensors Sample surveillance application task graph: filter and collage are the fusion functions. n Deployed on iPAQ farm! n Tested 4 cost-functions

14. Evaluation Application Timeline showing Network Traffic

15. Applause!!!