Advanced media-oriented systems research: Ubiquitous capture, access, and interpretation n Faculty involved with RI-related projects l Kishore Ramachandran, Mustaque Ahamad, Karsten Schwan, Richard Fujimoto, Ken Mackenzie, Sudha Yalamanchili, Irfan Essa, Jim Rehg, Gregory Abowd, Yannis Smaragdakis, Santosh Pande, Calton Pu, Ling Liu,... n Federal funding l NSF RI, NSF ITR, DARPA, DOE n State funding l Yamacraw, GT Broadband Institute n Industry funding (equipment and personnel) l HP, Intel, Microsoft, IBM

MediaBroker and DFuse: Powerful support for emerging applications Kishore Ramachandran Jim Rehg, Phil Hutto, Ken Mackenzie, Irfan Essa College of Computing, Georgia Tech Kath Knobe, Jamey Hicks, Raj Kumar HP Labs Students : Rajnish Kumar, Matt Wolenetz, Ilya Bagrak, Martin Modahl, Bikash Agarwalla, Junsuk Shin, Arnab Paul, Sameer Adhikari, Nissim Harel, Hasnain Mandviwala, Yavor Angelov, David Hilley

n Distributed system components l sensors, actuators, embedded processors, PDAs, laptops, clusters… “OCTOPUS” DIAGRAM head / arms / tentacles Skiff camera Data Aggregators Sensors Actuators Unix / Linux / NT cluster Sensor Fusion

Killer App? n Application context l distributed sensors with varying capabilities l control loop involving sensors, actuators l rapid response time at computational perception speeds

n Distributed Collaboration n Emergency Response l Collaborative search and rescue l Evacuation management n “Aware” Environments Key Requirements n Data distribution and Infrastructure adaptation Emerging Applications

Application Characteristics n Physically distributed heterogeneous devices n Interfacing and integrating with the physical environment n Diverse stream types (low to high BW) n Diverse computation, communication and power capabilities (from embedded sensors to clusters) n Stream fusion/transformation, with loadable code n Resource scarcities n Dynamic join/leave of application components Two projects: n MediaBroker n DFuse

Abstractions…abstractions n Goals: l Temporal guarantees, efficient streams, distributed synchronization, fusion support l Simplify application development n Key result: Fusion Channels l built on top of D-Stampede [ICDCS 2002] A Fusion Channel (a ‘Virtual Sensor’) Inputs (sensors or other fusion channels) Consumers (actuators or other fusion channels) F()...

The MediaBroker n An architecture for Data Distribution l Type management, Scalability, Stream “sharing” n A sample scenario for distributed collaboration l Physically distributed participants in a classroom setting l Interaction via a variety of devices l System takes care of any needed data transformations and distribution MediaBroker Infrastructure MediaBroker Infrastructure

MediaBroker Infrastructure ( ) D-Stampede audio video sonar Re-publish fusion sharing derivation bio applications virtual sensors registry discovery  Fusion Channels and Virtual Sensors  Resource Discovery and Sharing  Sensor-Provided Attributes  Sensor-Derived Attributes  Scalability  Access Control and Resource Scheduling  Security

MediaBroker and Typed Streams n Experimental “type-lattice” registry, discovery, and resolution structure Fusion Channel Performs text-to-speech Fusion Channel Outputs A/V as MPEG, Incorporates text as caption Today’s Lectur e:

DFuse [ACM SenSys 2003] n An architecture for Infrastructure Adaptation n A sample scenario for an aware environment l Field trip for a class! l Deployed power-constrained sensors l Dynamic wireless network consisting of the students’ PDAs l In-network stream filtering and aggregation

DFuse Fundamentals n Fusion Module: Deploys task graph on sensor network l Comprehensive API for fusion and migration l Low-overhead n Placement Module: Employs a self-stabilizing algorithm to place fusion points in the network l Energy and application aware cost functions l Localized decisions Sink (Display) Cameras Filter Collage Sensors Sample surveillance application task graph: filter and collage are the fusion functions. n Deployed on iPAQ farm! n Tested 4 cost-functions

Evaluation Application Timeline showing Network Traffic

Applause!!!