1. Stampede Overview Joint research between HP CRL and Georgia Tech (*) Kishore Ramachandran (*) Jim Rehg(*), Phil Hutto(*), Ken Mackenzie(*), Irfan Essa(*), Kath Knobe, Jamey Hicks Students (*) : Sameer Adhikari, Arnab Paul, Bikash Agarwalla, Matt Wolenetz, Nissim Harel, Hasnain Mandviwala, Yavor Angelov, Junsuk Shin, Rajnish Kumar, Ilya Bagrak, Martin Modahl, David Hilley

2. www.amonline.net.au/invertebrates/mal/gallery/octopus.htm

3. n Hardware Model l sensors, actuators, embedded processors, PDAs, laptops, clusters… “OCTOPUS” DIAGRAM head / arms / tentacles Skiff camera Data Aggregators Sensors Actuators Unix / Linux / NT cluster Channels / queues Sensor Fusion Distributed Ubiquitous Computing

4. Killer App? n Application context l distributed sensors with varying capabilities l control loop involving sensors, actuators l rapid response time at computational perception speeds

5. Application Scenarios n Mobile robots n Smart vehicles n Aware homes n Real-life emergencies l natural and man-made disaster response iearthquakes, twisters, fire, terrorist situations n Environmental monitoring l viruses, pollution, … l animals and birds in natural habitats n Augmented reality applications l training for hazardous situations l battlefield management n Interactive animation

6. Application Characteristics n Physically distributed heterogeneous devices n Distributed mobile sensing and actuation n Interfacing and integrating with the physical environment n Information acquisition, processing, synthesis, and correlation l streaming high BW data such as audio and video l low BW data such as from a haptic sensor l time-sequenced data n Dynamic computation continuum from low end device-level filtering to high end inference

7. Research Issues n Stream-oriented and time-sequenced data n Heterogeneity of Components n Resource management n High Availability n Clients leave and join arbitrarily n Security and Privacy

8. Stampede Project n Theme l seamless programming system spanning sensors and backend servers id-stampede: common programming paradigm across widely varying architectures [ICDCS 2002] isupports development of pervasive computing applications

9. Stampede computational model: a dynamic thread-channel graph thread Channel thread Channel i_conn o_conn many to many connections time sequenced data correlation of streams automatic GC put(ts, item) get(ts, item) consume(ts)

10. Experiences with Stampede n Color-based people tracker for SmartKiosk (Jim Rehg) Change Detection Model 1 Location Digitizer Video Frame Histogram Motion Mask Target Detection Target Detection Histogram Model Model 2 Location

11. Model 1 Model 2

12. Color-Based Tracking Example

13. n Video Textures (Irfan Essa) Generate an infinite video sequence from a finite set of video frames -embarrassingly parallel (comparison of images) -data distribution from source the main challenge -breaking image into strips to fit the computation in caches secondary challenge

14. Cluster skiff Stampede client (C) Stampede Application (C) skiff Stampede client (C) STM.... n Multipoint video/audio capture

15. Multipoint Video Demo

16. Ongoing Work n Media broker architecture Media broker l resource naming and discovery resource naming and discovery l data fusion (fusion channels) data fusion l asynchronous notification n Aspect-oriented programming support l STAGES language and compiler STAGES n Dynamic multi-cluster implementation n D-Stampede Web Service l.NET implementation n Models for reasoning about failures n Security and privacy issues

17. The Media Broker: A media stream registry and transformation engine Client lifecycle Broker discovery Connect Publish … Stream discovery Subscribe … Disconnect data cmd status producer Client detail data cmd status consumer media broker Re-publish Stampede audio video sonar Re-publish fusion sharing derivation bio real sensors applications virtual sensors registry discovery

18. data cmd status sensor data cmd status actuator media broker actuator sensor microphone camera sonar sensor & actuator local sensor registry sensor pan-tilt-zoom camera display virtual sensor virtual sensor Device Abstraction Media Broker A Media Broker B Media Broker C Media Broker D Media Broker A

19. Resource Discovery and Sharing n Naming API l Every sensor (real or virtual) has application-level name. Queries result in connections. l Any sensor may produce more than one ‘type’ of output (e.g. RTP raw audio, TCP mp3 audio, …) l ‘types’ are registered as part of sensor attributes l Dynamic attributes such as device load and remaining energy may be provided by sensors l Media Broker monitors sensors and adds attributes of latency, bandwidth, current consumers and type l Media Broker negotiates consumers’ requests, indicates requested ‘type’ to sensors iConflict resolution? – open question iSimpler if each sensor can only produce one type

20. Fusion Architecture 1 n Fusion channel l Abstract input streams into a single output stream l Virtualizes inputs into fused abstract sensor l Embeds supplied fusion process code (Kalman filter, etc.) l Input stream set can be dynamically amended n Aggregation of specified input streams n Correlation via virtual timestamps l Use real-time of synchronized capture hosts if necessary l Or other convenient sequence numbers (frame, sample #) n Application of arbitrary (user-specified) fusion function

21. Fusion Architecture 2 n Partial fusion tolerates losses, late delivery l Fuse after specified timeout … l If at least M out of N inputs available n Fusion pipelines (graphs) l Input streams may themselves be fusion channels l Fusion channel (output) supports multiple consumers n Fusion policy (when to fuse) l Fuse-on-request: demand-driven, lazy, pull model l Fuse-when-available: data-driven, eager, push model

22. Fusion Architecture 3 n Failure notification l Via failed get request return value l Via special failure-items in channel l Via failure event callback n Device interaction, control l Smart sensors support command sets iAlter parameters, activate/deactivate, query status l Use device driver model for sensor interactions iCommand, data, status “registers” l Example: video capture device iCompression, resolution, rate

23. STAGES (Stampede Application Generator Specification) n Addresses Stampede application development issues l Which computations go where? (Mapping) l How do computations communicate? (Plumbing) l How to easily adjust mapping and plumbing? (Tuning) n Uses Aspect Oriented Programming to separate performance concerns (aspects) from algorithmic implementation (components) n Implemented as a graph specification language and compiler n Targets Stampede cluster library n Extensions in progress l D-Stampede applications l Dynamic functional and performance concerns