IrisNet: A Planetary-Scale Architecture for Multimedia Sensors Phillip B. Gibbons Intel Research Pittsburgh January 19, 2007 Slides are © Phillip B. Gibbons

What is a Sensor Network? Tiny sensor nodes with very limited processing power, memory, battery. Scalar sensors (e.g., temperature) Closely co-located, communicating via an ad hoc low-bandwidth wireless network Singly tasked Microservers? not so tiny, PDA-class processor Fault-line monitoring? wide-area, not ad hoc Broadband? not low-bandwidth Webcams? not scalar, can be multi-tasked Tanker/Fab monitoring? powered, wired © Phillip B. Gibbons

Sensor Networks is a Rich Space Characteristics of sensor network depend on Requirements of the application Restrictions on the deployment Characteristics of sensed data Sampling the real world Tied to particular place and time Often write-dominated: Data Sampled far more than Data Queried UCLA CENS NIMS James Reserve © Phillip B. Gibbons

Multimedia Sensor Systems Rich collection of (cheap) sensors Cameras, Microphones, RFID readers, vibration sensors, etc Internet-connected. Potentially Internet-scale Tens to millions of sensor feeds over wide-area Pervasive broadband (wired & wireless) Goal: Unified system for accessing, filtering, processing, querying, and reacting to sensed data Programmed to provide useful sensing services © Phillip B. Gibbons

Example Multimedia Sensing Services Consumer services: Parking Space Finder Lost & Found / Lost pet Watch-my-child / Watch-my-parent Congestion avoidance © Phillip B. Gibbons

Example Multimedia Sensing Services Health, Security, Commerce, and Science services: Epidemic Early Warning System Homeland Security Low Atmosphere Climate Monitoring Asset/Supply Chain Tracking Our prototype Internet-scale Sensor Observatories © Phillip B. Gibbons

Sensor Network Challenges? Resource-constrained sensor networks present many unique challenges Battery limitations => limiting communication, duty cycles, etc Limited processing power & memory => new OS, languages, algorithms, etc On the other hand, in multimedia sensor systems Nodes are powerful, well-connected Where’s the Challenge? © Phillip B. Gibbons

Data & Query Scaling Challenges Data scaling Millions of sensors Globally-dispersed High volume feeds Historical data Query scaling May want sophisticated data processing on all sensor feeds May aggregate over large quantities of data, use historical data, run continuously Want latest data, NOW NetRad: 100Mb/s © Phillip B. Gibbons

Further Challenges IrisNet addresses some of these challenges System Scaling How to manage a million node system? How to write services for a million node system? System Heterogeneity Sensor types, Nodes, Networks, Admin. Domains Shared Resource Privacy & Security Competing actuation IrisNet addresses some of these challenges Many open problems remain… © Phillip B. Gibbons

Outline Multimedia Sensor Systems: Opportunities & Challenges IrisNet overview IrisNet key features Conclusions © Phillip B. Gibbons

IrisNet: Internet-scale Resource-intensive Sensor Network services General-purpose architecture for wide-area sensor systems A worldwide sensor web Key Goal: Ease of service authorship Provides important functionality for all services Intel Research Pittsburgh + many CMU collaborators First prototype in late 2002, in use In ACM Multimedia, BaseNets, CVPR, DCOSS, DSC, FAST, NSDI(2), Pervasive Computing, PODC, SenSys, Sigmod(2) www.intel-iris.net © Phillip B. Gibbons

Data & Query Scaling in IrisNet Store sensor feeds locally Too much data to collect centrally Push data processing & filtering to sensor nodes Reduce the raw data to derived info, in parallel near source Push queries to sensor nodes Data sampled >> Data queried Tied to particular place: Queries often local Exploit logical hierarchy of sensor data Compute answers in-network © Phillip B. Gibbons

Ease of Service Authorship What is a minimal service specification? Application-specific functionality needed Program sensor system as a whole Provide a simple, centralized view of the system Data as a single queriable unit Declarative (database-centric) approach Be the Google/Ask Jeeves for live content IrisNet uses XML: Self-describing tags, Hierarchy © Phillip B. Gibbons

Data Organized as Logical Hierarchy <State id=“Pennysylvinia”> <County id=“Allegheny”> <City id=“Pittsburgh”> <Neighborhood id=“Oakland”> <total-spaces>200</total-spaces> <Block id=“1”> <GPS>…</GPS> <pSpace id=“1”> <in-use>no</in-use> <metered>yes</metered> </pSpace> <pSpace id=“2”> … </Block> </Neighborhood> <Neighborhood id=“Shadyside”> Example XML Hierarchy … ID attributes are treated specially. Hierarchy is automatically partitioned among the OAs © Phillip B. Gibbons

IrisNet’s Two-Tier Architecture User Two components: SAs: sensor feed processing OAs: distributed database Query Web Server for the url . . . OA XML database . . . SA senselet Sensor SA senselet Sensor SA senselet . . . Sensornet © Phillip B. Gibbons

SAs and Senselets: Further Details Sensor SA senselet Each SA runs a C module that provides a common runtime environment for senselets Senselets are arbitrary Linux binaries Can leverage OpenCV image processing library Example senselet: images to parking availability Send updates to OAs Multiple senselets can share the same sensor feed Senselets are uploaded to running SAs Deployment costs greatly reduced: Services share same sensors and infrastructure © Phillip B. Gibbons

Creating a New IrisNet Service Image processing steps FFFFEFF Send to OA Updates DB Only 500 lines of new code for Parking Space Finder Extended code (application-specific aggregation) Senselet (program to filter sensor data) Hierarchy (XML schema) Front-end Query with standard DB language OAs SA SA © Phillip B. Gibbons

Outline Multimedia Sensor Systems: Opportunities & Challenges IrisNet overview IrisNet key features Conclusions © Phillip B. Gibbons

IrisNet Key Features Distributed data collection and storage Support for XML queries for distributed DB Transparently routing queries to data Automatic data partitioning Multi-camera calibration & Image stitching Efficient and protected sharing of sensor nodes Privacy features Triggering and actuation Novel fault tolerance schemes © Phillip B. Gibbons

Blind Men and Elephant Problem OA’s local DB contains some fragment of the overall service DB Not a federation – A single XML document Quickly determining which part of an (XPATH) query answer can be answered from an XML fragment is a challenging task, not previously studied E.g., can this predicate be correctly evaluated? Is the result returned from local DB complete? Where can the missing parts be gathered? © Phillip B. Gibbons

Query Evaluate Gather (QEG) /NE/PA/Allegheny/Pittsburgh/(Oakland | Shadyside) / rest of query Pittsburgh OA Q 1. Queries its XML DB Discovers Shadyside data is cached, but not Oakland Does DNS lookup to find IP addr for Oakland 2. Evaluate the result Q’ Q’: /NE/PA/Allegheny/Pittsburgh/Oakland/ rest of query 3. Gathers the missing data by sending Q’ to Oakland OA Oakland OA QEG Combines results & returns © Phillip B. Gibbons

Helping the Blind Men Tag the data [Sigmod’03] Tag the data IrisNet tags the nodes in its fragment with status info, indicating various degrees of completeness Maintains partitioning/tagging invariants E.g., when gather data, generalize subquery to fetch partitionable units For each missing part, construct global name from its id chain & do DNS lookup as before Specialize subquery to avoid duplications & ensure progress © Phillip B. Gibbons

IrisNet’s Query Processing (cont) XPATH query converted to an XSLT program that walks the local XML document & handles the various tags appropriately Conversion done without accessing the DB IrisNet supports all of XPATH 1.0 Supports (distribution transparent) continuous queries & application-specific aggregates, predicates, states (Technically, the unordered projection of XPATH) © Phillip B. Gibbons

IrisNet Key Features Distributed data collection and storage Support for XML queries for distributed DB Transparently routing queries to data Automatic data partitioning Multi-camera calibration & Image stitching Efficient and protected sharing of sensor nodes Privacy features Triggering and actuation Novel fault tolerance schemes © Phillip B. Gibbons

Adaptive Data Partitioning & Placement Distributed Storage Distributed Query Processing (Internet) Distributed Storage Partitioning Replication Placement … For robustness and performance Goal: Automate these data-management tasks © Phillip B. Gibbons

Desired Behavior Partitioning + Placement 2. Replication + Placement Query Query Query Partitioning + Placement 2. Replication + Placement Query Query Query Query © Phillip B. Gibbons

Problem Statement Given Hierarchical databases Machine capacities Data partitioning Replication factor Data placement Determine Cluster based on access locality Write-heavy data => few replicas Near data and query sources Query latency Query traffic Update traffic Optimize © Phillip B. Gibbons

IrisNet Data Placement algorithm [FAST’05] Distributed Algorithm using locally collected stats E.g., link use frequency (hierarchy), read/write ratio, GNP coordinates of readers & writers Balances desire to cluster vs. placing near sources Replica selection probability ≈ inversely proportional to recent response time Fast response to flash-crowds Reacts quickly Accounts for delay between off-load decision & corresponding load decrease Approximates globally optimal solution © Phillip B. Gibbons

IDP Adapting to Flash Crowds Real deployment, real workload, synthetic flashcrowd (real workload replayed 100x times faster) Live PlanetLab deployment with real workload © Phillip B. Gibbons

IrisNet Key Features Distributed data collection and storage Support for XML queries for distributed DB Transparently routing queries to data Automatic data partitioning Multi-camera calibration & Image stitching Efficient and protected sharing of sensor nodes Privacy features Triggering and actuation Novel fault tolerance schemes © Phillip B. Gibbons

Auto-Calibrating Cameras Automatic extrinsic calibration Many, unattended cameras External forces => out-of-band camera movement Goal: Determine physical positions & orientations of cameras wrt the scene Keypoint algorithm finds correspondences between cameras Key points: rotation & scaling invariant, plentiful, distinctive Calculate homographies & combine images © Phillip B. Gibbons

Improved Key Point Algorithm Threshold set manually to return 10 matches per image. SIFT (4/10) PCA-SIFT (9/10) [CVPR’04] White solid – correct matches Black dotted – incorrect matches © Phillip B. Gibbons

PCA-SIFT In Action © Phillip B. Gibbons

IrisNet Key Features Distributed data collection and storage Support for XML queries for distributed DB Transparently routing queries to data Automatic data partitioning Multi-camera calibration & Image stitching Efficient and protected sharing of sensor nodes Privacy features Triggering and actuation Novel fault tolerance schemes © Phillip B. Gibbons

Network Monitoring using IrisNet We have deployed IrisNet on the PlanetLab network 450+ nodes, 5 continents “Sensor” feeds are CPU, memory & network measurements at each node Demonstrates wide-area scaling, robustness features © Phillip B. Gibbons

IrisNet’s Deployment on PlanetLab: Lessons Learned Robustness, robustness, robustness Platform heterogeneity => VM-based? Load balancing Processes die; connectivity flaky (on re-connect incorporate into running aggregate query) Correlated failures: replication scheme matters: used Signed Quorum Systems (SQS) subtleties arise (can predict 99% of failure patterns and still not improve availability) Importance of relative object assignment © Phillip B. Gibbons

IrisNet’s Availability on PlanetLab Fraction of available data objects in IrisStore Fraction of available PlanetLab nodes Say: planetlab is the most unstable before major systems conferences Naming in IrisStore is done via DNS. IrisStore has about 3500 objects. Objects are grouped into ~300 groups that can be named. Individual objects cannot be named. This means about 300 DNS entries. We maintain two DNS servers locally at CMU. When doing the experiments, the DNS client is at CMU as well. The results do capture naming failures. However, because the DNS server and client is always up, there were no naming failures during the experiment. - Hours before SOSP’05 deadline 7 replicas/object with SQS quorum size = 2 [NSDI’06] © Phillip B. Gibbons

Outline Multimedia Sensor Systems: Opportunities & Challenges IrisNet overview IrisNet key features Conclusions © Phillip B. Gibbons

Pervasive Sensing is Emerging Planetary-Scale Multimedia Sensor Systems Beyond motes: wide-area, broadband, multimedia, multi-tasked Many opportunities for new services Many Challenges Data & Query Scaling System scaling (authoring, robustness) System heterogeneity Shared resource © Phillip B. Gibbons

A Distributed Systems Nightmare Planetary-scale Multimedia Sensor Systems are: Write-intensive, time-critical, high volume, mobile, heterogeneous, shared, subject to harsh conditions, include both sensing & actuation Continuous/Global/aggregating queries (macroscope) Programmed by many developers for many applications …dynamically changing mix …sensors shared by many applications Must hide this complexity from application developers © Phillip B. Gibbons

IrisNet: Architecture & Prototype User Two components: SAs: sensor feed processing OAs: distributed database Query Web Server for the url . . . www.intel-iris.net open source code, papers OA XML database . . . SA senselet Sensor SA senselet Sensor SA senselet . . . Sensornet © Phillip B. Gibbons

IrisNet Team Phil Gibbons [project leader, databases, Feb02- ] Rahul Sukthankar [computer vision, robotics, Jan03- ] Babu Pillai [real-time systems, Sep03- ] Jason Campbell [robotics, Oct03- ] Haifeng Yu [distributed systems, Dec03- ] Brad Karp [networking, Nov02-Nov03] + many valuable suggestions from M.Satyanarayanan Srini Seshan [networking, systems, Mar02- ] Suman Nath [grad student, May02-Aug05 ] Yan Ke [grad student, Sep02-Apr04 ] Summer interns: Amol Deshpande [Berkeley], Shimin Chen [CMU], Dilip Sundarraj [Portland St], Amit Manjhi [CMU],… Intel Research Pittsburgh Carnegie Mellon University © Phillip B. Gibbons

Backup Slides © Phillip B. Gibbons

Coastal Imaging using IrisNet Working with oceanographers at Oregon State Process coastline video to detect & analyze sandbar formation and riptides, etc Images from IrisNet prototype © Phillip B. Gibbons

Query-based Image Stitching Stitch images from multiple cameras What does the LA-region coast look like today? How is “this” rip current going to progress in the next 24 hours? Use “Key Points” to automatically find how images fit together (planar scene) + = © Phillip B. Gibbons

Disconnection-Tolerant Query Processing Intermittent sensor disconnections Wireless link failures Internet delays to remote sites Power-saving duty cycles Mobile nodes out-of-range Why is this important ? Disconnected sensors still collecting data Data resides only on that node: can’t replicate yet Weakest availability link in system Query responses based on stale data can be problematic For instance: In Oceanography, Unable to talk to tidal sensor Unable to get images from SAs involved in image stitching NETWORK SLEEP © Phillip B. Gibbons

Advantage of Disconnection-Tolerant QP © Phillip B. Gibbons

Relative Object Assignment ABCD EFGH ABCD EFGH GHAB CDEF NO, Left better Do they give us the same availability? How large is the difference? Over 2 nines for TPC-H © Phillip B. Gibbons

Synopsis of Related Work Sensor Networks Specialized local sensor networks , e.g., home security Satellite-based data collection Mote-based: local network of sensors with small CPUs Large-Scale Distributed Databases E.g., PIER (P2P DB), SQL transactions, XML collections Multi-Camera Systems & Algorithms E.g., VSAM – study detecting & tracking objects E.g., Lee at al [PAMI], Stauffer & Tieu [CVPR] – calibration © Phillip B. Gibbons

Logging & Replay of Queries Leaf OA detects loss of connection to SA Queries and partial results are logged locally Partial results sent based on user preference Query is replayed only along disconnected branches Leaf OA Query log SA-1 SA-2 Partial results Disconnected Mode Leaf OA Replay SA-1 SA-2 Query log Partial results Reconnected Mode © Phillip B. Gibbons

More recent data available at ‘t’ Temporal Repair Temporally Consistent Complete Result QUERY: Stitch latest complete set of images (arrives at ‘t’) 1 Latest complete set of images at ‘t-x’ 2 3 3 4 5 Consensus protocol : Fetch timestamp vectors from each leaf OA. At the LCA determine complete set that satisfies an epsilon window Image at ‘t’ unavailable At ‘t-x’ At ‘t’ At ‘t-x’ At ‘t-x’ At ‘t’ Most recent More recent data available at ‘t’ © Phillip B. Gibbons

Actuation Challenges Competing actuation: shared resource Economic models Multiple cameras, set of requests, fidelity based on resolution & angle Real-time control Control Theory Planning/Coordinating the actuation Multiple ways to accomplish the same goal Declarative API Query-actuated data collection © Phillip B. Gibbons