Some Distributed Coordination Schemes for Wireless Sensor Networks Deborah Estrin UCLA Computer Science Department and USC/ISI http://lecs.cs.ucla.edu/estrin destrin@cs.ucla.edu Collaborative work with SCADDS researchers Heidemann, Govindan, Bulusu, Cerpa, Elson, Ganesan, Girod, Intanagowat, Yu, and Zhao (USC/ISI and UCLA); and Shenker (ACIRI) 9/18/2018

I. Motivation Disaster Response Embed numerous distributed devices to monitor and interact with physical world: work-spaces, hospitals, homes, vehicles, and “the environment” Circulatory Net Network these devices so that they can coordinate to perform higher-level tasks. Requires robust distributed systems of hundreds or thousands of devices. 9/18/2018

Motivating Applications 2 meters Algae -scaled Tethered Robot Bio-Tank Laboratory Inner wall of storm drain Sensors Environmental Monitoring Model Development Sensors Complex Structures 9/18/2018

Theme: New Constraints Tight coupling to the physical world Need better physical models More experimentation Designing for energy constraints Coping with “apparent” loss of layering 9/18/2018

Theme: New Design Goals Designing for long-lived (and often energy-constrained) systems Exploiting redundancy Low-duty cycle operation Tiered architectures Self configuring systems Measure and adapt to unpredictable environment Exploit spatial diversity of sensor/actuator nodes Localization and Time synchronization are key building blocks 9/18/2018

Implications for Wireless Sensor Network Design Achieve desired global behavior through localized interactions, without global state Avoid communication over long distances [Pottie 2000] Energy propagation loss: E α R4 (10 m: 5000 ops/transmitted bit; 100 m: 50,000,000 ops/transmitted bit) Empirically adapt to observed environment Dynamic, messy, environments preclude pre-configured behavior Leverage data processing/aggregation inside the network We envision that sensor nodes will be battery-powered. So, energy is one of our main design considerations. As energy is a very limited resource, sending large amount of data over long distances should be avoided. Data should be processed and summarized into small event description at the sensing node. Only events should be sent out, especially using only multi-hop short range radio. Not only because short range radio consume less energy but also because it can get around obstacles. And as we pointed out that dynamics are common and the number of nodes is large. We can not assume that we have global knowledge. We don’t want changes in just some area affect the whole network either. So, we think that it would be nice if we use only localized interactions that lead to desired global behavior and empirically adapt to observed environment. We also expect that sensors are densely deployed so that the detecting sensor is physically close to sensed phenomena. Specially, sensor networks are not general-purposed networks. They are built to perform a very specific type of application. Don’t expect that they will do ‘telnet’. Sensor networks are not about communication among specific nodes. They are more about physical environment. Temperature. Motion. Location. Moisture. And so on. With application-specific networks, we can do several good things. Intermediate nodes can process, aggregate, transform, or cache data. In sensor networks, such data processing will not be limited to only ending nodes.

Roadmap Motivation Directed Diffusion Other enabling schemes: time synch, localization, self configuration Wrap up: tiered architecture, future work 9/18/2018

II. Example: Directed Diffusion In-network data processing (e.g., aggregation, caching) Application-aware communication primitives expressed in terms of named data (not in terms of the nodes generating or requesting data) Distributed algorithms using localized interactions and measurement based adaptation Here we propose directed diffusion, a communication paradigm that is designed with all those consideration in mind. With directed diffusion, data are named with attributes, not with node id. A data consumer expresses interest in data with certain attributes. The interest will be propagated towards producers using only local interactions. The interest propagation will leave traces or gradients in the network along the way. The producers will send data back to the consumers using the established gradients. Reinforcement and negative reinforcement will be used to reduce the number of data delivery path for efficient distribution. And because diffusion is application-aware, all nodes including intermediate nodes can process or cache interests and data at will.

Basic Directed Diffusion Setting up gradients Source - Introduce sink and source - The user requests for low-data-rate events - Interest is sent and propagated. (the black arrow) - Gradients are set up. (the blue arrow) - Events are sent along the gradients. (the red arrow) - The thin red arrow show a low data rate. - The user reinforces the best path in terms of delay (the green arrow) - The thick red arrow shows a high data rate. - Suppose the middle node fail. - There is no high-data-rate events received. - The sink reinforces a low-data-rate path to recover from the node failure. Sink Interest = Interrogation in terms of data attributes Gradient = direction and strength

Basic Directed Diffusion Sending data and Reinforcing the “best” path Source - Introduce sink and source - The user requests for low-data-rate events - Interest is sent and propagated. (the black arrow) - Gradients are set up. (the blue arrow) - Events are sent along the gradients. (the red arrow) - The thin red arrow show a low data rate. - The user reinforces the best path in terms of delay (the green arrow) - The thick red arrow shows a high data rate. - Suppose the middle node fail. - There is no high-data-rate events received. - The sink reinforces a low-data-rate path to recover from the node failure. Sink Low rate event Reinforcement = Increased interest

Directed Diffusion and Dynamics Source Sink Recovering from node failure - Introduce sink and source - The user requests for low-data-rate events - Interest is sent and propagated. (the black arrow) - Gradients are set up. (the blue arrow) - Events are sent along the gradients. (the red arrow) - The thin red arrow show a low data rate. - The user reinforces the best path in terms of delay (the green arrow) - The thick red arrow shows a high data rate. - Suppose the middle node fail. - There is no high-data-rate events received. - The sink reinforces a low-data-rate path to recover from the node failure. Low rate event Reinforcement High rate event

Directed Diffusion and Dynamics Source Sink - Introduce sink and source - The user requests for low-data-rate events - Interest is sent and propagated. (the black arrow) - Gradients are set up. (the blue arrow) - Events are sent along the gradients. (the red arrow) - The thin red arrow show a low data rate. - The user reinforces the best path in terms of delay (the green arrow) - The thick red arrow shows a high data rate. - Suppose the middle node fail. - There is no high-data-rate events received. - The sink reinforces a low-data-rate path to recover from the node failure. Stable path Low rate event High rate event

Local Behavior Choices For propagating interests In our example, flood More sophisticated behaviors possible: e.g. based on cached information, GPS For data transmission Multi-path delivery with selective quality along different paths probabilistic forwarding single-path delivery, etc. For setting up gradients data-rate gradients are set up towards neighbors who send an interest. Others possible: probabilistic gradients, energy gradients, etc. For reinforcement reinforce paths, or parts thereof, based on observed delays, losses, variances etc. other variants: inhibit certain paths because resource levels are low However, what we show you is just an example of local rules that can be used. There are several choices of local rules to use. Each of them lead to a different global behavior. In our example, we use flooding for interest propagation. But there are several sophisticated rules that can be used. For example, interest propagation that is based on cached information or GPS. There are also several choices of gradients. For our example, we use data-rate gradients. But probabilistic gradients are also possible. We may also deliver data using only a single path or multiple paths. Deterministically or probabilistically, Redundant or distinct, and so on. Our reinforcement is based on observed delay. We can also reinforce based on observed losses, delay variances, or energy level.

Initial simulation study of diffusion Key metric Average Dissipated Energy per event delivered indicates energy efficiency and network lifetime Compare diffusion to flooding centrally computed tree (omniscient multicast) To evaluate diffusion, we compare with 2 idealized schemes. Flooding and omniscient multicast. For omniscient multicast, the multicast tree is centrally computed from the simulator. No overhead is assigned. We use 3 metrics for our evaluation To show energy efficiency, we measure the average dissipated energy per distinct event received. We also measure the delay so that we can imply the temporal accuracy of information. The event delivery ratio is measure to compare their robustness

Diffusion Simulation Details Simulator: ns-2 Network Size: 50-250 Nodes Transmission Range: 40m Constant Density: 1.95x10-3 nodes/m2 (9.8 nodes in radius) MAC: Modified Contention-based MAC Energy Model: Mimic a realistic sensor radio [Pottie 2000] 660 mW in transmission, 395 mW in reception, and 35 mw in idle We evaluated our diffusion on several network size ranging from 50 to 250 nodes with constant density. We use IEEE 802.11 as our MAC but we are not quite happy with the choice at all. The main reason is that the energy consumption during idle intervals is too much, comparable to energy consumption in reception. So, we have modify the energy model such that it mimics a realistic sensor radio. In this energy model, the energy consumption during idle intervals is only 10% of energy consumption in reception.

Diffusion Simulation Surveillance application 5 sources are randomly selected within a 70m x 70m corner in the field 5 sinks are randomly selected across the field High data rate is 2 events/sec Low data rate is 0.02 events/sec Event size: 64 bytes Interest size: 36 bytes All sources send the same location estimate for base experiments Here are parameters we used in our simulation. There are 5 sources and 5 sinks. Event size is very small. In our simulated scenario, all sources will detect the same animal so they send the same location estimates.

Average Dissipated Energy (Standard 802.11 energy model) 0.14 Diffusion 0.12 Flooding Omniscient Multicast 0.1 0.08 Average Dissipated Energy (Joules/Node/Received Event) 0.06 0.04 - To show the impact of high idle energy, we re-simulate the previous scenarios but this time we set idle energy as high as receiving energy. - The graph indicates that, with the domination of idle energy, all schemes performs approximately the same. In this case, we can just flood the networks because it gives a high packet delivery ratio. 0.02 50 100 150 200 250 300 Network Size Standard 802.11 is dominated by idle energy

Average Dissipated Energy (Sensor radio energy model) 0.018 0.016 Flooding 0.014 0.012 0.01 Average Dissipated Energy (Joules/Node/Received Event) 0.008 Omniscient Multicast 0.006 - The X axis is network size - The Y axis is dissipated energy - As expected, flooding performed the worst. - Diffusion can outperform omniscient multicast because they can perform application-level duplicate suppression while the other can’t. Diffusion 0.004 0.002 50 100 150 200 250 300 Network Size Diffusion can outperform flooding and even omniscient multicast. WHY ?

Impact of In-network Processing 0.025 Diffusion Without Suppression 0.02 0.015 (Joules/Node/Received Event) Average Dissipated Energy 0.01 Diffusion With Suppression 0.005 - However, the main impact comes from application-level duplicate suppression. - Suppression helps reduce dissipated energy about 3-5 times. - This number depends on the number of sources. 50 100 150 200 250 300 Network Size Application-level suppression allows diffusion to reduce traffic and to surpass omniscient multicast.

Impact of Negative Reinforcement 0.012 0.01 Diffusion Without Negative Reinforcement 0.008 Average Dissipated Energy (Joules/Node/Received Event) 0.006 0.004 Diffusion With Negative Reinforcement - This graph shows the impact of negative reinforcement on directed diffusion. - Negative reinforcement helps reduce dissipated energy by a half. 0.002 50 100 150 200 250 300 Network Size Reducing high-rate paths in steady state is critical

Summary of Diffusion Results Under the investigated scenarios, diffusion outperformed omniscient multicast and flooding Application-level data dissemination has the potential to improve energy efficiency significantly Duplicate suppression is only one simple example out of many possible ways. Aggregation (in progress) All layers have to be carefully designed Not only network layer but also MAC and application level Experimentation on our testbed in progress - From our evaluation, we can conclude that diffusion outperform flooding and omniscient multicast under investigated scenarios. - We also show that all layers should be carefully designed with energy efficiency in mind. This includes MAC, network, and application level. - Finally, application-level data processing can improve energy efficiency significantly. Duplicate suppression is just one example. Several others are possible. For example data aggregation.

Implied direction: hierarchical queries Create processing points in the network High level interests/queries for activity triggers lower level local queries for particular data modalities and signatures (e.g. acoustic and vibration patterns that are mapped to the activity of interest) As opposed to generating detailed queries at sink points and relying on opportunistic aggregation alone. Acoustic? Source Large animal? 9/18/2018 Sink

Ongoing work in Diffusion Multipath: reinforcing multiple upstream neighbors for load balancing and robustness Braided vs. Disjoint paths Opportunistic aggregation of source data Managing gradients/resources Tiny diffusion for Motes Diffusion under mobility: objects, nodes 9/18/2018

III. Enabling Sensor Networks: works in progress Time synchronization Localization Self-configuration 9/18/2018

Time Synchronization Critical at many layers TDMA guard bands Data aggregation, collaborative processing Localization But time sync needs are non-uniform Precision Lifetime Scope & Availability Cost and form factor And time sync can be expensive in terms of communications…energy No single method optimal on all axes 9/18/2018

Pulse Synchronization “External” node generates pulse. Synchronizing nodes compare reception times. Create locality of synchronized nodes, quickly and energy-efficiently NTP good at correcting frequency Local pulse good at correcting phase Use combination Initial experiment using wired stimulus sent to 10 nodes…evaluated precision of achievable timestamp 1 usec clock resolution achieved (vs 100 usec with NTP alone) Combination is 10x better than either solution alone: multimodal is good Do as well when NTP used in pre-training! 9/18/2018

9/18/2018

Localization Needed for coordination of many 3-space related tasks Coordination/scoping of network operation as well Multi-modal ranging and localization: RF RSSI: inadequate for most environments due to multi-path, shadowing Acoustic ranging: measure time of flight of chirp, using RF for synchronization Non Line of Site propagation effects distort measurements Hard to determine source of geometrical inconsistencies Investigating imaging to identify NLOS sources and combine with acoustic 9/18/2018

Results This graph shows the results of a series of tests in a noisy machine room. Each point represents about 10 trials. The tests were conducted at 1 m intervals. The data in each point ranges about 1.5 cm. The variance is about 0.01 cm 9/18/2018

Self-configuration Each node assesses its connectivity and signals or actuates when it detects a depleted (BW/fidelity) region. 'Healing' is collaborative self-organized deployment of nodes Activate more/fewer nodes Mobilize more/fewer nodes Adjust duty cycle/power level of existing nodes… Assumptions: No centralized processing; all nodes act based on locally available information. A very large solution space; not seeking unique optimal solution. Some links have high packet loss.. 9/18/2018

IV. Wrapping up… Tiered Architecture We are implementing a sensor net hierarchy: PC-104s, tags, motes, ephemeral one-shot sensors Save energy by Running the lower power and more numerous nodes at higher duty cycles than larger ones Having low-power “pre-processors” activate higher power nodes or components (Sensoria approach) Components within a node can be tiered too Our “tags” are a stack of loosely coupled boards Interrupts active high-energy assets only on demand 9/18/2018

Tiered Platform for experimentation with SCADDS algorithms Embedded PC: COTS PC104 CPU module AMD ELANSC400, 16MB RAM+16MB FlashDisk, 4 serial/1 parallel ports Phasing out current radio: 418Mhz RPC from Radiometrix Moving to RFM OS: Slimmed Redhat 6.1. (2.2.x/Libc6) Incoporating PC104+ for higher end processing, image capture, etc Tags and Motes: 8 bit proc (ATMEL/PIC) RFM Radio Mote nicely packaged Tag for more experimentation Culler’s TOS ISI PC-104 UCB Mote (Pister) UCLA Tag (Girod) 9/18/2018

Technical challenges Ad hoc, self organizing, adaptive systems with predictable behavior Collaborative processing, data fusion, multiple sensory modalities Data analysis/mining to identify collaborative sensing, triggering thresholds, etc Combining experimentation, simulation, and analysis Engaging theory community (Algorithms? Controls?) 9/18/2018

Enormous Potential Impact Disaster Recovery and Urban Rescue Earth Science Exploration Condition Based Maintenance Medical monitoring Wearable computing Networked Embedded Systems Smart spaces Transportation Environmental Monitoring Active Structures Biological Monitoring Strand Stand Bio-Tank -scaled Tethered Robot Algae 9/18/2018 Sensors 2 meters

More information UCLA Laboratory for Embedded Collaborative Systems (LECS) http://lecs.cs.ucla.edu UCLA Distributed Embedded Systems Program (DESP) http://desp.cs.ucla.edu (joint EE and CS) SCADDS project http://www.isi.edu/scadds ns-2: network simulator (with diffusion supports) http://www.isi.edu/nsnam/dist/ns-src-snapshot.tar.gz Our testbed and software http://www.isi.edu/scadds/testbeds.html 9/18/2018

Some Other Related Work (NOT complete) Sensor networks www.isi.edu/scadds www.janet.ucla.edu/WINS wins.rsc.rockwell.com wind.lcs.mit.edu/~hari www.nesl.ee.ucla.edu/people/mbs tinyos.millennium.berkeley.edu Smart Matter www.parc.xerox.com/spl/projects/smart-matter www-swiss.ai.mit.edu/projects/amorphous Internet design inspiration irl.cs.ucla.edu/AWC/ www-mash.cs.berkeley.edu/mash 9/18/2018