1. Some Distributed Coordination Schemes for Wireless Sensor Networks
Deborah Estrin
UCLA Computer Science Department
and
USC/ISI
Collaborative work with SCADDS researchers Heidemann, Govindan, Bulusu, Cerpa, Elson, Ganesan, Girod, Intanagowat, Yu, and Zhao (USC/ISI and UCLA); and Shenker (ACIRI)
9/18/2018

2. I. Motivation
Disaster Response
Embed numerous distributed devices to monitor and interact with physical world: work-spaces, hospitals, homes, vehicles, and “the environment”
Circulatory Net
Network these devices so that they can coordinate to perform higher-level tasks.
Requires robust distributed systems of hundreds or thousands of devices.
9/18/2018

3. Motivating Applications
2 meters
Algae
-scaled
Tethered
Robot
Bio-Tank
Laboratory
Inner wall of storm drain
Sensors
Environmental Monitoring
Model Development
Sensors
Complex Structures
9/18/2018

4. Theme: New Constraints
Tight coupling to the physical world
Need better physical models
More experimentation
Designing for energy constraints
Coping with “apparent” loss of layering
9/18/2018

5. Theme: New Design Goals
Designing for long-lived (and often energy-constrained) systems
Exploiting redundancy
Low-duty cycle operation
Tiered architectures
Self configuring systems
Measure and adapt to unpredictable environment
Exploit spatial diversity of sensor/actuator nodes
Localization and Time synchronization are key building blocks
9/18/2018

6. Implications for Wireless Sensor Network Design
Achieve desired global behavior through localized interactions, without global state
Avoid communication over long distances [Pottie 2000]
Energy propagation loss: E α R4 (10 m: 5000 ops/transmitted bit; 100 m: 50,000,000 ops/transmitted bit)
Empirically adapt to observed environment
Dynamic, messy, environments preclude pre-configured behavior
Leverage data processing/aggregation inside the network
We envision that sensor nodes will be battery-powered. So, energy is one of our main design considerations.
As energy is a very limited resource, sending large amount of data over long distances should be avoided. Data should be processed and summarized into small event description at the sensing node. Only events should be sent out, especially using only multi-hop short range radio. Not only because short range radio consume less energy but also because it can get around obstacles.
And as we pointed out that dynamics are common and the number of nodes is large. We can not assume that we have global knowledge. We don’t want changes in just some area affect the whole network either. So, we think that it would be nice if we use only localized interactions that lead to desired global behavior and empirically adapt to observed environment.
We also expect that sensors are densely deployed so that the detecting sensor is physically close to sensed phenomena. Specially, sensor networks are not general-purposed networks. They are built to perform a very specific type of application. Don’t expect that they will do ‘telnet’. Sensor networks are not about communication among specific nodes. They are more about physical environment. Temperature. Motion. Location. Moisture. And so on.
With application-specific networks, we can do several good things. Intermediate nodes can process, aggregate, transform, or cache data. In sensor networks, such data processing will not be limited to only ending nodes.

7. Roadmap Motivation Directed Diffusion
Other enabling schemes: time synch, localization, self configuration
Wrap up: tiered architecture, future work
9/18/2018

8. II. Example: Directed Diffusion
In-network data processing (e.g., aggregation, caching)
Application-aware communication primitives
expressed in terms of named data (not in terms of the nodes generating or requesting data)
Distributed algorithms using localized interactions and measurement based adaptation
Here we propose directed diffusion, a communication paradigm that is designed with all those consideration in mind.
With directed diffusion, data are named with attributes, not with node id.
A data consumer expresses interest in data with certain attributes. The interest will be propagated towards producers using only local interactions. The interest propagation will leave traces or gradients in the network along the way. The producers will send data back to the consumers using the established gradients. Reinforcement and negative reinforcement will be used to reduce the number of data delivery path for efficient distribution.
And because diffusion is application-aware, all nodes including intermediate nodes can process or cache interests and data at will.

9. Basic Directed Diffusion
Setting up gradients
Source
- Introduce sink and source
- The user requests for low-data-rate events
- Interest is sent and propagated. (the black arrow)
- Gradients are set up. (the blue arrow)
- Events are sent along the gradients. (the red arrow)
- The thin red arrow show a low data rate.
- The user reinforces the best path in terms of delay (the green arrow)
- The thick red arrow shows a high data rate.
- Suppose the middle node fail.
- There is no high-data-rate events received.
- The sink reinforces a low-data-rate path to recover from the node failure.
Sink
Interest = Interrogation in terms of data attributes
Gradient = direction and strength

10. Basic Directed Diffusion
Sending data and Reinforcing the “best” path
Source
- Introduce sink and source
- The user requests for low-data-rate events
- Interest is sent and propagated. (the black arrow)
- Gradients are set up. (the blue arrow)
- Events are sent along the gradients. (the red arrow)
- The thin red arrow show a low data rate.
- The user reinforces the best path in terms of delay (the green arrow)
- The thick red arrow shows a high data rate.
- Suppose the middle node fail.
- There is no high-data-rate events received.
- The sink reinforces a low-data-rate path to recover from the node failure.
Sink
Low rate event
Reinforcement = Increased interest

11. Directed Diffusion and Dynamics
Source
Sink
Recovering
from node failure
- Introduce sink and source
- The user requests for low-data-rate events
- Interest is sent and propagated. (the black arrow)
- Gradients are set up. (the blue arrow)
- Events are sent along the gradients. (the red arrow)
- The thin red arrow show a low data rate.
- The user reinforces the best path in terms of delay (the green arrow)
- The thick red arrow shows a high data rate.
- Suppose the middle node fail.
- There is no high-data-rate events received.
- The sink reinforces a low-data-rate path to recover from the node failure.
Low rate event
Reinforcement
High rate event

12. Directed Diffusion and Dynamics
Source
Sink
- Introduce sink and source
- The user requests for low-data-rate events
- Interest is sent and propagated. (the black arrow)
- Gradients are set up. (the blue arrow)
- Events are sent along the gradients. (the red arrow)
- The thin red arrow show a low data rate.
- The user reinforces the best path in terms of delay (the green arrow)
- The thick red arrow shows a high data rate.
- Suppose the middle node fail.
- There is no high-data-rate events received.
- The sink reinforces a low-data-rate path to recover from the node failure.
Stable path
Low rate event
High rate event

13. Local Behavior Choices
For propagating interests
In our example, flood
More sophisticated behaviors possible: e.g. based on cached information, GPS
For data transmission
Multi-path delivery with selective quality along different paths
probabilistic forwarding
single-path delivery, etc.
For setting up gradients
data-rate gradients are set up towards neighbors who send an interest.
Others possible: probabilistic gradients, energy gradients, etc.
For reinforcement
reinforce paths, or parts thereof, based on observed delays, losses, variances etc.
other variants: inhibit certain paths because resource levels are low
However, what we show you is just an example of local rules that can be used. There are several choices of local rules to use. Each of them lead to a different global behavior.
In our example, we use flooding for interest propagation. But there are several sophisticated rules that can be used. For example, interest propagation that is based on cached information or GPS.
There are also several choices of gradients. For our example, we use data-rate gradients. But probabilistic gradients are also possible.
We may also deliver data using only a single path or multiple paths. Deterministically or probabilistically, Redundant or distinct, and so on.
Our reinforcement is based on observed delay. We can also reinforce based on observed losses, delay variances, or energy level.

14. Initial simulation study of diffusion
Key metric
Average Dissipated Energy per event delivered
indicates energy efficiency and network lifetime
Compare diffusion to
flooding
centrally computed tree (omniscient multicast)
To evaluate diffusion, we compare with 2 idealized schemes. Flooding and omniscient multicast.
For omniscient multicast, the multicast tree is centrally computed from the simulator. No overhead is assigned.
We use 3 metrics for our evaluation
To show energy efficiency, we measure the average dissipated energy per distinct event received.
We also measure the delay so that we can imply the temporal accuracy of information.
The event delivery ratio is measure to compare their robustness

15. Diffusion Simulation Details
Simulator: ns-2
Network Size: Nodes
Transmission Range: 40m
Constant Density: 1.95x10-3 nodes/m2 (9.8 nodes in radius)
MAC: Modified Contention-based MAC
Energy Model: Mimic a realistic sensor radio [Pottie 2000]
660 mW in transmission, 395 mW in reception, and 35 mw in idle
We evaluated our diffusion on several network size ranging from 50 to 250 nodes with constant density.
We use IEEE as our MAC but we are not quite happy with the choice at all. The main reason is that the energy consumption during idle intervals is too much, comparable to energy consumption in reception.
So, we have modify the energy model such that it mimics a realistic sensor radio. In this energy model, the energy consumption during idle intervals is only 10% of energy consumption in reception.

16. Diffusion Simulation Surveillance application
5 sources are randomly selected within a 70m x 70m corner in the field
5 sinks are randomly selected across the field
High data rate is 2 events/sec
Low data rate is 0.02 events/sec
Event size: 64 bytes
Interest size: 36 bytes
All sources send the same location estimate for base experiments
Here are parameters we used in our simulation.
There are 5 sources and 5 sinks.
Event size is very small.
In our simulated scenario, all sources will detect the same animal so they send the same location estimates.

17. Average Dissipated Energy (Standard 802.11 energy model)
0.14
Diffusion
0.12
Flooding
Omniscient Multicast
0.1
0.08
Average Dissipated Energy
(Joules/Node/Received Event)
0.06
0.04
- To show the impact of high idle energy, we re-simulate the previous scenarios but this time we set idle energy as high as receiving energy.
- The graph indicates that, with the domination of idle energy, all schemes performs approximately the same. In this case, we can just flood the networks because it gives a high packet delivery ratio.
0.02 50
100
150
200
250
300
Network Size
Standard is dominated by idle energy

18. Average Dissipated Energy (Sensor radio energy model)
0.018
0.016
Flooding
0.014
0.012
0.01
Average Dissipated Energy
(Joules/Node/Received Event)
0.008
Omniscient Multicast
0.006
- The X axis is network size
- The Y axis is dissipated energy
- As expected, flooding performed the worst.
- Diffusion can outperform omniscient multicast because they can perform application-level duplicate suppression while the other can’t.
Diffusion
0.004
0.002 50
100
150
200
250
300
Network Size
Diffusion can outperform flooding and even omniscient multicast. WHY ?

19. Impact of In-network Processing
0.025
Diffusion Without Suppression
0.02
0.015
(Joules/Node/Received Event)
Average Dissipated Energy
0.01
Diffusion With Suppression
0.005
- However, the main impact comes from application-level duplicate suppression.
- Suppression helps reduce dissipated energy about 3-5 times.
- This number depends on the number of sources. 50
100
150
200
250
300
Network Size
Application-level suppression allows diffusion to reduce traffic and to surpass omniscient multicast.

20. Impact of Negative Reinforcement
0.012
0.01
Diffusion Without Negative Reinforcement
0.008
Average Dissipated Energy
(Joules/Node/Received Event)
0.006
0.004
Diffusion With Negative Reinforcement
- This graph shows the impact of negative reinforcement on directed diffusion.
- Negative reinforcement helps reduce dissipated energy by a half.
0.002 50
100
150
200
250
300
Network Size
Reducing high-rate paths in steady state is critical

21. Summary of Diffusion Results
Under the investigated scenarios, diffusion outperformed omniscient multicast and flooding
Application-level data dissemination has the potential to improve energy efficiency significantly
Duplicate suppression is only one simple example out of many possible ways.
Aggregation (in progress)
All layers have to be carefully designed
Not only network layer but also MAC and application level
Experimentation on our testbed in progress
- From our evaluation, we can conclude that diffusion outperform flooding and omniscient multicast under investigated scenarios.
- We also show that all layers should be carefully designed with energy efficiency in mind. This includes MAC, network, and application level.
- Finally, application-level data processing can improve energy efficiency significantly. Duplicate suppression is just one example. Several others are possible. For example data aggregation.

22. Implied direction: hierarchical queries
Create processing points in the network
High level interests/queries for activity triggers lower level local queries for particular data modalities and signatures (e.g. acoustic and vibration patterns that are mapped to the activity of interest)
As opposed to generating detailed queries at sink points and relying on opportunistic aggregation alone.
Acoustic?
Source
Large animal?
9/18/2018
Sink

23. Ongoing work in Diffusion
Multipath: reinforcing multiple upstream neighbors for load balancing and robustness
Braided vs. Disjoint paths
Opportunistic aggregation of source data
Managing gradients/resources
Tiny diffusion for Motes
Diffusion under mobility: objects, nodes
9/18/2018

24. III. Enabling Sensor Networks: works in progress
Time synchronization
Localization
Self-configuration
9/18/2018

25. Time Synchronization Critical at many layers
TDMA guard bands
Data aggregation, collaborative processing
Localization
But time sync needs are non-uniform
Precision
Lifetime
Scope & Availability
Cost and form factor
And time sync can be expensive in terms of communications…energy
No single method optimal on all axes
9/18/2018

26. Pulse Synchronization
“External” node generates pulse. Synchronizing nodes compare reception times.
Create locality of synchronized nodes, quickly and energy-efficiently
NTP good at correcting frequency
Local pulse good at correcting phase
Use combination
Initial experiment using wired stimulus sent to 10 nodes…evaluated precision of achievable timestamp
1 usec clock resolution achieved (vs 100 usec with NTP alone)
Combination is 10x better than either solution alone: multimodal is good
Do as well when NTP used in pre-training!
9/18/2018

27. 9/18/2018

28. Localization Needed for coordination of many 3-space related tasks
Coordination/scoping of network operation as well
Multi-modal ranging and localization:
RF RSSI: inadequate for most environments due to multi-path, shadowing
Acoustic ranging: measure time of flight of chirp, using RF for synchronization
Non Line of Site propagation effects distort measurements
Hard to determine source of geometrical inconsistencies
Investigating imaging to identify NLOS sources and combine with acoustic
9/18/2018

29. Results
This graph shows the results of a series of tests in a noisy machine room. Each point represents about 10 trials. The tests were conducted at 1 m intervals. The data in each point ranges about 1.5 cm. The variance is about 0.01 cm
9/18/2018

30. Self-configuration
Each node assesses its connectivity and signals or actuates when it detects a depleted (BW/fidelity) region.
'Healing' is collaborative self-organized deployment of nodes
Activate more/fewer nodes
Mobilize more/fewer nodes
Adjust duty cycle/power level of existing nodes…
Assumptions:
No centralized processing; all nodes act based on locally available information.
A very large solution space; not seeking unique optimal solution.
Some links have high packet loss..
9/18/2018

31. IV. Wrapping up… Tiered Architecture
We are implementing a sensor net hierarchy: PC-104s, tags, motes, ephemeral one-shot sensors
Save energy by
Running the lower power and more numerous nodes at higher duty cycles than larger ones
Having low-power “pre-processors” activate higher power nodes or components (Sensoria approach)
Components within a node can be tiered too
Our “tags” are a stack of loosely coupled boards
Interrupts active high-energy assets only on demand
9/18/2018

32. Tiered Platform for experimentation with SCADDS algorithms
Embedded PC:
COTS PC104 CPU module
AMD ELANSC400, 16MB RAM+16MB FlashDisk, 4 serial/1 parallel ports
Phasing out current radio: 418Mhz RPC from Radiometrix
Moving to RFM
OS: Slimmed Redhat 6.1. (2.2.x/Libc6)
Incoporating PC104+ for higher end processing, image capture, etc
Tags and Motes:
8 bit proc (ATMEL/PIC)
RFM Radio
Mote nicely packaged
Tag for more experimentation
Culler’s TOS
ISI PC-104
UCB Mote (Pister)
UCLA Tag (Girod)
9/18/2018

33. Technical challenges
Ad hoc, self organizing, adaptive systems with predictable behavior
Collaborative processing, data fusion, multiple sensory modalities
Data analysis/mining to identify collaborative sensing, triggering thresholds, etc
Combining experimentation, simulation, and analysis
Engaging theory community (Algorithms? Controls?)
9/18/2018

34. Enormous Potential Impact
Disaster Recovery
and Urban Rescue
Earth Science
Exploration
Condition Based Maintenance
Medical monitoring
Wearable computing
Networked Embedded Systems
Smart spaces
Transportation
Environmental Monitoring
Active Structures
Biological Monitoring
Strand
Stand
Bio-Tank
-scaled
Tethered
Robot
Algae
9/18/2018
Sensors
2 meters

35. More information
UCLA Laboratory for Embedded Collaborative Systems (LECS)
UCLA Distributed Embedded Systems Program (DESP)
(joint EE and CS)
SCADDS project
ns-2: network simulator (with diffusion supports)
Our testbed and software
9/18/2018

36. Some Other Related Work (NOT complete)
Sensor networks
wins.rsc.rockwell.com
wind.lcs.mit.edu/~hari
tinyos.millennium.berkeley.edu
Smart Matter
www-swiss.ai.mit.edu/projects/amorphous
Internet design inspiration
irl.cs.ucla.edu/AWC/
www-mash.cs.berkeley.edu/mash
9/18/2018