1. DRIFT: Efficient Message Ordering in Ad Hoc Networks
Stefan Pleisch
Joint work with
Thomas Clouser, Mikhail Nesterenko, André Schiper
EPFL, Kent State University
SRDS 2006

2. Background: Ad Hoc Networks
Communication is ad hoc, i.e., no fixed communication infrastructure
Two nodes can communicate if
they are within transmission range, or
there is a sequence of intermediate nodes that can forward the message from the source to the destination (set up by a routing protocol)
Communication is by broadcast (to all neighbors), or by point-to-point (to one neighbor, but still a broadcast)
Shared transmission media and low bandwidth leads to interference and message collisions, and thus to message losses
Hidden terminal problem (Allan 1993)
Maintaining routing information may not be feasible if nodes are mobile or have limited resources
flooding is an effective mechanism of reaching all nodes in the network without routing information
a source broadcasts a message to its neighbors
every node rebroadcasts the message once

3. The Need for Total Ordering in Ad Hoc Networks
Consider a temporary military sensor network deployed to protect an extended asset
communication is multihop and ad hoc
directives are periodically issued by mobile operators
commands must be delivered in the same order at each sensor node to prevent conflicting behavior of different network regions
requires Total Order Multicast
More applications from traditional fixed networks find their way to wireless networks
Properties of ad hoc networks make total ordering of messages challenging

4. Outline Total Order Multicast
Lamport’s Total Order Multicast Algorithm
Virtual Flooding
DRIFT Description
Example
Simulation and Experimentation
Conclusion and Future Work

5. Total Order Multicast Communication primitives
TO-multicast is invoked to send a message to all the nodes of the multicast group; executed by a source node
TO-deliver is executed to convey the message to the application; executed by a destination node
Problem: Ensure all messages TO-multicast by any source are TO-delivered in the same order at all destinations

6. Related Work
Many total order multicasts in fixed networks, see ACM Survey by Defago et al. 2003
sequencer-based: single sequencer decides ordering
asymmetric load on the network
privilege-based: token-holder establishes order
expensive route maintenance among the token users
destination: ordering by agreement among destinations
expensive when man destinations
communication history ordering: based on message timestamps
Few algorithm in ad hoc networks
sequencer-based in single hop networks
Bartoli 1998 (Mobile Networks and Applications)
Anastati et al 1999 (SRDS’99)
privilege-based
Malpani et al. (IEEE ToMC)
communication history
Prakash et al. (ICDCS’97), dependency on fixed infrastructure
Lou et al. (IEEE ToMC) use probabilistic guarantees

7. Lamport’s Total Order Multicast Algorithm
CACM 1978, belongs to the class of communication history ordering
Delivers messages based on the causal order of multicasts
causal relation establishes a partial ordering
total order achieved by deterministically ordering concurrent messages (e.g. by source id)
Assumes FIFO communication channels and reliable messaging, no failures
Uses logical clocks (Lamport’s clocks) for capturing causal relationship between multicast messages
Source nodes update their logical clock
prior to sending a multicast
when the source receives a multicast message it has not received before
Delivery rule: Node n can TO-deliver a particular message m only after it receives a message with a higher or equal timestamp from every source
Disadvantage: the delivery rate of all destinations depends on the sending rate of the source that multicasts least frequently

8. Virtual Flooding
Node attaches data to an unrelated message it has to broadcast
Example
Node a has message to physically flood
Node c has a message to virtually flood
Node a sends m
Node b forwards m
Node c attaches virtually flooded message and forwards m
Node d forwards combined message
Node e forwards combined message

9. DRIFT: Key Concepts
Combines virtual flooding with Lamport’s algorithm to achieve lower delivery latencies
Virtual flooding propagates the latest logical clock of each source
For node n to TO-deliver message m it is sufficient that n learns that it will not receive a message from any source with a timestamp less than or equal to m’s timestamp
DRIFT assumes reliable flooding, as e.g. in DELUGE (Hui and Culler, Sensys’04)

10. Example
Initial condition – a and c are sources, b is a destination
lc = 0
sn = 0
lc = 0
sn = 0 a b c
RcvdSN = [0,0]
Seen = { }
RcvdSN = [0,0]
Seen = { }
RCVD = { }
DRIFT DLVD = { }
TOF DLVD = { }
RcvdSN = [0,0]
Seen = { }
Note: TOF is Lamport’s algorithm

11. Example a b c a TO-multicasts <am1, a, 1, 1, {<a, 1, 1>}>
Sequence number (sn)
a TO-multicasts <am1, a, 1, 1, {<a, 1, 1>}>
Logical clock (lc)
Source
lc = 1
sn = 1
lc = 0
sn = 0 a b c
RcvdSN = [1,0]
Seen = {}
RcvdSN = [0,0]
Seen = { }
RCVD = { }
DRIFT DLVD = { }
TOF DLVD = { }
RcvdSN = [0,0]
Seen = {}

12. Example a b c b has received <am1, a, 1, 1, {<a, 1, 1>}>
lc = 1
sn = 1
lc = 0
sn = 0 a b c
RcvdSN = [1,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}
RCVD = {am1}
DRIFT DLVD = {}
TOF DLVD = {}
RcvdSN = [0,0]
Seen = {}
b updates RcvdSN and Seen

13. Example a b c b rebroadcasts <am1, a, 1, 1, {<a, 1, 1>}>
lc = 1
sn = 1
lc = 0
sn = 0 a b c
RcvdSN = [1,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}
RCVD = {am1}
DRIFT DLVD = {}
TOF DLVD = {}
RcvdSN = [0,0]
Seen = {}

14. Example a b c c has received <am1, a, 1, 1, {<a, 1, 1>}>
lc = 1
sn = 1
lc = 2
sn = 0 a b c
RcvdSN = [1,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}
RCVD = {am1}
DRIFT DLVD = {}
TOF DLVD = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}

15. Example
c rebroadcasts <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}>
lc = 1
sn = 1
lc = 2
sn = 0 a b c
RcvdSN = [0,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}
RCVD = {am1}
DRIFT DLVD = {}
TOF DLVD = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}

16. Example
b receives <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}>
lc = 1
sn = 1
lc = 2
sn = 0 a b c
RcvdSN = [0,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>, <c,2,0>}
RCVD = {am1}
DRIFT DLVD = {am1}
TOF DLVD = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}
b TO-delivers am1 with DRIFT, but not with TOF

17. Example
c TO-multicasts <cm1, c, 3, 1, {<a, 1, 1>, <c, 3, 1>}>
lc = 1
sn = 1
lc = 3
sn = 1 a b c
RcvdSN = [1,0]
Seen = {}
RcvdSN = [1,0]
Seen = {<a,1,1>, <c,2,0>}
RCVD = {am1}
DRIFT DLVD = {am1}
TOF DLVD = {}
RcvdSN = [1,0]
Seen = {<a,1,1>}

18. Example a b c lc = 4 lc = 3 sn = 1 sn = 1 RcvdSN = [1,1]
Seen = {<c,3,1>}
RcvdSN = [1,1]
Seen = {<a,1,1>, <c,2,0>,<c,3,1>, <a,4,1> }
RCVD = {am1, cm1}
DRIFT DLVD = {am1, cm1}
TOF DLVD = {am1, cm1}
RcvdSN = [1,1]
Seen = {<a,1,1>}

19. Simulation -- Setup
Using Java-based JiST/SWANS network simulator (v1.0.4)
Developed by Rimon Barr, 2004 (
Applications written for real deployment can be ported to the simulation environment and be placed under variety of simulated scenarios
Communication is by broadcast as defined by IEEE b, transmission range is set to 88m
Setup
100 nodes in a field of 400x400m
Nodes are stationary
Nodes start up at random times and positions. Each node floods 20 messages (128Bytes) at regular interval (= base rate) through the entire field

20. Simulation – Results and Metric
Message loss due to hidden terminal problem minimized due to low base rates
Results: Average of 20 runs with 95% confidence interval
Delivery latency – the time needed to TO-deliver a message after it was received at a destination
We compare the performance of
Total Order multicast with Flooding only (TOF) , Lamport’s Algorithm
Total Order multicast with Virtual Flooding (TOVF), DRIFT using physically flooded multicast messages as virtual flooding carriers
=> Speedup = latencyTOF / latencyTOVF
Unless otherwise noted – the measurement for TOF and TOVF are taken in the same experimental trial

21. Rate Delay Speedup with different base rates Varying rate delay
The delivery latency is impacted by
base rate – the rate at which the source TO-multicast messages
rate delay – the relative difference in the base rate between the sources
To evaluate the effect of relative rate differences, for each source i we set the TO-multicast rate as follows
TO-multicastRatei = baseRate + (i  rateDelay)

22. Position of Flooding Source (1/2)
Single source [0,0]
Average
Max
order of network diameter
One to four sources physically flood messages at aggregated frequency 1s-1
Other nodes use only virtual flooding
Varying positions of (physical) flooding source
All nodes are destinations

23. Position of Flooding Source (2/2)
4 sources in 4 corners
Average
Max
Average
Average
Max
Max
Single source [350,350]
2 sources [0,0], [600,600]

24. Gossiping Number of virtual flooding entries per message Base rate 10s
Overhead increases linearly
Speedup decreases with increasing number of transmitted tuples
I = 0, network contains 50 nodes, all are sources and destinations.
Applications need to chose a trade-off point between overhead and speedup

25. Experimental Setup
We evaluate the performance of DRIFT using BenchNet, our wireless sensor testbed
Setup: 16 MICA2 motes, arranged in a 4x4 grid, running the TinyOS operating system and programmed using the nesC programming language
Each mote is connected via its communication port to an Ethernet programming board – which allows monitoring applications and the motes to communicate
4 interior nodes are sources, all nodes are destinations
Each source multicasts 10 messages
Base rate of 30 seconds
Reliable 1-hop communication emulated
TOF and TOVF implemented separately

26. Conclusion and Future Work
DRIFT uses virtual flooding to propagate logical clock information
We demonstrated through simulation and experimentation the effectiveness of DRIFT as a total order multicast delivery mechanism for ad hoc networks
Although based on flooding, DRIFT can be modified to work with structured routing mechanisms
Virtual flooding can be used to propagate data of any type
Future work
measure different failure scenarios, especially failures of sources
scale the experimental evaluation up to many nodes

27. Questions