DRIFT: Efficient Message Ordering in Ad Hoc Networks Stefan Pleisch Joint work with Thomas Clouser, Mikhail Nesterenko, André Schiper EPFL, Kent State University SRDS 2006

Background: Ad Hoc Networks Communication is ad hoc, i.e., no fixed communication infrastructure Two nodes can communicate if they are within transmission range, or there is a sequence of intermediate nodes that can forward the message from the source to the destination (set up by a routing protocol) Communication is by broadcast (to all neighbors), or by point-to-point (to one neighbor, but still a broadcast) Shared transmission media and low bandwidth leads to interference and message collisions, and thus to message losses Hidden terminal problem (Allan 1993) Maintaining routing information may not be feasible if nodes are mobile or have limited resources flooding is an effective mechanism of reaching all nodes in the network without routing information a source broadcasts a message to its neighbors every node rebroadcasts the message once

The Need for Total Ordering in Ad Hoc Networks Consider a temporary military sensor network deployed to protect an extended asset communication is multihop and ad hoc directives are periodically issued by mobile operators commands must be delivered in the same order at each sensor node to prevent conflicting behavior of different network regions requires Total Order Multicast More applications from traditional fixed networks find their way to wireless networks Properties of ad hoc networks make total ordering of messages challenging

Outline Total Order Multicast Lamport’s Total Order Multicast Algorithm Virtual Flooding DRIFT Description Example Simulation and Experimentation Conclusion and Future Work

Total Order Multicast Communication primitives TO-multicast is invoked to send a message to all the nodes of the multicast group; executed by a source node TO-deliver is executed to convey the message to the application; executed by a destination node Problem: Ensure all messages TO-multicast by any source are TO-delivered in the same order at all destinations

Related Work Many total order multicasts in fixed networks, see ACM Survey by Defago et al. 2003 sequencer-based: single sequencer decides ordering asymmetric load on the network privilege-based: token-holder establishes order expensive route maintenance among the token users destination: ordering by agreement among destinations expensive when man destinations communication history ordering: based on message timestamps Few algorithm in ad hoc networks sequencer-based in single hop networks Bartoli 1998 (Mobile Networks and Applications) Anastati et al 1999 (SRDS’99) privilege-based Malpani et al. (IEEE ToMC) communication history Prakash et al. (ICDCS’97), dependency on fixed infrastructure Lou et al. (IEEE ToMC) use probabilistic guarantees

Lamport’s Total Order Multicast Algorithm CACM 1978, belongs to the class of communication history ordering Delivers messages based on the causal order of multicasts causal relation establishes a partial ordering total order achieved by deterministically ordering concurrent messages (e.g. by source id) Assumes FIFO communication channels and reliable messaging, no failures Uses logical clocks (Lamport’s clocks) for capturing causal relationship between multicast messages Source nodes update their logical clock prior to sending a multicast when the source receives a multicast message it has not received before Delivery rule: Node n can TO-deliver a particular message m only after it receives a message with a higher or equal timestamp from every source Disadvantage: the delivery rate of all destinations depends on the sending rate of the source that multicasts least frequently

Virtual Flooding Node attaches data to an unrelated message it has to broadcast Example Node a has message to physically flood Node c has a message to virtually flood Node a sends m Node b forwards m Node c attaches virtually flooded message and forwards m Node d forwards combined message Node e forwards combined message

DRIFT: Key Concepts Combines virtual flooding with Lamport’s algorithm to achieve lower delivery latencies Virtual flooding propagates the latest logical clock of each source For node n to TO-deliver message m it is sufficient that n learns that it will not receive a message from any source with a timestamp less than or equal to m’s timestamp DRIFT assumes reliable flooding, as e.g. in DELUGE (Hui and Culler, Sensys’04)

Example Initial condition – a and c are sources, b is a destination lc = 0 sn = 0 lc = 0 sn = 0 a b c RcvdSN = [0,0] Seen = { } RcvdSN = [0,0] Seen = { } RCVD = { } DRIFT DLVD = { } TOF DLVD = { } RcvdSN = [0,0] Seen = { } Note: TOF is Lamport’s algorithm

Example a b c a TO-multicasts <am1, a, 1, 1, {<a, 1, 1>}> Sequence number (sn) a TO-multicasts <am1, a, 1, 1, {<a, 1, 1>}> Logical clock (lc) Source lc = 1 sn = 1 lc = 0 sn = 0 a b c RcvdSN = [1,0] Seen = {} RcvdSN = [0,0] Seen = { } RCVD = { } DRIFT DLVD = { } TOF DLVD = { } RcvdSN = [0,0] Seen = {}

Example a b c b has received <am1, a, 1, 1, {<a, 1, 1>}> lc = 1 sn = 1 lc = 0 sn = 0 a b c RcvdSN = [1,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>} RCVD = {am1} DRIFT DLVD = {} TOF DLVD = {} RcvdSN = [0,0] Seen = {} b updates RcvdSN and Seen

Example a b c b rebroadcasts <am1, a, 1, 1, {<a, 1, 1>}> lc = 1 sn = 1 lc = 0 sn = 0 a b c RcvdSN = [1,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>} RCVD = {am1} DRIFT DLVD = {} TOF DLVD = {} RcvdSN = [0,0] Seen = {}

Example a b c c has received <am1, a, 1, 1, {<a, 1, 1>}> lc = 1 sn = 1 lc = 2 sn = 0 a b c RcvdSN = [1,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>} RCVD = {am1} DRIFT DLVD = {} TOF DLVD = {} RcvdSN = [1,0] Seen = {<a,1,1>}

Example c rebroadcasts <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}> lc = 1 sn = 1 lc = 2 sn = 0 a b c RcvdSN = [0,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>} RCVD = {am1} DRIFT DLVD = {} TOF DLVD = {} RcvdSN = [1,0] Seen = {<a,1,1>}

Example b receives <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}> lc = 1 sn = 1 lc = 2 sn = 0 a b c RcvdSN = [0,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>, <c,2,0>} RCVD = {am1} DRIFT DLVD = {am1} TOF DLVD = {} RcvdSN = [1,0] Seen = {<a,1,1>} b TO-delivers am1 with DRIFT, but not with TOF

Example c TO-multicasts <cm1, c, 3, 1, {<a, 1, 1>, <c, 3, 1>}> lc = 1 sn = 1 lc = 3 sn = 1 a b c RcvdSN = [1,0] Seen = {} RcvdSN = [1,0] Seen = {<a,1,1>, <c,2,0>} RCVD = {am1} DRIFT DLVD = {am1} TOF DLVD = {} RcvdSN = [1,0] Seen = {<a,1,1>}

Example a b c lc = 4 lc = 3 sn = 1 sn = 1 RcvdSN = [1,1] Seen = {<c,3,1>} RcvdSN = [1,1] Seen = {<a,1,1>, <c,2,0>,<c,3,1>, <a,4,1> } RCVD = {am1, cm1} DRIFT DLVD = {am1, cm1} TOF DLVD = {am1, cm1} RcvdSN = [1,1] Seen = {<a,1,1>}

Simulation -- Setup Using Java-based JiST/SWANS network simulator (v1.0.4) Developed by Rimon Barr, 2004 (http://jist.ece.cornell.edu) Applications written for real deployment can be ported to the simulation environment and be placed under variety of simulated scenarios Communication is by broadcast as defined by IEEE 802.11b, transmission range is set to 88m Setup 100 nodes in a field of 400x400m Nodes are stationary Nodes start up at random times and positions. Each node floods 20 messages (128Bytes) at regular interval (= base rate) through the entire field

Simulation – Results and Metric Message loss due to hidden terminal problem minimized due to low base rates Results: Average of 20 runs with 95% confidence interval Delivery latency – the time needed to TO-deliver a message after it was received at a destination We compare the performance of Total Order multicast with Flooding only (TOF) , Lamport’s Algorithm Total Order multicast with Virtual Flooding (TOVF), DRIFT using physically flooded multicast messages as virtual flooding carriers => Speedup = latencyTOF / latencyTOVF Unless otherwise noted – the measurement for TOF and TOVF are taken in the same experimental trial

Rate Delay Speedup with different base rates Varying rate delay The delivery latency is impacted by base rate – the rate at which the source TO-multicast messages rate delay – the relative difference in the base rate between the sources To evaluate the effect of relative rate differences, for each source i we set the TO-multicast rate as follows TO-multicastRatei = baseRate + (i  rateDelay)

Position of Flooding Source (1/2) Single source [0,0] Average Max order of network diameter One to four sources physically flood messages at aggregated frequency 1s-1 Other nodes use only virtual flooding Varying positions of (physical) flooding source All nodes are destinations

Position of Flooding Source (2/2) 4 sources in 4 corners Average Max Average Average Max Max Single source [350,350] 2 sources [0,0], [600,600]

Gossiping Number of virtual flooding entries per message Base rate 10s Overhead increases linearly Speedup decreases with increasing number of transmitted tuples I = 0, network contains 50 nodes, all are sources and destinations. Applications need to chose a trade-off point between overhead and speedup

Experimental Setup We evaluate the performance of DRIFT using BenchNet, our wireless sensor testbed Setup: 16 MICA2 motes, arranged in a 4x4 grid, running the TinyOS operating system and programmed using the nesC programming language Each mote is connected via its communication port to an Ethernet programming board – which allows monitoring applications and the motes to communicate 4 interior nodes are sources, all nodes are destinations Each source multicasts 10 messages Base rate of 30 seconds Reliable 1-hop communication emulated TOF and TOVF implemented separately

Conclusion and Future Work DRIFT uses virtual flooding to propagate logical clock information We demonstrated through simulation and experimentation the effectiveness of DRIFT as a total order multicast delivery mechanism for ad hoc networks Although based on flooding, DRIFT can be modified to work with structured routing mechanisms Virtual flooding can be used to propagate data of any type Future work measure different failure scenarios, especially failures of sources scale the experimental evaluation up to many nodes

Questions