Flush: A Reliable Bulk Transport Protocol for Multihop Wireless Networks Sukun Kim †#, Rodrigo Fonseca †, Prabal Dutta †, Arsalan Tavakoli †, David Culler †, Philip Levis*, Scott Shenker †‡, and Ion Stoica † University of California at Berkeley Samsung Electronics International Computer Science Institute Stanford University SenSys 2007 † ‡ * #

1/26  All data from all nodes are needed As quickly as possible Collecting data from one node at a time is completely acceptable Over 46 hop network ! Motivating Example 8 nodes 56 nodes 1125 ft4200 ft 500 ft 246 ft SF (south) Sausalito (north) Structural Health Monitoring of the Golden Gate Bridge

2/26 Introduction  Target applications Structural health monitoring, volcanic activity monitoring, bulk data collection  One flow at a time Remove inter-path interference Easier to optimize for intra-path interference  Built on top of MAC layer No merging with MAC layer for easy porting

3/26 Table of Contents  Introduction  Algorithm  Implementation  Evaluation  Discussion  Related Work  Conclusion

4/26 Flush Algorithm Overview  Receiver-initiated  Selective-NACK  Hop-by-hop Rate Control  Empirically Measure Interference Range

5/26 Rate Control / Rate = Packet Interval = δ 8 + δ 7 + δ 6 + δ 5 δ X : Packet transmission time at node X Interferer:6 5 4

6/26 Interference Range > Reception Range However, Signal Strength Noise Floor + SNR Threshold Noise Floor + 2 X SNR Threshold SNR Threshold – minimum SNR to decode a packet Jammer – a node which can conflict with the transmission, but cannot be heard JammerVulnerable to JammerNo problem to Jammer

7/26 Identifying the Interference Set Fraction of Links CDF of the difference between the received signal strength from a predecessor and the local noise floor A large fraction of interferers are detectable and avoidable

8/26 Implementation – Control Information  Control information is snooped δ: packet transmission time, 1 byte f: sum of δ’s of interfering nodes, 1 byte D: Packet Interval = 1 / Rate, 1 byte  δ, f, D are put into packet header, and exchanged through snooping

9/26 Implementation – Rate-limited Queue  16-deep Rate-limited Queue Enforces departure delay D(i) When a node becomes congested (depth 5), it doubles the delay advertised to its descendants  But continues to drain its own queue with the smaller delay until it is no longer congested

10/26 Table of Contents  Introduction  Algorithm  Implementation  Evaluation  Discussion  Related Work  Conclusion 100 MicaZ nodes – Purple nodes Diameter of 6~7 hops * Mirage Testbed in Intel Research Berkeley Sink

11/26 Packet Throughput of Different Fixed Rates Effective Throughput (pkt/s) Packet throughput of fixed rate streams over different hop counts No fixed rate is always optimal

12/26 Packet Throughput of Flush Overall Throughput (pkt/s) Effective packet throughput of Flush compared to the best fixed rate at each hop Flush tracks the best fixed packet rate

13/26 Bandwidth of Flush Overall Bandwidth (B/s) Effective bandwidth of Flush compared to the best fixed rate at each hop Flush’s protocol overhead reduces the effective data rate

14/26 Fraction of Data Transferred in Different Phases Fraction of data transferred from the 6th hop during the transfer phase and acknowledgment phase Greedy best-effort routing is unreliable, and exhibits a loss rate of 43.5%. A higher than sustainable rate leads to a high loss rate

15/26 Amount of Time Spent in Different Phases Fraction of time spent in different stages A retransmission during the acknowledgment phase is expensive, and leads to a poor throughput

16/26 Packet Throughput at Transfer Phase Transfer Phase Throughput (pkt/s) Effective goodput during the transfer phase Flush provides comparable goodput at a lower loss rate which reduces the time spent in the expensive acknowledgment phase, which increases the effective bandwidth

17/26 Packet Rate over Time for a Source Source is 7 hops away, Data is smoothed by averaging 16 values Flush approximates the best fixed rate with the least variance Flush-e2e has no in-network rate control

18/26 Maximum Queue Occupancy across All Nodes for Each Packet Flush exhibits more stable queue occupancies than Flush-e2e

19/26 Sending Rate at Lossy Link Both Flush and Flush-e2e adapt while the fixed rate overflows its queue Packets were dropped from hop 3 to hop 2 with 50% probability between 7 and 17 seconds

20/26 Queue Length at Lossy Link Flush and Flush-e2e adapt while the fixed rate overflows its queue

21/26 Route Change Experiment We started a transfer over a 5 hop path Approximately 21 seconds into the experiment forced the node 4 hops from the sink to switch its next hop Node 4’s next hop is changed, changing all nodes in the subpath to the root No packets were lost, and Flush adapted quickly to the change 0 1a 2a 3a 1b 2b 3b 4 5

22/26 Scalability Test Overall Throughput (B/s) Effective bandwidth from the real-world outdoor scalability test where 79 nodes formed 48 hop network with 35B payload size Flush closely tracks or exceeds the best possible fixed rate across at all hop distances that we tested

23/26 Table of Contents  Introduction  Algorithm  Implementation  Evaluation  Discussion  Related Work  Conclusion

24/26 Discussion  High-power node reduces hop count and interference Not an option on many structural health monitoring due to power and space problems  Interactions with Routing Link estimator of a routing layer breaks down under heavy traffic

25/26 Related Work  Li et al – capacity of a chain of nodes limited by interference using  ATP, W-TCP – rate-based transmission in the Internet  Wisden – concurrent transmission without a mechanism for a congestion control  Fetch – single flow, selective-NACK, no mention about rate control

26/26 Conclusion  Rate-based flow control Directly measure intra-path interference at each hop Control rate based on interference information  Light-weight solution reduces complexity Overhearing is used to measure interference and to exchange information Two rules to determine a rate  At each node, Flush attempts to send as fast as possible without causing interference at the next hop along the flow  A node’s sending rate cannot exceed the sending rate of its successor  In combination, Flush provides as good bandwidth as fixed rate, and also gives a better adaptability

Questions

28/26

29/26 Reliability , 4, 5 4, 9 SourceSink

30/26 Rate Control: Conceptual Model Rate: Assuming disk model N: Number of nodes, I: Interference range

31/26 Rate Control d 8 = δ 8 + H 7 = δ 8 + δ 7 + f 7 = δ 8 + δ 7 + δ 6 + δ 5 1. At each node, Flush attempts to send as fast as possible without causing interference at the next hop along the flow 2. A node’s sending rate cannot exceed the sending rate of its successor

32/26 Implementation  RSSI is measured by snooping  Information is also snooped δ, f, D are put into packet header, and exchanged through snooping δ, f, D take 1 byte each, 3 bytes total  Cutoff A node i considers a successor node (i− j) an interferer of node i+1 if, for any j > 1, rssi(i+1) − rssi(i−j) < 10 dBm The threshold of 10 dBm was chosen after empirically evaluating a range of values

33/26 Implementation  16-deep Rate-limited Queue Enforces departure delay D(i) When a node becomes congested (depth 5), it doubles the delay advertised to its descendants  But continues to drain its own queue with the smaller delay until it is no longer congested  Protocol Overhead Out of 22B provided by Routing layer, 2B sequence number + 3B control field + 17B payload

34/26 Test Methodology  Mirage testbed in Intel Research Berkeley, consists of 100 MicaZ  -11 dBm  Diameter of 6~7 hops  Average of 4 runs

35/26 Bottom line performance  High-power node reduces hop count and interference Not an option on the Golden Gate Bridge due to power and space problems  Interactions with Routing Link estimator of a routing layer breaks down under heavy traffic  Bottom line performance???

36/26 Average Number of Transmissions per node for sending 1,000 packets

37/26 Bandwidth at Transfer Phase Transfer Phase Throughput (B/s) Effective goodput during the transfer phase Effective goodput is computed as the number of unique packets received over the duration of the transfer phase

38/26 Details of Queue Length for Flush-e2e

39/26 Memory and Code Footprint

40/26

41/

42/

43/26 0 1a 2a 3a 1b 2b 3b 4 5

44/26  Every data from every node is needed Partial data has no or little value  Should work over 46 hops 8 nodes 56 nodes 1125 ft4200 ft 500 ft 246 ft SF (south) Sausalito (north) Motivating Example