1. TCP over Wireless Networks

2. Puzzle In C, what is the output of the following code: main() {
int a[5] = {0,1,2,3,4};
a[2] & printf(“%d %d”, 3[a], 3[a]++); }

3. Outline TCP over wireless networks
TCP assumptions
Wireless network characteristics
Impact on TCP performance
Approaches to improve TCP performance
Link layer approaches
TCP-aware link layer approaches
Split connection approaches
End-to-end approaches
TCP over satellite and ad-hoc networks

4. TCP on Wireless – Random Losses
TCP uses losses as indication of congestion
Reduces congestion window by half (LIMD) when it experiences congestion
Even when no congestion, if packets are dropped due to random losses, TCP will cut down its rate (is this right?)

5. Other Losses
TCP will interpret hand-offs related losses also as congestion based losses
Hence, it will reduce the congestion window every time hand-offs related losses occur
Black-outs will further result in TCP experiencing multiple timeouts of increasing granularity

6. Large and Varying Delay
TCP uses RTTavg + 4 * RTTmdev as the retransmission timeout
If there is large variance in delay, mean deviation is high resulting in inflated timeout values
Hence, if there are burst losses resulting in a timeout, the sender would take longer time to detect losses and recover

7. Path Asymmetry
TCP relies on ACK arrivals for congestion window progression
If path asymmetry exists, a TCP connection’s performance will be influenced by the reverse path characteristics also
Indirect effects of path asymmetry (ACK bunching)

8. Low Bandwidths TCP uses window based congestion control
If there is free space in the congestion window, TCP will transmit
Hence, TCP’s output can be bursty
This coupled with the low bandwidths can result in queue build-ups in the network adversely affecting RTT calculations and causing packet drops

9. Large Bandwidth Delay Products
TCP’s header has 16 bits allocated for receiver window advertisement
Maximum of 500Kb can be advertised
Consider a satellite link with 1Mbps bandwidth and 1 second RTT (BDP = 1Mb)
But, TCP can only achieve 500Kbps (resulting in only 50% utilization)

10. Slow-start
TCP uses slow-start to ramp up rate to the available capacity
Whenever timeouts occur, TCP uses slow-start
If hand-offs, black-outs, or route re-computations are frequent, TCP will constantly be in slow-start, lowering performance

11. Recap TCP over wireless networks
Wireless losses
Other losses
Large and varying delay
Path asymmetry
Low bandwidths
Large bandwidth delay product
Slow start
Approaches to improve TCP performance …

12. Approaches Reliable link layers TCP-aware link layers
Split connection protocols

13. Reliable Link Layers
Help in recovering from losses that occur on the wireless link
Can potentially hide such losses from the TCP layer
Can be implemented without requiring any changes at all to the sender and the receiver

14. Reliable Link Layer (contd.)
Losses can still occur
Retransmission can interfere with TCP retransmissions worsening the performance
TCP timeouts similar LL timeouts
DUPACKs
Can introduce variations in TCP’s rtt estimation increasing the RTO

15. Snoop Module Resides in the base-station
Caches packets sent from fixed host to mobile host
TCP-aware functionality for retransmissions, and ACK suppression
Improves on the performance of reliable link layers

16. Snoop (Contd.) Two modules:
Snoop_data() – for processing data packets on the forward path (from FH – MH)
Snoop_ACK() – for processing ACK packets on the reverse path (from MH – FH)

17. Snoop_data() 3 scenarios Normal packet in sequence Cache packet
Forward to MH
Timestamp if necessary (once per window) – for RTOs

18. Snoop_data() Scenario 2 Out of sequence (S) but already cached
If highestACK < S
Forward packet
Else
Generate an ACK from the snoop module for the highestACK

19. Snoop_data() Scenario 3
Out of sequence (S) and has not been cached earlier
Either out-of-order or packet that was lost earlier
Heuristically assume retransmission
Packet forwarded, and marked as having been retransmitted due to congestion loss

20. Snoop_ACK() Scenario 1 New ACK Spurious ACK Clean snoop cache
Round-trip estimate updated
Spurious ACK
ACK# less than highestACK#
Discard ACK

21. Snoop_ACK() Duplicate ACK (DUPACK)
If sender retransmitted packet or packet not in cache, forward ACK to sender
Unexpected DUPACK (loss between BS and MH)
Retransmit packet
Keep track of maximum number of relevant DUPACKs possible
Expected DUPACKs
Suppress

22. Mobile host to Fixed host
Cannot use only base-station based mechanisms
Need to change the mobile host
NACKs from the base-station to the mobile-station
Mobile-station retransmits immediately and does not perform window reduction for NACKed losses

23. Split Connection Approaches
Fixed Host
Base Station
Mobile Host
TCP
Wireless aware TP

24. Advantages Wireless aware congestion control and flow control
Wireless link characteristics completely decoupled from the progress of TCP on the wired leg
Better service to mobile applications possible
Potentially simpler protocol stack at mobile
Improved performance

25. Cons Application layer re-linking End-to-end semantics
Software overheads
Hand-off latency

26. Recap Wireless networks and TCP Reliable link layers Snoop module
Indirect TCP (split connection)
WTCP …

27. WTCP Reliable transport protocol for wide-area wireless networks
Alternative to TCP (but completely different unlike the schemes we have seen so far)
End-to-end approach

28. WWANs Non-congestion related packet losses (0 to 10% in CDPD networks)
Very low bandwidths (20 Kbps CDPD, 8 Kbps RAM, 5Kbps Ardis)
Large round-trip times and variance in round-trip time (800ms to 4 seconds)
Asymmetric channel (ACK bunching)
Occasional blackouts

29. Arguments against non-E2E approaches
Overhead at the base-station (or mobile switching center)
Need to be TCP aware (different flavors of TCP and consequent problems …)
Comparable timeout values when delay over the wireless network is a significant portion of the end to end delay
E2E semantics critical for applications
Other overheads such as hand-off overheads in I-TCP

30. WTCP Approach Rate based transmission control
Eliminates the RTO miscalculation problem in TCP due to self-clocking/window-based CC/ACK-bunching
No waiting for ACKs before sending out new packets
Packets separated by a period T seconds where T = 1/R where R is the rate of the flow
Cons?
Why is this not a problem in WWANs?

31. Inter-packet delay as main congestion indicator
Use inter-packet delay (as opposed to losses) as indication of congestion
Specifically, the relative differences between the average receiver inter packet delay and the average sender inter packet delay is used to detect congestion
Helps in keeping the network un-congested and at the same time fully utilize the network

32. Distinguishing cause of packet loss
Losses still possible because of congestion, random wireless errors, and blackouts
Ideal reaction for each of the probable causes different (what?)
Need to distinguish the cause of packet loss to facilitate correct reaction

33. Distinguishing … Consider packets x, x+1, x+2
Let r be the rate of transmission (in pps)
If packet x+1 is lost on the wireless leg (random error), the interpacket separation between x and x+2 will be 2/r
If packet x+1 is lost on the wired leg, the interpacket separation between x and x+2 will likely be 1/r
Use to distinguish cause of losses

34. Transmission Control Computations at the Receiver
Unlike TCP, where computations happen solely at the sender, WTCP uses the receiver to perform rate control computations
Eliminates ACK related problems in the congestion control process

35. Variable Granularity Rate Adjustment
TCP uses LIMD (with 1 and 0.5 as increase and decrease parameters)
In WTCP, the initial value of the multiplicative decrease factor is small, and is increased for consecutive decreases (incipient congestion detection)
A decrease factor of 0.5 is used for congestion related losses

36. Start-up Behavior
Round-trip times large, connections potentially short-lived
Faster ramp-up period desirable
WTCP uses the “packet-pair” approach for start-up
Two packets (MSS size) sent back-to-back
Inter-packet delay at receiver used to indicate available rate
Same mechanism used when recovering from black-outs (any other approach?)

37. Reliability Selective acknowledgements
SACKs very useful in TCP when random losses occur
Better utilization of the network resources (no unnecessary retransmissions)

38. Reliability (contd.) No retransmit timeouts
Since RTOs cannot be relied upon, WTCP does not use RTOs
Use probe packets to recover from suffix losses
ACK frequency tuned by sender based on factors such as (a) observed loss rate on reverse path, (b) round-trip time, (c ) half-duplex or full-duplex nature of channel

39. WTCP Algorithm Congestion control
3 phases: increase, decrease, and maintain
Chosen based on long-term and short term averages in inter-packet separation
RTT computation (for probing)
Using loss profile
Increase, decrease, maintain phases

40. WTCP Reliability
SACK algorithm
Probe packets

41. pTCP: A Transport Layer Approach for Achieving Aggregate Bandwidths on Multi-homed Mobile Hosts

42. Introduction A multitude of wireless access technologies
How can these technologies co-exist in providing the best service to a multi-homed mobile user?
Use only the interface with the highest data rate at any given time
Objective
Allow simultaneous use of multiple wireless access technologies (striping)
Design a transport layer protocol that provides reliable and sequenced delivery like in TCP, and achieves effective bandwidth aggregation
existing solutions have been considered using the best…
for example, …
However, the objective of our work is to consider simultaneous use…
For example, 7Mbps
To achieve this objective, we design… --
effective cost-qos tradeoffs
an interesting problem that arises is …
the concept of vertical handoff has been proposed to address this problem, where a mobile user is provided access through the higher data connection when it migrates across coverage areas.
For example, when a user with 2Mbps Wireless WAN access moves within the coverage of 10Mbps Wireless LAN, …
why transport layer
emphasize simultaneous use and bandwidth aggregation
illustrate bandwidth aggregation?

43. Why Not Lower Layers? Link layer striping [Adiseshu ’96, Snoeren ’99]
Stripe according to estimates of link bandwidths
Optimal link layer striping  optimal bandwidth aggregation across multi-hop paths
Not applicable to multi-homed mobile hosts
Network layer striping [Phatak ’02]
IP performs tunneling and striping
TCP’s RTT estimates lose meaning
TCP adversely reacts to packet reordering
bandwidth aggregation at the link layer has been proposed --
But more importantly, TCP’s adversely… causes the problem.
adverse reaction: mention fast retransmit causes window cutdown
Lower layer approaches fail due to the multi-hop nature of paths and TCP’s reliance on FIFO delivery!

44. Why Not Multiple Sockets?
Application layer striping
Open multiple TCP sockets
Each socket is in charge of only one path (pipe)
Perform bandwidth estimation at the sending end and packet re-sequencing at the receiving end
Optimal aggregate bandwidth = sum of maximum achievable bandwidths on the individual pipes
pipe 1
pipe 2
an alternative approach is to use application layer striping by opening multiple …
to achieve bandwidth aggregation, the sending end…, and to provide in-sequence, ..
Note that what we mean by effective bandwidth aggregation is to achieve.. --
smart application coarse level
Problems
Data rate differential
Data rate fluctuations
Blackouts

45. Data Rate Differential
Head of line blocking will result in the slower pipes stalling the faster pipes
Example
Two pipes with mismatched bandwidths
Unaware application
Write until blocked
Sequence Number (Unaware Application) 1 2 3 4 5 6 35 36 37 38 39 40
Time (sec)
Sequence Number (x1000)
Pipe 1 (2500kbps)
Pipe 2 (500kbps)
Throughput vs. Bandwidth Ratio 1 2 3 4 5 6 7 8 9 10
Bandwidth Ratio (to 500kbps)
Throughput (Mbps)
Ideal
Smart Application
Unaware Application
mention ideal here
snapshot , a more generic scenario …
describe the problem
put up ideal with unaware application
smart application stripes based on 5:1
Unaware application does not work. Need some kind of bandwidth estimation
Smart application
Coarse-grained bandwidth estimation
The aggregate bandwidth is limited by the slower pipe in the unaware application

46. Data Rate Fluctuations
Data rate fluctuations cause disproportional striping
Data Rate Fluctuations
1.5
1.7
1.9
2.1
2.3
2.5
2.7
2.9
3.1
3.3
3.5 10 9 8 7 6 5 4 3 2 1
Fluctuation Period (sec)
Throughput (Mbps)
Ideal
Smart Application
Unaware Application 1 2 3 4 5 6 20 40 60 80
100
Bandwidth (Mbps)
Time (sec)
Bandwidth Fluctuations (5Mbps Pipe)
Example
Two pipes with random data rate fluctuations every t seconds
Smart application
Performs bandwidth estimation every T seconds
put up smart application after the fluctuation curve
1. general description
2. describe the link fluctuations (1 to 5Mbps, 200Kbps to 1Mbps)
3. smart application performs coarse grain for every T seconds
Need aggressive bandwidth estimation, which will cause overhead
Packets will be unnecessarily held up in each socket
Smart application’s performance is limited by disproportional striping

47. Blackouts
Undelivered packet in the blackout pipe will cause head of line blocking
Smart Application during Blackouts 5 10 15 20 25 30 40 50 60 70 80 90
100
Time (sec)
Sequence Number (x1000)
Pipe 1 (does not experience blackout)
Pipe 2 (experiences blackout)
Example
One of the two pipes experiences blackout
The other pipe does not experience any blackout
stalled due to pipe 1 blackout
blackout is a phenomenon where the data rate goes to zero --
1. one generic statement
2. scenario
true even for multiple pipes
blackout
Blackouts on one pipe will stall the entire aggregate connection

48. Design Elements Decoupling of functionalities
Per-pipe (TCP-v) behavior vs. aggregate connection (pTCP) behavior
TCP-v: virtual packets; pTCP: data packets
TCP-v: congestion control; pTCP: reliability
TCP-v: how much to send; pTCP: what to send
The fundamental design element of the pTCP protocol is the decoupling of functionalities…
in terms of key functionalities, TCP-v handles…
Note that although we present pTCP as a wrapper around TCP-v, it can be used in tandem with any single-pipe transport layer protocol
TCP-v congestion control, pTCP reliability
Allow plug-in of different congestion control schemes for different pipes

49. Design Elements Congestion window based striping Dynamic reassignment
A packet is given to a TCP-v only when its congestion window has space
Reuse TCP’s congestion control for bandwidth estimation
Dynamic reassignment
Reassign packets that fall out of TCP-v’s window
Avoid packets being held up by one pipe
-> differential [say here that pTCP can use any congestion control; also mention that this is fine-grained bandwidth-based striping]
-> fluctuations [mention that this addresses the problem caused by fluctuations (disproportional striping)]
-> blackouts
binding mention
Redundant striping
Redundantly stripe the first packet after timeouts
Allow the concerned pipe to probe while not potentially stalling the rest

50. pTCP Architecture Application pTCP TCP-v IP data packet virtual packet
control
read
write
ptcp-recv
ip-output
Application IP
pTCP
TCP-v
open/close
established/closed
receive
send
shrunk
resume
recv buffer
send buffer
bindings
active
pipes
virtual recv buffer
virtual send buffer
briefly mention open/close/established/closed only
can link back to the design?

51. pTCP Architecture Application pTCP TCP-v IP data packet virtual packet
control
read
write
ptcp-recv
ip-output
Application IP
pTCP
TCP-v
open/close
established/closed
recv buffer
send buffer
bindings
active
pipes
receive
send
virtual recv buffer
virtual send buffer
resume
mention application reads at the end
briefly mention open/close/established/closed only
can link back to the design?
shrunk

52. pTCP Protocol Congestion control Flow control Reliability
Performed by each TCP-v on a per-pipe basis
Any congestion control mechanism can be used
Flow control
Managed by pTCP through window advertisement
No flow control by TCP-v
Reliability
pTCP delegates reliability to each TCP-v unless the packet has to be reassigned
congestion control: Any congestion control mechanism can be used (current prototype and results using TCP’s CC)
flow control: No flow control by TCP-v
which component handles it
how does it do it
Connection management
pTCP manages the aggregation connection while TCP-v manages the individual pipes

53. Simulation Environment
Topology
Pipe 1: BW = 500kbps ~ 5Mbps, RTT = 100ms
Pipe 2: BW = 2Mbps, RTT = 400ms
Pipe 3: BW = 500kbps, RTT = 400ms
Approaches
Ideal
pTCP
Smart application
Unaware application
3 types of pipes representative of different wireless evironments
and we consider 4 different types of approaches
why such values
simulation does not matter

54. Data Rate Differential
Throughput vs. Bandwidth Ratio 1 2 3 4 5 6 7 8 9 10
Bandwidth Ratio (to 500kbps)
Throughput (Mbps)
Ideal
pTCP
Smart Application
Unaware Application

55. Data Rate Fluctuations
Throughput vs. Number of Pipes (with Fluctuations) 2 4 6 8 10 12 14 16 3 5 7 9
Number of Pipes
Throughput (Mbps)
Ideal
pTCP
Smart Application
Unaware Application
take the same scenario as the “different number of pipes”, and we now introduce data rate fluctuations

56. Blackouts (pTCP) pTCP during Blackouts 10 20 30 40 50 60 70 80 90 10010 20 30 40 50 60 70 80 90
100
Time (sec)
Sequence Number (x1000)
Pipe 1 (does not experience blackout)
Pipe 2 (experiences blackout)
continues progression
blackout

57. Multiple Congestion Control Schemes
Use TCP-ELN for lossy link (Pipe 2)
Multiple Congestion Control Schemes 1 2 3 4 5 6 7
1.00E-05
1.00E-04
1.00E-03
1.00E-02
Packet Drop Probability (Pipe 2)
Throughput (Mbps)
Ideal: TCP (Pipe 1) + ELN (Pipe 2)
pTCP: TCP (Pipe 1) + ELN (Pipe 2)
Smart Application: TCP (Pipe 1) + ELN (Pipe 2)
Smart Application: TCP (Pipe 1) + TCP (Pipe 2)
mention bw for each pipe
ideal first
link back to decoupling of functionalities

58. Related Work Link layer Network layer Transport layer
Scalable striping [Adiseshu ’96]
Adaptive inverse multiplexing [Snoeren ’99]
Network layer
Multi-IP links streaming [Phatak ’02]
Transport layer
SCTP [Stewart ’00]
RMTP
no animation?
RMTP [Magalhaes ’01]
Application layer
PSockets [Sivakumar ’00]

59. Recap
Bandwidth aggregation can be used to provide better service to multi-homed mobile hosts
A transport layer approach is more effective than other layers in the target environment
pTCP achieves effective bandwidth aggregation despite the data rate mismatch, fluctuations, and blackouts of different pipes
mention 3 characteristics first
then target environment
For more information:

60. Puzzle
Which direction is white playing?