A Survey of Recent High Speed TCP Variants Philip Fisher-Ogden U.C. Davis March 16th, 2007

Outline Why do we need high speed TCP variants? Three recent variants Compound TCP CUBIC TCP-Adaptive Reno Recent Evaluations Q&A

Standard TCP - advantages Widely used Standardized Promises, promises Guaranteed, in-order delivery Avoids congestion collapse Evolved: Tahoe, Reno/NewReno, SACK

Standard TCP - disadvantages Underutilization of the bandwidth in fast long distance networks (FLDnets) Low window size resiliency to packet loss in FLDnets

Prior High Speed TCPs Effectively use available bandwidth Unfriendly – “doesn’t play nicely with others” Unfair to different RTT flows Examples: HSTCP, STCP, BIC-TCP

Goals TCP friendly RTT fair Efficient use of available bandwidth Minimize bandwidth-stolen RTT fair Co-existing flows with different RTTs are treated fairly Efficient use of available bandwidth

Compound TCP

Compound TCP - Motivation Loss-based - uses packet loss as an indicator of congestion and modifies the increase/decrease congestion avoidance TCP parameters. Delay-based - infers congestion and bottleneck queue size from changes in the RTT and modifies the transfer rates to mitigate the effects of congestion. Combine both approaches Focus on efficiency & friendliness

Compound TCP - Design Maintains a delay window and a congestion window – uses both to determine send window Estimates the bottleneck queue size: Diff = (Expected – Actual) * baseRTT Has a threshold, γ, at which it says “the network is congested”

Compound TCP Delay Window Calculations Formula Meaning dwnd(t+1) = dwnd(t) + (α * win(t)k – 1)+, if diff < γ In the increase phase, CTCP scales up the win (a composite of cwnd and dwnd) by α * win(t)k. (dwnd(t) – ζ * diff )+, if diff ≥ γ When a bottleneck queue is detected, dwnd decreases proportional to the estimated bottleneck size diff. (win(t) * (1 - β) – cwnd/2)+, if loss is detected (3 dup acks) win(t) * (1 - β) is the desired window size after loss occurs, so dwnd is set to the desired size minus the portion that cwnd already provides. Note: (...)+ = max(..., 0)

Compound TCP - Design Builds on loss-based (standard) TCP Reverts to standard TCP at low window sizes Uses gamma-auto tuning to dynamically adjust γ, the estimated number of packets that indicate a bottleneck in the queue, based on the network configuration

Compound TCP - Evaluation Production FLD Network Link Tests (comparing CTCP & TCP) Test Results Throughput CTCP improves the throughput by 28% to 52% over regular TCP. TCP friendliness With gamma auto-tuning, CTCP keeps good TCP friendliness (<20% bandwidth stolen) even in the presence of multiple concurrent flows. Reverse traffic CTCP improves the throughput over TCP even in the presence of many reverse flows.

CUBIC

CUBIC - Motivation Enhance BIC! Maintain BIC’s scalability & stability Simplify the window control Improve BIC’s friendliness Use real-time, rather than ACK-clocked, updates to window Improve the detection of the “TCP Region”

CUBIC - Design Based on BIC:

CUBIC - Design Window growth function:

CUBIC – Design Aspects Stability – The window grows slowly around Wmax. Scalability – It experiences fast “probing” growth away from Wmax. Intra-protocol fairness - Two competing CUBIC flows will converge to fair share window sizes. Fairness – The window growth rate is time dependent and RTT independent, allowing for fairer sharing.

CUBIC – Design Aspects Standard TCP outperforms CUBIC’s window growth function in short RTT networks. CUBIC emulates the time-independent TCP window adjustment algorithm so that it can select the greater of the two windows (emulated versus cubic) to use.

NS-2 Tests (comparing TCP, CUBIC, BIC, HSTCP, STCP, & HTCP) CUBIC - Evaluation NS-2 Tests (comparing TCP, CUBIC, BIC, HSTCP, STCP, & HTCP) Test Results TCP Friendliness in Short-RTT Networks CUBIC TCP maintained the best overall fair-share throughput ratio with regular TCP, and even had less throughput than regular TCP in the higher speed networks. TCP Friendliness in Long-RTT Networks As the link speed increased the throughput ratios of high speed to standard decreased. CUBIC TCP showed the best friendly ratio to regular TCP across all bottleneck link bandwidths in this experiment. Stability The CoV measurements showed that CUBIC TCP provides good stability in the presence of network perturbations and varying router buffer sizes.

TCP-Adaptive Reno (AReno)

TCP-AReno - Motivation Enhance TCP-Westwood-BBE Improve on TCP-Reno’s response function by incorporating a congestion estimate that is based on RTT measurements. Identify if a packet loss was due to congestion or not.

TCP-AReno - Design Congestion Estimate RTTcong = the RTT that would indicate a congestion event. Classifies packet losses as due to congestion or not RTT values vary between RTTmin and RTTcong. The distance of recent RTT measurements from RTTcong and RTTmin determines the level of congestion in the network.

TCP-AReno - Design Congestion Window Increase

TCP-AReno - Design Congestion Window Reduction

TCP-AReno - Evaluation Laboratory Tests (comparing TCP-AReno & TCP-Reno) Test Results Congestion windows of lone flows TCP-AReno is able to increase its congestion window when the link was under-utilized and then limit the increase when the window approached or exceeded the congestion point. TCP-AReno distinguishes between packet loss events that were congestion based or not, and then adapts its congestion window reduction accordingly. Congestion windows of co-existing flows TCP-AReno averaging 227.4Mbps (vs. 232.1) and TCP-Reno averaging 18.5Mbps (vs. 18.3). TCP-AReno is resilient to random packet losses and is able to attain high throughput without negatively affecting TCP-Reno flows. Throughput of a lone flow under various bottleneck capacities TCP-AReno is effective in high-speed and lossy environments, obtaining 10x more throughput than TCP-Reno when the packet loss rate is 10-4.

Recent Evaluations

Recent Evaluations Linux beats Windows! TCP variant Operating System BIC Current Linux CUBIC Possibly future Linux Compound TCP (CTCP) Possibly future Windows Vista TCP Any other operating system

Recent Evaluations Linux beats Windows! BIC/CUBIC are over aggressive and steal bandwidth. Serious unfairness

Recent Evaluations Experimental Evaluation of Cubic-TCP Conducted partially in response to “should CUBIC be adopted in Linux?” CUBIC suffers from a slow convergence of congestion windows. Controversial! A rebuttal was issued by Injong Rhee (CUBIC co-author)

Recent Evaluations Experimental Evaluation of Cubic-TCP Example of slow convergence:

Recent Evaluations Assessing Interactions among Legacy & High-Speed TCPs Evaluated efficiency, fairness, & friendliness

Recent Evaluations Assessing Interactions among Legacy & High-Speed TCPs Delay-based control are not effective in improving RTT-fairness due to the slow-start behavior of short flows that induces RTT measurement problems TCP-AReno was modified to avoid having the short flow problem impact its delay based mechanism. TCP-AReno outperformed all others. Compound TCP came close, but experienced the delay-based control problem.

Summary Protocol Approach Advantages Disadvantages Compound TCP Combines a delay based component with TCP Reno’s loss based component. Outperforms the link utilization of HSTCP and TCP Reno while still maintaining fairness to TCP Reno flows. The delay based component works best when in a steady state, but its effectiveness is reduced by the slow start behavior of lots of short flows. CUBIC Builds on BIC by simplifying the window response function and improving its TCP friendliness and RTT fairness through RTT-independent congestion window updates. Expands the TCP region to utilize standard TCP when it performs well. Maintains BIC’s scalability and stability regardless of RTT. Suffers from long convergence times between competing CUBIC flows. TCP-Adaptive Reno Incorporates a congestion estimation component that allows it to differentiate between packet losses from congestion or not, while also using RTT measurement to dynamically adjust its window response function. Offers the best link utilization, RTT fairness, and TCP friendliness of all the high speed variants. The most recent modifications have not yet been evaluated by an outside party.

Q&A