Helping TCP Work at Gbps Cheng Jin the FAST project at Caltech

Talk Outline TCP Reno does not perform well at Gbps TCP protocol has stability problems How FAST improves stability of TCP Project status

High Energy Physics Clear and present need for high bandwidth –Global cooperation 2000 physicists from 150 institutions in the world physicists in US from > 30 universities and labs –Large file transfers ~ 1 TB At 622 Mbps ~ 4 hrs At 2.5 Gbps ~ 1 hr At 10 Gbps ~ 15 minutes

Highspeed TCP Performance ns-2: 100 sources, 100 ms round trip propagation delay 155 Mbps 622 Mbps 2.5 Gbps 5 Gbps 10 Gbps J. Wang (Caltech)

TCP Window Evolution ns-2: capacity = 10 Gbps FASTTCP/RED J. Wang (Caltech)

Current TCP Protocol Stability problems: –Slow-timescale oscillation as delay or capacity increases –Independent of packet-level AIMD dynamics –Independent of network noise

Instantaneous Queue Stable: Small (20 ms) Delay individual window average window Window 50 identical FTP sources, single link 9 pkts/ms, RED marking

individual window average window Unstable: Large (200ms) Delay Instantaneous Queue 50 identical FTP sources, single link 9 pkts/ms, RED marking Congestion Window

Other Effects on Queue Length same RTT 20ms same RTT 200ms 30% noise mean RTT 16ms mean RTT 208ms

Stability Regions capacity (pkts/ms) delay (ms) N = 40 N = 30 N = 20 N = 50 N = 60 Unstable for Large delay Large capacity Small load

Loss vs. Delay TCP Reno uses loss as congestion measure Loss becomes noisy as capacity increases TCPs increase and decrease of cwnd not adaptive to system response FAST can use either queueing delay or loss Queueing delay has the right scaling with respect to capacity FAST adapts to capacity or end-to-end delay

FAST: Fast AQM Scalable TCP If loss is used –Both sender TCP and router AQM need to be changed If queueing delay is used –Only sender TCP needs to be changed –Injecting x ms of queueing delay into the network and change the send rate based on the observed queueing delay and its rate of change

Project Status Designed improved TCP/AQM protocols with the right scaling Compare FAST to existing approaches for highspeed TCP Linux kernel implementation of FAST Router implementation of AQM Experiments on real networks

Linux Kernel Implementation RedHat 7.3 with kernel Modifications to the TCP layer Monitoring tool as loadable kernel module Incorporate features such as tunable socket buffer size and MTU

Floyds Highspeed TCP Slow increase of congestion window requires extremely small loss probability Tuning TCPs AIMD window adjustment –More rapid increase of cwnd –Less aggressive reduction of cwnd More simulations/experiments are needed

Equation-Based Approach Remove packet-level AIMD effect Estimate congestion signal (usually packet loss) and compute transmission rate Needs the right scaling with respect to delay and capacity

Effect of Protocol Instability Large jitters, bad for real-time traffic Creating bursty queues, causing packet losses Lower network utilization at high speed