1. Helping TCP Work at Gbps Cheng Jin the FAST project at Caltech http://netlab.caltech.edu/FAST

2. Talk Outline TCP Reno does not perform well at Gbps TCP protocol has stability problems How FAST improves stability of TCP Project status

3. High Energy Physics Clear and present need for high bandwidth –Global cooperation 2000 physicists from 150 institutions in the world 300 - 400 physicists in US from > 30 universities and labs –Large file transfers ~ 1 TB At 622 Mbps ~ 4 hrs At 2.5 Gbps ~ 1 hr At 10 Gbps ~ 15 minutes

4. Highspeed TCP Performance ns-2: 100 sources, 100 ms round trip propagation delay 155 Mbps 622 Mbps 2.5 Gbps 5 Gbps 10 Gbps J. Wang (Caltech)

5. TCP Window Evolution ns-2: capacity = 10 Gbps FASTTCP/RED J. Wang (Caltech)

6. Current TCP Protocol Stability problems: –Slow-timescale oscillation as delay or capacity increases –Independent of packet-level AIMD dynamics –Independent of network noise

7. 0 200040006000 8000 10000 0 100 200 300 400 500 600 700 800 Instantaneous Queue Stable: Small (20 ms) Delay 0 200040006000800010000 0 10 20 30 40 50 60 70 individual window average window Window 50 identical FTP sources, single link 9 pkts/ms, RED marking

8. 0 2000400060008000 10000 0 10 20 30 40 50 60 70 individual window average window Unstable: Large (200ms) Delay 0200040006000800010000 0 100 200 300 400 500 600 700 800 Instantaneous Queue 50 identical FTP sources, single link 9 pkts/ms, RED marking Congestion Window

9. Other Effects on Queue Length same RTT 20ms same RTT 200ms 30% noise mean RTT 16ms mean RTT 208ms

10. Stability Regions 89101112131415 50 55 60 65 70 75 80 85 90 95 100 capacity (pkts/ms) delay (ms) N = 40 N = 30 N = 20 N = 50 N = 60 Unstable for Large delay Large capacity Small load

11. Loss vs. Delay TCP Reno uses loss as congestion measure Loss becomes noisy as capacity increases TCPs increase and decrease of cwnd not adaptive to system response FAST can use either queueing delay or loss Queueing delay has the right scaling with respect to capacity FAST adapts to capacity or end-to-end delay

12. FAST: Fast AQM Scalable TCP If loss is used –Both sender TCP and router AQM need to be changed If queueing delay is used –Only sender TCP needs to be changed –Injecting x ms of queueing delay into the network and change the send rate based on the observed queueing delay and its rate of change

13. Project Status Designed improved TCP/AQM protocols with the right scaling Compare FAST to existing approaches for highspeed TCP Linux kernel implementation of FAST Router implementation of AQM Experiments on real networks

14. Linux Kernel Implementation RedHat 7.3 with 2.4.18 kernel Modifications to the TCP layer Monitoring tool as loadable kernel module Incorporate features such as tunable socket buffer size and MTU

15. Floyds Highspeed TCP Slow increase of congestion window requires extremely small loss probability Tuning TCPs AIMD window adjustment –More rapid increase of cwnd –Less aggressive reduction of cwnd More simulations/experiments are needed

16. Equation-Based Approach Remove packet-level AIMD effect Estimate congestion signal (usually packet loss) and compute transmission rate Needs the right scaling with respect to delay and capacity

17. Effect of Protocol Instability Large jitters, bad for real-time traffic Creating bursty queues, causing packet losses Lower network utilization at high speed