1. 28/09/2016 Congestion Control Ian McDonald imcdnzl@gmail.com (with many other WAND members)

2. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 2 Outline Congestion and Congestion Control Queuing Van Jacobson Linux congestion control TCP Throughput and Tuning Bandwidth Delay TCP Windows TCP Throughput Perry's Calculator

3. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 3 Outline continued Other TCP stuff ECN WAND Research TCP measurement DCCP

4. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 4 TCP congestion control - queuing

5. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 5 TCP congestion control Van Jacobson resolved by implementing a congestion window Once system is overloaded hard to recover so do not use full window size immediately If congestion occurs fast action is needed Window to transmit is minimum of congestion window and advertised window

6. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 6 TCP congestion control – congestion window

7. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 7 TCP congestion control Changes in Linux kernel BIC default in 2.6.7 for vanilla kernel. Fixed in 2.6.11! Pluggable congestion control from 2.6.13 Linux is now used more widely for congestion control research

8. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 8 TCP congestion control TCP variants in Linux New Reno (van Jacobson's) BIC (binomial search to get speed back) CUBIC (improved BIC as of 2.6.16) Vegas & Westwood (wireless) Highspeed variants (H-TCP, Highspeed TCP, Scalable TCP) Hybla (for long RTT)

9. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 9 TCP congestion control How do I change which variant? echo name > /proc/sys/net/ipv4/tcp_congestion_control can also do on a per socket basis via socket options

10. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 10 Bandwidth Delay Product It is the amount of data in transit = rate * latency e.g. 2 Mbit and 300 msec latency = 2,000,000 / 8 * 0.3 = 75,000 bytes (e.g. ADSL to USA) 100 Mbit and 100 msec latency = 1.25 Mbytes (e.g. cross NZ Ethernet)

11. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 11 TCP Windows and tuning TCP window must be bigger than the sum of the bandwidth delay products or else you will not go full speed Different operating systems have different defaults but they are not adequate for high speed/high latency networks To tune Linux see http://www-didc.lbl.gov/TCP-tuning/linux.html

12. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 12 TCP Windows and tuning If using TCP Segmentation Offload (TSO) to increase network performance (e.g. Broadcom Gbit cards, Intel e1000) then upgrade your kernel as early implementations buggy or turn off TSO (ethtool -K eth0 tso off). Another problem was that TSO prevented the congestion window from increasing properly. This was fixed in 2.6.15.

13. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 13 TCP throughput TCP throughput (in packets) is: where b = number of packets acknowledged by one acknowledgement and p is probability of packet loss (Source: Padhye, J., Firoiu, V., Towsley, D. and J. Kurose) As throughput is in packets try to use as large packets as possible (e.g. jumbograms)

14. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 14 TCP throughput Instead of formula you can use Perry's throughput calculator - http://wand.net.nz/~perry/max_download.php http://wand.net.nz/~perry/max_download.php Checks whether constrained by: upstream bandwidth (delayed acks, MSS, ack size) loss & RTT Window size (bandwidth delay product)

15. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 15 More TCP stuff ECN (Explicit congestion notification) notifies of congestion before packets are dropped. Causes congestion window to halve and slow start threshold to decrease previous backward compatibility issues with old firewalls due to using formerly reserved bits to turn it on “echo 1 > /proc/sys/net/ipv4/tcp_ecn” for Linux. Please do for routers and consider for servers

16. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 16 Comparative TCP startup TCP connection establishment and slow start Large differences between implementations Some difference due to administrative defaults, some due to TCP features and algorithms in place Measured on the WAND Emulation Network

17. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 17 TCP bugs Bugs found while measuring TCP stacks Windows XP sends outside of the receive window FreeBSD 5.3 produces bad selective acknowledgment ranges in response to this. This bug has been fixed. Measured on the WAND Emulation Network

18. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 18 TCP implementations vary The previous two slides show some sample measurements of how TCP implementations vary We are also able to simulate these implementations in one research project called the “Network Simulation Cradle” An example result on this follows, this shows how TCP fairness differs between TCP implementations in the same scenario Also shown is a traditional simulated TCP model

19. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 19 Example simulation results: TCP fairness analysis

20. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 20 DCCP New protocol at draft RFC status Connection based protocol (like TCP) but unreliable (like UDP) Has pluggable congestion control Designed for services such as VoIP, streaming media where both UDP and TCP have issues In Linux kernel from 2.6.14 onwards and rapidly improving Being worked on within WAND

21. © THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 21 WAND Network Research Group Department of Computer Science The University of Waikato Private Bag 3105 Hamilton, New Zealand www.crc.net.nz www.wand.net.nz www.waikato.ac.nz