TCP Westwood: Experiments over Large Pipes Cesar Marcondes Anders Persson Prof. M.Y. Sanadidi Prof. Mario Gerla NRL – Network Research Lab UCLA

PATHNETS San Jose CA Background TCP NewReno is challenged on large pipes: –Slow convergence to full utilization –Not intended to handle non-congestion packet loss Large Pipes performance criteria: –Utilization –Stability –Fast Ramp Up to “Cruising Speed” from Slow start –Fairness under differing RTTs –Friendliness to NewReno Alternatives include: HS TCP, FAST, TCPW Goal of this study: Measurements of TCPW, FAST and HS TCP over large pipes

PATHNETS San Jose CA TCPW Goal: high utilization, fairness, and friendliness over large leaky dynamic pipes Sender side only estimation of Eligible Rate Estimate (ERE) Estimation takes into account congestion level, capacity of the bottleneck, achieved rate Exponential filtering to time average estimates and avoid network conditions instability ERE is used to: –(1) set congestion window after packet loss –(2) repeatedly reset ssthresh to reach “cruising speed” fast from slow start

PATHNETS San Jose CA RE Sampling : Packet train, fair estimate under congestion, underestimates under random loss TCPW ABSE BE Sampling : Packet pair, effective under random loss, overestimates under congestion Under Congestion Under No Congestion TkTk TkTk To obtain ERE: adapt the sample interval T k according to congestion level Congestion level is similar to that in Vegas: Expected Rate-Achieved Rate

PATHNETS San Jose CA Experiments Environment (Powerful Machines) CPU: Xeon 3.06GHz Cache: 512 L2/ 1MB L3 Intel 1000PRO PCI-X BUS 133MHz NewRenoSender AdvancedTCPSender Gigabit link UCLAGigabitSwitch Gigabit link NewRenoReceiver(Alabama) Internet2 NewRenoReceiver(Caltech)

PATHNETS San Jose CA UCLA Internet2 Link Traffic Our Experiments Traffic Other UCLA Users in Background

PATHNETS San Jose CA Test Methodology Automated Scripts –Scheduled by Unix crontab –Automatically reinitiate the O.S. with each protocol and conduct new measurements Linux: FAST, HS-TCP and NewReno FreeBSD: TCPW Sender/Receiver buffer is set to 2 MB to enable high utilization of Gbps links Iperf traffic generation, TCPdump, Nistnet emulator

PATHNETS San Jose CA Benchmark Tests Case Study I: –UCLA-Alabama (155 Mbps, 64 msec) Case Study II: –UCLA-CalTech (1 Gbps, 4msec) Group of 10 successive night time runs for each test Throughput, fairness, friendliness Artificial non-congestion loss (PER 0.1 to 0.5%)

PATHNETS San Jose CA Case Study I: UCLA–Alabama NewRenoSender AdvancedTCPSender Internet2(Gigabit) ATM Atlanta – Alabama NewRenoReceiver(Alabama) 155Mbps ATM Link Bottleneck Link as measured by PathRate And confirmed later by the network admin

PATHNETS San Jose CA Throughput Convergence to cruising speed varies among protocols High deviation among multiple runs in HSTCP and NewReno HSTCP deviations decrease over time (as the AIMD behavior changes) UCLA-Alabama

PATHNETS San Jose CA UCLA-Alabama

PATHNETS San Jose CA Transfer Completion Times On average: TCPW and FAST: 0 to 100 MB in 5.8 Sec! HSTCP: 0 to 100 MB in 7.5 Sec! NewReno: 0 to 100 MB in 11 Sec! UCLA-Alabama

PATHNETS San Jose CA Friendliness UCLA-Alabama

PATHNETS San Jose CA TCP FAST – Preliminary Analysis RTT Variation over Time as Observed by TCPdump Outstanding Window as Observed by TCPdump UCLA-Alabama

PATHNETS San Jose CA Random Loss Emulation Induced non-congestion packet loss in emulator (PER 0.1% up to 0.5%) TCPW throughput much higher than all other schemes AdvancedTCPSender NewRenoReceiver(Alabama) UCLA – Alabama UCLA-Alabama NistnetNetworkEmulator

PATHNETS San Jose CA Random Loss Emulation (Results) UCLA-Alabama

PATHNETS San Jose CA Case Study II: UCLA–CalTech NewRenoSender(UCLA) AdvancedTCPSender(UCLA) Internet2(Gigabit) TCPReceiver(CalTech) 1 Gbps 4 ms

PATHNETS San Jose CA Throughput TCP NewReno starts-up really high since it relies in the cached threshold and the feedback is really fast Cached Slow Start Threshold versus Adaptive Start-Up (Pros and Cons) Westwood is delayed by its own Stability Filter –Stability-based Filter dampens estimates in proportion to the variance of observation UCLA-CalTech

PATHNETS San Jose CA UCLA-CalTech

PATHNETS San Jose CA TCP Westwood Stability Filter versus Fixed Gain Filter Sample Estimations vary a lot due to NIC coalescing and OS issues at Gigabit/s. As variability increases, stability filter relies on a more *stable* moving average filter Solution: Use a fixed gain instead of an adaptive when we know we are dealing with Gbps range speeds TCPW ramp up as HS-TCP and FAST UCLA-CalTech

PATHNETS San Jose CA TCPW Start-Up using Fixed Exponential Average UCLA-CalTech

PATHNETS San Jose CA Friendliness UCLA-CalTech

PATHNETS San Jose CA Conclusions TCPW and FAST performed equally well in terms of average throughput All Advanced TCP protocols have an excellent intra- protocol fairness Friendliness –FAST appears to suffer a synchronization problem Under non-congestion error scenario, TCPW shows greater robustness At Gigabit speed, measurements could be messed up by Interrupt Coalescing and other HW/Kernel bottlenecks, affecting moving average filters

PATHNETS San Jose CA Future Work New algorithm that is Interrupt Coalescence-Aware for Gbps environment New Agile and Stable Filter Improve the Automated TCP Test Tool (Benchmark and New Tests)

PATHNETS San Jose CA Thanks Netlab CalTech Xiaoyan Hong – CS / Alabama Univ.