Alok Shriram and Jasleen Kaur Presented by Moonyoung Chung Empirical Evaluation of Techniques for Measuring Available Bandwidth

Outline Introduction – Available Bandwidth – ABETs Related Work Motivation and Goal Experimental Framework Experimental Results – Accuracy – Cost Conclusion Infocom '072

Available Bandwidth (AB) End-to-End AB: – minimum unused capacity of path. – varies with time Infocom '073 Narrow Link Tight Link or Available Bandwidth (AB) 100 Mbps 10 Mbps 2500 Mbps 1200 Mbps 1000 Mbps 950 Mbps Tight link: minimum avail-bw link  ui : utilization of link i in time interval t ( 0 ≤ ui ≤ 1 )  Available bandwidth in link i:  Available bandwidth in path (Avail-bw): Applications –Network monitoring –Congestion control –Design of transport protocol –Streaming applications

Methodology for AB estimation Infocom '074 ToolProbesInference Metric PathloadEqui-spacedOne-way delay PathchirpExp-SpacedDispersion SprucePacket-PairDispersion IGI Packet TrainDispersion IperfTcp-StreamReceiving Rate Cprobe Packet TrainReceiving Rate 1. Design of Probes 2. Inference Logic End-to-End Path Feedback for successive Iterations Send probe packet(s) into the network and measure a response closed-loop tools

Packet Pair [Jacobson ’88, Keshav ’91] – Spruce Packet Train – Pathload, PathChirp, IGI, Cprobe Algorithmic Techniques dispersion T1 T0 Narrow Link Tn+1 Tn Gap t t Infocom '075 one way delay dispersion receiving rate cross traffic

Implementation techniques High time-stamping accuracy – variations in end-to-end delays, in the sub-millisecond rage 1.OS support for detecting and discarding probe streams that appear to not have been time-stamped accurately 2.Collect observations from several probe streams before converging on a robust estimate of AB Infocom '076

Questions Infocom '077 Which algorithmic technique performs the best? To what extend does current implementation technology limit tool performance? – How well would tools perform if technology advances in the future?

Related Works a small subset of ABETs – Robeiro et. al. [6] compares PathChirp to Pathload and TOPP – Hu et. al. [4] compares IGI/PTR to Pathload and Iperf – Strauss et. al. [7] compares Spruce to Pathload and IGI only simple network and traffic scenarios biased by current implementation technology ignores two key quantities: MT and SI Infocom '078

Motivation and Goal Motivation – Existing evaluations are either non-comprehensive and biased, or are affected by implementation issues Goal – Conducting an extensive experimental study of existing techniques for measuring AB Approach – Evaluation independent of current implementation technology simulation environments – Evaluation against diverse probing and network conditions Gigabyte network path/ diverse conditions Infocom '079

Evaluating Conditions 1. dynamic traffic load 2. measurement timescales (MT) – the time-scale at which AB is observed. 3. sampling intensities (SI) – the duration for which the AB is sampled per unit time. 4. number of bottleneck links 5. location of bottleneck Both MT and SI impact the accuracy and variability of the AB sampled by an ABET. [Shriram et al. 2006] Infocom '0710

Definition – Timescale at which we observe the AB process – the duration of a single probe stream = length of the probe stream Measurement Timescale (MT) Infocom '0711 Time AB MT Packet PairPacket Train MT

MT effect on AB Infocom '0712 Smaller MT expect more variability, expect lower accuracy

Sampling Intensity (SI) Definition – the duration for which the AB is sampled per unit time. – the number of probes – the product of the MT and the number of probe streams sent per unit time Infocom '0713 Higher SI expect better accuracy

ABET Implementation Tools: – Pathload, PathChirp, Spruce, IGI, Fast-IGI, and Cprobe Implement in the NS-2 network simulation environment Incorporating MT Incorporating Si – Open-loop tools : Cprobe, Spruce, PathChirp SI = MT/(MT+G) G: the gap between successive probe-streams – Closed-loop tools: Pathload, IGI, Fast-IGI The construction of a probe-stream is determined by the delays experienced by the previous probe-stream. RTT instead of SI Infocom '0714

Performance Metrics Accuracy-related – AB estimation error : the estimated AB – the actual AB (* actual AB = the number of bits that traverse the link during the tool run/the tool run-time) Cost-related – run time : the time taken by a tool to return an estimate – probing overhead : the total amount of network probe traffic sent by the tool in order to arrive at a single estimate of AB – intrusiveness : the average bit-rate of a tool (overhead/runtime) Impact on responsive cross-traffic – probe traffic on the response time of ongoing TCP connections Infocom '0715

Single Bottleneck Topology with a Single Bottleneck Link – Tool Traffic: traffic by ABETs – Cross Traffic: traffic with a constant bit-rate (CBR) Infocom '0716 link capacity = 1Gbps link delay = 1ms sufficient buffer

Validation of ABET Implementation Infocom '0717 Pathload, Spruce  quite accurate pathChirp  slightly higher Cprobe based on Receiving Rate  poor IGI -> R-IGI  good Cross wit a constant bit-rate (CBR) Cprobe based on Receiving Rate  poor

Dynamic Traffic Load Trace used for evaluation – Collect five 1-hour packet traces from four different Internet links Infocom '0718 for 1 Gbps links

Single Bottleneck Tool errors with default parameters Infocom ' percentiles average 5-percentiles Accuracy The average estimation errors are higher with dynamic cross- traffic than with CBR. Pathload, PathChirp, Fast-IGI have similar average error. R-IGI has lower error. Spruce has higher error. Variability The estimation errors vary widely around the average. least for Pathload quite high for Spruce and PathChirp MT=1ms MT=10ms MT=0.5ms MT=10ms

Impact of MT IPLS-CLEV: Impact of MT (SI=0.1, RTT=60ms) Infocom '0720 Increasing the MT improves the accuracy of all ABETs The gain are negligible beyond an MT of 50ms. MT impact on PathChirp is lower. Spruce now is the most accurate (it was the least with default settings)

Impact of SI IPLS-CLEV: Impact of SI (MT=10ms) Infocom '0721 SI and RTT has a negligible impact on the accuracy open-looped tools

Bottleneck Location Infocom '0722 Different tight and narrow links tight link narrow link cross traffic: IPLS-CLEV(410Mbps), IPLS-KSCY(530Mbps)

Bottleneck Location : Result MT=50ms, SI=0.1 Infocom '0723 The error of PathChirp and Spruce increases by a factor of 2-3. Other ABETs are not impacted much. Single Bottleneck Spruce PathChirp Different tight and narrow links IPLS-CLEV

Multiple bottlenecks Infocom '0724 Single narrow link: two tight link tight link narrow link IPLS-KSCYIPLS-CLEV

Multiple bottlenecks : Result MT=50ms, SI=0.1 Infocom '0725 Single narrow link: two tight link PathChirp and Spruce further degrades and the most inaccurate. The accuracy of the others are not significantly impacted. Single Bottleneck Spruce PathChirp IPLS-CLEV IPLS-CLEV, IPLS- KSCY

Overhead Infocom '0726 Spruce PathChirp Fast-IGI R-IGI Pathload PathChirp, R-IGI, Fast-IGI have the least overhead. Overhead increase with MT. SI and RTT has no impact on the overhead.

Run-time Infocom '0727 Spruce is the fastest tool. Pathload is the slowest tool. Spruce Pathload Increase the MT  a proportional increase in the runtime.

Intrusiveness Non-intrusiveness: cross traffic should not be affected Infocom '0728 All closed-loop tools are quite intrusive. Spruce has the highest value of intrusiveness. PathChirp is the most non-intrusive tool

Responsive Cross-Traffic Responsive cross-traffic – TCP uses congestion-control mechanism to reduce the data sending rate on detecting network congestion. queuing delays losses on the subsequent packet transmissions How adversely do these tools impact the performance of applications that rely on such responsive transport protocols? – Tmix: traffic-generation tool that incorporate the responsive behavior of TCP Infocom '0729 link capacity = 1Gbps average traffic = 300Mbps buffer size = 100 MSS-sized packets

Impact on Responsive Cross-Traffic CDF of response times with default parameters Infocom '0730 PathChirp has no noticeable impact on the response time. Pathload and Fast-IGI can significantly impact on the response times. CDF of connections no tool & PathChirp Pathload Fast-IGI

Conclusion Conduct a comprehensive empirical evaluation of existing algorithmic techniques used for measuring end-to-end AB. Key Observations – Accuracy The accuracy can be improved by using an MT of 50ms. SI and path RTT have negligible impact on the accuracy. While Spruce is the most accurate for paths with a single bottleneck link, its accuracy worsens for paths for multiple bottleneck links. – Cost PathChirp has the lowest overhead, and it has no impact on the response times of TCP. Spruce is the fastest, but has the highest value of intrusiveness. The cost of Pathload, R-IGI, and Fast-IGI seems to be highest. – Responsive Cross-traffic If an application needs to run an ABET repeatedly on a given internet path, it should use PathChirp. Infocom '0731