Bandwidth Estimation in Broadband Access Networks Venkat Padmanabhan Systems & Networking Group Microsoft Research Joint work with: Karthik Lakshminarayanan (Berkeley) & Jitu Padhye (MSR) June 2004

Outline Bandwidth estimation Previous work Challenges in broadband access networks ProbeGap Experimental evaluation –802.11a testbed –cable modem testbed Conclusion

Bandwidth Estimation Active area of networking research for 15+ years “Bandwidth” refers to data rate –“CS bandwidth” (bps), not “EE bandwidth” (Hz) Several notions of bandwidth –bottleneck bandwidth, or capacity raw bandwidth of narrow link –available bandwidth spare capacity of tight link –other notions fair share bandwidth bulk transfer capacity

Bandwidth Estimation Of interest in several contexts –congestion control (e.g., TCP) –admission control (e.g., A/V streaming) –background transfer (e.g., TCP Nice) –server/peer selection (e.g., overlay multicast) Desirable attributes of an estimation scheme –depends only on end hosts –accurate –fast –lightweight & non-intrusive

Previous Work on Capacity Estimation Packet-pair method –Jacobson ’88, Keshav ’91 –cross-traffic ⇒ underestimation/overestimation Refinement: filtering to eliminate noise –nettimer [Lai ’00], pathrate [Dovrolis ’01] –key observation: capacity mode may not be dominant Single-packet techniques –pathchar [Jacobson ’97], clink [Downey ’99] –dependence on ICMP msgs. limits applicability and accuracy dodo narrow link

Previous Work on Available Bandwidth Estimation Packet Rate Method (PRM) –e.g., pathload [Jain ’02], PTR [Hu ’03] –probe at gradually increasing rates –increasing trend in OWD indicates that pipe is full –accurate but somewhat heavyweight Packet Gap Method (PGM) –e.g., IGI [Hu ’03], Spruce [Strauss ’03] –send several carefully spaced probe pairs –estimate cross-traffic based on the increase in spacing –assumes that the tight link is also the narrow link –relatively lightweight but susceptible to delays elsewhere RTT-based estimation [Gunawardena ’03] –derive analytical relationship between load and RTT –perturb network by introducing known amount of additional load –quite heavyweight, susceptible to delays elsewhere & departure from the assumed traffic model

Packet Rate Method (PRM) tight link probescross-traffic Probing rate > available bandwidth ⇒ increasing OWD Probing rate < available bandwidth ⇒ no trend in OWD

Packet Gap Method (PGM) didi dodo didi dodo probes cross-traffic tight & narrow link d i < d o ⇒ cross-traffic = C*(d o -d i )/d i d i = d o ⇒ no cross-traffic

Traditional Link Model Assumptions made in previous work: –link has well-defined capacity –point-to-point link with FIFO scheduling –fluid cross-traffic (infinitesimal packet size) But these assumptions break down in broadband network settings

Broadband Access Networks Various technologies –cable modem, DSL, wireless (WiFi, WiMax) Why is broadband different? –“managed” links (pricing flexibility) –typically shared medium (lower cost) –DSL is an exception conforms to the traditional link model Specific issues –link may not have well-defined capacity –contention and non-FIFO scheduling –bursty cross-traffic

Broadband Issues Link may not have well-defined capacity –rate regulation (e.g., token bucket) –dynamic multirate (e.g., ) –⇒ measured capacity may not be same as sustained capacity Non-FIFO scheduling due to frame-level contention –fully distributed contention-based MAC (e.g., ) –centrally coordinated MAC (e.g., cable uplink) –⇒ difficult for packet pairs to go through back-to-back –⇒ probe packets may not see full impact of cross-traffic –⇒ relative sizes of probe packets & cross-traffic packets matter Bursty cross-traffic –interference between links operating at different rates –e.g., in a, a single packet CT packet on 6 Mbps link would appear as a large burst on 54 Mbps link –⇒ makes it difficult to accurately sample the cross-traffic

Is AvlbBw Still Interesting? With a “fair” MAC it may be feasible to estimate the fair share bandwidth –e.g., Keshav’s original packet-pair work However, available bandwidth remains interesting –TCP ramp-up safe option is to quickly ramp-up to available bandwidth and then probe gradually for fair share –admission control for A/V streams letting new stream exercise its fair share might cause disruption of existing streams

ProbeGap New technique for estimating available bandwidth –designed to address some of these issues –non-FIFO scheduling, bursty cross-traffic Key idea: probe for idle “gaps” in the link –gather OWD samples –knee in CDF identifies idle fraction –multiply by capacity to obtain available bandwidth estimate Issues –very lightweight 200 probes of 20-bytes each –clock drift is a concern can estimate and neutralize –susceptible to delays at other links like PGM and RTT-based method x 0 1 CDF OWD

Experimental Evaluation We focus on the broadband network in isolation Testbeds –802.11a –cable modem controlled testbed commercial connections Tools evaluated –capacity: pathrate –available bandwidth: pathload, spruce, probegap Validation: –capacity: measured using intrusive packet train probes –available bandwidth: determined by observing impact on cross-traffic

802.11a Evaluation Experimental setup –6 nodes in ad hoc configuration one pair used for bandwidth estimation other two pairs used to generate cross-traffic –cross-traffic: link rate = 6 Mbps traffic rate = 0-4 Mbps, packet size = 300 or 1472 B –estimation link: single-rate case: link rate = 6 Mbps multi-rate case: link rate = 54 Mbps

Impact of Packet Size (802.11a) Significant per-packet overhead, especially at 54 Mbps

Capacity Estimation (802.11a) Pathrate uses 1472-byte probe packets Single-rate case: –capacity mode identified consistently in the Mbps range, even with cross-traffic –enough packet pairs go through back-to-back, despite non- FIFO “fair” MAC –situation might be different with a larger number of contending stations Multi-rate case: –capacity mode identified in the Mbps range in most cases –exception with heavy cross-traffic (4 Mbps, 300 B) capacity mode identified was Mbps Packet-pair sampling with suitable filtering mostly works

AvlbBw Estimation (802.11a single-rate) Overestimation due to tendency towards fair share (Pathload) and differential packet size (Spruce). Probegap only overestimates slightly under high load.

ProbeGap (802.11a single-rate) Overestimation at high loads. Possible fix: send probes in bunches and pick max OWD.

AvlbBw Estimation (802.11a multi-rate) The single-rate issues persist. But new anomalies with both Pathload and Spruce due to the burstiness of cross-traffic.

Pathload (802.11a multi-rate) Pathload fails to detect consistent increasing trend even though probing rate (9.79 Mbps) exceeds avlb bw (6.5 Mbps).

Impact of Token Bucket (Cable Modem) Experimental setup –raw bandwidth of downlink = 27 Mbps –token bucket rate = 6 Mbps, depth = 9600 bytes –cross-traffic rate = 0-6 Mbps Capacity estimation –pathrate consistently estimates 26 Mbps regardless of cross-traffic Available bandwidth estimation –pathload overestimates slightly token bucket can accommodate large train of 300-byte probes –spruce overestimates significantly a pair of probes is less likely to be regulated than a train unclear what right capacity to assume is

Pathload (cable modem) Slight overestimation because of token bucket

Spruce (cable modem) Significant overestimation because of token bucket. Unclear what the right capacity to assume is.

Conclusion Broadband access networks present new challenges to bandwidth estimation –performance experienced by probes may not be indicative of true performance –tendency to estimate fair share rather than available bandwidth ProbeGap looks promising More info: eerMetric/ eerMetric/ –MSR tech report (MSR-TR ) –IMC 2003 paper (macroscopic properties of broadband networks)