1. Estimating Available Bandwidth with pathload and abget Professor Nelson Fonseca nfonseca@ic.unicamp.br

2. Estimators - Professor Nelson Fonseca2 Outline Introduction Measuring available bandwidth SLoPS method pathload abget Interrupt coalescence Examples Conclusions References

3. Estimators - Professor Nelson Fonseca3 Introduction

4. Estimators - Professor Nelson Fonseca4 Bandwidth Bandwidth defines how much data (bits) can be transferred between two points in a time interval (seconds) ‏ Related terms: nominal capacity, capacity, narrow link, available bandwidth, tight link, bulk transfer capacity (BTC),...

5. Estimators - Professor Nelson Fonseca5 Available Bandwidth Important to traffic engineering Important to take scheduling decisions (grids and clouds) ‏ Important to take decisions about links upgrade

6. Estimators - Professor Nelson Fonseca6 Measuring available bandwidth

7. Estimators - Professor Nelson Fonseca7 Varies with time u i : utilization of link i in a time interval ( 0 <= u i <= 1 ) Available bandwidth in link i: Available bandwidth in path (Avail-bw): minimum Ai Available Bandwidth: unutilized capacity Calculating Available Bandwidth

8. Estimators - Professor Nelson Fonseca8 τ defines sampling/averaging timescale Average avail-bw in τ Does not tell how avail-bw varies Variation range gives more information t A(t ) ‏ τ T Available Bandwidth:time varying metric

9. Estimators - Professor Nelson Fonseca9 Can be measured at each link from interface utilization data using SNMP MRTG (Multi Router Traffic Grapher) graphs: 5- minute averages But users do not normally have access to SNMP data And MRTG graphs give only per-hop avail-bandwidth Measuring per-hop available bandwidth

10. Estimators - Professor Nelson Fonseca10 Blast path with UDP packets Intrusive Carter & Crovella: cprobe (Infocom 1996) ‏  Packet train dispersion does not measure available bandwidth (Dovrolis et.al. Infocom’01) ‏ Measure throughput of large TCP transfer TCP throughput depends on network buffer Ribeiro et.al. : Delphi (ITC’00) ‏ Correct estimation when queuing occurs only at single link Assumes that cross traffic can be modelled by MWM model Measuring path Available Bandwidth

11. Estimators - Professor Nelson Fonseca11 SloPS method

12. Estimators - Professor Nelson Fonseca12 Examine One-Way Delay (OWD) variations of a fixed rate stream Relate rate to avail-bw OWD : D i = T arrive -T = T arrive - T send + Clock_Offset(S,R) SLoPS uses relative OWDs, ΔD i = D i+1 – D i-1 (independent of clock offset) ‏ With a stationary & fluid model for the cross traffic, and FIFO queues: If R > min Ai, then Δ D i > 0 for I = 1…N Else Δ D i = 0 for for I = 1…N send S R R R Main Idea

13. Estimators - Professor Nelson Fonseca13 Periodic Stream: K packets, size L bytes, rate R = L/T If R>A, OWDs gradually increase due to self-loading of stream Illustration

14. Estimators - Professor Nelson Fonseca14 For some rate R Increasing trend in OWDs  R > Avail-bw No trend in OWDs  R < Avail-bw Trend in real data

15. Estimators - Professor Nelson Fonseca15 At sender: Send periodic stream n with rate R n At receiver: Measure OWDs D i for i=1…K At receiver: Notify sender of trend in OWDs At sender: If trend is : increasing (i.e. R n >A )  repeat with R n+1 < R n non-increasing (i.e. R n R n  Selection of R n+1 : Rate adjustment algorithm Terminate if R n+1 – R n < ω ω : resolution of final estimate Iterative Algorithm in SloPS

16. Estimators - Professor Nelson Fonseca16 Grey region: Rate R not clearly greater or smaller than Avail-bw during the duration of stream Rate R is within variation range of avail-bw If things were black and white...

17. Estimators - Professor Nelson Fonseca17 Increasing trend  R > variation range of Avail-bw No trend  R < variation range of Avail-bw Grey trend  R inside variation range Big Picture

18. Estimators - Professor Nelson Fonseca18 Grey region Increasing trend : R max = R(n)‏ R(n+1) = (G max + R max) /2 Non-increasing trend: R min = R(n)‏ R(n+1) = (G max +R min )/2 Grey region & R(n) > G max: G max = R(n)‏ R(n+1) = (G max + R max )/2 Grey region & R(n) < G min: G min = R(n)‏ R(n+1) = (G min + R min )/2 Terminate if: (R max – G max ) && (R min – G min ) <  R max > A R min < A G max G min Variation Range Rate Adjustment Algorithm Initial G min = 0 Initial G max = Average throughput of a small initial transfer of UDP data

19. Estimators - Professor Nelson Fonseca19 pathload (1) ‏ Implements SLoPS Estimates end-to-end available bandwidth Based on one-way delay trend of periodic streams Reports a range of avail-bw  Corresponds to variation, measured in stream duration According the developers, the interference is almost nonexistent

20. Estimators - Professor Nelson Fonseca20 pathload (2) ‏ Version 1.3.2 (2006) ‏ Features Open Source Two processes (one at sender, one at receiver) ‏ No need to root privileges No need to define several parameters pathload2 released recently (Apr 2009) ‏ Installation Download from http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/bw- est/pathload.tar.gz http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/bw- est/pathload.tar.gz Uncompress in a Linux operating system./configure && make

21. Estimators - Professor Nelson Fonseca21 pathload (3) ‏ Two connections TCP – control UDP – data stream Execution order At sender:  pathload_snd  Waits receiver connection via TCP port 55002 At receiver:  pathload_rcv -s  Connects to sender at TCP port 55002  Sender sends UDP data to receiver at UDP port 55001

22. Estimators - Professor Nelson Fonseca22 pathload (4) ‏ Problems Static ports User needs access to the two end points Ports must be opened in firewalls

23. Estimators - Professor Nelson Fonseca23 abget (1) ‏ Also implements SloPS Solves some pathload problems Runs only in one end-point Can make the estimations using TCP ports already opened in firewalls (e.g. 80) ‏

24. Estimators - Professor Nelson Fonseca24 abget (2) ‏ Version 1.0 (2007) ‏ Features Open Source One process Need root privileges Can estimate both upstream and downstream availability Installation Install libpcap Download from http://www.ics.forth.gr/~papadog/abget/abget-1.0.tar.gz Uncompress in a Linux operating system./configure && make && make install

25. Estimators - Professor Nelson Fonseca25 TCP connections Both control and data stream abget uses TCP control mechanisms Execution order (at user end-point) ‏ Downstream  abget -d -f  Retrieves the file and control the rate simulating the ACKs Upstream  abget -u abget (3) ‏

26. Estimators - Professor Nelson Fonseca26 Downstream abget emulates TCP by sending fake ACKs to the server (It captures the received packets and bypasses the default TCP implementation) ‏ The advertise window of the server is limited to 1 MSS Each ACK “requests” 1 MSS By defining the ACKs period, it is possible to define the downstream rate  To force the server to send at rate R, abget sends ACKs at T=MSS/R abget (4) ‏

27. Estimators - Professor Nelson Fonseca27 Process illustration (downstream) ‏ abget (5) ‏

28. Estimators - Professor Nelson Fonseca28 Upstream abget sends an HTTP request in many overlapping segments. It repeats for many iterations with different rates in order to estimate a variation range for the available bandwidth in that path. abget (6) ‏

29. Estimators - Professor Nelson Fonseca29 abget (7) ‏ Problems Many parameters Results not so precise as pathload root privileges

30. Estimators - Professor Nelson Fonseca30 Interrupt Coalescence

31. Estimators - Professor Nelson Fonseca31 Interrupt Coalescence 1 Network card event != 1 Interrupt Enabled to avoid overload to the operating system Should be disabled to avoid incorrect estimations With interrupt coalescence: more packets in a time interval

32. Estimators - Professor Nelson Fonseca32 pathload and abget By reading the source code, pathload avoids interrupt coalescence. abget doesn't But each manufacturer implements a different method

33. Estimators - Professor Nelson Fonseca33 Examples

34. Estimators - Professor Nelson Fonseca34 Examples 1 and 2 - Scenario

35. Estimators - Professor Nelson Fonseca35 Examples 1 – Precision at 10Mbps

36. Estimators - Professor Nelson Fonseca36 Examples 1 – Overhead at 10Mbps

37. Estimators - Professor Nelson Fonseca37 Examples 2 – Precision at 1Gbps

38. Estimators - Professor Nelson Fonseca38 Examples 2 – Overhead at 1Gbps

39. Estimators - Professor Nelson Fonseca39 Conclusions

40. Estimators - Professor Nelson Fonseca40 Conclusions Available Bandwidth estimators must estimate using a range instead a average Interrupt coalescence must be disabled pathload and abget Unchanged Clock offset pathload gives more precise results abget runs only in one end-point abget uses HTTP (no problem with firewalls) ‏ pathload doesn't need root privileges pathload is ideal if there is access to the two end-points. Abget is ideal if there isn't.

41. Estimators - Professor Nelson Fonseca41 References http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/pathload.html Official pathload website http://www.measurementlab.net/measurement-lab-tools#pathload2 pathload2 http://www.ics.forth.gr/~papadog/abget/ Official abget website http://www.caida.org/tools/taxonomy/performance.xml CAIDA (The Cooperative Association for Internet Data Analysis) website about network performance tools http://www.29west.com/docs/THPM/latency-interrupt-coalescing.html Interrupt coalescence and GNU/Linux