1 BWdetail: A bandwidth tester with detailed reporting Masters of Engineering Project Presentation Mark McGinley April 19, 2007 Advisor: Malathi Veeraraghavan Acknowledgement: This work was carried out under the sponsorship of NSF ITR , NSF ANI , and DOE DE-FG02-04ER25640 grants.

2 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

3 Problem Statement Growing eScience traffic Terabytes / month ESnet Monthly Accepted Traffic, January, 2000 – June, 2006 Ref. 1

4 Problem Statement Scientific research efforts such as Atlas characterized by: Large datasets Geographically distributed collaborators 1800 physicists, 150 universities and laboratories Supporting high speed transfers of datasets often requires system tuning, diagnostics [2]

5 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

6 Related Work Iperf from NLANR Successor to ttcp, similar to nttcp, nuttcp Web100 Host system software (autotuning) Lack of instrumentation/tools for diagnosing performance issues Iperf100 Web100-based Iperf

7 Related Work Nettest Incorporates Iperf Net100 Leverages Web100 to eliminate “wizard gap” with autotuning tcpdump

8 Related Work Iperf

9 Related Work tcpdump

10 Related Work BWdetail

11 Related Work IperfIperf100Bwdetail Bandwidth estimation Buffer tuning Parameter tuning Cwnd, etc No kernel modifications

12 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

13 Overview of BWdetail Bandwidth estimation ability of Iperf Instrumentation capabilities similar to Web100, without kernel modification

14 Overview of BWdetail Software architecture C program TCP_INFO struct associated with each socket Kernel parameters read/written using getsockopt(), setsockopt(), sysctl(), system()

15 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

16 Features and Uses Data transfer for goodput estimation Memory-to-memory is most common usage Disk/memory-to-disk/memory Memory transfer can either be a specific size or duration Disk transfer using a file of specific size, to eliminate caching effects

17 Features and Uses Effect of caching 1 GB disk-to-disk successive transfer

18 Features and Uses Multi-threaded client Client uses separate threads to send data and monitor flow Sendfile(), send loopMonitoring loop Transfers data, records goodput from application’s perspective Records fields from TCP_INFO, particularly cwnd and RTT

19 Features and Uses Tunable parameters from command line Socket buffer size TCP congestion control algorithm CTCP-enabled Web100 capable SACK, Nagle, no metrics save

20 Features and Uses Adjustable reporting granularity Client and server report goodput over some small, configurable interval (100ms) Client reports cwnd, RTT, etc, every smaller, configurable interval (5ms)

21 Features and Uses Applications Research regarding transport protocols CTCP/Reno/BIC Cwnd and RTT measurements Improving performance on a path Records relevant kernel parameters Ongoing recording of cwnd, RTT, goodput Change parameters, repeat

22 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

23 Demonstration Results

24 Outline Problem Statement Related Work Overview of BWdetail Features and Uses Demonstration Conclusion

25 Conclusion Available bandwidth estimation Instrumentation such as cwnd without requiring kernel modification or privileged access

26 Thank you Questions?

27 References [1] W.E. Johnston, ESnet4: Networking for the Future of DOE Science, 1.ppt, Dec. 5, ppt [2] The Atlas Experiment,

28 Limitations Does not replace tcpdump tcpdump handles content Needed to detect retransmissions, Ethernet PAUSE frames, verification Incorporation of libpcap capabilities would degrade the non-interference status of BWdetail BWdetail is interested in statistics, not content

29 Limitations BWdetail does not replace an expert Still requires knowledge to tune, does not provide autotuning Use of TCP_INFO struct presents portability issues

30 Impact of monitoring Minimal, even at ~1 Gbps

31 RTT