1. Broadband Internet Performance: A View from the Gateway Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford, Antonio Pescapè Georgia Tech University of Napoli CNRS/UPMC Sorbonne Universités SamKnows

2. What Affects Broadband Performance? Important for regulators, consumers, ISPs Notion of performance is fuzzy – What metrics should we measure? – How to measure them? Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 2

3. Accurate Measurements are Difficult End host measurements are not continuous, and affected by confounding factors Home Network: AT&T DSL 6 Mbps Down, 512 Kbps Up Last Mile ISP Network speedtest.net: 4.4 Mbps, 140 Kbps Netalyzr: 4.8 Mbps, 430 Kbps Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 3

4. The Case For the Gateway Gateway enables periodic measurements, and can account for confounding factors Home Network: AT&T DSL 6 Mbps Down, 512 Kbps Up Last Mile ISP Network speedtest.net: 4.4 Mbps, 140 Kbps Netalyzr: 4.8 Mbps, 430 Kbps Gateway: 5.6 Mbps, 460 Kbps Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 4

5. Contributions Study of access networks from the gateway Insight into how to benchmark performance – Effect of measurement technique, traffic shaping – Effect of access link properties, home equipment Insight into broadband performance – Consistency of performance – Loss, jitter, time-of-day effects Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 5

6. The Deployments Breadth: The FCC/SamKnows study – 4,000 gateways, 16 ISPs, multiple service plans Depth: The BISmark study – 16 gateways in Atlanta, on-demand measurements Duration: Dec 2010 – Jan 2011 SamKnows/BISmark Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 6 Last Mile Internet Nearby Server

7. Results: Overview Throughput measurement technique depends on usage scenario Traffic shaping is highly variable across users Access link characteristics affect performance Modem buffers induce high latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 7

8. Results: Overview Throughput measurement technique depends on usage scenario Traffic shaping is highly variable across users Access link characteristics affect performance Modem buffers induce high latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 8

9. Interpreting Throughput Results Different techniques measure different aspects of throughput Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 9

10. Results: Overview Throughput measurement technique depends on usage scenario Traffic shaping is highly variable across users Access link characteristics affect performance Modem buffers induce high latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 10

11. Traffic Shaping: PowerBoost Cable companies advertise “PowerBoost” – Short bursts of high bandwidth Non-existent in DSL Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 11

12. Traffic Shaping Varies Across Users Short-term throughput significantly different from sustainable throughput Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 12

13. Results: Overview Throughput measurement technique depends on usage scenario Traffic shaping is highly variable across users Access link characteristics affect performance Modem buffers induce high latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 13

14. Latency Measurements Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 14 ParameterWhat it captures End-to-end Latency to nearby server Last-mile Latency to edge of ISP network Under Load Buffer delays due to cross traffic

15. DSL last-mile latencies can be very high Impact of Last-mile on Latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 15 Cable ISPs DSL ISPs

16. DSL Interleaving Affects Latency 9 9 8 8 7 7 6 6 5 5 4 4 3 3 2 2 1 1 Fastpath is susceptible to bursty loss 9 9 6 6 3 3 8 8 5 5 2 2 7 7 4 4 1 1 Fastpath sends data in order, can recover from single losses Interleaving sends data out-of-order, can recover from bursty loss Fastpath vs. Interleaved last-mile data path Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 16 Extra Delay

17. Example: Latency-Throughput Tradeoff Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 17 User 2: high latency, low loss User 1: low latency, high loss Throughput (Mbits/s) Latency (ms) 10 2.5 2.2 30 Both users have same service plan Interleaving decreases loss, increases latency, improves throughput Interleaving creates a trade-off between latency and throughput

18. Results: Overview Throughput measurement technique depends on usage scenario Traffic shaping is highly variable across users Access link characteristics affect performance Modem buffers induce high latency Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 18

19. Modem Buffers are Too Large Service plans can interact badly with modem buffers Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 19 10 seconds!

20. Conclusion The gateway provides unique insight into home network Throughput measurements are affected by measurement technique, shaping Latency is affected by last-mile, buffering ISPs generally deliver consistent throughput, with some time-of-day effects How to mitigate buffering effects Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 20

21. Future Work BISmark node deployment across the world – Currently nodes in US, South Africa Performance monitoring from the edge – Effect of peering on performance – IPv6 performance – Effect of CDN location, traffic engineering on application performance Measure the home network Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 21

22. http://projectbismark.net Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 22

23. Previous Studies Measurement from outside – Dischinger et al [1] initiated measurements from wide area server – Indirect measurements, not continuous Measurements from inside – Endhost based measurements – Grenouille [2], Netalyzr [3] – Can’t account for confounding factors [1] Characterizing Residential Broadband Networks IMC 2007 [2] http://www.grenouille.comhttp://www.grenouille.com [3] http://netalyzr.icsi.berkeley.edu Netalyzr: Illuminating the Network Edge IMC 2010http://netalyzr.icsi.berkeley.edu Broadband Internet Performance: A View from the Gateway ACM SIGCOMM, Toronto., August 2011 23