1. Broadband Internet Performance: A View from the Gateway Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford, Antonio Pescapè Georgia Tech University of Napoli CNRS/UPMC Sorbonne Universités SamKnows

2. What Affects Broadband Performance? Notion of performance is fuzzy – What metrics should we measure? – How to measure them? Important for regulators, consumers, ISPs, content providers – 500ms delay causes 20% decrease in Google search traffic Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 2

3. Accurate Measurements are Difficult End host measurements are not continuous, and affected by confounding factors Last Mile ISP Network Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 3

4. The Case For the Gateway Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 4 Actual Service plan: AT&T DSL 3 Mbps Down, 384 Kbps Up Measured average: 2.85 Mbps Down, 338 Kbps Up Gateway enables periodic measurements, and can account for confounding factors

5. The Deployments Breadth: The FCC/SamKnows study – 7,800 gateways, 18 ISPs, multiple service plans Depth: The BISmark study – 120+ gateways in 28 countries worldwide, periodic and on-demand measurements SamKnows/BISmark Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 5 Last Mile Internet Nearby Server

6. Results: Overview Throughput: – Depending on technique, results can vary up to 25% – Traffic shaping varies across users, affects results Latency: – Access link characteristics introduce loss/latency/throughput trade-offs – Modem buffers can induce latencies up to 10 seconds Application Performance – Web: – Latency becomes bottleneck beyond 16 Mbits/s Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 6

7. Results: Overview Throughput: – Depending on technique, results can vary up to 25% – Traffic shaping varies across users, affects results Latency: – Access link characteristics introduce loss/latency/throughput trade-offs – Modem buffers can induce latencies up to 10 seconds Application Performance – Web: – Latency becomes bottleneck beyond 16Mbits/s Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 7

8. Interpreting Throughput Results Different techniques measure different aspects of throughput Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 8

9. Traffic Shaping: PowerBoost Cable companies advertise “PowerBoost” – Short bursts of high bandwidth Non-existent in DSL Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 9

10. Traffic Shaping Varies Across Users Short-term throughput significantly different from sustainable throughput Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 10

11. Results: Overview Throughput: – Depending on technique, results can vary up to 25% – Traffic shaping varies across users, affects results Latency: – Access link characteristics introduce loss/latency/throughput trade-offs – Modem buffers can induce latencies up to 10 seconds Application Performance – Web: – Latency becomes bottleneck beyond 16 Mbits/s Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 11

12. Latency Measurements Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 12 ParameterWhat it captures End-to-end Latency to nearby server Last-mile Latency to edge of ISP network Under Load Buffer delays due to cross traffic

13. DSL last-mile latencies can be very high – 20% of users > 40ms for some ISPs Impact of Last-mile on Latency Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 13 Cable ISPs DSL ISPs

14. DSL Interleaving Affects Latency 9 9 8 8 7 7 6 6 5 5 4 4 3 3 2 2 1 1 Fastpath is susceptible to bursty loss 9 9 6 6 3 3 8 8 5 5 2 2 7 7 4 4 1 1 Fastpath sends data in order, can recover from single losses Interleaving sends data out-of-order, can recover from bursty loss Fastpath vs. Interleaved last-mile data path Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 14 Extra Delay

15. Example: Latency-Throughput Tradeoff Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 15 User 2: high latency, low loss User 1: low latency, high loss Throughput (Mbits/s) Latency (ms) 10 2.5 2.2 30 Both users have same service plan Interleaving decreases loss, increases latency, improves throughput Interleaving creates a trade-off between latency and throughput

16. Modem Buffers are Too Large Service plans can interact badly with modem buffers Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 16 10 seconds!

17. Results: Overview Throughput: – Depending on technique, results can vary up to 25% – Traffic shaping varies across users, affects results Latency: – Access link characteristics introduce loss/latency/throughput trade-offs – Modem buffers can induce latencies up to 10 seconds Application Performance – Web: – Latency becomes bottleneck beyond 16 Mbits/s Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 17

18. Implications: Web Performance Page load times can exceed 3 seconds – even for popular sites Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 18

19. More Throughput Isn’t Everything Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 19 Page load times stop improving above about 8-16 Mbit/s Page load times stop improving

20. Last-Mile Latency Matters Page load times increase with last mile latency Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 20

21. Results: Summary Throughput: – Depending on technique, results can vary up to 25% – Traffic shaping varies across users, affects results Latency: – Access link characteristics introduce loss/latency/throughput trade-offs – Modem buffers can induce latencies up to 10 seconds Application Performance – Web: – Latency becomes bottleneck beyond 16 Mbits/s Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 21

22. Future Work Performance monitoring from the edge – Effect of peering on performance – Effect of CDN location, traffic engineering on application performance – Effect of home wireless networks on performance Improve end-host performance – How can we minimize the effect of last mile bottlenecks? Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 22

23. Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 23 http://projectbismark.net http://www.samknows.com srikanth@gatech.edu Interested in a BISmark router? Interested in contributing?

24. Previous Studies Measurement from outside – Dischinger et al [1] initiated measurements from wide area server – Indirect measurements, not continuous Measurements from inside – Endhost based measurements – Grenouille [2], Netalyzr [3] – Can’t account for confounding factors [1] Characterizing Residential Broadband Networks IMC 2007 [2] http://www.grenouille.comhttp://www.grenouille.com [3] http://netalyzr.icsi.berkeley.edu Netalyzr: Illuminating the Network Edge IMC 2010http://netalyzr.icsi.berkeley.edu Broadband Internet Performance: A View from the Gateway IRTF ANRP, Atlanta, November 2012 24