1. MBA - 2015 Working in open collaboration, to build a transparent measurement collection and reporting methodology, and to publish a freely available dataset of US consumer broadband performance, since 2010. 8 April 2015 / 18 August 2015

2. AGENDA Overview Project Objective Selected Services (beta) Timeline Review of Technical Methodology Joint Opportunities UCSC initial results

4. Project Objective Create a working group whose purpose is to develop a common methodology of measuring video streaming performance to allow for a comparative measurement and analysis of available services.

7. Review of Technical Methodology

8. Whitebox

9. Network BEFORE whitebox

10. Network with whitebox

11. Whitebox operation Whitebox defers scheduled tests when the home network is active Wired traffic is detected passing through the whitebox Wi-Fi radio is used to detect network activity from wireless clients Whitebox does not need your key to detect activity Whitebox reports total GB/month traffic FCC interest in actual v advertised bandwidth

12. YouTube Methodology Streams real videos from YouTube Uses the real YouTube servers (including Google Global Cache, if installed by the ISP Tests most popular videos Decodes videos on the probes – it’s not just a speed test

13. Netflix Methodology Streams synthetic content from Netflox Uses the real Netflix ‘Open Connect Appliances’ inside the ISP network Downloads content at full-rate for 20 seconds. A bitrate reliably streamed’ metric can be derived from analysis of this transfer Developed in collaboration with Netflix Complaints about use of synthetic data

15. Beta Observations (April 2015) Early rollout showed peak-hour congestion issues in certain areas for both YouTube and Netflix Many tests operate over IPv6 rather than IPv4, again for both YouTube and Netflix Tests new report the maximum available bitrate of the video as well, so it is possible to present the bitrate reliably streamed as a percentage.

16. UCSC test plan UCSC has a 200 node DOCSIS 2 cable modem service Built in 1970, before the Internet, before dirt This cable TV coax was not designed for data We will compare DOCSIS modem service to campus Ethernet One test box in Engineering building One test box in a Grad Student Appt

19. Netflix Streaming CENIC has Netflix caches in LA and Sacramento

23. More info The FCC uses contractor SamKnows to code and deploy measurement infrastructure https://www.samknows.com/ https://www.samknows.com/broadband/uploads/methodology/SQ3 01-005-EN-Test-Suite-Whitepaper-4.pdf https://www.samknows.com/broadband/uploads/methodology/SQ3 01-005-EN-Test-Suite-Whitepaper-4.pdf They are still sending out whiteboxes to candidate participants Testing of a 1 Gb/s whitebox has started FCC’s annual broadband report is late, but soon – they say.