MBA Working in open collaboration, to build a transparent measurement collection and reporting methodology, and to publish a freely available dataset of US consumer broadband performance, since April 2015 / 18 August 2015
AGENDA Overview Project Objective Selected Services (beta) Timeline Review of Technical Methodology Joint Opportunities UCSC initial results
Project Objective Create a working group whose purpose is to develop a common methodology of measuring video streaming performance to allow for a comparative measurement and analysis of available services.
Review of Technical Methodology
Whitebox
Network BEFORE whitebox
Network with whitebox
Whitebox operation Whitebox defers scheduled tests when the home network is active Wired traffic is detected passing through the whitebox Wi-Fi radio is used to detect network activity from wireless clients Whitebox does not need your key to detect activity Whitebox reports total GB/month traffic FCC interest in actual v advertised bandwidth
YouTube Methodology Streams real videos from YouTube Uses the real YouTube servers (including Google Global Cache, if installed by the ISP Tests most popular videos Decodes videos on the probes – it’s not just a speed test
Netflix Methodology Streams synthetic content from Netflox Uses the real Netflix ‘Open Connect Appliances’ inside the ISP network Downloads content at full-rate for 20 seconds. A bitrate reliably streamed’ metric can be derived from analysis of this transfer Developed in collaboration with Netflix Complaints about use of synthetic data
Beta Observations (April 2015) Early rollout showed peak-hour congestion issues in certain areas for both YouTube and Netflix Many tests operate over IPv6 rather than IPv4, again for both YouTube and Netflix Tests new report the maximum available bitrate of the video as well, so it is possible to present the bitrate reliably streamed as a percentage.
UCSC test plan UCSC has a 200 node DOCSIS 2 cable modem service Built in 1970, before the Internet, before dirt This cable TV coax was not designed for data We will compare DOCSIS modem service to campus Ethernet One test box in Engineering building One test box in a Grad Student Appt
Netflix Streaming CENIC has Netflix caches in LA and Sacramento
More info The FCC uses contractor SamKnows to code and deploy measurement infrastructure EN-Test-Suite-Whitepaper-4.pdf EN-Test-Suite-Whitepaper-4.pdf They are still sending out whiteboxes to candidate participants Testing of a 1 Gb/s whitebox has started FCC’s annual broadband report is late, but soon – they say.