BISmark: the Broadband Internet Service benchmark Walter de Donato Department of Computer Engineering and Systems University of Napoli Federico II

The Internet scenario (1/2) 2 billion Internet users worldwide 500 million broadband users Linux Day, Napoli, 27 October 2012

The Internet scenario (2/2) Heterogeneous access technologies Wired Wireless UMTS xDSL Fiber WiMax LTE PSTN Satellite Cable Wi-Fi ISDN Internet applications are more and more complex and resource demanding Linux Day, Napoli, 27 October 2012

What performance do users see? Important for regulators, consumers, ISPs Are users really getting what they pay for? Can visibility into problems help reduce service calls? Notion of performance is fuzzy What metrics should we measure? How to measure them? Linux Day, Napoli, 27 October 2012

IP Performance evaluation Measurement methodologies and techniques Active → purposely forge synthetic traffic Passive → exploit user generated traffic Most common Quality of Service metrics Throughput VoIP Quality Acceptable Good Latency < 300 ms < 150 ms Jitter < 50 ms < 20 ms Packet loss < 3% < 1% Latency Jitter Packet loss Linux Day, Napoli, 27 October 2012

Studying Internet access networks Taxonomy of approaches Server-based Router-based Host-based ISP-based Core-based Client-based Web-based Service-based plugin-based embedded Last Mile Linux Day, Napoli, 27 October 2012

The BISmark project University of Napoli Federico II Walter de Donato, Antonio Pescapè University of Napoli Federico II Srikanth Sundaresan, Nick Feamster Georgia Institute of Technology

BISmark: a router-based platform continous periodical measures observes all traffic passing through network can take into account interfereces difficult to obtain large scale deployments BISmark firmware Last Mile Linux Day, Napoli, 27 October 2012

BISmark: architecture overview Management Server Measurement Server BISmark daemon BISmark measurement services Hi-speed Network DB Internet ISP BISmark firmware Control Measure Linux Day, Napoli, 27 October 2012

BISmark: main features Completely open-source (sources maintained on github.com) Built on top of OpenWRT Linux Real-time monitoring of routers (60 sec heartbeat) Remote router firmware update Server-side configuration (pull mode) On-demand remote troubleshooting Geographic selection of measurement servers http://projectbismark.net Linux Day, Napoli, 27 October 2012

BISmark: the platform growing up Atlanta USA Worldwide 2010 2010 2011 2012 2013 6 20 60 260 Routers deployed Routers deployed 1 2 4 6 12 Developers involved Netgear WNDR3700v2 Linksys WRT54GL NoxBox Router models Linux Day, Napoli, 27 October 2012

BISmark: challenges and solutions Limited memory and CPU resources Hard to let OS, tools and data coexits Solution: optimizations optimized and stripped binaries most binaries on ram-disk (pull mode) reuse of ash interpeter for most tasks processing on-the-fly of results Remote management through NATs Unable to directly contact routers from the Internet Solution: ad-hoc management protocol UDP heartbeat packets allow the server to reply with simple commands Optimized OpenWRT firmware Netgear WNDR3700 Linksys WRT54GL 680Mhz MIPS CPU 64 MB RAM 8MB Flash 200Mhz MIPS CPU 16 MB RAM 4MB Flash Linux Day, Napoli, 27 October 2012

BISmark: active measurements Parameters Type L4 Proto Frequency Tool Latency End-to-end Last-mile Upstream load Downstream load DNS ICMP, UDP ICMP UDP 5 min 30 min ping, D-ITG ping nslookup Packet loss 15 min D-ITG Jitter Downstream throughput Single-thread Passive Capacity TCP N/A 12 hrs, 60 min Curl (GET), D-ITG /proc/net/dev shaperprobe, D-ITG Upstream throughput Curl (PUT), D-ITG Collected statistics minimum, maximum, average, standard deviation, median, inter quartile range Measurement servers hosted by GeorgiaTech, University of Napoli and MLab Linux Day, Napoli, 27 October 2012

BISmark: scientific results Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford, Antonio Pescapè, Measuring Broadband Home Performance, Communications of ACM, Volume 55 - Issue 11, November 2012. Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford, Antonio Pescapè, Broadband Internet Performance: A View From the Gateway, ACM SIGCOMM 2011, Toronto, ON, Canada, August 15-19, 2011 Selected by IRTF for the Applied Networking Research Prize (ANRP) 2012 Srikanth Sundaresan, Nick Feamster, Renata Teixeira, Anthony Tang, W. Keith Edwards, Rebecca E. Grinter, Marshini Chetty, Walter de Donato, Helping Users Shop For ISPs with Internet Nutrition Labels, ACM SIGCOMM Workshop on Home Networks proceedings, Toronto, ON, Canada, August 15-19, 2011 Linux Day, Napoli, 27 October 2012

BISmark: live examples Official website Virtual router Linux Day, Napoli, 27 October 2012

Thank you! ?

Remote troubleshooting Linux Day, Napoli, 27 October 2012