Taming the Torrent (Can’t P2P and ISPs just get along?) David R. Choffnes Fabián E. Bustamante Northwestern University Mention the subtitle of the paper – paraphrase the paper’s subtitle SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Peer-to-Peer Systems Enable a range of important services End-system multicast, file sharing, content distribution, … Leverage everybody’s resources Disk, processing power, bandwidth Natural scalability, inherent robustness, high flexibility … Studies have shown that users upgrade their conneciton or change ISPs for better P2P performance David Choffnes, Taming the Torrent, SIGCOMM 2008

The BitTorrent protocol Over 66% of P2P users & growing Distribute files through piece exchanges between collaborating peers A file for download is described by a torrent Peers sharing content are connected to the same torrent Which peers? Trackers provide a random subset of peers in the torrent For a peer’s initial set of connections For new connections, as transfer progresses and other connections are dropped Stronger connection to previous slide Comment on BT taking over a good fraction of the multimedia flows David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 The ISP Perspective P2P performance - key factor for service upgrade & selection by users A major engineering challenge for ISPs ≈70% of the Internet traffic Random peer connections → growing ISP operation costs Internet Video to TV Internet Video to PC VoIP Video Gaming P2P Web/E-mail Users pay flat rate, but ISPs see increasing, expensive traffic volumes Make positive and negative clear To say :oblivious of net topology & link costs Source: Cisco David Choffnes, Taming the Torrent, SIGCOMM 2008

All fun and games until someone gets a subpoena ISPs shape traffic directed to standard ports P2Ps move to dynamic, non-standard ports ISPs turn to deep-packet inspection to identify & shape P2P flows P2Ps encrypt their connections ISPs place caches and/or spoofs TCP RST msgs Lawsuits + bad publicity First lesson An effective solution needs users to buy in Three class-action lawsuits filed against Comcast ars technica. June 6th, 2008 Wikipedia has a full page about how to defeat DPI Comcast is reading blogs to try to fix user perceptions David Choffnes, Taming the Torrent, SIGCOMM 2008

A straightforward approach ISPs guide peers when choosing neighbors Help reduce cross-ISP traffic First proposed by Bindal et al. (ICDCS 2006) and Aggarwal et al. (CCR 2007) More recently – Xie et al. P4P (SIGCOMM 2008) Two remaining issues Assumes P2P users & ISPs trust each other Misses incentive for user adoption

An alternative, practical approach CDNs are already collecting the necessary info Various CDNs, e.g. Akamai, Limelight Serve popular sites e.g. NYTimes, Apple, Yahoo … Move load from providers to edge of network How do they do it? Using thousands of replica servers worldwide Extensive network measurements to inform replica server selection Redirecting web-clients to web content replicas Reuse CDNs’ net views for peer selection How? By simply comparing DNS redirections Present time

Reusing CDNs’ network views Client’s request redirected to “nearby” server Client gets web site’s DNS CNAME entry with domain name in CDN network Hierarchy of CDN’s DNS servers direct client to nearby servers Multiple redirections to find nearby edge servers Internet (3) Hierarchy of CDN DNS servers Customer DNS servers (1) (2) (4) (5) (6) LDNS Web replica servers Clients and replica servers are “nearby” [SIGCOMM ’06] Another web client As we reported in SIGCOMM Client is given 2 web replica servers (fault tolerance) Client gets CNAME entry with domain name in Akamai Client requests translation for Yahoo Web client David Choffnes, Taming the Torrent, SIGCOMM 2008

Reusing CDNs’ network views (Cont) Hypotheses Links between “nearby” hosts cross few ISPs If two hosts are close to the same CDN replica servers, they are close to each other Advantages Requires no additional infrastructure Needs no topological information Avoids the trust issue between ISPs and P2P Reduces cross-ISP traffic … while improving users’ transfer performance Before this slide, don’t read the text as much, say things different David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Deployment Available as Ono, an extension to a popular BitTorrent client First released April 2007 Currently > 195,000 users worldwide … in hundreds of countries … observing > 2 million peers per day … with flows traversing ~ 10,000 ASs … collecting ~15GB of data per day David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Coverage Click image to load interactive version David Choffnes, Taming the Torrent, SIGCOMM 2008

CDN-based peer selection Implementation Ono Extension to Azureus/Vuze BitTorrent client (GPL) Now also part of a tracker CDN-based peer selection Uses multiple CDN customers (DNS names) Only DNS resolution, no content download needed Adaptive lookup rates on CDN names Exchanges ratio information w/ other Ono peers Sends Ono information to supporting trackers Over 12,000 SLOC (it’s Java) 3,000 for data collection, 3,000 for GUI Make point that tracker came after paper

Figures based on a 2-week study in December, 2007 Experimental dataset Types of data Per-connection sampled transfer rates (> 100 million per day) Ping RTTs (> 100 million per day) Traceroutes (> 2 million per day) Figures based on a 2-week study in December, 2007 Each data point is the average for a peer over a 6-hour interval Results of real, working system

Reducing cross-ISP traffic Average number of AS hops to reach Ono-recommended/random peers > 30% of paths to Ono-recommended peers do not leave the AS of origin Note BT curve includes all peers, either Ono-recommended or randomly selected We use AS hops as a proxy for ISP hops, median is 1 hop, that’s the case for less than 10% of the random one. This is the first study of the actual number of cross-ISP links, only 8.2% stay in same AS Median for Ono is 1 AS hop, 3 for random The quantization evident in the Ono curve is not from a lack of data (they are several thousands of data point there, > 5,000) . Rather, because each Ono peer sees a relatively small number of recommended Ono peers during an observation interval and because those peers are typically along short paths, the average of which takes on small range of values, often integers. Each point, the average of statistics recorded from one Ono peer over a six-hour interval. David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Finding nearby peers Two orders of magnitude difference And, on average, 31% lower loss rates! Not only does it reduce cross ISP traffic, but also finds better paths. Directly say the values below. Also say the value for traceroute medians, and about 60% that have 0 loss which median for random is 2.1% 6ms to 530ms; the large value of 1ms is ‘cause Windows returns only integer values and <1ms for those bellow 1. David Choffnes, Taming the Torrent, SIGCOMM 2008

Improving transfer performance Heavy Tail – Average performance improves by 31% DSL in England -- 4/8Mbps down, only 768Kbps up ISP bandwidth allocation policy brings bottleneck to the access link Median difference is ~2KB/s Even when Ono doesn’t help, it allows BT to naturally select faster peers To our surprise, this is what we found out … Of course 31% is good enough, we could leave it there. But this doesn’t seem to fit with the path qualities. Talk about why we saw this w/ BT getting good system-wide performance, but low per-connection rates per connection, bottleneck at the access link. If this is the case, then we should be able to find an ISP w/ the policy. What do you know, we have one. David Choffnes, Taming the Torrent, SIGCOMM 2008

… with the right bandwidth allocation policy Romania: 50 Mb/s in metro-area, 4 Mb/s outside 883% median improvement We have a user said this was great, Portugal sees this too David Choffnes, Taming the Torrent, SIGCOMM 2008

Helpful ISPs can help themselves What’s more interesting, the ISP that helps you, helps itself. Before was bandwidth allocation, now it’s ISPs with policy that reduces cross-ISP traffic, mutually assured benefits David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Discussion Scalability/Overhead No path probing to locate nearby peers Requires only periodic DNS lookups Max: 18KB up and 36KB down per day Practical No new infrastructure required Ready to use now, even a year ago No need for trust between ISPs & P2P Performance improvement is the best incentive for adoption! Cross-ISP traffic No comment about complaint Change to “Discussion”. Mention some ISPs want cross-ISP traffic David Choffnes, Taming the Torrent, SIGCOMM 2008

That’s all fine and good, but why not …? Get rid of peers that are “far away” (?) Latency is weakly correlated with path distance Requires everyone to measure Use AS numbers AS-level info can be too … Fine-grained - Cross-AS != Cross-ISP Coarse - Intra-AS could be coast-to-coast Ask your ISP Decisions guided by aggregated traffic policies But what about adoption/deployment cost/operation …? Requires extensive latency measurements (not N^2) Last one, say explicitly that you this is P4P Comcast has >40 different ASNs – Cross-AS != Cross-ISP AS7132 spreads over most of the continental US David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Summary Ono – A practical & effective way to reduce P2P networking costs Reuses CDNs’ network views An alternative approach for ISPs? Recommend Ono to your P2P users Change bandwidth allocation to favor in-network connections Part of 3R project - Reduce, Reuse, Recycle Reuse/recycle the views of long-running services in building large distributed systems Ono or Ono-like solution, don’t forget limwire David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Extra, Extra… Network Early Warning System (NEWS) Reuse P2P systems’ aggregated views and natural traffic Detect network anomalies locally Use corroboration to confirm them globally Join the 4,207 users who have already done so! http://aqualab.cs.northwestern.edu/projects/NEWS.html David Choffnes, Taming the Torrent, SIGCOMM 2008

David Choffnes, Taming the Torrent, SIGCOMM 2008 Fin David Choffnes, Taming the Torrent, SIGCOMM 2008