1. Taming the Torrent (Can’t P2P and ISPs just get along?)
David R. Choffnes
Fabián E. Bustamante
Northwestern University
Mention the subtitle of the paper – paraphrase the paper’s subtitle
SIGCOMM 2008

2. David Choffnes, Taming the Torrent, SIGCOMM 2008
Peer-to-Peer Systems
Enable a range of important services
End-system multicast, file sharing, content distribution, …
Leverage everybody’s resources
Disk, processing power, bandwidth
Natural scalability, inherent robustness, high flexibility …
Studies have shown that users upgrade their conneciton or change ISPs for better P2P performance
David Choffnes, Taming the Torrent, SIGCOMM 2008

3. The BitTorrent protocol
Over 66% of P2P users & growing
Distribute files through piece exchanges between collaborating peers
A file for download is described by a torrent
Peers sharing content are connected to the same torrent
Which peers? Trackers provide a random subset of peers in the torrent
For a peer’s initial set of connections
For new connections, as transfer progresses and other connections are dropped
Stronger connection to previous slide
Comment on BT taking over a good fraction of the multimedia flows
David Choffnes, Taming the Torrent, SIGCOMM 2008

4. David Choffnes, Taming the Torrent, SIGCOMM 2008
The ISP Perspective
P2P performance - key factor for service upgrade & selection by users
A major engineering challenge for ISPs
≈70% of the Internet traffic
Random peer connections → growing ISP operation costs
Internet Video to TV
Internet Video to PC
VoIP
Video
Gaming
P2P
Web/
Users pay flat rate, but ISPs see increasing, expensive traffic volumes
Make positive and negative clear
To say :oblivious of net topology & link costs
Source: Cisco
David Choffnes, Taming the Torrent, SIGCOMM 2008

5. All fun and games until someone gets a subpoena
ISPs shape traffic directed to standard ports
P2Ps move to dynamic, non-standard ports
ISPs turn to deep-packet inspection to identify & shape P2P flows
P2Ps encrypt their connections
ISPs place caches and/or spoofs TCP RST msgs
Lawsuits + bad publicity
First lesson
An effective solution needs users to buy in
Three class-action lawsuits filed against Comcast ars technica. June 6th, 2008
Wikipedia has a full page about how to defeat DPI
Comcast is reading blogs to try to fix user perceptions
David Choffnes, Taming the Torrent, SIGCOMM 2008

6. A straightforward approach
ISPs guide peers when choosing neighbors
Help reduce cross-ISP traffic
First proposed by Bindal et al. (ICDCS 2006) and Aggarwal et al. (CCR 2007)
More recently – Xie et al. P4P (SIGCOMM 2008)
Two remaining issues
Assumes P2P users & ISPs trust each other
Misses incentive for user adoption

7. An alternative, practical approach
CDNs are already collecting the necessary info
Various CDNs, e.g. Akamai, Limelight
Serve popular sites e.g. NYTimes, Apple, Yahoo …
Move load from providers to edge of network
How do they do it?
Using thousands of replica servers worldwide
Extensive network measurements to inform replica server selection
Redirecting web-clients to web content replicas
Reuse CDNs’ net views for peer selection
How? By simply comparing DNS redirections
Present time

8. Reusing CDNs’ network views
Client’s request redirected to “nearby” server
Client gets web site’s DNS CNAME entry with domain name in CDN network
Hierarchy of CDN’s DNS servers direct client to nearby servers
Multiple redirections to find nearby edge servers
Internet
(3)
Hierarchy of CDN DNS servers
Customer DNS servers
(1)
(2)
(4)
(5)
(6)
LDNS
Web replica servers
Clients and replica servers are “nearby” [SIGCOMM ’06]
Another web client
As we reported in SIGCOMM
Client is given 2 web replica servers (fault tolerance)
Client gets CNAME entry with domain name in Akamai
Client requests translation for Yahoo
Web client
David Choffnes, Taming the Torrent, SIGCOMM 2008

9. Reusing CDNs’ network views (Cont)
Hypotheses
Links between “nearby” hosts cross few ISPs
If two hosts are close to the same CDN replica servers, they are close to each other
Advantages
Requires no additional infrastructure
Needs no topological information
Avoids the trust issue between ISPs and P2P
Reduces cross-ISP traffic …
while improving users’ transfer performance
Before this slide, don’t read the text as much, say things different
David Choffnes, Taming the Torrent, SIGCOMM 2008

10. David Choffnes, Taming the Torrent, SIGCOMM 2008
Deployment
Available as Ono, an extension to a popular BitTorrent client
First released April 2007
Currently > 195,000 users worldwide
… in hundreds of countries
… observing > 2 million peers per day
… with flows traversing ~ 10,000 ASs
… collecting ~15GB of data per day
David Choffnes, Taming the Torrent, SIGCOMM 2008

11. David Choffnes, Taming the Torrent, SIGCOMM 2008
Coverage
Click image to load interactive version
David Choffnes, Taming the Torrent, SIGCOMM 2008

12. CDN-based peer selection
Implementation
Ono
Extension to Azureus/Vuze BitTorrent client (GPL)
Now also part of a tracker
CDN-based peer selection
Uses multiple CDN customers (DNS names)
Only DNS resolution, no content download needed
Adaptive lookup rates on CDN names
Exchanges ratio information w/ other Ono peers
Sends Ono information to supporting trackers
Over 12,000 SLOC (it’s Java)
3,000 for data collection, 3,000 for GUI
Make point that tracker came after paper

13. Figures based on a 2-week study in December, 2007
Experimental dataset
Types of data
Per-connection sampled transfer rates (> 100 million per day)
Ping RTTs (> 100 million per day)
Traceroutes (> 2 million per day)
Figures based on a 2-week study in December, 2007
Each data point is the average for a peer over a 6-hour interval
Results of real, working system

14. Reducing cross-ISP traffic
Average number of AS hops to reach Ono-recommended/random peers
> 30% of paths to Ono-recommended peers do not leave the AS of origin
Note BT curve includes all peers, either Ono-recommended or randomly selected
We use AS hops as a proxy for ISP hops, median is 1 hop, that’s the case for less than 10% of the random one.
This is the first study of the actual number of cross-ISP links, only 8.2% stay in same AS
Median for Ono is 1 AS hop, 3 for random
The quantization evident in the Ono curve is not from a lack of data (they are several thousands of data point there, > 5,000) . Rather, because each Ono peer sees a relatively small number of recommended Ono peers during an observation interval and because those peers are typically along short paths, the average of which takes on small range of values, often integers. Each point, the average of statistics recorded from one Ono peer over a six-hour interval.
David Choffnes, Taming the Torrent, SIGCOMM 2008

15. David Choffnes, Taming the Torrent, SIGCOMM 2008
Finding nearby peers
Two orders of magnitude difference
And, on average, 31% lower loss rates!
Not only does it reduce cross ISP traffic, but also finds better paths. Directly say the values below. Also say the value for traceroute medians, and about 60% that have 0 loss which median for random is 2.1%
6ms to 530ms; the large value of 1ms is ‘cause Windows returns only integer values and <1ms for those bellow 1.
David Choffnes, Taming the Torrent, SIGCOMM 2008

16. Improving transfer performance
Heavy Tail – Average performance improves by 31%
DSL in England -- 4/8Mbps down, only 768Kbps up
ISP bandwidth allocation policy brings bottleneck to the access link
Median difference is ~2KB/s
Even when Ono doesn’t help, it allows BT to naturally select faster peers
To our surprise, this is what we found out …
Of course 31% is good enough, we could leave it there. But this doesn’t seem to fit with the path qualities. Talk about why we saw this w/ BT getting good system-wide performance, but low per-connection rates per connection, bottleneck at the access link. If this is the case, then we should be able to find an ISP w/ the policy. What do you know, we have one.
David Choffnes, Taming the Torrent, SIGCOMM 2008

17. … with the right bandwidth allocation policy
Romania: 50 Mb/s in metro-area, 4 Mb/s outside
883% median improvement
We have a user said this was great, Portugal sees this too
David Choffnes, Taming the Torrent, SIGCOMM 2008

18. Helpful ISPs can help themselves
What’s more interesting, the ISP that helps you, helps itself.
Before was bandwidth allocation, now it’s ISPs with policy that reduces cross-ISP traffic, mutually assured benefits
David Choffnes, Taming the Torrent, SIGCOMM 2008

19. David Choffnes, Taming the Torrent, SIGCOMM 2008
Discussion
Scalability/Overhead
No path probing to locate nearby peers
Requires only periodic DNS lookups
Max: 18KB up and 36KB down per day
Practical
No new infrastructure required
Ready to use now, even a year ago
No need for trust between ISPs & P2P
Performance improvement is the best incentive for adoption!
Cross-ISP traffic
No comment about complaint
Change to “Discussion”.
Mention some ISPs want cross-ISP traffic
David Choffnes, Taming the Torrent, SIGCOMM 2008

20. That’s all fine and good, but why not …?
Get rid of peers that are “far away” (?)
Latency is weakly correlated with path distance
Requires everyone to measure
Use AS numbers
AS-level info can be too …
Fine-grained - Cross-AS != Cross-ISP
Coarse - Intra-AS could be coast-to-coast
Ask your ISP
Decisions guided by aggregated traffic policies
But what about adoption/deployment cost/operation …?
Requires extensive latency measurements (not N^2)
Last one, say explicitly that you this is P4P
Comcast has >40 different ASNs – Cross-AS != Cross-ISP
AS7132 spreads over most of the continental US
David Choffnes, Taming the Torrent, SIGCOMM 2008

21. David Choffnes, Taming the Torrent, SIGCOMM 2008
Summary
Ono – A practical & effective way to reduce P2P networking costs
Reuses CDNs’ network views
An alternative approach for ISPs?
Recommend Ono to your P2P users
Change bandwidth allocation to favor in-network connections
Part of 3R project - Reduce, Reuse, Recycle
Reuse/recycle the views of long-running services in building large distributed systems
Ono or Ono-like solution, don’t forget limwire
David Choffnes, Taming the Torrent, SIGCOMM 2008

22. David Choffnes, Taming the Torrent, SIGCOMM 2008
Extra, Extra…
Network Early Warning System (NEWS)
Reuse P2P systems’ aggregated views and natural traffic
Detect network anomalies locally
Use corroboration to confirm them globally
Join the 4,207 users who have already done so!
David Choffnes, Taming the Torrent, SIGCOMM 2008

23. David Choffnes, Taming the Torrent, SIGCOMM 2008
Fin
David Choffnes, Taming the Torrent, SIGCOMM 2008