1. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion A Measurement Study of Piece Population in BitTorrent Cameron Dale and Jiangchuan Liu Simon Fraser University Burnaby, BC, Canada camerond@cs.sfu.ca Globecom, November 29 th, 2007, Washington, D.C.

2. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Overview ● background on some BitTorrent features ● describe the experiment ● results for piece population snapshots and the evolution of the piece population for some real BitTorrent swarms* ● PlanetLab simulations of BitTorrent swarms ● discussion and future work *swarm: a group of peers connected to one another via the BitTorrent file distribution protocol

3. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Piece Population ● BitTorrent splits a large file into many pieces ● each piece will have a certain number of copies in all the peers in the swarm ● ideally, all pieces should have approximately the same number of copies ● local knowledge limitations prevent the ideal and introduce an imbalance ● too much variation can lead to inefficiencies and even starvation

4. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Rarest-First Policy ● responsible for choosing the next piece that a peer will download ● each client chooses the piece that it believes is the rarest one ● choices are made based only on local knowledge of neighboring peers ● goal is to maintain an even distribution of pieces throughout the swarm

5. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Piece Population Distribution ● expect the population to form a distribution around a mean value ● the mean will be determined solely by the macro-characteristics: peer arrival rate, download time, departure rate, etc. ● the width of the distribution will indicate the effectiveness of the rarest-first policy – the narrower, the better

6. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Real Swarm Snapshots ● used a modified BitTorrent client to gather data from real swarms ● client constantly requests peers from the tracker* and connects to all returned peers ● client connects to most (90%) peers in a swarm and collects their piece information ● collection occurs very quickly and is terminated, usually after less than an hour *tracker: a server that coordinates communication between peers attempting to download a file

7. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Real Swarm Evolution ● collection was done as in the snapshots ● collection continued for many hours, or even days ● observe changes in the Piece Population as the swarm evolves ● focussed on the early stages of a swarm when there are a large number of leechers* *leecher: a user who does not yet have a complete copy of the file, and so is currently downloading (and uploading)‏

8. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Swarm Characteristics *clients: the number of clients administered by us

9. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Piece Population Plots ● x-axis shows the number of copies within the downloaders in the swarm ● y-axis shows the number of pieces that have that number of copies ● x-axis is normalized to range from 0 to 1, by the total number of downloaders ● y-axis can be normalized by the number of pieces, or so that the area under the graph is 1 (to facilitate comparisons)‏

10. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Real Internet Swarm Snapshots

11. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: feisty

12. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: feisty

13. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: feisty

14. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: feisty

15. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: openSUSE-2

16. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: openSUSE-2

17. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: openSUSE-2

18. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Evolution: openSUSE-2

19. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulated Swarms ● run on PlanetLab research network testbed ● all peers are controlled by us ● experiment starts with a single seed* and peers join randomly over first 4 hours ● PlanetLab: identical peers, no peers join after 4 hours ● PlanetLab-2: distribution of peers, new peers arriving continuously *seed: a peer that has the complete file, and is only uploading

20. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab ● all peers are identical ● peers stay in the system for about 9 hours, then leave forever ● each peer's maximum number of connections is limited to 40 to enhance the local effect ● it takes 9 hours for the initial seed to upload a single copy of the file

21. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab

22. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab

23. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab

24. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab

25. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab-2 ● peers have a distribution of download and upload speeds ● peers download the file, seed for a random time, then leave and rejoin as a new peer ● arrivals and departures are grouped together to increase the amount of churn* ● each peer's maximum number of connections is limited to 80 (default)‏ *churn: the arrival and departure of peers in a P2P system

26. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab-2

27. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab-2

28. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Simulation: PlanetLab-2

29. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Rarest-first Policy ● is effective – results in normal distributions that are narrow ● improves on random piece selection by 3-4x – recovers quickly (exponentially) from events that cause a widened distribution ● could be improved – larger swarms show tails on the high end due to the limited local knowledge of peers – churn causes increases in the width of the population

30. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Suggested Improvement ● peers knowledge of pieces could be extended through gossiping – share all piece information with neighbors ● increased communication cost – share information on which pieces are the rarest ● piece with the most votes gets downloaded next ● should reduce the size of the tails for the larger swarms

31. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion Future Work ● use these measurements to evaluate the effect of new piece selection strategies ● further analysis of swarms in the presence of churn to determine the cause of the increased population width ● work is already under way to create an analytical piece-level model describing the piece population of a swarm – complicated due to the complex interactions between peers

32. Cameron Dale and Jiangchuan LiuA Measurement Study of Piece Population in BitTorrent Introduction BitTorrent Experiment Results Simulation Discussion A Measurement Study of Piece Population in BitTorrent Cameron Dale and Jiangchuan Liu Simon Fraser University Burnaby, BC, Canada camerond@cs.sfu.ca Globecom, November 29 th, 2007, Washington, D.C.