1. Stochastic Analysis of File Swarming Systems The Chinese University of Hong Kong John C.S. Lui Collaborators: D.M. Chiu, M.H. Lin, B. Fan

2. Background Traditional Client/Server Sharing  Performance deteriorates rapidly as the number of clients increases IP Multicast Application Multicast (e.g., CDN, ESM)  reliability, unused resources at leaf nodes P2P (e.g., Naspter, Gnutella)  Free riders only download without contributing to the network. BitTorrent P2P systems:  Good scalability  Built-in incentive mechanism to contribute

3. BT Components On a public domain site, obtain torrent file, for example:  http://bt.btchina.net http://bt.btchina.net  http://bt.ydy.com/ Web Server Harry Potter.torrent Transformer.torrent The Lord of Ring.torrent

4. BT Components The.torrent file  Static “metainfo” file to contain necessary information : File name # of chunks, size checksum Tracker IP address of the Tracker,…etc A BitTorrent tracker A BitTorrent tracker  Non-content-sharing node  Track peers File: File:  Chunk size (256KB), has individual hash code in the torrent file Types of peers: Types of peers:  Leechers  Seeders

5. BT: publishing a file Web ServerMoe Tracker Downloader: Larry Seeder: John Downloader: Curly Harry Potter.torrent

6. Simple example Seeder: John Downloader Moe {1,2,3,4,5,6,7,8,9,10} {} {1,2,3} Downloader Larry {} {1,2,3} {1,2,3,4} {1,2,3,5} {1,2,3,4,5}

7. BT: internal Chunk Selection mechanisms Strict Priority  First Priority Rarest First  General rules Random First Piece  Special case, at the beginning Endgame Mode  Special case

8. BT: internal mechanism Built-in incentive mechanism (where all the magic happens):  Choking Algorithm  Optimistic Unchoking

9. BT: internal mechanism Choking is a temporal refusal to upload Each peer unchokes a fixed number of peers Reasons for choking: –Avoid free riders –Network congestion –Contribute to “useful” peers Yaokun Wu John C.S Lui Choked

10. BT: internal mechanism (optimistic unchoking) A BitTorrent peer has a single “optimistic unchoke” which uploads regardless of the current download rate from it. This peer rotates every 30s Reasons:  To discover currently unused connections are better than the ones being used  To provide minimal service to new peers

11. Example: optimistic unchoking Andy Yao Downloader: John Lui Downloader: Melinda Downloader: Larry Downloader: Curly 40kb/s 30kb/s 10kb/s 100kb/s 20kb/s 70kb/s 15kb/s 10kb/s 70kb/s. 110kb/s 70kb/s 5kb/s Downloader Moe

12. P2P content distribution BitTorrent Sending a file to a large number of peers, with the help of peers Producing the most Internet traffic today (over 50% of traffic, creates contention but....) What IP multicast tried to support Modeling these systems => insights

13. Why Study BitTorrent-like System? BitTorrent is very efficient.  Which features make it perform so well? Motivating questions  What is the effect of bandwidth constraints?  Is the Rarest First policy really necessary?  Must nodes perform seeding after file downloading?  How serious is the Last Piece Problem?  Is source coding useful?  Does the incentive mechanism affect the performance much? Our aim is to develop mathematical models of file swarming systems, allowing us to investigate these issues via analytical means.

14. Model for the File Swarming System A file has K non-overlapping chunks. Peers arrive according to a Poisson process. Each peer is initialized with one random chunk. Peers leave the system immediately when finish downloading. download constraint The system is slotted: downlink bandwidth is one chunk per time slot for all peers. ( download constraint ) In each time slot, each peer contacts m neighbors uniformly from the system to see whether they are useful. If some neighbors are useful, it randomly chooses one and requests a random useful chunk. (without/with upload constraint) If a peer receives several requests, it will satisfy all / random one request(s). (without/with upload constraint)

15. Model for the File Swarming System peer A peer B peer C peer D peer E Request C1 Request C5 C1 C5 Example: m=2 The case “m = 1 & no upload constraint” was studied by L.Massoulie et.al in ”Coupon replication systems”. HELLO Bitmap Without upload constraint With upload constraint

16. Download Constraint Only Model 1: Download Constraint Only Classify peers into K−1 types. Peers holding i chunks are named type i peers. Denote the number of type i peers, We are interested in the average sojourn time T i for type i peers. The average downloading time For a type i peer, the probability that a type j peer is useful : For a type i peer, the probability that a randomly picked peer is useful:

17. Model 1: Download Constraint Only Given the system state, is a Multi-dimensional infinite state-space Markov Process : It is hard to solve this Markov Chain directly Density Dependent jump Markov Process Transform the Markov Chain to a “Density Dependent jump Markov Process” Focusing on its steady state and asymptotic behavior tight bounds We derive tight bounds.

18. Download Constraint Only Model 1: Download Constraint Only The case m=1 has been studied in [1], in which the authors gave a looser bound: [1] L.Massoulie, M.VojnoviC, ”Coupon replication systems”, SIGMETRICS, 2005. The average downloading time.

19. Lower bound v.s. Upper bound (K=200) m=1m=2 Last Piece Problem It takes a peer a longer time to download the last few chunks of the file, since it gets increasingly more difficult to find other peers that can help.

20. Bounds v.s. Simulation (K=200) m=1m=2 The simulation shows the accuracy of our model. How to relief the last piece problem?

21. System with Source Coding K=4Q=6 Source peer A peer B peer C peer D peer E C1

22. System with Source Coding The source encodes the original K chunks into Q chunks, Any peer could reconstruct the original file after he receives any K distinct chunks.

23. Source Coding vs. No Coding(K=200) m=1, no coding Source coding eliminates the Last Piece Problem !!! m=1, source coding ( )

24. Download constraint only K=200; m=1 K=500; m=1

25. Download Constraint K=200; m=2 K=500; m=2

26. Model 2: Download & Upload Constraints —— m=1 peer A peer B peer C peer D peer E Request C1 Request C5 C1 HELLO Bitmap

27. Model 2: Download & Upload Constraints —— m=1 Stage One: Requesting  The same as Model 1. Stage Two: Downloading  The distribution of the number of requests one peer would receive (depending on its type ).  Only one request will be satisfied. Still a density dependent jump Markov process  The transition rates are more complicated.

28. Model 2: Download & Upload Constraints —— m=1 ≈ 1.58

29. Bounds v.s. Simulation (K=200, without source coding) m=1 & satisfying one request Ti is NOT close 1 any more, i.e. downloading time is far from being optimal.

30. Model 3: An Incentive Mechanism peer A peer B peer C peer D peer E Request C1 Request C5 C5 Assuming peers are matched randomly at the beginning of each time slot. Each pair will perform chunk transfer iff both of them are useful to each other. Request C2 C2

31. Model 3: An Incentive Mechanism

32. Bounds v.s. Simulation (K=200, without source coding) First Piece Problem It is not easy to download the first few chunks when a peer enters the system, but one can solve this in various of ways….

33. Incentive Mechanism K=200; m=1 K=500; m=1

34. Conclusion Many peers, steady state, certain mechanism to ensure file availability (e.g. some seeders), then The nature of swarming makes P2P systems very efficient. Rarest First policy is not necessary for performance. If peers are cooperative, “random policy” is good enough, though it may be helpful to enhance file availability. Peers are not necessary to perform seeding after file downloading. Simple strategies (everything is random) can make the downloading time near optimal. Source coding is useful, to relief the last piece problem. With certain incentive mechanism, the downloading time can still approach optimal. Our mathematical models provide a basis for designing new BT-like protocol.

35. Research Questions What about fairness? How to extend file swarming to multimedia streaming? For Joost? What about wide area network exchange? What happen if there is ``network congestion’’? What is the impact? Network Coding? Security?

36. Q & A Thank You