1. Jin Li, Principal Researcher (Collaborators: Cheng Huang, Keith Ross) Communication and Collaboration Systems Microsoft Research 1

2. 2 Introduction: Internet Video on the Rise

3. Internet Video is on the Rise Video streams served increased 38.8% in 2006 to 24.92 billion (Source: AccuStream iMedia Research) 53 web-video startup in 2006 (US), $521M VC funding (Source: DowJones VentureOne) Major studio goes into Web Video $410M video ads revenue in 2006 and grow by 89% in 2007 (0.6% of $74B TV ad market, Internet ads $16.4B in 2006, expect to grow 19% in 2007) [source: Emarketer] 3

4. Internet Video is Growing: in Popularity & Quality 4 Apr. 2006Dec. 2006Up (%) # of views (million)41.164.757.4 # of users (million)9.0312.0233.1 video quality evolution popularity evolution

5. 5 Internet Video Delivery: Data Center vs. CDN vs. P2P

6. Just Build More Powerful Data Center? 6

7. Data Center Capacity VHS quality video streaming: 500 kbps (H.264) 200,000 viewers = 100 Gbps Data center capacity: Tera Grid (UIUC) 30 petabyte of storage 40 Gbps backbone: 80k video viewers 7

8. CDN Is Not The Answer 8 Akamai 20,000 servers, 900 point of presence, 71 countries 400Gbps bandwidth Network optimized for latency Limelight 25 point of presence, hundreds servers per presence 1,000Gbps bandwidth Akamai+Limelight: 2.8 million viewers. Current TV audience Olympics: 2.5 billion viewers Each viewer may have his/her own interest (different sport event, athlete nationality, etc.)

9. Peer Assisted Delivery is the Way To Scale 9 Economical to run Saves server/CDN bandwidth, disk I/O, CPU, memory Robust no single point of failure in network Super-scalable system capacity increases with number of nodes peer resource bandwidth CPU memory hard drive

10. P2P Benefit Consumers: Better Video Quality, More Selection 10

11. 11 Case Study 1: On Demand Internet Video

12. Peer Assisted Delivery: Mode File Sharing broadcast On Demand Streaming (Interactive TV) Live Messenger FolderShare Groove 12

13. Peer Buffer Map: File Sharing 13 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

14. Peer Buffer Map: Broadcast 14 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

15. Peer Buffer Map: On Demand 15 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

16. MSN VoD Service 16 Traces from the on-demand service of MSN Video 9-month period: Apr. – Dec. 2006 520M streaming requests 59,000 unique videos

17. Peer-assisted VoD Model 17 Guaranteed QoS: always available server Performance metric: server bandwidth Peers upload what / when they are watching conservative assumption servers in data centers/CDN

18. Peer Bandwidth Download BW is measured by Windows Media Server no accurate measurement beyond 3.5Mbps Upload BW is inferred Average upload: 500+ kbps 18 DSL2 Cable Ethernet Modem ISDN DSL1

19. Bandwidth Allocation Policies 19 Assumptions peers always start watching from beginning VoD: earlier peers upload to later peers 1 st policy: no-prefetching only satisfy demand for smooth playback, do not further build up the buffer used by commercial live streaming companies to offer VoD servers in data centers 1 2 3 4 arrival ask ask server ask ask server 3 2 1

20. Bandwidth allocation policies (2) 20 Prefetching – to utilize remaining upload capacity 2 nd policy: water-leveling 3 rd policy: greedy Lower bound allow later peers upload to earlier ones no arrival order constraints servers in data centers 1 2 3 4 arrival water-leveling: greedy: water-leveling: greedy: 4 2

21. Observations on Policies (Simulated: Peer Poisson Arrival) Prefetching is crucial “free” to increase video bitrate “balanced mode” is most difficult S ≈ D Greedy policy works best lowest server load very close to bound more available upload

22. Server BW Reduction – Two Videos select top two most popular videos ~800,000 views during April, 2006 significant server bandwidth reduction using peer assistance less server BW even increase quality 3 times (@3x bitrate) 22 gold streamsilver stream

23. Server BW reduction – two videos select top two most popular videos ~800,000 views during April, 2006 significant server bandwidth reduction using peer assistance less server BW even increase quality 3 times (@3x bitrate) 23 gold streamsilver stream P2P @3x

24. Server BW reduction – all videos 24 12,000+ videos server bandwidth reduction in all categories 1.23Gbps  36.9Mbps (97%) 1.23Gbps  770Mbps @3X bitrate (38%) April 2006

25. 25 Locality Aware P2P Delivery

26. P2P Traffic Today 26 1999 to present: fueled by Napster, KaZaA, eDonkey and BitTorrent CacheLogic Research Internet Protocol Breakdown 1993 - 2006

27. Internet Traffic on the Rise 27 Internet traffic trend: grow at a compound monthly average of 7.4% in 2006 Internet traffic doubles per year Traffic at Amsterdam Internet Exchange (AMS-IX)

28. Locality to the Rescue 28 Internet Hierarchy AS ISP POP Home/corporation Branch office of a corporation Delivery content in a locality aware fashion Beyond ISP aware delivery

29. Internet : Grand View 29

30. Impact on ISPs 30 Tier 1 ISP Tier 2 ISP AS sibling peering peering entity boundary sibling entity boundary transit Tier 2 ISP AS  Economics of ISP relationships  sibling relationship several ISPs belong to same org  peering relationship mutual beneficial free agreement (to certain extent)  transit relationship one ISP pays another

31. Inside ISP 31

32. ISP POP (Point of Presence) 32

33. Inside Home/Branch Office 33 neighborhood homecorporation Branch office

34. Identify Peers Locality 34 Information used External IP address Internal IP address Subnet mask Peer locality ISP (AS) ISP POP Home/corporation Corporate branch office Peers are considered closer if they are in a smaller common neighborhood

35. MAP External IP Address to AS 35 Using BGP dump

36. Identify POP 36 POP neighborhood Identify one peer that is directly connected to the Internet at some point of time Collect its external IP address and the subnet mask Infer the subnet neighborhood where other peers belong, even if they are not directly connected to the Internet

37. Below POP 37 Home/corporation neighborhood All peers with the same external IP address Corporation branch office All peers with the same external IP address, and on the same internal subnet (based on subnet mask)

38. Locality Aware Topology Building 38 Preferentially link peers within the same ISP neighborhood Say if we need to establish 20 connections We assign 50% of links to be within branch office neighborhood If there are less peers than the allocated links, we simply put the unused links back to the pool We then assign 50% of unused links to be within home/office neighborhood The next 50% of unused links are assigned within POP neighborhood The next 50% of unused links are assigned within AS neighborhood The rest of the links are used for cross-AS connections

39. Example 39 ScenarioNeighborhoodBranch officeHome/ corporation ISPASOutside AS 1 Total peers208009009000 Connected peers105032 2 Total peers01010010009000 Connected peers010532 3 Total peers02010009000 Connected peers02099

40. Locality Aware P2P Scheduling 40 Preferentially deliver content to peers within closer neighborhood Propagate neighborhood availability information Exchange with a outside peer preferentially content that is not available in the neighborhood

41. Preliminary Result: ISP Friendly Without ISP-friendly Much more cross sibling than peering boundary Significant crossing boundary traffic 41 Without ISP-friendly

42. Preliminary Result: ISP Friendly Pure ISP-friendly 1 video  5000+ separate distributions still surprising reductions but unnecessarily conservative ISP could help by sharing information 42 svr rate (Mbps) no P2P sibling partition peering partition silver39.019.615.8 top 10295.290.375.1 cut cross boundary traffic completely

43. 43 Conclusions

44. Conclusion 44 Peer assisted delivery is the way to go for mass content delivery over the world Peer assistance can significantly reduce server bandwidth requirement Demonstrated in real world for file sharing & broadcast Shown in our work for on demand streaming Locality aware P2P delivery is the way to scale