Jin Li, Principal Researcher (Collaborators: Cheng Huang, Keith Ross) Communication and Collaboration Systems Microsoft Research 1

2 Introduction: Internet Video on the Rise

Internet Video is on the Rise Video streams served increased 38.8% in 2006 to billion (Source: AccuStream iMedia Research) 53 web-video startup in 2006 (US), $521M VC funding (Source: DowJones VentureOne) Major studio goes into Web Video $410M video ads revenue in 2006 and grow by 89% in 2007 (0.6% of $74B TV ad market, Internet ads $16.4B in 2006, expect to grow 19% in 2007) [source: Emarketer] 3

Internet Video is Growing: in Popularity & Quality 4 Apr. 2006Dec. 2006Up (%) # of views (million) # of users (million) video quality evolution popularity evolution

5 Internet Video Delivery: Data Center vs. CDN vs. P2P

Just Build More Powerful Data Center? 6

Data Center Capacity VHS quality video streaming: 500 kbps (H.264) 200,000 viewers = 100 Gbps Data center capacity: Tera Grid (UIUC) 30 petabyte of storage 40 Gbps backbone: 80k video viewers 7

CDN Is Not The Answer 8 Akamai 20,000 servers, 900 point of presence, 71 countries 400Gbps bandwidth Network optimized for latency Limelight 25 point of presence, hundreds servers per presence 1,000Gbps bandwidth Akamai+Limelight: 2.8 million viewers. Current TV audience Olympics: 2.5 billion viewers Each viewer may have his/her own interest (different sport event, athlete nationality, etc.)

Peer Assisted Delivery is the Way To Scale 9 Economical to run Saves server/CDN bandwidth, disk I/O, CPU, memory Robust no single point of failure in network Super-scalable system capacity increases with number of nodes peer resource bandwidth CPU memory hard drive

P2P Benefit Consumers: Better Video Quality, More Selection 10

11 Case Study 1: On Demand Internet Video

Peer Assisted Delivery: Mode File Sharing broadcast On Demand Streaming (Interactive TV) Live Messenger FolderShare Groove 12

Peer Buffer Map: File Sharing 13 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

Peer Buffer Map: Broadcast 14 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

Peer Buffer Map: On Demand 15 Peer 1 Peer 2 Peer 3 Peer 4 Peer 5 Peer 6

MSN VoD Service 16 Traces from the on-demand service of MSN Video 9-month period: Apr. – Dec M streaming requests 59,000 unique videos

Peer-assisted VoD Model 17 Guaranteed QoS: always available server Performance metric: server bandwidth Peers upload what / when they are watching conservative assumption servers in data centers/CDN

Peer Bandwidth Download BW is measured by Windows Media Server no accurate measurement beyond 3.5Mbps Upload BW is inferred Average upload: 500+ kbps 18 DSL2 Cable Ethernet Modem ISDN DSL1

Bandwidth Allocation Policies 19 Assumptions peers always start watching from beginning VoD: earlier peers upload to later peers 1 st policy: no-prefetching only satisfy demand for smooth playback, do not further build up the buffer used by commercial live streaming companies to offer VoD servers in data centers arrival ask ask server ask ask server 3 2 1

Bandwidth allocation policies (2) 20 Prefetching – to utilize remaining upload capacity 2 nd policy: water-leveling 3 rd policy: greedy Lower bound allow later peers upload to earlier ones no arrival order constraints servers in data centers arrival water-leveling: greedy: water-leveling: greedy: 4 2

Observations on Policies (Simulated: Peer Poisson Arrival) Prefetching is crucial “free” to increase video bitrate “balanced mode” is most difficult S ≈ D Greedy policy works best lowest server load very close to bound more available upload

Server BW Reduction – Two Videos select top two most popular videos ~800,000 views during April, 2006 significant server bandwidth reduction using peer assistance less server BW even increase quality 3 times bitrate) 22 gold streamsilver stream

Server BW reduction – two videos select top two most popular videos ~800,000 views during April, 2006 significant server bandwidth reduction using peer assistance less server BW even increase quality 3 times bitrate) 23 gold streamsilver stream

Server BW reduction – all videos 24 12,000+ videos server bandwidth reduction in all categories 1.23Gbps  36.9Mbps (97%) 1.23Gbps  bitrate (38%) April 2006

25 Locality Aware P2P Delivery

P2P Traffic Today to present: fueled by Napster, KaZaA, eDonkey and BitTorrent CacheLogic Research Internet Protocol Breakdown

Internet Traffic on the Rise 27 Internet traffic trend: grow at a compound monthly average of 7.4% in 2006 Internet traffic doubles per year Traffic at Amsterdam Internet Exchange (AMS-IX)

Locality to the Rescue 28 Internet Hierarchy AS ISP POP Home/corporation Branch office of a corporation Delivery content in a locality aware fashion Beyond ISP aware delivery

Internet : Grand View 29

Impact on ISPs 30 Tier 1 ISP Tier 2 ISP AS sibling peering peering entity boundary sibling entity boundary transit Tier 2 ISP AS  Economics of ISP relationships  sibling relationship several ISPs belong to same org  peering relationship mutual beneficial free agreement (to certain extent)  transit relationship one ISP pays another

Inside ISP 31

ISP POP (Point of Presence) 32

Inside Home/Branch Office 33 neighborhood homecorporation Branch office

Identify Peers Locality 34 Information used External IP address Internal IP address Subnet mask Peer locality ISP (AS) ISP POP Home/corporation Corporate branch office Peers are considered closer if they are in a smaller common neighborhood

MAP External IP Address to AS 35 Using BGP dump

Identify POP 36 POP neighborhood Identify one peer that is directly connected to the Internet at some point of time Collect its external IP address and the subnet mask Infer the subnet neighborhood where other peers belong, even if they are not directly connected to the Internet

Below POP 37 Home/corporation neighborhood All peers with the same external IP address Corporation branch office All peers with the same external IP address, and on the same internal subnet (based on subnet mask)

Locality Aware Topology Building 38 Preferentially link peers within the same ISP neighborhood Say if we need to establish 20 connections We assign 50% of links to be within branch office neighborhood If there are less peers than the allocated links, we simply put the unused links back to the pool We then assign 50% of unused links to be within home/office neighborhood The next 50% of unused links are assigned within POP neighborhood The next 50% of unused links are assigned within AS neighborhood The rest of the links are used for cross-AS connections

Example 39 ScenarioNeighborhoodBranch officeHome/ corporation ISPASOutside AS 1 Total peers Connected peers Total peers Connected peers Total peers Connected peers02099

Locality Aware P2P Scheduling 40 Preferentially deliver content to peers within closer neighborhood Propagate neighborhood availability information Exchange with a outside peer preferentially content that is not available in the neighborhood

Preliminary Result: ISP Friendly Without ISP-friendly Much more cross sibling than peering boundary Significant crossing boundary traffic 41 Without ISP-friendly

Preliminary Result: ISP Friendly Pure ISP-friendly 1 video  separate distributions still surprising reductions but unnecessarily conservative ISP could help by sharing information 42 svr rate (Mbps) no P2P sibling partition peering partition silver top cut cross boundary traffic completely

43 Conclusions

Conclusion 44 Peer assisted delivery is the way to go for mass content delivery over the world Peer assistance can significantly reduce server bandwidth requirement Demonstrated in real world for file sharing & broadcast Shown in our work for on demand streaming Locality aware P2P delivery is the way to scale