Architectures for Peer-to-Peer Content Distribution Christos Gkantsidis Microsoft Research, Cambridge, UK Joint work with: Pablo Rodriguez, John Miller, Dinan Gunawardena, and others

Peer-to-Peer Applications Peer-to-Peer Content Distribution File Sharing Media Streaming Collaboration Distributed Computing TelephonyPlatforms BBC iMP (PeerChannel, WCF) JXTA P2P content distribution is hugely popular:  >40-60% of Internet traffic  Easy access to an amazing variety of content Challenges of P2P:  P2P traffic difficult to manage  ISP complaints  Dependence of free bandwidth e.g. from universities  Massive copyright infringement 2

Microsoft scenarios where P2P can help  Large scale software distribution:  Windows / Microsoft updates  Beta versions of software (e.g. Visual Studio)  Free software  MSN Video  File sharing (e.g. Foldershare, Groove, Live Messenger) Windows XP  P2P Name Resolution Protocol (PNRP)  P2P Graphing and Grouping  Peer Identity Manager Windows Vista  PNRP, Graphing and Grouping, Peer Identity Manager, P2P Contacts, People Near Me Windows Communication Foundation  Peer Channel Incidentally, MS has been interested in P2P for some time … 3

Typical Objections P2P does not bring enough value (especially relative the risk)! I know how to deploy servers, why learn something new? How do I control it? How will I monitor it? How can I provision my network? 4

Typical Objections P2P does not bring enough value (especially relative the risk)! I know how to deploy servers, why learn something new? How do I control it? How will I monitor it? How can I provision my network? External objections Respect network operators (e.g. ISPs) Protect content owners Impact of firewalls, NATs, etc 5

Peer-to-Peer Distribution Avalanche: P2P file distribution –Reduce download times, use network resources optimally –Secure distribution: Publisher and content protection Only authorized users can join the distribution networks –Cache-friendly distribution –Enable network locality RedCarpet: P2P Video-on-Demand –Watch movies while downloading … –… without loosing delivery efficiency. –New P2P capability! RewindTV: Enhanced Live TV –Watch live TV … –… and be able to re-play missed scenes (from other peers) 6

Peer-to-Peer Distribution Avalanche: P2P file distribution –Reduce download times, use network resources optimally –Secure distribution: Publisher and content protection Only authorized users can join the distribution networks –Cache-friendly distribution –Enable network locality 7

File Swarm = Block Scheduling 8

System’s progress in current File Swarming systems 9 (From Tian et al., Infocom’06) A lot of time spent at the beginning and finish of download: Beginning of download: finding good blocks to exchange End of download: discovering the last missing blocks

Network Coding Simplified 10 File to Transfer Block 1Block 2Block 3 Encoding

With Network Coding 11

System’s progress 12 Smooth download progress: –No start-up delay –No last-block problem

13 Bandwidth Contribution Easily withstands flash crowds Server contribution is fixed, Client contribution scales >10 fold savings in content provider’s bandwidth using peer-to-peer.

Results from distributing Visual Studio 14 Data from distribution of beta versions of Visual Studio 2008 Beta (Nov’07) Median speeds: ~1.5Mbps for VS Pro ~2.7Mbps for the others

Peer-to-Peer Distribution Avalanche: P2P file distribution –Reduce download times, use network resources optimally –Secure distribution: Publisher and content protection Only authorized users can join the distribution networks –Cache-friendly distribution –Enable network locality 15

Strong security model All connections TLS- authenticated Data is self-certifying –Block data validated by signed homomorphic hash equivalent Strong identity enables solid DOS defenses –Misbehaving parties cannot misrepresent themselves 16 Certificate Engine Registrars Publishers Clients Caches

Peer-to-Peer Distribution Avalanche: P2P file distribution –Reduce download times, use network resources optimally –Secure distribution: Publisher and content protection Only authorized users can join the distribution networks –Cache-friendly distribution –Enable network locality 17

ISP Cache Integration Protocol designed for cacheability –Network operators can redirect clients to Avalanche-compatible caches –Reduce ISP pain –Improve experience for clients, publishers Experience can be accelerated Cache can proxy or relay for poorly connected nodes 18

Effect of P2P on inter-ISP traffic 19 Transit ISP Broadband ISP $$ server Internet Increase in egress traffic!! $$ => $$$$$

Does locality work? 20 Transit ISP Broadband ISP $$ server Internet

Benefits of locality for current P2P scenarios Each ISP is required to upload just over a copy of the file (1.9 GB) May require internal re-engineering 21

Architecture for P2P distribution 22

Architecture for P2P distribution 23 Peer matching based on locality Provides extensibility points for ISPs and content owners Content caches Preference to local connections

Peer-to-Peer Distribution Avalanche: P2P file distribution –Reduce download times, use network resources optimally –Secure distribution: Publisher and content protection Only authorized users can join the distribution networks –Cache-friendly distribution –Enable network locality RedCarpet: P2P Video-on-Demand –Watch movies while downloading … –… without loosing delivery efficiency. –New P2P capability! RewindTV: Enhanced Live TV –Watch live TV … –… and be able to re-play missed scenes (from other peers) 24

Large Scale Content Delivery In current Peer-to-Peer systems: >60% of traffic is for video files Users need to wait to download the entire file before watching  Emerging: P2P Live Streaming: Very popular: >200K concurrent users in one swarm Huge selection of content (mostly Asian) 25

Large Scale Content Delivery Unstructured Peer-to-Peer –Scalability –Easy to construct/maintain Typical applications: –File distribution E.g. eMule, BitTorrent –Live streaming E.g. CoolStreaming et al. Q) Can we use unstructured P2P to deliver a video-on-demand experience? Our approach: Pseudo-sequential downloads Pseudo-sequential downloads Pre-fetching of data Pre-fetching of data Network coding for efficient swarming Network coding for efficient swarming Q) Can we use unstructured P2P to deliver a video-on-demand experience? Our approach: Pseudo-sequential downloads Pseudo-sequential downloads Pre-fetching of data Pre-fetching of data Network coding for efficient swarming Network coding for efficient swarming 26

Challenges of Video on Demand Small start-up times –Fast channel switching Good sustainable playback rate Users join at arbitrary points in time Video Seekingfuturefuture 27

Network Coding vs. Non-NetCoding Flash crowd of 20 clients Best non-Network Coding scheme –Global-rarest Network Coding –14% improvement in playback rates –Low jitter in download times  Better user experience Global rarest vs. Network Coding (Based on a prototype implementation) 28

Other topic: Topology Management 29 -Issue: Peers at different stages of the download. -Issue: Heterogeneous capacities: -Fast nodes should not be penalized. Data (%)

30 Mass Scale Content Distribution Multi-purpose P2P platform File Sharing: Avalanche Software Updates: DeltaCast Video-on-Demand: RedCarpet Live Streaming: RewindTV Network Coding Secure Distribution Topology Construction ISP Friendliness

Lessons learned P2P technologies are growing in acceptance –Being deployed by Movie Studios, Television broadcasters, Music companies, Software distributors P2P systems need to respect the current ecosystem –Network operators: locality, caching –Content producers: strong security Network coding is feasible and practical today, and leads to performance improvements The unstructured P2P paradigm helps even for video distribution 31