Challenges, Design and Analysis of a Large-scale P2P-VoD System Dr. Yingwu Zhu

P2P Overview Advantage – Reduce server load by having peers contributing their resources – Scalability, availability, robustness, … P2P services –P2P file downloading : BitTorrent and Emule –P2P live streaming : Coolstreaming and PPLive –P2P video-on-demand (P2P-VoD) : PPLive Unlike P2P streaming systems, P2P-VoD systems lacks synchrony, as peers can watch different parts of a video at the same time. Require each user to contribute a small amount of storage (usually 1GB) instead of only the playback buffer in memory as in the P2P streaming system

Challenges in P2P VoD Lack of synchrony in user behavior Content replication to provide movie data for other peers – Data availability by caching at peers – ATD: Availability to Demand Ratio – Cache replacement Transmission strategy to deliver data in real time – Satisfaction level – Playrate

PPLive-VoD Architecture Design considerations Measurement study

Architecture Major components –Peers –Content Servers : the source of content –Trackers : help peers connect to other peers to share the same content –A bootstrap server : helps peers to find a suitable tracker and to perform other bootstrapping functions –Other servers log servers : log significant events for data measurement transit servers : help peers behind NAT boxes

Design Decisions Segment sizes Content replication strategy Content Discovery Piece Selection Transmission Strategy Other issues

Segment sizes How to divide a video into multiple pieces –Small segment size gives more flexibility to schedule which piece should be uploaded from which neighboring peer. –The larger the segment size the smaller the overhead. Header overhead (checksum) Bitmap overhead (advertisement) Protocol overhead (TCP/IP header) –The video player expects a certain minimum size for a piece of content to be viewable (playback rate: frame rate). Segmentation of a movie in PPLive’s VoD syste m

Replication Strategy Goal –To make the chunks as available to the user population as possible to meet users’ viewing demand while without incurring excessive additional overheads Considerations –Whether to allow multiple movies be cached Multiple movie cache (MVC) / single movie cache (SVC) –Whether to pre-fetch or not (NO! due to short view duration) –Cache replacement: Which chunk/movie to remove when the disk cache is full (movie-based replacement) Least recently used (LRU) / least frequently used (LFU) Weight-based replacement (How complete the movie is cached locally; and ATD = c/n  from tracker) Reduce sever load from 19% to 11-7%

Content Discovery Content advertising and look-up methods –Trackers keep track of which peers replicate a given movie As soon as a user starts watching a movie, the peer informs its tracker that it is replicating that movie. Also inform tracker of content removal When a peer wants to start watching movie, it goes to the tracker to find out which other peers have that movie. –Gossip method Discovering where chunks are is by the gossiping chunk bitmap. This cuts down on the reliance on the tracker, and makes the system more robust. –DHT Used to automatically assign movies to trackers to achieve some level of load balancing.

Piece Selection Which piece to download first –Sequential : select the piece that is closest to what is needed for the video playback First priority –Rarest first : selecting the rarest piece helps speeding up the spread of pieces, hence indirectly helps streaming quality. Second priority –Anchor-based : when a user tries to jump to a particular location in the movie, if the piece for that location is missing then the closest anchor point is used instead. Currently not used User do not jump much, 1.8 times/movie Optimizing transmission alg., buffering time after a jump is satisfactory

Transmission Strategy Goals –Maximize downloading rate –Minimize the overheads due to duplicate transmissions and requests Strategies (by levels of aggressiveness) –A peer can send a request for the same content to multiple neighbors simultaneously –A peer can request for different content from multiple neighbors simultaneously (PPLive’s choice) For playback rate of 500Kbps, 8-20 neighbors is the best. More than this number can still improve the achieved rate, but at the expense of heavy duplication rate. –A peer can work with one neighbor at a time.

Other Design Issues NAT and firewalls –Discovering different types of NAT boxes (similar to STUN protocol) –Pacing the upload rate and request rate (avoid being labeled as attacks and blocked) Content authentication –Chunk level authentication Some pieces may be polluted and cause poor viewing experience locally at a peer. If a peer detects a chunk is bad, discard it. –Piece level authentication

What to measure User behavior –includes the user arrival patterns, and how long they stayed watching a movie –used to improve the design of the replication strategy External performance metrics –includes user satisfaction and server load –used to measure the system performance perceived externally Health of replication –measures how well a P2P-VoD system is replicating a content –Used to infer how well the important component of the system is doing

User Behavior MVR (movie viewing record)

User Satisfaction Simple fluency –measures the fraction of time a user spends watching a movie out of the total time he spends waiting for and watching that movie R(m, i) : the set of all MVRs for a given movie m and user i n(m, i) : the number of MVRs in R(m, i) r : one of the MVRs in R(m, i)

User Satisfaction (cont’) User satisfaction index –considers the quality of the delivery of the content r(Q) : a grade for the average viewing quality for an MVR r, using infer/estimate

Health of Replication Three levels –Movie level The number of active peers who have advertised storing chunks of that movie The information that the tracker collects about movies –Weighted movie level Considers the fraction of chunks a peer has in computing the index –Chunk bitmap level (chunk vector) The number of copies each chunk of a movie is stored by peers Various other statistics can be computed; the average number of copies of a chunk in a movie, the minimum number of chunks, the variance of the number of chunks.

Statistics on video objects Overall statistics of the three typical movies

Statistics on user behavior (1) Interarrival time distribution of viewers

Statistics on user behavior (2) View duration distribution Many MVRs <10 mins Prefetching?

Statistics on user behavior (3) How long does a user stay in system? – important to data replication >=70% users stay longer than 15 mins, able to provide upload serv.

Statistics on user behavior (4) Start position distribution

Observations from last slide A large fraction of users start watching movies from beginning User jump to different positions uniformly, so anchor points for jump operations could be uniformly spaced, guide for chunk selection strategy

Health index of Movies (1) Number of peers that cahce the movie

Health index of Movies (2) Average owning ratios for different chunks (avg. over 24 hours)

What the observations imply? Many users watching movies from the beginning Many users do not finish the whole movie? Why? – Movie epilog!

Health index of Movies (3) Chunk availability and chunk demand

Health index of Movies (4) The available to demand ratios (good, >= 1)

User Satisfaction Index (1) Generating fluency index –The computation of F(m, i) is carried out by the client software. –The client software reports all MVRs and the fluency F(m, i) to the log server whenever a “stop-watching” event occurs. The STOP button is pressed Another movie/program is selected The user turns off the P2P-VoD software

User Satisfaction Index (2) The number of fluency records –A good indicator of the number of viewers of the movie

User Satisfaction Index (3) The distribution of fluency index ( =0.8 good) –Good, but need to improve buffering time!

Future works Further research in P2P-VoD systems –How to design a highly scalable P2P-VoD system to support millions of simultaneous users –How to perform dynamic movie replication, replacement, and scheduling so as reduce the workload at the content servers –How to quantify various replication strategies so as to guarantee a high health index –How to select proper chunk and piece transmission strategies so as to improve the viewing quality –How to accurately measure and quantify the user satisfaction level

Thinking What makes PPLive-VoD distinguish from BitTorrent? – From design perspectives – Similarity – Difference