Schedules and Overlays: Interactions Between Information Scheduling and Topology Management in P2P Streaming Renato Lo Cigno University of Trento (UNITN) Torino, 21th January 2011

NAPA-WINE Final Workshop – Torino Jan 20-21, Agenda Selecting peers and planning transfers: topology management or information scheduling? Scheduling in structured systems: What is the freedom? The NAPA-WINE architecture: a reprise An approach based on offer/trade protocol Analysis of Push based scheduling on quasi- random topologies

NAPA-WINE Final Workshop – Torino Jan 20-21, Structures or Swarms Structured systems follow regular topologies  Imply a (relatively) small number of contacts  Restrict freedom in scheduling information  Simplify scheduling choices (if any) Swarms do not have a regular topology  Large number of neighbors  Reduced sensitivity to topology changes (churn)  Multiple choices for information exchange  Resilient but … can they perform?

NAPA-WINE Final Workshop – Torino Jan 20-21, Structured systems: Scheduling follows topology Trees & Multi-Trees The burden or performance is entirely on the tree maintenance algorithm Non bandwidth-optimal (trees)

NAPA-WINE Final Workshop – Torino Jan 20-21, Structured systems: Scheduling follows topology Regular trees (and forests) Suffer from heterogeneity Trees & Multi-Trees The burden or performance is entirely on the tree maintenance algorithm Non bandwidth-optimal (trees)

NAPA-WINE Final Workshop – Torino Jan 20-21, Structured systems: Scheduling follows topology Uneven/unbalanced trees are hard to manage/predict Trees & Multi-Trees The burden or performance is entirely on the tree maintenance algorithm Non bandwidth-optimal (trees)

NAPA-WINE Final Workshop – Torino Jan 20-21, Structured systems: Scheduling follows topology Leaves do not contribute (trees) Multi-trees are hard to match Trees & Multi-Trees The burden or performance is entirely on the tree maintenance algorithm Non bandwidth-optimal (trees)

NAPA-WINE Final Workshop – Torino Jan 20-21, Structured systems: Scheduling follows topology Hypercubes Bandwidth-optimal in homogeneous settings  Minimum delivery delay when bandwidth=stream rate What if nodes are not 2 N ? What if resources are not homogeneous? Even more complex than trees to manage

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Scheduling decisions are required The information flow does not follow the structure of the topology Peers have a neighborhood, not parents & children ¿ What chunk to send / request ? ¿ To / From which peer ? ¿ When ?

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Blind Selection Normally based on Pull  Blind Push is too wasteful  Derived from file-sharing and BitTorrent Choose neighbors at random (nearly/biased)  Often following a selfish utility function Search in neighborhood the information (chunks) more needed from the peers that gives them fast  Bad Neighborhood? Sorry, watch TV next time … or try to change it  Inefficient Neighborhood? The networks (and other peers) will suffer … who cares!

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Blind Selection ? ?

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Beyond Random Pull Required functionalities  A Peer discovery protocol If good gives a random topology (may include a tracker)  A chunk exchange protocol Includes decisions on When / Who / What and Pull / Push Additional (useful) functionalities  A Topology Management protocol Obtains topologies with given characteristics May make use of ALTO-like oracles  A robust, efficient, chunk/peer scheduler Must be coupled to the Topology Management Requires knowledge (signaling overhead) Can be Pull/Push or mixed

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Useful Selection Signaling for the exchange of peer “ state ” is required  Playout point  Buffermaps  Advices on improving each other neighborhoods Main decision about the logical structure:  Pull (BitTorrent-like) information retrieval  Push information diffusion  Mixed

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Useful Selection ? ? ? ?

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Useful Selection

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Useful Selection

NAPA-WINE Final Workshop – Torino Jan 20-21, Unstructured systems: Random Topologies & Useful Selection

NAPA-WINE Final Workshop – Torino Jan 20-21, NAPA-WINE architecture Scheduler layer Overlay Layer Active peers’ InfoBase Chunk buffer Peer Selection Trading Logic Neighbour set Topology controller Peer-Rep Net-Rep Ext-Rep REP controller Video Source(s)Display(s) User Layer Content Ingestion PlayerControl Interface NAPA-WINE Second Video Conference 22 Oct IPv4 / IPv6 + UDP / TCP / SCTP /... Messaging Layer + NAT/FW traversal Monitoring layer Monitoring Controller Pasv. measAct. meas

NAPA-WINE Final Workshop – Torino Jan 20-21, Monitoring layer Monitoring Controller Topology; Scheduling and Monitoring Scheduler Overlay Management Active peers’ InfoBase Chunk buffer Peer Selection Trading Logic Neighbor set Topology controller To/From ALTO

NAPA-WINE Final Workshop – Torino Jan 20-21, Monitoring layer A more abstract view The T-Man offers a set of peers The Scheduler feedbacks good ones / bad ones T-Man offers more & merge based on feedbak Neighbors T-Man Scheduler

NAPA-WINE Final Workshop – Torino Jan 20-21, NAPA-WINE architecture: Flexibility at Work Different implementations of  Topology Controller  Peer Selection  Trading Logic Give rise to different architectures and interaction models of Topology Management and Scheduling  Random Topology with Offer/Trade  Random Topology with Deadline Based Chunk Push and Earliest/Latest Peer selection (DLc/ELp … more later)  ALTO-based Topology and Offer/Trade ...

NAPA-WINE Final Workshop – Torino Jan 20-21, Random Topology and Offer Trade T-Man try to build a random graph with high connectivity (Nn neighbors) The scheduler  Offers buffermaps with owned chunks to Np<<Nn peers  Answer FIFO to requests selecting chunks from the buffermao  Modify Np to keep a small stable queue of chunk transmissions

NAPA-WINE Final Workshop – Torino Jan 20-21, Predictive trading time Chunk #1 to Peer 2 Chunk #1 to Peer 5 Chunk #2 to Peer 1 New Chunk Arrival Receive Select Send Offer Chunk Transmission  T diff is the time between a new chunk arrival and the moment in which the tx queue becomes empty.  T offer is the time between a new chunk arrival and the moment in which starts a new offer session.  T queue is the interval that runs from the reception of last select message until the moment in which the tx queue becomes empty.  N p is the number of neighbors that a peer contacts in every offer session. T diff T offer T queue NpNp

NAPA-WINE Final Workshop – Torino Jan 20-21, The Adaptative Signaling Protocol Crucial parameters  N p should match peer upload capacity If N p is too small  Peers’ upload bandwidth is not exploited at best  The transmission queue empties quickly  Long periods of inactivity If N p is too large  Transmission queue becomes too long  Large delivery delays and, possibly, losses  A lot of signaling overhead is produced  T diff should match the minimum RTT among neighbors to avoid long idle periods (this means T offer = T queue = 0 )

NAPA-WINE Final Workshop – Torino Jan 20-21, Comparison with fixed N p schemes Chunk loss probability varying ρ.

NAPA-WINE Final Workshop – Torino Jan 20-21, Comparison with fixed N p schemes II 26 Total number of received signaling messages versus ρ

NAPA-WINE Final Workshop – Torino Jan 20-21, Offer/Trade scheduling A form of Random Peer selection  Bias towards peers that are closer & with more bandwidth (faster to answer)  Blind, or with explicit preference for peers with higher upload (must be diffused with signaling) Chunk selection left to the destination  Selfish – does not take into account further distribution chances  Random Useful (results shown) or Latest Useful

NAPA-WINE Final Workshop – Torino Jan 20-21, More on Scheduling Lots of works done  Most ignore interaction with topology, assuming either full mesh or random with high connectivity  Most ignore difference between Pull (the receiver decides what to receive and from whom) Push (the sender decides what to send and to whom) Explored in NAPA-WINE  Push/Pull differences; Mixed schemes  Chunk or Peer First strategies  One distributed scheme, based on “Push, Chunk First” has been proven optimal

NAPA-WINE Final Workshop – Torino Jan 20-21, Scheduling Chunks and Peers Chunks  Random Useful (RU)  Latest Useful (LU) – found fragile for pushing  Earliest Useful (EU)  DeadLine (DL), updated dynamically Peers  Random Useful (RU)  Most Deprived (MD)  Earliest Latest (EL) DLc/ELp Proven Optimal All can be combined with “network awareness” (bandwidth, delay,...) thus interacting with and modifying the topology

NAPA-WINE Final Workshop – Torino Jan 20-21, DLc/ELp (push, chunk first) DLc: select the chunk i with the minimum deadline d i  d i = chunk emission time + T’  Ntx  T’: any time larger than the chunk duration T  Ntx: number of times this copy or the chunk has been retransmitted by any peer  Intuition: the smallest deadline identifies a chunk that is old AND has not diffused in the system ELp: select the Peer which has the oldest most recent chunk  It is the peer which is distributing the oldest information  Intuition: it is the peer that will be the first to start distributing the chunk we give it

NAPA-WINE Final Workshop – Torino Jan 20-21, DLc/ELp (push, chunk first) Distributed, but requires buffermap diffusion Robust to small neighborhoods Sensitive to stale buffermaps  Confirmation before sending? OPTIMAL  Uniform scenario with bandwidth = streaming rate  All peer receive all stream with delay t d t d <  log N + 1 

NAPA-WINE Final Workshop – Torino Jan 20-21, DLc/ELp: Worst case delay comparison DLc/ELp and LUc/ELp are optimal, but...

NAPA-WINE Final Workshop – Torino Jan 20-21, DLc/ELp: Worst case delay comparison... LUc/ELp is fragile in face of neighborhood reduction

NAPA-WINE Final Workshop – Torino Jan 20-21, More on Scheduling The goals we have are three 1. Minimize delays: Can we do better than offer/trade? 2. Minimize bandwidth wastes: No wrong decisions 3. Be Network-Aware: refine and optimize topology DL δ c/BA β ELP  δ is the postponing delay and can be used also for priorities (embedded in chunks – see the presentation on QoS for that)  β is a weight for the bandwidth parameter

NAPA-WINE Final Workshop – Torino Jan 20-21, Bandwidth-Aware ELp Algorithm weighted combination Maximize: t − L(P j, t) + β (s(P j )/s(P i ))  Where β is a weight assigned to the bandwidth component  L(P j, t) is the expected arrival of the chunk to P j, through the bandwidth of the sender s(P i )  Redistribution potential of P j through the bandwidth of the target peer s(P j ).

NAPA-WINE Final Workshop – Torino Jan 20-21, Uniform scenario

NAPA-WINE Final Workshop – Torino Jan 20-21, th percentile as a function of heterogeneity with 3 classes of users Bandwidth-Aware ELp Algorithm: Sample results 600 peers Nn 20

NAPA-WINE Final Workshop – Torino Jan 20-21, Selected Literature (from NAPA-WINE) C. Kiraly, R. Lo Cigno, and L. Abeni, “Deadline-based Differentiation in P2P Streaming,” in IEEE GLOBECOM 2010, Miami, Florida, USA, Dec A. Carta, M. Mellia, M. Meo, and S. Traverso, “Efficient Uplink Bandwidth Utilization in P2P-TV Streaming Systems,” in IEEE GLOBECOM 2010, Miami, Florida, US), Dec J. Seedorf, S. Niccolini, M. Stiemerling, E. Ferranti, and R. Winter, “Quantifying operational Cost-Savings through ALTO- Guidance for P2P Live Streaming,” in 3rd Workshop on Economic Traffic Management (ETM 2010), Sept R. Fortuna, E. Leonardi, M. Mellia, M. Meo, and S. Traverso, “QoE in Pull Based P2P-TV Systems: Overlay Topology Design Tradeoffs,” in Proceedings of the 10th International Conference on Peer-to-Peer Computing 2010 (P2P'10), Delft, The Netherlands, August 2010

NAPA-WINE Final Workshop – Torino Jan 20-21, Selected Literature (from NAPA-WINE) A. Russo and R. Lo Cigno, “Delay-Aware Push/Pull Protocols for Live Video Streaming in P2P Systems,” in IEEE ICC 2010, Cape Town, South Africa, May 2010 L. Abeni, C. Kiraly, and R. Lo Cigno, “Robust Scheduling of Video Streams in Network-Aware P2P Applications,” in IEEE ICC 2010, Cape Town, South Africa, May 2010 C. Kiraly, L. Abeni, and R. Lo Cigno, “Effects of P2P Streaming on Video Quality,” in IEEE ICC 2010, Cape Town, South Africa, May 2010 A. Couto da Silva, E. Leonardi, M. Mellia, and M. Meo, “Chunk Distribution in Mesh-Based Large Scale P2P Streaming Systems: a Fluid Approach,” IEEE Transactions on Parallel and Distributed Systems, To appear

NAPA-WINE Final Workshop – Torino Jan 20-21, Selected Literature (from NAPA-WINE) R. Birke, E. Leonardi, M. Mellia, A. Bakay, T. Szemethy, C. K. amd R. Lo Cigno, F. Mathieu, L. Muscariello, S. Niccolini, J. Seedorf, and G. Tropea, “ Architecture of a Network-Aware P2P-TV Application: the NAPA-WINE Approach,” IEEE Communication Magazine, To appear A. Couto da Silva, E. Leonardi, M. Mellia, and M. Meo, “Exploiting Heterogeneity in P2P Video Streaming,” IEEE Transactions on Computers, vol. To appear F. Mathieu and D. Perino, “On resource aware algorithms in epidemic live streaming,” in 22nd International Teletraffic Congress (ITC22), Amsterdam, NL, 2010 M. Stiemerling and S. Kiesel, “A System for Peer-to-Peer Video Streaming in Resource Constrained Mobile Environments,” in CoNext U-NET Workshop, Rome, IT, Dec. 2009

NAPA-WINE Final Workshop – Torino Jan 20-21, Selected Literature (from NAPA-WINE) L. Abeni, C. Kiraly, and R. Lo Cigno, “Scheduling P2P Multimedia Streams: Can We Achieve Performance and Robustness?,” in IMSAA 2009, Bangalore, India, Dec L. Abeni and A. Montresor, “Scheduling in P2P Streaming: from Algorithms to Protocols,” in IWSOS 2009, Zurich, CH, Dec J. Seedorf, S. Kiesel, and M. Stiemerling, “Traffic Localization for P2P-Applications: The ALTO Approach,” in IEEE P2P 2009, Seattle, OR, USA, Sept L. Abeni, C. Kiraly, and R. Lo Cigno, “SSSim: a Simple and Scalable Simulator for P2P Streaming Systems,” in 14th IEEE CAMAD, Pisa, Italy, June 2009

NAPA-WINE Final Workshop – Torino Jan 20-21, Selected Literature (from NAPA-WINE) R. Lobb, A. P. Couto da Silva, E. Leonardi, M. Mellia, and M. Meo, “Adaptive Overlay Topology for Mesh-Based P2P-TV Systems,” in ACM NOSSDAV 2009, Williamsburg, VA, USA, June 2009 C. Kiraly and R. Lo Cigno, “On the Effects of Overlay Localization on P2P Networks,” in IEEE Infocom 2009 Student Workshop, Rio de Janeiro, BR, Apr A. Russo and R. Lo Cigno, “Push/Pull Protocols for Streaming in P2P Systems,” in IEEE Infocom 2009 Student Workshop, Rio de Janeiro, BR, Apr L. Abeni, C. Kiraly, and R. Lo Cigno, “On the Optimal Scheduling of Streaming Applications in Unstructured Meshes,” in IFIP Networking, Aachen, DE, May 2009

NAPA-WINE Final Workshop – Torino Jan 20-21, 2011 THE END Thank you! Questions? Comments?