1. Multimedia Streaming Protocols1 Multimedia Streaming: Jun Lu Xinran (Ryan) Wu CSE228 Multimedia Systems Challenges and Protocols

2. Multimedia Streaming Protocols2 Presentation Outline Architecture for Multimedia Streaming Summary of Challenges and Techniques Protocol Stacks Quick Introduction to RTSP, RTP/RTCP and RSVP Reference

3. Multimedia Streaming Protocols3 What is Multimedia Streaming? Streaming ServerClient/Receiver Storage Device Compressed Video Compressed Video Compressed Audio Compressed Audio Multimedia Streaming Protocols Transport Protocols Transport Protocols Raw Video Raw Video Raw Audio Raw Audio Internet Video Decoder Video Decoder Multimedia Streaming Protocols Transport Protocols Transport Protocols Audio Decoder Audio Decoder Multimedia Streaming: Clients request audio/video files from servers and pipeline reception over the network and display User’s perspective: Quick start without waiting for full download. Coming continuously without interruption. VCR operation (pause, resume, fast forward, rewind, etc.)

4. Multimedia Streaming Protocols4 Challenges in Media Streaming Protocols Streaming Server Ethernet Clients/Receivers Broadband Modem 1. Rate Control: Determine the sending rate based on the available bandwidth in the network. 2. Error Control: Improve video presentation quality in the presence of packet loss. 3. Continuous Distribution: TCP/UDP/IP suite provides best-effort, no guarantees on expectation or variance of packet delay

5. Multimedia Streaming Protocols5 Techniques in Multimedia Streaming Protocols (1) Rate Control Scalable compression Base substream and enhancement substreams. SNR scalability / spatial scalability / temporal scalability Rate filter Frequency filter Frame-dropping filter Re-quantization filter QoS Feedback, e.g. RTCP.

6. Multimedia Streaming Protocols6 Techniques in Multimedia Streaming Protocols (2) Error Control Add redundant data in coding MDC, (Multiple Description Coding) FEC (Forward Error Coding) Receiver End Error Concealment Receiver conceal data loss. Spatial interpolation, used in intra-coded frame. Temporal interpolation, used in inter-coded frame.

7. Multimedia Streaming Protocols7 Techniques in Multimedia Streaming Protocols (3) Continuous Distribution Network Filter, e.g. frame-dropping filter. Put filters at nodes connected to network bottleneck. Content Replication, caching and mirroring. Resource Reservation: RSVP

8. Multimedia Streaming Protocols8 A General View of the Multimedia Streaming Protocols Stream descriptionSDP, SMIL... Describe the session and content Stream controlRTSP Remote control the session Media transportRTP Error control and flow control Resource reservation (if any!): RSVP, DiffServ provide QoS for media streaming packets

9. Multimedia Streaming Protocols9 Protocol stacks for media streaming Internet Compressed Video/Audio IPv4, IPv6 UDP RTP Layer Data Plane Control Plane RTCP Layer RTSP Layer TCP

10. Multimedia Streaming Protocols10 Real Time Streaming Protocol (RTSP) SETUP rtsp://xxxxxxxxx:554/rtsp.wav/streamid=0 RTSP/1.0 CSeq: 3 Transport: rtp/avp;unicast;client_port=6970-6971 RTSP/1.0 200 OK CSeq: 3 Date: Thu, 29 May 2003 19:30:44 GMT Session: 9978-3 Transport: rtp/avp;unicast;client_port=6970 6971;server_port=10620-10621

11. Multimedia Streaming Protocols11 Real Time Streaming Protocol (RTSP) (Con) Supports the following operations: Supports VCR-like control operations Chooses delivery channels, e.g. UDP, TCP. Supports any session description. Establish and control stream of continuous audio and video media. Retrieves requested media. Add Media to an existing session.

12. Multimedia Streaming Protocols12 Real-Time Protocol (RTP/RTCP) RTP is a data transfer protocol and RTCP is a control protocol. RTP provides Payload type identification: Identify which kind of information is being transmitted, RTP provides 128 possible different types of encoding; eg MPEG2 video, etc. Sequencing: Reassemble the stream and detect packet loss. Timestamping: Assure synchronization. Source identification: Provide a means for the receiver to distinguish different sources.  RTP DO NOT provide  Quality of service  Reliability in packet delivery.  Security. Payload Type Sequence Number Timestamp Source Identifier Misc.

13. Multimedia Streaming Protocols13 Real-Time Protocol (RTP/RTCP) (Con) Exchange information about the connection to assure quality of data distribution. Control connection is held over a different channel than the RTP one. RTCP provides QoS Feedback: In form of sender reports/receiver reports. Senders adjust transmission rate based on reports. Participant Identification: Human-friendly source identification. Control Packets Scaling: Typically, limit the RTCP bandwidth to 5% of the session bandwidth, divided between the sender reports (25%) and the receivers reports (75%) Minimal Session Control Information: Advanced control functions must be implemented in a higher level protocol.

14. Multimedia Streaming Protocols14 Quality of Service – Use of RSVP Access Backbone Diffserv Region Per flow policing DSCP marking Classify & schedule based on DSCP RSVP signalling Trust Boundary RSVP is used for signaling end to end (admission control based on bandwidth, QOS requirements)

15. Multimedia Streaming Protocols15 Selected bibliography General Schulzrinne, IRT (Internet Real-Time) lab, http://www.cs.columbia.edu/~hgs/research/IRT RTP Schulzrinne, Casner, Frederick, Jacobson, « RTP, a tranport protocol for real-time applications », IETF, work in progress,, March 2001. Schulzrinne, RTP home page, http://www.cs.columbia.edu/~hgs/rtp/ RTSP Schulzrinne, Rao, Lanphier, « Real-Time Streaming Protocol (RTSP) », IETF, Request For Comments 2327, April 1998. Schulzrinne, RTSP home page, http://www.cs.columbia.edu/ ~hgs/rtsp/