1. Multimedia and QoS#1#1 Multimedia Applications

2. Multimedia and QoS#2#2 Multimedia Applications r Multimedia requirements r Streaming r Recovering from Jitter and Loss r RTP

3. Multimedia and QoS#3#3 Application Classes r Typically: m sensitive to delay, but m can tolerate packet loss would cause minor glitches that can be concealed r Data contains audio and video content (“continuous media”), three classes of applications: m Streaming m Unidirectional Real-Time m Interactive Real-Time

4. Multimedia and QoS#4#4 Application Classes (more) r Streaming m Clients request audio/video files from servers and pipeline reception over the network and display m Interactive: user can control operation (similar to VCR: pause, resume, fast forward, rewind, etc.) m Delay: from client request until display start can be 1 to 10 seconds

5. Multimedia and QoS#5#5 Application Classes (more) r Unidirectional Real-Time: m similar to existing TV and radio stations, but delivery on the network m Non-interactive, just listen/view r Interactive Real-Time : m Phone conversation or video conference m More stringent delay requirement than Streaming and Unidirectional because of real-time nature m Video: < 150 msec acceptable m Audio: < 150 msec good, <400 msec acceptable

6. Multimedia and QoS#6#6 Multimedia Challenges r TCP/UDP/IP suite provides best-effort, no guarantees on expectation or variance of packet delay r Streaming applications delay of 5 to 10 seconds is typical and has been acceptable, but performance deteriorate if links are congested (transoceanic) r Real-Time Interactive requirements on delay and its jitter have been satisfied by over-provisioning (providing plenty of bandwidth), what will happen when the load increases?...

7. Multimedia and QoS#7#7 Challenges (more) r Most router implementations: m use only First-Come-First-Serve (FCFS) m Limited packet processing and transmission scheduling r To mitigate impact of “best-effort” protocols, we can: m Use UDP to avoid TCP and its slow-start phase… m Buffer content at client and control playback to remedy jitter m Adapt compression level to available bandwidth

8. Multimedia and QoS#8#8 Solution Approaches in IP Networks r Just add more bandwidth and enhance caching capabilities (over-provisioning)! r Need major change of the protocols : m Incorporate resource reservation (bandwidth, processing, buffering), and new scheduling policies m Set up service level agreements with applications, monitor and enforce the agreements, charge accordingly r Need moderate changes (“Differentiated Services”): m Use two traffic classes for all packets and differentiate service accordingly m Charge based on class of packets m Network capacity is provided to ensure first class packets incur no significant delay at routers

9. Multimedia and QoS#9#9 Encoding and compressing Audio r Pulse Code Modulation (PCM) m Sample at a fixed rate value is an arbitrary real number m Quantization: map samples to a given set of values m Encoding: encode the values as bits. r Examples: m sample rate 8000/sec. & 256 values : 64Kbps m CD: sample 44,100/sec. & 16 bits : 705.6 Kbps r Compressions: m Depend on the output rate: GSM (13 Kbps) G.729 (8 Kbps) G.723.3 (6.4 or 5.3 Kbps) MP3 (128 or 112 Kbps)

10. Multimedia and QoS#10 Video Compression r Raw rate: m images: 24 or 30 per second. m Size: 1024 x 1024 = 1 M pixels m pixel encoding 24 bit m Rate: 576 or 900 Mbps !!! r Compression schemes (many): m MPEG 1 (1.5 Mbps) CD quality m MPEG 2 (3-6 Mbps) DVD quality m Motion JPEG m H.261 (for ISDN) r Variable rate compression r Compression is a world of its own

11. Multimedia and QoS#11 Streaming r Important and growing application due to m reduction of storage costs, m increase in high speed net access from homes, m enhancements to caching and m introduction of QoS in IP networks r Audio/Video file is segmented and sent over either TCP or UDP, public segmentation protocol: Real-Time Protocol (RTP)

12. Multimedia and QoS#12 Streaming r User interactive control is provided, e.g. the public protocol Real Time Streaming Protocol (RTSP) r Helper Application: displays content, which is typically requested via a Web browser; e.g. RealPlayer; typical functions: m Decompression m Jitter removal m Error correction: use redundant packets to be used for reconstruction of original stream m GUI for user control

13. Multimedia and QoS#13 Streaming From Web Servers: I r Audio: in files sent as HTTP objects r Video (interleaved audio and images in one file, or two separate files and client synchronizes the display) sent as HTTP object(s) r A simple architecture is to have the Browser requests the object(s) and after their reception pass them to the player for display - No pipelining

14. Multimedia and QoS#14 Streaming From Web Server: II r Alternative: set up connection between server and player, then download r Web browser requests and receives a Meta File (a file describing the object) instead of receiving the file itself; r Browser launches the appropriate Player and passes it the Meta File; r Player sets up a TCP connection with Web Server and downloads the file

15. Multimedia and QoS#15 Meta file requests

16. Multimedia and QoS#16 III: Using a Streaming Server r This gets us around HTTP, allows a choice of UDP vs. TCP and the application layer protocol can be better tailored to Streaming; many enhancements options are possible (see next slide)

17. Multimedia and QoS#17 Options When Using a Streaming Server r Use UDP, and Server sends at a rate (Compression and Transmission) appropriate for client; to reduce jitter, Player buffers initially for 2-5 seconds, then starts display r Use TCP, and sender sends at maximum possible rate under TCP; retransmit when error is encountered; Player uses a much large buffer to smooth delivery rate of TCP

18. Multimedia and QoS#18 Real Time Streaming Protocol (RTSP) r For user to control display: rewind, fast forward, pause, resume, etc… r Out-of-band protocol (uses two connections, one for control messages (Port 554) and for media stream) r RFC 2326 permits use of either TCP or UDP for the control messages connection, sometimes called the RTSP Channel r As before, meta file is communicated to web browser which then launches the Player; Player sets up an RTSP connection for control messages in addition to the connection for the streaming media

19. Multimedia and QoS#19 Meta File Example Twister <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> <track type="video/jpeg" src="rtsp://video.example.com/twister/video">

20. Multimedia and QoS#20 RTSP Operation

21. Multimedia and QoS#21 Jitter: Definitions r Input: t 0, …, t n r Delay Jitter: m Delay jitter J m For every k: |t 0 + kX - t k |  J m X = (t n - t 0 ) / n r Rate Jitter m Rate Jitter A m I k = t k - t k-1 m For every k and j: |I j - I k |  A r Jitter and buffering m delay versus jitter

22. Multimedia and QoS#22 Fixed Playout Delay (Delay Jitter)

23. Multimedia and QoS#23 Adaptive Playout Delay (Rate Jitter) r Objective is to use a delay value that tracks the network delay performance as it varies m Modify the inter-packet times r Estimate the inter-arrival time m Similar to estimates of RTT and deviation in TCP m Use the estimate for playout rate

24. Multimedia and QoS#24 Recovery From Packet Loss r Loss is in a broader sense: m packet never arrives or m arrives later than its scheduled playout time r Retransmission is inappropriate for Real Time applications, r Other methods are used to reduce loss impact.

25. Multimedia and QoS#25 I: Recovery From Packet Loss - FEC r FEC is Forward Error Correction r Simplest FEC scheme adds a redundant chunk made up of exclusive OR of a group of n chunks; m redundancy is 1/n; m can reconstruct if at most one lost chunk; m playout time schedule assumes a loss per group r More errors...

26. Multimedia and QoS#26 II: Recovery From Loss - Mixed Quality r Include two qualities m High quality m Low quality r If high quality lost, play low quality. m Better lower quality than nothing r Send the low quality with the next high quality r Can recover from single losses r Bandwidth blowup: m Depends on the low quality

27. Multimedia and QoS#27 Piggybacking Lower Quality Stream

28. Multimedia and QoS#28 III: Recovery From Loss - Interleaving r Divide audio data to smaller units and interleave r Upon loss, the missing data is spread out r Has no redundancy, but can cause delay in playout beyond Real Time requirements

29. Multimedia and QoS#29 Real-Time Protocol (RTP) r Provides standard packet format for real-time application r Typically runs over UDP r Specifies header fields below r Payload Type: 7 bits, providing 128 possible different types of encoding; eg PCM, MPEG2 video, etc. r Sequence Number: 16 bits; used to detect packet loss

30. Multimedia and QoS#30 Real-Time Protocol (RTP) r Timestamp: 32 bytes; gives the sampling instant of the first audio/video byte in the packet; used to remove jitter introduced by the network r Synchronization Source identifier (SSRC): 32 bits; an id for the source of a stream; assigned randomly by the source

31. Multimedia and QoS#31 RTP Control Protocol (RTCP) r Protocol specifies report packets exchanged between sources and destinations of multimedia information r Three reports are defined: Receiver reception, Sender, and Source description r Reports contain statistics such as the number of packets sent, number of packets lost, inter-arrival jitter r Used to modify sender transmission rates and for diagnostics purposes