1. 1 Multimedia Outline Compression RTP Scheduling

2. 2 Compression Overview Encoding and Compression –Huffman codes Lossless –data received = data sent –used for executables, text files, numeric data Lossy –data received does not != data sent –used for images, video, audio

3. 3 Lossless Algorithms Run Length Encoding (RLE) –example: AAABBCDDDD encoding as 3A2B1C4D –good for scanned text (8-to-1 compression ratio, used for fax) –can increase size for data with variation (e.g., some images) Differential Pulse Code Modulation (DPCM) –example AAABBCDDDD encoding as A0001123333 diff. can be encoded by fewer bits than original symbol –change reference symbol if delta becomes too large –works better than RLE for many digital images (1.5-to-1)

4. 4 Dictionary-Based Methods Build dictionary of common terms –variable length strings Transmit index into dictionary for each term Lempel-Ziv (LZ) is the best-known example Commonly achieve 2-to-1 ration on text Variation of LZ used to compress GIF images –first reduce 24-bit color to 8-bit color (most closely approximate) –treat common sequence of pixels as terms in dictionary –not uncommon to achieve 10-to-1 compression

5. 5 Image Compression JPEG: Joint Photographic Expert Group (ISO/ITU) Lossy still-image compression Three phase process –process in 8x8 block chunks (macro-block) –DCT: transforms signal from spatial domain to the frequency domain (loss-less) higher freq. finer detail (multi-resolution analysis) –apply a quantization to the results (lossy) –RLE-like encoding (loss-less) –Color image each pixel is three values: YUV (luminance + 2 chrominance), more suitable for human visual system than RGB –Compression ratio 30-to-1

6. 6 Quantization and Encoding Quantization Table 3 5 7 9 11 13 15 17 5 7 9 11 13 15 17 19 7 9 11 13 15 17 19 21 9 11 13 15 17 19 21 23 11 13 15 17 19 21 23 25 13 15 17 19 21 23 25 27 15 17 19 21 23 25 27 29 17 19 21 23 25 27 29 31 Encoding Pattern For the coefficient at 0,0 Quantum = 3 If the value is 25 Quantized value [25/3]=8 Decompressed value 8*3=24 Use RLE for 0’s, Huffman code for others

7. 7 MPEG Motion Picture Expert Group Lossy compression of video First approximation: JPEG on each frame Also remove inter-frame redundancy

8. 8 MPEG (cont) Frame types –I frames: intrapicture –P frames: predicted picture –B frames: bidirectional predicted picture Example sequence transmitted as I P B B I B B

9. 9 MPEG (cont) B and P frames –coordinate for the macroblock in the frame –motion vector relative to previous reference frame (B, P) –motion vector relative to subsequent reference frame (B) –delta for each pixel in the macro block Effectiveness –typically 90-to-1 –as high as 150-to-1 –30-to-1 for I frames –P and B frames get another 3 to 5 x

10. 10 Other Video Encoding Standards The version previous described is MPEG-2 MPEG-4 –Support for lower speed links such as wireless –Treat a scene as a collection of video objects rather than macro-blocks as in MPEG-2 H.261, H.262 –Standardized by ITU-T –Similar compression methods as MPEG –Used for lower speed links (ISDN, 64 kbps increments) Compare to MPEG, 1.5 Mbps

11. 11 Audio Compression: MP3 CD Quality –44.1 kHz sampling rate –2 x 44.1 x 1000 x 16 = 1.41 Mbps stereo, 16 bits per channel, higher quality than dig. Tel. (64kbps) –49/16 x 1.41 Mbps = 4.32 Mbps Overhead ratio 49/16, for synchronization, error correction Layering for transmission on low speed links –Layer I, 384 kbps, comp. ratio 4 –Layer II, 192 kbps, comp. ratio 8 –Layer III, 128 kbps, comp. ratio 12 (MP3)

12. 12 MP3 Encoding Strategy –similar to MPEG –split into some number of frequency bands –divide each subband into a sequence of blocks –encode each block using DCT + Quantization + Huffman –trick: how many bits assigned to each subband Determined by human listening sensitivity and experience

13. 13 The Real-Time Transport Protocol RTP is a transport protocol for multimedia applications but implemented in the application space. RTP multiplexes several real-time streams into a single stream of UDP packets A companion protocol, Real-Time Transport Control Protocol (RTCP), provides control functions (RTP is for data transport) –Provide feedback (delay, jitter, bandwidth, congestion, etc.), synchronization, text info about source, etc.

14. 14 RTP RTP has no flow control, error control, acknowledgement and retransmission; RTP has sequence numbers –the receiver will not ask for retransmission but will interpolate instead. RTP provides timestamps –relative to the start of the stream –enable buffering and correct playback at the receiver, and synchronizing among constituent streams (indicated by the synchronizing source identifier in the header). RTP defines application profiles –i.e., MP3, indicated by the payload type field in the header.

15. 15 RTP header

16. 16 Transmitting MPEG Adapt the encoding to network bandwidth –Resolution, frame rate, quantization table, GOP (group of picture, I, B, P frames) mix Packetization –Application layer framing, packetizing at, e.g., macroblock boundaries (to isolate loss to one block) Dealing with loss –B frame loss damages a single frame, I frame loss multiple frames –Retransmission not an option, can adapt Diffserv dropping probability GOP-induced latency –B frame delayed until the next I/P frame (n frames away) @15 fps, can be n*67ms, perceivable to human Many video conference application use Motion-JPEG, whose frames are standalone.

17. 17 Layered Video Layered encodeing –e.g., wavelet encoded Receiver Layered Multicast (RLM) –transmit each layer to a different group address –receivers subscribe to the groups they can “afford” –Probe to learn if you can afford next higher group/layer Smart Packet Dropper (multicast or unicast) –select layers to send/drop based on observed congestion –observe directly or use RTP feedback

18. 18 Real-Time Scheduling Priority Earliest Deadline First (EDF) Rate Monotonic (RM) Proportional Share –with feedback –with adjustments for deadlines