Media Compression

You are Here Encoder Decoder Middlebox Sender Receiver Network NUS.SOC.CS5248 Ooi Wei Tsang

Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie = NUS.SOC.CS5248 Ooi Wei Tsang

Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208.6GB NUS.SOC.CS5248 Ooi Wei Tsang

JPEG Compression

Original Image (1153KB) 1:1

Original Image (1153KB) 3.5:1

Original Image (1153KB) 17:1

Original Image (1153KB) 27:1

Original Image (1153KB) 72:1

Original Image (1153KB) 192:1

Compression Ratio Quality Size Ratio Raw TIFF 1153KB 1:1 Zipped TIFF 1.2:1 Q=100 331KB 3.5:1 Q=70 67KB 17:1 Q=40 43KB 27:1 Q=10 16KB 72:1 Q=1 6KB 192:1 NUS.SOC.CS5248 Ooi Wei Tsang

Magic of JPEG Throw away information we cannot see Color information “High frequency signals” Rearrange data for good compression Use standard compression NUS.SOC.CS5248 Ooi Wei Tsang

Discard color information Y YUV is also known as LUV or YCbCr V U NUS.SOC.CS5248 Ooi Wei Tsang

Color Sub-sampling 4:4:4 4:2:0 4:2:2 4:1:1 420 is used by MPEG. 411 is used by DV. 4:2:2 4:1:1 NUS.SOC.CS5248 Ooi Wei Tsang

4:2:2 Sub-sampling Y V U YUV is also known as LUV or YCbCr NUS.SOC.CS5248 Ooi Wei Tsang

Original Image (1153KB) 4:2:0

Original Image (1153KB) “4:1:0”

Discrete Cosine Transform Demo NUS.SOC.CS5248 Ooi Wei Tsang

Quantization DC 242 65 23 5 8 8 8 8 30 8 2 -54 -10 -4 -2 8 8 8 16 -6 -1 / = 13 6 3 5 8 8 16 32 1 2 1 -1 -2 8 16 32 64 Quantization Table AC NUS.SOC.CS5248 Ooi Wei Tsang

Differential Coding 30 8 6 -1 2 1 25 3 2 1 4 27 3 2 1 4 30 8 6 -1 2 1 1 25 3 2 1 4 27 3 2 1 4 30 8 6 -1 2 1 -5 3 2 1 4 2 3 1 4 NUS.SOC.CS5248 Ooi Wei Tsang

Zig-zag ordering 27 3 2 1 4 27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 NUS.SOC.CS5248 Ooi Wei Tsang

Run-Length Encoding 27 3 2 1 4 27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4) NUS.SOC.CS5248 Ooi Wei Tsang

Idea: Motion JPEG Compress every frame in a video as JPEG DVD-quality video = 208.6GB Reduction ratio = 27:1 Final size = 7.7GB NUS.SOC.CS5248 Ooi Wei Tsang

Video Compression

Temporal Redundancy NUS.SOC.CS5248 Ooi Wei Tsang

Motion Estimation NUS.SOC.CS5248 Ooi Wei Tsang

Bi-directional Prediction NUS.SOC.CS5248 Ooi Wei Tsang

Watch for Motion Vectors Demo NUS.SOC.CS5248 Ooi Wei Tsang

H.261 P-Frame I-Frame NUS.SOC.CS5248 Ooi Wei Tsang

MPEG-1 B-Frame NUS.SOC.CS5248 Ooi Wei Tsang

MPEG Frame Pattern Example display sequence: IBBPBBP … Example encoding sequence: IPBBPBB NUS.SOC.CS5248 Ooi Wei Tsang

Compression Ratio Frame Type Typical Ratio I 10:1 P 20:1 B 50:1 NUS.SOC.CS5248 Ooi Wei Tsang

Sequence sequence header: width height frame rate bit rate : NUS.SOC.CS5248 Ooi Wei Tsang

GOP: Group of Picture gop header: time : NUS.SOC.CS5248 Ooi Wei Tsang

Picture pic header: number type (I,P,B) : NUS.SOC.CS5248 Ooi Wei Tsang

Picture NUS.SOC.CS5248 Ooi Wei Tsang

Slice NUS.SOC.CS5248 Ooi Wei Tsang

Macroblock NUS.SOC.CS5248 Ooi Wei Tsang

Block Y U 1 Macroblock = V NUS.SOC.CS5248 Ooi Wei Tsang

For I-Frame Every macroblock is encoded independently (“I-macroblock”) NUS.SOC.CS5248 Ooi Wei Tsang

For P-Frame Every macroblock is either I-macroblock a motion vector + error terms wrt a prev I/P-frame (“P-macroblock”) NUS.SOC.CS5248 Ooi Wei Tsang

For B-Frame Every macroblock is either I-macroblock P-macroblock a motion vector + error terms wrt a future I/P-frame 2 motion vectors + error terms wrt a prev/future I/P-frame NUS.SOC.CS5248 Ooi Wei Tsang

Review: MPEG structure Sequence GOP Picture Slice Macroblock Block NUS.SOC.CS5248 Ooi Wei Tsang

MPEG Decoding (I-Frame) Entropy Decoding Dequantize 101000101 IDCT NUS.SOC.CS5248 Ooi Wei Tsang

MPEG Decoding (P-Frame) Entropy Decoding Dequantize 101000101 IDCT Prev Frame + NUS.SOC.CS5248 Ooi Wei Tsang

MPEG Decoding (B-Frame) Entropy Decoding Dequantize 101000101 Future Frame IDCT AVG Prev Frame + NUS.SOC.CS5248 Ooi Wei Tsang

There are more.. Half-pel Motion Prediction Skipped Macroblock etc. NUS.SOC.CS5248 Ooi Wei Tsang

MPEG in daily life MPEG Standards Bit-rate Usage MPEG-1 1.5Mbps VCD DVD, SVCD, HDTV MPEG-4 Scalable QuickTime, DivX;) NUS.SOC.CS5248 Ooi Wei Tsang

The Future: H.264 Same quality at half the rate How: Variable block size Multiple reference frame ... NUS.SOC.CS5248 Ooi Wei Tsang

Hands-On Download source code, compile and play with ffmpeg mpeg_stat Try different MPEG-1 encoding parameter NUS.SOC.CS5248 Ooi Wei Tsang

Impact on Systems Design How to package data into packets? How to deal with packet loss? How to deal with bursty traffic? How to predict decoding time? : NUS.SOC.CS5248 Ooi Wei Tsang