MPEG-1 Video Coding Standard Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223 jfan@uncc.edu http://www.cs.uncc.edu/~jfan

1. What does MPEG mean? MPEG: Moving Picture Experts Group 2. Why we do not use MJPEG? 12 13 14 Spatial correlations among neighboring pixels! Temporal correlations among neighboring frames!

3. Why we need MPEG? Temporal Redundancy Exploit temporal relationship among frames for coding! save our money for: a. Storage space; b. Bandwidth CIF: 352*288 MPEG Frame rate: 25f/s 1.5Mbits/s Space: ~21.6Mbits/s

4. MPEG Video Structure Temporal relationships could be in short period! GOP 1 GOP 2 GOP 3 GOP 4 time Why we need this? I B B P B B P B B B P GOP

4. MPEG Video Structure 16*16 8*8 block macroblock frame Universal structure is very important for universal decoder!!

5. MPEG-1 Encoder MPEG-1 has defined three frame coding types: I, P, B. I frame: Intra-coding frame P frame: motion-based predictive coding frame B frame: motion-based bidirectional predictive coding frame The difference between I and P or B: I can be encoded/decoded without reference The difference between P and B: P can be encoded/decoded with one reference

5. MPEG-1 Encoder DCT Q VLC Buffer 01001…. IQ Q: quantization ME IDCT IDCT IQ VLC Buffer 01001…. ME: motion estimation ME IQ: reverse quantization

5. MPEG-1 Encoder I B P

6. How to Encode I Frame? slice Y DCT U V Entropy coding block DCT U V macroblock Entropy coding 10010100010…………

(3) CIE: more close to human vision 6. How to Encode I Frame? a. Color Space (1) RGB: display (2) YUV: coding (3) CIE: more close to human vision b. DCT (Discrete Cosine Transformation) 2550 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 DCT 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0

b. DCT (Discrete Cosine Transformation) 6. How to Encode I Frame? b. DCT (Discrete Cosine Transformation) 2550 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 DCT 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0

Q(u) = integer round (u/scale) 6. How to Encode I Frame? 255 217 120 110 0 0 0 0 7 2 2 1 0 0 0 0 120 110 0 0 0 0 0 0 5 3 0 0 0 0 0 0 DCT 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 120 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Zigzag scale Quantization (7,0), (5,0), (2,0), (2,0), (3,4), (1,4), (1,41) Q(u) = integer round (u/scale) Entropy coding

6. How to Encode I Frame?

6. How to Encode I Frame? Entrpoy coding: Code probability 1 1 Pc=1/2 1 01 Pc=1/4 1 001 Pc=1/8 000 Pc=1/8 Huffman Tree

7. How to Encode P Frame? Motion-based prediction I or P frame Current P frame I B B P B B P B B P

7. How to Encode P Frame? Motion compensation: motion vectors + errors

a. Find the reference block at the same position 7. How to Encode P Frame? Motion estimation: a. Find the reference block at the same position (x,y) (x,y) Reference Frame Current Frame I, P blocks & Skip

7. How to Encode P Frame? b. Take the reference block as the search center

8. How to Encode B Frame? Forward prediction Backward prediction Previous reference Current frame Future reference I B B P B B P

8. How to Encode B Frame? Why we need B frame? Higher compression ratio I, P, B blocks & Skips

8. How to Encode B Frame? Why we cannot put too much B frames? Display order: 1 2 3 4 5 6 7 8 9 10 11 12 13 I B B P B B P B B P B B P Encoding order: 1 4 2 3 7 5 6 10 8 9 13 11 12 I P B B P B B P B B P B B Buffer Size: 3 frames

9. How to Design MPEG Decoder? Video sequence Sequence header Sequence end code Sequence layer GOP GOP GOP GOP GOP header GOP layer I-frame P-frame B-frame B-frame Picture header slice slice slice Picture layer Slice header macroblock macroblock Slice layer Macroblock header Macroblock layer Block(0) Block(1) Block(5)

9. How to Design MPEG Decoder? a. Search for the start codes: Sequence header code: 0x000001B5 32bits Sequence end code: 0x000001B7 32bits Group_start_code: 0x000001B8 32bits Picture_start_code: 0x00000100 32bits Slice_start_code: 0x000001AF 32bits

9. How to Design MPEG Decoder? b. Functions for decoding the video data: Sequence header() Sequence_start_code 32 Horizontal size 12 12 Vertical size 4 Pel_aspect_ratio 4 Picture_rate 18 Bit rate Marker_bit 1 Buffer_size 10 Q table 22 Quantization table Done

9. How to Design MPEG Decoder? Group_of_picture () Group_start_code 32 Drop_frame_flag 1 Time_code_hours 5 Time_code_minutes 6 Market_bit 1 Time_code_seconds 6 6 Time_code_pictures Close_gop 1 1 Broken_link Picture( ) Done

9. How to Design MPEG Decoder? Picture ( ) 32 Picture_start_code 10 Temporal_reference 3 Picture_coding_type 16 Decoding_delay 3 Forward_vector_type 3 Backward_vector_type Slice( ) Done

9. How to Design MPEG Decoder? Slice ( ) 32 Slice_start_code 5 Quantization scale Macroblock ( ) Done

9. How to Design MPEG Decoder? Macroblock ( ) 11 Macroblock_escape Macroblock_address_increase 1-11 Macroblock_type 1-6 5 Quantizer_scale Motion_vector ( ) 3-9 Coded_block_pattern Block (i) Done

10. Where you can find MPEG? a. http://www.mpeg.org b. http://www.cs.wayne.edu/~dil/research Question: If you are offered a job on designing MPEG decoder, how you can do? Show me the video frames from 1228 to 1338? IBM, Microsoft, Intel, NEC, Cisco, Eastman Kodak, ……, .com