1. 1 Image and Video Compression: An Overview Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.in

2. 2 Data Compression Alternative description of data requiring less storage and bandwidth. Uncompressed 1 Mbyte Compressed (JPEG) 50 Kbyte (20:1)

3. 3 IMAGE COMPRESSION The process of obtaining a representation of the image with less number of bits than the original. DECOMPRESSION The process of recovering the image from its compressed form to its (almost) original form. LOSSLESS COMPRESSION Compression without any loss of information-There is no difference between the Original Image and Reconstructed one. LOSSY COMPRESSION Compression with partial loss of information – Reconstructed Image differs from the Original one (within and accepted limit).

4. 4 Compression / Decompression

5. 5 Different Forms of Alternative Representation Image Transforms - DCT - DWT Fractal Representation - Partial Iterated Function System Multi-resolution Representation - Laplacian Pyramid - Wavelet Subbands Differential Representation - DPCM

6. 6

7. 7 JPEG Scheme

8. 8 Quantization Table

9. 9 DC and AC encoding

10. 10 JPEG: Code Structure

11. 11 JPEG2000 Scheme

12. 12 Dyadic Decomposition

13. 13 JPEG2000 9/7 wavelet filters Analysis Filter Coefficients Synthesis Filter Coefficients i Lowpass H(i) Highpass G(i) Lowpass H ’ (i) Highpass G ’ (i) 0 0.6029 -1.11511.1151-0.6029 0.2669 0.5913 0.2669 -0.0782 0.0575-0.0575 0.0782 -0.0169 -0.0913 -0.0169 0.0267-0.0267

14. 14 JPEG2000 Code Structure

15. 15 Fractals Infinite Detailing Self Similarity: Rules of production similar at every scale Continuous at every point but nowhere differentiable Whole can be described from part and vice versa

16. 16 R1R1 D1 D2 BASIC COMRESSION SCHEME W1 For every RANGE BLOCK (R) find suitable DOMAIN BLOCK(D) so that R  W(D) [W is the Transformation applied on D]

17. 17 ADAPTIVE SCHEMES RANGE & DOMAIN SIZE –QUADTREE PARTITION DOMAIN POOL –TRIANGULATED PARTITION –LATTICE OF SEPARATION TRANSFORMATION SET –IGNORE ISOMETRIES –NON-LINEAR CONTRACTIVE TRANSFORM

18. 18 DECOMPRESSION ALGORITHM Start with any arbitrary image For each Range Block, apply transformation on corresponding Domain Block in every iteration. After sufficiently large number of iteration it converges to the target image (lossy version of original image)

19. 19 MPEG2 Uses DCT based JPEG for temporal compression. Lossy video compression and lossy audio compression. Core of most digital television and DVD formats. Designed to code standard-definition television at bit rates from about 3-15 Mbit/s and high-definition television at 15- 30 Mbit/s.

20. 20 INTRA Motion Compensated Inter Frames Encoding Decoding (DCT, Quant., Motion Estimation & Compensation, VLC) (IDCT, IQuant., Inverse Motion Compensation, VLC)

21. 21 Frame Types I Frames (Intra Pictures) - encoded using only information within that frame. - used as reference frames. - provides least amount of compression. P Frames (Predicted Pictures) - encoded with reference to a previous I or P frame. - used as reference frame. - provides moderate compression B Frames (Bidirectional Pictures) - requires both past and future reference frames. - never used as reference. - provides highest amount of compression

22. 22 MPEG2 : Coding Sequence 1I1I 2B2B 3B3B 4B4B 5P5P 6B6B 7B7B 8B8B 1I1I Forward Prediction Bidirectional Prediction

23. 23 MPEG2 Encoder _ DCTQVLC IQ IDCT + MCP _ Video in Coded bitstream out (I)DCT=(Inverse) discrete cosine transform (I)Q=(Inverse) Quantization MCP=Motion Compensated Prediction VLC= Variable Length Coder

24. 24 MPEG2 Decoder VLDIQIDCT MCP + Coded Bitstream in Decoded video out (I)DCT=(Inverse) discrete cosine transform (I)Q=(Inverse) Quantization MCP=Motion Compensated Prediction VLD = Variable Length Decoder

25. 25 For each 8x8 block DCTQuant. RLE { Zig-Zag Scan Huffman ……011000011010 Intra Frame Encoding

26. 26 Motion Estimation & Prediction to construct Inter Frame (P/B- frames) 12345 678910 11m131415 1617181920 2122232425 Predicted P Reference m’ e = m – m’

27. 27 Reference Best Match Motion Vector } Difference DCT + Quant. + RLE Huffman 01101100 Past reference Future reference Target [ + ] = - 0.5 x DCT + Quant. + RLE Huffman coder 01101100 Motion vectors P-Frame Encoding B-Frame Encoding

28. 28 Seq. Header GOP Header GOP Header GOPMPEG End Code ----------------------- Pic. Header PIC ----------------------- Macroblock header Macroblock ----------------------- Block 8x8 Block 8x8 Block 8x8 ----------------------- I B B P B B P B B P B B I B B... I P P P P P P P P P P P I P P... I I I I I I ….

29. 29 MPEG-4

30. 30 MJPEG2000 (Motion JPEG2000)

31. 31 MCJ2K codec Structure Compressed Stream JPEG2000 Compression Motion Estimation /Compensation JPEG2000 Compression JPEG2000 Decompression of I or P Frames + Video in I frame P or B

32. 32 H.264/AVC: Encoder

33. 33 Image Transform MPEG-2 uses 8x8 DCT(Discrete Cosine Transform) A separable integer 4x4 transform an approximation to 4x4 DCT Can be computed with only 16 bit additions, subtractions, and shifts Repeated transform of DC coeffs for 8x8 chroma and 16x16 Intra luma blocks

34. 34 Intra - frame Prediction Two modes for luma block – Intra 4x4 9 modes Used in texture area – Intra 16x16 4 modes Used in flat area One mode of chroma block – Similar to Intra 16x16 I_PCM: bypass the prediction and transform coding

35. 35 Intra 16x16 & Chroma prediction 4 modes

36. 36 Intra 4x4 9 modes

37. 37 Inter - Frame Prediction –Multiple reference pictures and decoupling of reference order from display order. –Quarter sample accuracy motion vector

38. 38 Macro Block 16x16 8x88x8 Macro Block and Block size

39. 39 Performance comparison MPEG-2 Frame H264/AVC Frame

40. 40 MPEG-2 Vs H264/AVC Baseline profile(foreman.cif video) Performance comparison contd.