T.Sharon-A.Frank 1 Multimedia Image Compression

2 T.Sharon-A.Frank Coding Techniques – Hybrid

3 T.Sharon-A.Frank Hybrid coding Images: –JPEG Video/Audio –MJPEG –MPEG (1, 2, 4) –Other codings –H.26x

4 Image files may be too big for network transmission, even at low resolutions. Use more sophisticated data representation or discard information to reduce data size. Effectiveness of compression will depend on actual image data. For any compression scheme, there will always be some data for which 'compressed' version is actually bigger than the original. Image Compression

5 T.Sharon-A.Frank Compression Steps 1. Preparation: analog to digital conversion. 2. Processing: transform data into a domain easier to compress. 3. Quantization: reduce precision at which the output is stored. 4. Entropy encoding: remove redundant information in the resulting data stream. Picture Preparation Picture Processing Quanti- zation Entropy Encoding Image Uncompressed Image Compressed

6 T.Sharon-A.Frank JPEG Goals: Support still images True color/grayscale Continuous-tone pictures Usually lossy Subjects: Quality Steps Modes Joint Photographic Experts Group

7 T.Sharon-A.Frank JPEG Picture Quality 0.25 – 0.5 bits/pixel Moderate to good quality. 0.5 – 0.75 bits/pixel Good to very good quality – 1.5 bits/pixel Excellent quality. 1.5 – 2.0 bits/pixel Usually indistinguishable from the original.

8 Lossy technique, well suited to photographs, images with fine detail and continuous tones. Consider image as a spatially varying signal that can be analysed in the frequency domain. Experimental fact: people do not perceive the effect of high frequencies in images very accurately. Hence, high frequency information can be discarded without perceptible loss of quality. JPEG Compression

9 T.Sharon-A.Frank Stage 1: Image Preparation C1C1 C2C2 CnCn * * * YjYj XiXi Sample s Left Right Line 1<= n <=255, usually 3 Top Bottom Components

10 T.Sharon-A.Frank 3 components having the same resolution A1A1 A2A2 AnAn Y X B1B1 B2B2 BnBn Y X C1C1 C2C2 CnCn Y X

11 T.Sharon-A.Frank A1A1 A2A2 AnAn Y X B1B1 B2B2 B n/2 Y X/2 C1C1 C2C2 C n/2 Y X/2 3 components having different resolution

12 If image resolution < output device resolution, must interpolate extra pixels –Always leads to loss of quality. If image resolution > output device resolution, must downsample (discard pixels): –Quality will often be better than that of an image at device resolution (uses more information). – Image sampled at a higher resolution than that of intended output device is over sampled. Changing Resolution

13 T.Sharon-A.Frank Block Preparation (a) RGB input data. (b) After block preparation.

14 T.Sharon-A.Frank Non Interleaved Order of Data Units Left Right Top ******** ******** ******** ******** ******** Bottom MCU = Minimum Coded Unit

15 T.Sharon-A.Frank Interleaved Data Units * * * **** **** **** **** **** **** 0101 *** ***

16 T.Sharon-A.Frank Stage 2: Image Processing Using DCT (Discrete Cosine Transform) Convert from Spatial to Frequency Domain Input : 8x8 matrix of 64 values Output: –1 DC value (basic color of block = average color) –63 AC values low frequency coefficients, have high values high frequency coefficients – represent sharp edges, have low values

17 T.Sharon-A.Frank Discrete Cosine Transform (DCT) A Fourier-related transform similar to the Discrete Fourier Transform (DFT), but using only real numbers. Often used in signal and image processing. Maps values from spatial domain to frequency domain – image areas with low frequency (large blocks of single color) are compressed more efficiently. Used in JPEG, MJPEG, MPEG, and DV Compression.

18 Analyses a signal into its frequency components. Takes array of pixel values, produces an array of coefficients of frequency components in the image. Computationally expensive process – time proportional to square of number of pixels. Usually applied to 8x8 blocks of pixels. DCT Dynamics

19 T.Sharon-A.Frank JPEG: DCT-Based Encoder 8x8 blocks Compressed Image Data Quantizer Entropy Encoder FDCT Table Specification Source Image Data

20 T.Sharon-A.Frank Block i-1 Block i DC i DC i-1 DIFF=DC i -DC i-1 Preparation of DC Coefficients

21 T.Sharon-A.Frank (a) One block of the Y matrix. (b) The DCT coefficients. DCT Transformation

22 T.Sharon-A.Frank Preparation of AC Coefficients AC 07 AC 01 DC AC 77 AC 70 Zig-Zag Sequence

23 T.Sharon-A.Frank Probability Distribution of DCT Coefficients Probability of being nonzero Zigzag index

24 T.Sharon-A.Frank Next Stages Stage 3: Quantization Based on quantization table: –Quantization table: values from 1 to 255 –DCT coefficient/Table value –If value > 0, keep DCT coefficient else, don’t keep DCT coefficient Stage 4: Entropy Encoding

25 Applying DCT does not reduce data size –Array of coefficients is same size as array of pixels. Allows information about high frequency components to be identified and discarded. Use fewer bits (distinguish fewer different values) for higher frequency components. Number of levels for each frequency coefficient may be specified separately in a quantization matrix. JPEG Quantization

26 T.Sharon-A.Frank Computation of the quantized DCT coefficients

27 T.Sharon-A.Frank Order in which the quantized values are transmitted

28 After quantization, there will be many zero coefficients. –Use RLE on zig-zag sequence (maximizes runs). Use Huffman coding of other coefficients (best use of available bits). JPEG Entropy Encoding

29 T.Sharon-A.Frank Steps of JPEG Compression Picture Preparation Pixel Block, MCU Picture Processing Predictor FDCT Quantization Entropy Encoding Run-Length Huffman Arithmetic Steps taking into account the different JPEG modes

30 Expand runs of zeros and decompress Huffman- encoded coefficients to reconstruct array of frequency coefficients. Use Inverse Discrete Cosine Transform to take data back from frequency to spatial domain. Data discarded in quantization step of compression procedure cannot be recovered. Reconstructed image is an approximation (usually very good) to the original image. JPEG Decompression

31 T.Sharon-A.Frank JPEG: DCT-Based Decoder Dequantizer Entropy Encoder IDCT Table Specification Reconstructe d Image Data Compressed Image Data

32 T.Sharon-A.Frank JPEG Modes of Operation Lossless Encoding: the image is encoded to guarantee exact recovery of every source image sample value (low compression ratio). Sequential Encoding: each image is encoded in a single left-to-right, top-to-bottom scan. Progressive Encoding: the image is encoded in multiple scans for applications in which transmission time is long. Hierarchical Encoding: the image is encoded at multiple resolutions.

33 T.Sharon-A.Frank Sequential Presentation

34 T.Sharon-A.Frank Progressive Presentation

35 T.Sharon-A.Frank Types of image processing in DCT Image display Bits per sample Entropy coding Sequential 8 Huffman coding Sequential 8 Arithmetic coding Sequential 12 Huffman coding Sequential 12 Arithmetic coding Progressive successive 8 Huffman coding Progressive spectral 8 Huffman coding Progressive successive 8 Arithmetic coding Progressive spectral 8 Arithmetic coding Progressive successive 12 Huffman coding Progressive spectral 12 Huffman coding Progressive successive 12 Arithmetic coding Progressive spectral 12 Arithmetic coding

36 T.Sharon-A.Frank Hybrid Coding Examples JPEG –image compression using a discrete cosine transform, then quantization, then Huffman coding. JPEG 2000 –image compression using wavelets, then quantization, then entropy coding. MP3 –A part of the MPEG-1 standard for sound and music compression, using subbanding and MDCT, perceptual modeling, quantization, and Huffman coding.