1. Image (and Video) Coding and Processing Lecture: DCT Compression and JPEG Wade Trappe Again: Thanks to Min Wu for allowing me to borrow many of her slides.

2. Transform Coding

3. Lecture Overview We will talk first about the structure of transform coding –Zonal vs. Threshold Coding –Block Size Issues –Why should we Zig-Zag? JPEG –The Steps in the Process –Quantization –Subsampling of chrominance –Lossless part of JPEG

4. Transform Coding Use transform to pack energy to only a few coeff. How many bits to be allocated for each coeff.? –More bits for coeff. with high variance  k 2 to keep total MSE small –Also determined by perceptual importance From Jain’s Fig.11.15

5. Zonal Coding and Threshold Coding Zonal coding –Only transmit a small predetermined zone of transformed coeff. Threshold coding –Transmit coeff. that are above certain thresholds Compare – Threshold coding is inherently adaptive u introduce smaller distortion for the same # of coded coeff. – Threshold coding needs overhead in specifying index of coded coeff. u run-length coding helps to reduce overhead

6. Determining Block Size Why block based? –High transform computation complexity for large block u O( m log m  m ) per block in tranf. for (MN/m 2 ) blocks u complexity in bit allocation –Block transform captures local info. better than global transform Rate & complexity vs. block size –Commonly used block size ~ 8x8 From Jain’s Fig.11.16 complexity

7. Block-based Transform Coding Encoder –Step-1 Divide an image into m x m blocks and perfrom transform –Step-2 Determine bit-allocation for coefficients –Step-3 Design quantizer and quantize coefficients (lossy!) –Step-4 Encode quantized coefficients Decoder From Jain’s Fig.11.17

8. How to Encode Quantized Coeff. in Each Block Basic tools –Entropy coding (Huffman, etc.) and run-length coding –Predictive coding ~ esp. for DC Ordering –zig-zag scan for block-DCT to better achieve run-length coding gain Horizontal frequency Vertical frequency DC AC 01 AC 07 AC 70 AC 77  low-frequency coefficients, then high frequency coefficients

9. Summary: List of Compression Tools Lossless encoding tools –Entropy coding: Huffman, Lemple-Ziv, and others (Arithmetic coding) –Run-length coding Lossy tools for reducing redundancy –Quantization: scalar quantizer vs. vector quantizer –Truncations: discard unimportant parts of data Facilitating compression via Prediction –Encode prediction parameters and residues with less bits Facilitating compression via Transforms –Transform into a domain with improved energy compaction

10. Put Basic Tools Together: JPEG Image Compression Standard

11. JPEG Compression Standard (early 1990s) JPEG - Joint Photographic Experts Group –Compression standard of generic continuous-tone still image –Became an international standard in 1992 Allow for lossy and lossless encoding of still images –Part-1 DCT-based lossy compression u average compression ratio 15:1 –Part-2 Predictive-based lossless compression Sequential, Progressive, Hierarchical modes –Sequential ~ encoded in a single left-to-right, top-to-bottom scan –Progressive ~ encoded in multiple scans to first produce a quick, rough decoded image when the transmission time is long –Hierarchical ~ encoded at multiple resolution to allow accessing low resolution without full decompression

12. Baseline JPEG Algorithm “Baseline” –Simple, lossy compression u Subset of other DCT-based modes of JPEG standard A few basics –8x8 block-DCT based coding –Shift to zero-mean by subtracting 128  [-128, 127] u Allows using signed integer to represent both DC and AC coeff. –Color ( YCbCr / YUV ) and downsample u Color components can have lower spatial resolution than luminance –Interleaving color components (Based on Wang’s video book Chapt.1)

13. Block Diagram of JPEG Baseline From Wallace’s JPEG tutorial (1993)

14. 475 x 330 x 3 = 157 KB luminance From Liu’s EE330 (Princeton)

15. RGB Components From Liu’s EE330 (Princeton)

16. Y U V (Y Cb Cr) Components Assign more bits to Y, less bits to Cb and Cr From Liu’s EE330 (Princeton)

17. Lossless Coding Part in JPEG Differentially encode DC –(lossy part: DC differences are then quantized.) AC coefficients in one block – Zig-zag scan after quantization for better run-length u save bits in coding consecutive zeros – Represent each AC run-length using entropy coding u use shorter codes for more likely AC run-length symbols

18. Lossy Part in JPEG Quantization (adaptive bit allocation) –Different quantization step size for different coeff. bands –Use same quantization matrix for all blocks in one image –Choose quantization matrix to best suit the image –Different quantization matrices for luminance and color components Default quantization table –“Generic” over a variety of images Quality factor “Q” –Scale the quantization table –Medium quality Q = 50% ~ no scaling –High quality Q = 100% ~ unit quantization step size –Poor quality ~ small Q, larger quantization step u visible artifacts like ringing and blockiness

19. How Quantization is Performed Suppose you have an 8x8 DCT image X(u,v) The quantizer output is: I(u,v)=Round(X(u,v)/Q(u,v)) This rounds to the nearest integer Here, Q(u,v) is a quantization table. The default luminance table for JPEG is presented on the right What to note: –Smaller Q(u,v) means a smaller step size and hence more resolution –Vice-versa –Q(u,v) may be scaled by a quality factor 1611101624405161 12 141926586055 1413162440576956 1417222951878062 182237566810910377 243555648110411392 49647887103121120101 7292959811210010399

20. Uncompressed (100KB) JPEG 75% (18KB) JPEG 50% (12KB)JPEG 30% (9KB)JPEG 10% (5KB)

21. JPEG Compression (Q=75% & 30%) 45 KB 22 KB From Liu’s EE330 (Princeton)

22. Y Cb Cr After JPEG (Q=30%) From Liu’s EE330 (Princeton) JPEG Cb JPEG Cr

23. Lossless Coding Part in JPEG: Details Differentially encode DC –( SIZE, AMPLITUDE ), with amplitude range in [-2048, 2047] AC coefficients in one block – Zig-zag scan for better run-length – Represent each AC with a pair of symbols u Symbol-1: ( RUNLENGTH, SIZE )  Huffman coded u Symbol-2: AMPLITUDE  Variable length coded RUNLENGTH  [0,15] # of consecutive zero-valued AC coefficients preceding the nonzero AC coefficient  [0,15] SIZE  [0 to 10 in unit of bits] # of bits used to encode AMPLITUDE AMPLITUDE  in range of [-1023, 1024]

24. Table is from slides at Gonzalez/ Woods DIP book website (Chapter 8)