1. EE465: Introduction to Digital Image Processing1 Data Compression Techniques Text: WinZIP, WinRAR (Lempel-Ziv compression’1977) Image: JPEG (DCT-based), BMP (run- length coding) Video: VCD/DVD (MPEG), Real Player/Windows Media Player (H.263/H.264), BluRay Audio and Speech: MP3, G.728 (CELP)

2. EE465: Introduction to Digital Image Processing2 Lossless vs. Lossy Compression One is original and the other is compressed, which one is original? Mathematical lossy can still be visually lossless

3. EE465: Introduction to Digital Image Processing3 Engineering is About Design and Implementation cost performance Product A Product B

4. EE465: Introduction to Digital Image Processing4 A Tour of JPEG Coding Algorithm Flow-chart diagram of DCT-based coding algorithm specified by Joint Photographic Expert Group (JPEG) TQ C

5. EE465: Introduction to Digital Image Processing5 First Building Block: 8-by-8 DCT i j DC T

6. EE465: Introduction to Digital Image Processing6 Transform Coding of Images Why not transform the whole image together?  Require a large memory to store transform matrix  It is not a good idea for compression due to spatially varying statistics within an image Idea of partitioning an image into blocks  Each block is viewed as a smaller-image and processed independently  It is not a magic, but a compromise

7. EE465: Introduction to Digital Image Processing7 Block Processing under MATLAB Type “help blkproc” to learn the usage of this function  B = BLKPROC(A,[M N],FUN) processes the image A by applying the function FUN to each distinct M-by-N block of A, padding A with zeros if necessary. Example I = imread('cameraman.tif'); fun = @dct2; J = blkproc(I,[8 8],fun);

8. EE465: Introduction to Digital Image Processing8 Block-based DCT Example J I note that white lines are artificially added to the border of each 8-by-8 block to denote that each block is processed independently

9. EE465: Introduction to Digital Image Processing9 Boundary Padding padded regions Example 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 12 13 14 15 16 17 18 19 When the width/height of an image is not the multiple of 8, the boundary is artificially padded with repeated columns/rows to make them multiple of 8

10. EE465: Introduction to Digital Image Processing10 Work with a Toy Example Any 8-by-8 block in an image is processed in a similar fashion

11. EE465: Introduction to Digital Image Processing11 Encoding Stage I: Transform Step 1: DC level shifting 128 (DC level) _

12. EE465: Introduction to Digital Image Processing12 Step 2: 8-by-8 DCT Encoding Step 1: Transform (Con’t) 8  8 DCT

13. EE465: Introduction to Digital Image Processing13 Second Building Block: Quantization Q-table : specifies quantization stepsize (see slide #28) Notes:  Q-table can be specified by customer  Q-table is scaled up/down by a chosen quality factor  Quantization stepsize Q ij is dependent on the coordinates ( i,j ) within the 8-by-8 block  Quantization stepsize Q ij increases from top-left to bottom-right

14. EE465: Introduction to Digital Image Processing14 Encoding Stage II: Quantization (Con’t) Example f x ij s ij

15. EE465: Introduction to Digital Image Processing15 The Third Building Block: Entropy Coding Zigzag Scan (20,5,-3,-1,-2,-3,1,1,-1,-1, 0,0,1,2,3,-2,1,1,0,0,0,0,0, 0,1,1,0,1,EOB) zigzag scan End Of the Block: All following coefficients are zero

16. EE465: Introduction to Digital Image Processing16 Run-length Coding (20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) DC coefficient AC coefficient - DC coefficient : DPCM coding - AC coefficient : run-length coding (run, level) (5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) (0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1), (0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB Huffman coding encoded bit stream

17. EE465: Introduction to Digital Image Processing17 JPEG Decoding Stage I: Entropy Decoding (20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) Huffman decoding encoded bit stream AC coefficients DC coefficient DPCM decoding (0,5),(0,-3),(0,-1),(0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1),(2,0),(0,1), (0,2),(0,3),(0,-2),(0,1),(0,1),(6,0),(0,1),(0,1),(1,0),(0,1),EOB encoded bit stream

18. EE465: Introduction to Digital Image Processing18 JPEG Decoding Stage II: Inverse Quantization (20,5,-3,-1,-2,-3,1,1,-1,-1,0,0,1,2,3,-2,1,1,0,0,0,0,0,0,1,1,0,1,EOB) zigzag f -1

19. EE465: Introduction to Digital Image Processing19 JPEG Decoding Stage III: Inverse Transform 8  8 IDCT 128 (DC level) +

20. EE465: Introduction to Digital Image Processing20 Quantization Noise X X ^ MSE=|| X-X || 2 ^ Distortion calculation: Rate calculation: Rate=length of encoded bit stream/number of pixels (bps)

21. EE465: Introduction to Digital Image Processing21 JPEG Examples 1000 90 (58k bytes)50 (21k bytes)10 (8k bytes) best quality, lowest compression worst quality, highest compression

22. EE465: Introduction to Digital Image Processing22 Rate-Distortion Tradeoff

23. EE465: Introduction to Digital Image Processing23 Block artifacts of JPEG when the bit rate is low (0.25bpp)

24. EE465: Introduction to Digital Image Processing24 JPEG Coding Algorithm Summary

25. EE465: Introduction to Digital Image Processing25 Lossy Compression of Color Images Color-space transformation luminance chrominance inverse transform forward transform Luminance channel can be treated just like a gray-scale image

26. EE465: Introduction to Digital Image Processing26 Compression of Chrominance Channels Human eye is relatively insensitive to the high-frequency content of the chrominance channels. Down-sampling 2 C r /C b Q-Table

27. EE465: Introduction to Digital Image Processing27 Technology is About New Tools The same design principle but each building block can be replaced by a more advanced technique  Transform: DCT becomes Wavelet Transform  Quantization: uniform quantization becomes bitplane coding (to support scalability)  Entropy coding: Huffman coding becomes arithmetic coding (more flexible) JPEG becomes JPEG2000

28. EE465: Introduction to Digital Image Processing28 A Glimpse into Wavelet Transform* Multi-Resolution Analysis in Vision Human vision system is capable of interpreting the visual information regardless of the resolution

29. EE465: Introduction to Digital Image Processing29 Progressive Image Transmission I hate to wait – show me a piece and see if I like it or not Low-resolution (LR) High-resolution (HR)

30. EE465: Introduction to Digital Image Processing30 Haar Transform – the simplest wavelet transform Example

31. EE465: Introduction to Digital Image Processing31 Reordering Coefficients Apply Haar transform to each 2-by-2 block regrouping              00 00 00 00 00 00 100 200 Y approximation horizontal details diagonal detailsvertical details

32. EE465: Introduction to Digital Image Processing32 Multi-level Wavelet Transform Apply one-level Haar transform              00 00 00 00 00 00 100 200 Y Apply Haar transform to Low-Low band only

33. EE465: Introduction to Digital Image Processing33 Image Example

34. EE465: Introduction to Digital Image Processing34 Wavelet Image Compression Wavelet Transform Scalar Quantization Entropy Coding image compressed bit stream Major improvements  Wavelet transform instead of DCT  Arithmetic coding instead of Huffman coding  Modeling location instead of intensity Those are the topics in EE565: Advanced Image Processing

35. EE465: Introduction to Digital Image Processing35 FBI Wavelet Scalar Quantization (WSQ) Standard Established by FBI in 1990s as the standard for fingerprint image compression Wavelet-based: Daubechies’ wavelets were built just in time to win the race Demonstrate better Rate-Distortion performance than DCT-based JPEG standard

36. EE465: Introduction to Digital Image Processing36 JPEG2000 Standard Developed by ISO in late 1990s to replace the DCT-based JPEG standard Accumulative research progress on wavelet image compression has dramatically pushed the art of lossy image compression forward Demonstrate much better Rate-Distortion performance than DCT-based JPEG standard for typical photographic images

37. EE465: Introduction to Digital Image Processing37 Wavelet vs. DCT JPEG (CR=64)JPEG2000 (CR=64) discrete cosine transform basedwavelet transform based

38. EE465: Introduction to Digital Image Processing38 BIGGER Question Why haven’t we seen JPEG being replaced by JPEG2000? Technical superiority DO NOT always imply economical wins (think about Unix vs. Windows) THE next image compression standard has to wait for the right timing Anyone including you can be a part of that business