1. M. Wu: ENEE631 Digital Image Processing (Spring'09) Subband and Wavelet Coding Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park   bb.eng.umd.edu (select ENEE631 S’09)   minwu@eng.umd.edu ENEE631 Spring’09 Lecture 12 (3/9/2009)

2. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [2] Overview and Logistics Last Time: –Basis images for 2-D separable unitary transform –Basics on transform coding –JPEG compression standard: Baseline block-DCT based algorithm u lossy part: quantization with different step size for each coeff. band u lossless part: run-length coding, Huffman coding, differential coding Today –Wrap up JPEG compression –Subband and Wavelet based compression u Subband decomposition u Exploit the structures between coefficients for removing redundancy UMCP ENEE631 Slides (created by M.Wu © 2004)

3. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [3] Revisit: Exercise on Basis Images For 2-D separable unitary transform: Y = A X A T => X = A H Y A * –Represent X with NxN basis images weighted by coefficients in Y –Obtain basis image by setting Y={  (k-k 0, l-l 0 )} & getting X  In matrix form A* k,l = a* k a l *T ~ a* k is k th column vector of A H Exercise: –A is Unitary transform or not? –If so, find basis images –Represent an image X with basis images (Jain’s e.g.5.1, pp137: A’ [5, –1; – 2, 0] A; outer product of columns of A H : [1,1]’[1 1]/2, …) UMCP ENEE631 Slides (created by M.Wu © 2001)

4. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [4] Review and Exercise on Basis Images Exercise: –A is unitary transform or not? –Find basis images –Represent an image X with basis images (Jain’s e.g.5.1, pp137: A’ [5, –1; – 2, 0] A; outer product of columns of A H : [1,1]’[1 1]/2, …) UMCP ENEE631 Slides (created by M.Wu © 2001)

5. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [5] Review: Baseline JPEG Algorithm “Baseline”: simple, lossy compression u Subset of various DCT-based modes of JPEG standard A few basics: 8x8 block-DCT based coding –Shift to zero-mean by subtracting 128  [-128, 127] u Allows using signed integer to represent both DC and AC coefficients –Color representation: YCbCr / YUV u Color components can have lower spatial resolution than luminance u Interleaving color components: 4 Y blocks, 1 U block, 1 V block => Flash demo on Baseline JPEG algorithm by Dr. Ken Lam (HK PolyTech Univ.) (Based on Wang’s video book Chapt.1) UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

6. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [6] More on Color Interleaving – Different components are ordered into Minimum Coding Unit (MCU) – MCUs define repeating interleaving patterns Minimum coding unit (MCU) MCU1 = {Y00, Y01, Y10, Y11, U00, V00} MCU2 = {Y02, Y03, Y12, Y13, U01, V01}    YUV UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

7. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [7] Recap: JPEG Still Image Coding From B. Liu PU EE488 F’06 Lossy, block based, transform coding

8. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [8] Review: Block Diagram of JPEG Baseline From Wallace’s JPEG tutorial (1992)

9. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [9] Lossy Part in JPEG Important tradeoff between bit rate and visual quality Quantization (adaptive bit allocation) –Different quantization step size for different coefficient bands –Use same quantization matrix for all blocks in one image –Choose quantization matrix to best suit a specific image –Different quantization matrices for luminance and color components Default quantization table –“Generic” over a variety of images Quality factor “Q”  [1, 100] –Scale the quantization table –Medium quality Q = 50 ~ no scaling –High quality Q = 100 ~ quantization step is 1 –Low quality ~ small Q, larger quantization step u visible artifacts like ringing and blockiness UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

10. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [10] Quantization Table Recommended in JPEG –Take account of human visual properties and statistics from representative natural images –“Optimal” quantization tables vary, depending on image content, desired bit rate, and distortion criterion => need to send decoder the tables used 8x8 Quantization Table for Luminance 16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99 8x8 Quantization Table for Chrominance 17 18 24 47 99 99 99 99 18 21 26 66 99 99 99 99 24 26 56 99 99 99 99 99 47 66 99 99 99 99 99 99 99 99 99 99 99 99 99 99

11. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [11] Achieving Different Coding Bit Rate vs Distortion Adjust quantization by multiplying scale factor to base quantization tables (below is a commonly used scaling) –A convenient way to achieve different encoding bit rate vs distortion u Medium quality Q = 50 ~ no scaling u High quality Q = 100 ~ quantization step is 1 (i.e. just round coeff. to integer) u Low quality ~ larger quantization step Determine MSE introduced –Energy preservation by unitary transf. => MSE in DCT coefficients equal MSE in image signal samples –Artifacts contributed by DCT basis images of strongly quantized freq. bands “Optimal” quantization tables –Depend on image content, desired bit rate, and distortion criterion –Need to inform decoder what the quantization tables were used u Encode the customized table, or quality factor if standard table is used

12. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [12] Lossless Coding Part in JPEG Differentially encode DC –(lossy part: DC differences are then quantized.) AC coefficients in one block – Zig-zag scan after quantization for better run-length u save bits in coding consecutive zeros – Represent each AC run-length using entropy coding u use shorter codes for more likely AC run-length symbols UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

13. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [13] Lossless Coding Part in Baseline JPEG: Details Differentially encode DC –( SIZE, AMPLITUDE ), with amplitude range in [-2048, 2047] AC coefficients in one block – Zig-zag scan for better run-length – Represent each AC with a pair of symbols u Symbol-1: ( RUNLENGTH, SIZE )  Huffman coded u Symbol-2: AMPLITUDE  Variable length coded RUNLENGTH  [0,15] # of consecutive zero-valued AC coefficients preceding the nonzero AC coefficient  [0,15] SIZE  [0 to 10 in unit of bits] # of bits used to encode AMPLITUDE AMPLITUDE  in range of [-1023, 1024] UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

14. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [14] Encoding Non-zero AC Coefficients DCT AC coefficients of natural images follow Laplacian- like distribution –use shorter codewords for coeff. with small magnitude, and vice versa UMCP ENEE631 Slides (created by M.Wu © 2004) Table is from slides at Gonzalez/ Woods DIP 2/e book website (Chapter 8); Baseline JPEG use a smaller part of the coefficient range.

15. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [15] Recall: Properties of 1-D Unitary Transform y = A x Energy Conservation || y || 2 = || x || 2 u || y || 2 = || Ax || 2 = (Ax) *T (Ax)= x *T A *T A x = x *T x = || x || 2 So if y +  y = A ( x +  x ) => ||  y || 2 = ||  x || 2 u This allows for measuring distortion in either domain Orthogonal matrix: A -1 = A T –Real-valued unitary matrix is also an orthogonal matrix –Row vectors of real orthogonal matrix A form orthonormal basis vectors u Not so for complex orthogonal matrix => inner product is defined with complex conjugation UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

16. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [16] Subband/Wavelet Coding UMCP ENEE631 Slides (created by M.Wu © 2004)

17. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [17] Wavelet Transform for Image Compression ENEE631 emphasis –Focus on conceptual aspects related to image compression –Multiresolution analysis through wavelet transforms also useful for image modeling, denoising, enhancement, and content analysis –Build upon filterbank and subband coding from ENEE630’s multirate signal processing (For more in-depth info. on wavelet: wavelet course offered in Math Dept.) K-level 1-D wavelet/subband decomposition –Successive lowpass/highpass filtering and downsampling u on different level: capture transitions of different frequency bands u on the same level: capture transitions at different locations UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

18. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [18] Review: Subband Coding Concept

19. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [19] Successive Wavelet/Subband Decomposition Successive lowpass/highpass filtering and downsampling u on different level: capture transitions of different frequency bands u on the same level: capture transitions at different locations Figure from Matlab Wavelet Toolbox Documentation UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

20. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [20] Examples of 1-D Wavelet Transform From Matlab Wavelet Toolbox Documentation UMCP ENEE631 Slides (created by M.Wu © 2001) Figures are from slides at Gonzalez/ Woods DIP book 2/e website (Chapter 7)

21. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [21] UMCP ENEE631 Slides (created by M.Wu © 2004) Review: Filterbank & Multiresolution Analysis

22. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [22] Figures are from slides at Gonzalez/ Woods DIP book website (Chapter 7)

23. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [23] 2-D Wavelet Transform via Separable Filters From Matlab Wavelet Toolbox Documentation UMCP ENEE631 Slides (created by M.Wu © 2001)

24. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [24] UMCP ENEE631 Slides (created by M.Wu © 2004) 2-D Example From Usevitch’s article in IEEE Sig.Proc. Mag. 9/01 Separable transform by successively applying 1-D DWT to rows and columns

25. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [25] Subband Coding Techniques General coding approach –Allocate different bits for coefficients in different frequency bands –Encode different bands separately –Example: DCT-based JPEG and early wavelet coding   Some difference between subband coding and early wavelet coding ~ Choices of filters –Subband filters aims at (approx.) non-overlapping frequent response –Wavelet filters has interpretations in terms of basis and typically designed for certain smoothness constraints => will discuss more Shortcomings of subband coding –Difficult to determine optimal bit allocation for low bit rate applications –Not easy to accommodate different bit rates with a single coded stream –Difficult to encode at an exact target rate UMCP ENEE631 Slides (created by M.Wu © 2001)

26. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [26] Recall: Filterbank Perspective of Block Processing From Assignment#1

27. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [27] UMCP ENEE631 Slides (created by M.Wu © 2004) Smoothness Conditions on Wavelet Filter –Ensure the low band coefficients obtained by recursive filtering can provide a smooth approximation of the original signal From M. Vetterli’s wavelet/filter-bank paper

28. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [28] Embedded Zero-Tree Wavelet Coding (EZW) “Modern” lossy wavelet coding exploits multi-resolution and self-similar nature of wavelet decomposition –Energy is compacted into a small number of coefficients –Significant coeff. tend to cluster at the same spatial location in each frequency subband u time-freq. or space–freq. analysis Two set of info. to code: –Where are the significant coefficients? –What values are the significant coefficients? From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

29. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [29] Key Concepts in EZW Parent-children relation among coeff. –Each parent coeff at level k spatially correlates with 4 coeff at level (k-1) of same orientation –A coeff at lowest band correlates with 3 coeff. Coding significance map via zero-tree –Encode only high energy coefficients u Need to send location info.  large overhead –Encode “insignificance map” w/ zero-trees Successive approximation quantization –Send most-significant-bits first and gradually refine coefficient value –“Embedded” nature of coded bit-stream u get higher fidelity image by adding extra refining bits From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

30. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [30] EZW Algorithm and Example Initial threshold ~ 2 ^ floor(log 2 x max ) –Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands) –Assign symbol to each coeff. and entropy encode symbols u ps – positive significance u ns – negative significance u iz – isolated zero u ztr – zero-tree root –Significant coefficients u Move to subordinate list u Set them to zero in dominant list Subordinate Pass –Output one bit for subordinate list u According to position in up/low half of quantization interval Repeat with half threshold –Until bit budget achieved From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

31. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [31] After 1 st Pass From Usevitch (IEEE Sig.Proc. Mag. 9/01) Divide quantization interval of [32,64) by half: [32,48) => 40 and [48, 64) => 56 UMCP ENEE631 Slides (created by M.Wu © 2001)

32. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [32] After 2 nd Pass From Usevitch (IEEE Sig.Proc. Mag. 9/01) [16,24) => 20, [24, 32) => 28; [32,40) => 36, [40,48) => 44; [48, 56) => 52, [56, 64) => 60 UMCP ENEE631 Slides (created by M.Wu © 2001)

33. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [33] EZW and Beyond Can apply DWT to entire images or larger blocks than 8x8 Symbol sequence can be entropy encoded –e.g. arithmetic coding Cons of EZW –Poor error resilience; Difficult for selective spatial decoding SPIHT (Set Partitioning in Hierarchal Trees) –Further improvement over EZW to remove redundancy EBCOT (Embedded Block Coding with Optimal Truncation) –Used in JPEG 2000 –Address the shortcomings of EZW (random access, error resilience, …) –Embedded wavelet coding in each block + bit-allocations among blocks UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

34. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [34] A Close Look at Wavelet Transform Exercise: Show Haar Transform is unitary and orthogonal UMCP ENEE631 Slides (created by M.Wu © 2004)

35. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [35] Construction of Haar Functions Unique decomposition of integer k  (p, q) – k = 0, …, N-1 with N = 2 n, 0 <= p <= n-1 – q = 0, 1 (for p=0); 1 0) e.g., k=0 k=1 k=2 k=3 k=4 … (0,0) (0,1) (1,1) (1,2) (2,1) … h k (x) = h p,q (x) for x  [0,1] k = 2 p + q – 1 “remainder” power of 2 1 x UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

36. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [36] Haar Transform Haar transform H –Sample h k (x) at {m/N} u m = 0, …, N-1 –Real and orthogonal –Transition at each scale p is localized according to q Basis images of 2-D (separable) Haar transform –Outer product of two basis vectors UMCP ENEE631 Slides (created by M.Wu © 2001)

37. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [37] Summary of Today’s Lecture Compression via subband/wavelet coding –Subband decomposition –EZW: exploit structures between coefficients for removing redundancy Next Lecture: –More on wavelet transforms Readings –Gonzalez’s 3/e book Section 8.2.8; 8.2.10, 7.1, 7.4-7.5 Wallace’s paper on JPEG compression standard Usevitch’s SPM Sept.2001 tutorial paper on wavelet compression To explore further: Gonzalez’ 3/e book 7.2-7.3 (wavelet) IEEE Sig. Proc. Magazine Special Issue on transform coding (Sept.2001); Bovik’s Handbook 5.4 (wavelet compression) & 5.5 (JPEG lossy) UMCP ENEE631 Slides (created by M.Wu © 2004)

38. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [38] Reference [Copy] Compandor design of Waveform coding book [Copy] Transform coding article in IEEE Sig. Proc. Magazine Sept.’01 (special issue on transf. coding) [Copy] Bovik’s book Chapt.5.1 (Lossless coding) … huffman, lempel-ziv, etc. [links of E-copy] Wallace’s JPEG paper [VQ] –A. Gersho and R. M. Gray, Vector Quantization and Signal Compression.Vector Quantization and Signal Compression –R. M. Gray, ``Vector Quantization,'' IEEE ASSP Magazine, pp. 4--29, April 1984. –Y. Linde, A. Buzo, and R. M. Gray, ``An Algorithm for Vector Quantizer Design,'' IEEE Transactions on Communications, pp. 702--710, January 1980. Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

39. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [39] A Close Look at Wavelet Transform Haar Transform – unitary Orthonormal Wavelet Filters Biorthogonal Wavelet Filters UMCP ENEE631 Slides (created by M.Wu © 2004)

40. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [40] Compare Basis Images of DCT and Haar See also: Jain’s Fig.5.2 pp136 UMCP ENEE631 Slides (created by M.Wu © 2001)

41. M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [41] Summary on Haar Transform Two major sub-operations –Scaling captures info. at different frequencies –Translation captures info. at different locations Can be represented by filtering and downsampling Relatively poor energy compaction –Equiv. filter response doesn’t have good cutoff & stopband attenuation 1 x UMCP ENEE631 Slides (created by M.Wu © 2001)