Wavelet Based Image Coding 12/9/2018 Based on ENEE631 Spring’04 Section 11 Wavelet Based Image Coding 1st Lecture: Course logistics, etc. – 7 slides, about 15min Why img. proc. and why digital – 3 slides, about 8min Examples – 20min What’s an img? math. modeling of img, sampling and quantization – 5 slides, 15min 10min – fill out questionnaire Spring ’04 Instructor: Min Wu ECE Department, Univ. of Maryland, College Park www.ajconline.umd.edu (select ENEE631 S’04) minwu@eng.umd.edu

Wavelet Transform for Image Compression ENEE631 emphasis Focus on conceptual aspects related to image compression Wavelet is also useful for denoising, enhancement, and image analysis Build upon filterbank and subband coding from ENEE630 (For more in-depth info. on wavelet: wavelet course offered in Math Dept.) K-level 1-D wavelet/subband decomposition Successive lowpass/highpass filtering and downsampling on different level: capture transitions of different frequency bands on the same level: capture transitions at different locations UMCP ENEE631 Slides (created by M.Wu © 2001/2004) ENEE631 Digital Image Processing (Spring'04)

Successive Wavelet/Subband Decomposition UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Successive lowpass/highpass filtering and downsampling on different level: capture transitions of different frequency bands on the same level: capture transitions at different locations Figure from Matlab Wavelet Toolbox Documentation ENEE631 Digital Image Processing (Spring'04)

Examples of 1-D Wavelet Transform UMCP ENEE631 Slides (created by M.Wu © 2001) From Matlab Wavelet Toolbox Documentation ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2004) 2-D Example UMCP ENEE631 Slides (created by M.Wu © 2004) From Usevitch (IEEE Sig.Proc. Mag. 9/01) ENEE631 Digital Image Processing (Spring'04)

Subband Coding Techniques General coding approach Allocate different bits for coeff. in different frequency bands Encode different bands separately Example: DCT-based JPEG and early wavelet coding Some difference between subband coding and early wavelet coding ~ Choices of filters Subband filters aims at (approx.) non-overlapping freq. response Wavelet filters has interpretations in terms of basis and typically designed for certain smoothness constraints (=> will discuss more ) Shortcomings of subband coding Difficult to determine optimal bit allocation for low bit rate applications Not easy to accommodate different bit rates with a single code stream Difficult to encode at an exact target rate UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Review: Filterbank & Multiresolution Analysis UMCP ENEE631 Slides (created by M.Wu © 2004) ENEE631 Digital Image Processing (Spring'04)

Smoothness Conditions on Wavelet Filter Ensure the low band coefficients obtained by recursive filtering can provide a smooth approximation of the original signal UMCP ENEE631 Slides (created by M.Wu © 2004) From M. Vetterli’s wavelet/filter-bank paper ENEE631 Digital Image Processing (Spring'04)

Embedded Zero-Tree Wavelet Coding (EZW) “Modern” lossy wavelet coding exploits multi-resolution and self-similar nature of wavelet decomposition Energy is compacted into a small number of coeff. Significant coeff. tend to cluster at the same spatial location in each frequency subband Two set of info. to code: Where are the significant coefficients? What values are the significant coefficients? UMCP ENEE631 Slides (created by M.Wu © 2001) From Usevitch (IEEE Sig.Proc. Mag. 9/01) ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) Key Concepts in EZW Parent-children relation among coeff. Each parent coeff at level k spatially correlates with 4 coeff at level (k-1) of same orientation A coeff at lowest band correlates with 3 coeff. Coding significance map via zero-tree Encode only high energy coefficients Need to send location info.  large overhead Encode “insignificance map” w/ zero-trees Successive approximation quantization Send most-significant-bits first and gradually refine coeff. value “Embedded” nature of coded bit-stream get higher fidelity image by adding extra refining bits UMCP ENEE631 Slides (created by M.Wu © 2001) From Usevitch (IEEE Sig.Proc. Mag. 9/01) ENEE631 Digital Image Processing (Spring'04)

EZW Algorithm and Example Initial threshold ~ 2 ^ floor(log2 xmax) Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands) Assign symbol to each coeff. and entropy encode symbols ps – positive significance ns – negative significance iz – isolated zero ztr – zero-tree root Significant coeff. move to subordinate list put zero in dominant list Subordinate Pass Output one bit for subordinate list According to position in up/low half of quantization interval Repeat with half threshold Until bit budget achieved UMCP ENEE631 Slides (created by M.Wu © 2001) From Usevitch (IEEE Sig.Proc. Mag. 9/01) ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) 12/9/2018 After 1st Pass From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001) Divide quantization interval of [32,64) by half: [32,48) => 40 and [48, 64) => 56 ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) 12/9/2018 After 2nd Pass UMCP ENEE631 Slides (created by M.Wu © 2001) [16,24) => 20, [24, 32) => 28; [32,40) => 36, [40,48) => 44; [48, 56) => 52, [56, 64) => 60 From Usevitch (IEEE Sig.Proc. Mag. 9/01) ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Beyond EZW Cons of EZW Poor error resilience Difficult for selective spatial decoding SPIHT (Set Partitioning in Hierarchal Trees) Further improvement over EZW to remove redundancy EBCOT (Embedded Block Coding with Optimal Truncation) Used in JPEG 2000 Address the shortcomings of EZW (random access, error resilience, …) Embedded wavelet coding in each block + bit-allocations among blocks UMCP ENEE631 Slides (created by M.Wu © 2001/2004) ENEE631 Digital Image Processing (Spring'04)

JPEG 2000 Image Compression Standard UMCP ENEE631 Slides (created by M.Wu © 2004) ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) From Christopoulos (IEEE Trans. on CE 11/00) UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

JPEG 2000: A Wavelet-Based New Standard 12/9/2018 JPEG 2000: A Wavelet-Based New Standard Targets and features Excellent low bit rate performance without sacrifice performance at higher bit rate Progressive decoding to allow from lossy to lossless Region-of-interest (ROI) coding Error resilience For details David Taubman: “High Performance Scalable Image Compression with EBCOT”, IEEE Trans. On Image Proc, vol.9(7), 7/2000. JPEG2000 Tutorial by Skrodras @ IEEE Sig. Proc Magazine 9/2001 Taubman’s book on JPEG 2000 (on library reserve) Links and tutorials @ http://www.jpeg.org/JPEG2000.htm UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Ref: see also JPEG 2000 Tutorial by Christopolous et al @ IEEE Trans. on consumer electronics 11/00 ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) Examples JPEG2K vs. JPEG UMCP ENEE631 Slides (created by M.Wu © 2001) From Christopoulos (IEEE Trans. on CE 11/00) ENEE631 Digital Image Processing (Spring'04)

DCT vs. Wavelet: Which is Better? 3dB improvement? Wavelet compression was claimed to have 3dB improvement over DCT-based compression Comparison is done on JPEG Baseline Improvement not all due to transforms Main contribution from better rate allocation, advanced entropy coding, & smarter redundancy reduction via zero-tree DCT coder can be improved to decrease the gap [Ref] "A comparative study of DCT- and wavelet-based image coding",    Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang,    IEEE Trans. on Circuits and Systems for Video Tech.,    v.9, no.5, 8/99, pp692-695. UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

A Close Look at Wavelet Transform UMCP ENEE631 Slides (created by M.Wu © 2004) ENEE631 Digital Image Processing (Spring'04)

Construction of Haar functions Unique decomposition of integer k  (p, q) k = 0, …, N-1 with N = 2n, 0 <= p <= n-1 q = 0, 1 (for p=0); 1 <= q <= 2p (for p>0) e.g., k=0 k=1 k=2 k=3 k=4 … (0,0) (0,1) (1,1) (1,2) (2,1) … hk(x) = h p,q(x) for x  [0,1] k = 2p + q – 1 “reminder” power of 2 UMCP ENEE631 Slides (created by M.Wu © 2001/2004) 1 x ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) Haar Transform Haar transform H Sample hk(x) at {m/N} m = 0, …, N-1 Real and orthogonal Transition at each scale p is localized according to q Basis images of 2-D (separable) Haar transform Outer product of two basis vectors UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Compare Basis Images of DCT and Haar UMCP ENEE631 Slides (created by M.Wu © 2001) See also: Jain’s Fig.5.2 pp136 ENEE631 Digital Image Processing (Spring'04)

Summary on Haar Transform Two major sub-operations Scaling captures info. at different frequencies Translation captures info. at different locations Can be represented by filtering and downsampling Relatively poor energy compaction UMCP ENEE631 Slides (created by M.Wu © 2001) 1 x ENEE631 Digital Image Processing (Spring'04)

UMCP ENEE631 Slides (created by M.Wu © 2001) Orthonormal Filters Equiv. to projecting input signal to orthonormal basis Energy preservation property Convenient for quantizer design MSE by transform domain quantizer is same as reconstruction MSE Shortcomings: “coefficient expansion” Linear filtering with N-element input & M-element filter  (N+M-1)-element output  (N+M)/2 after downsample Length of output per stage grows ~ undesirable for compression Solutions to coefficient expansion Symmetrically extended input (circular convolution) & Symmetric filter UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Solutions to Coefficient Expansion Circular convolution in place of linear convolution Periodic extension of input signal Problem: artifacts by large discontinuity at borders Symmetric extension of input Reduce border artifacts (note the signal length doubled with symmetry) Problem: output at each stage may not be symmetric From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Solutions to Coefficient Expansion (cont’d) Symmetric extension + symmetric filters No coefficient expansion and little artifacts Symmetric filter (or asymmetric filter) => “linear phase filters” (no phase distortion except by delays) Problem Only one set of linear phase filters for real FIR orthogonal wavelets  Haar filters: (1, 1) & (1,-1) do not give good energy compaction UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Biorthogonal Wavelets From Usevitch (IEEE Sig.Proc. Mag. 9/01) Biorthogonal Wavelets “Biorthogonal” Basis in forward and inverse transf. are not the same but give overall perfect reconstruction (PR) recall EE624 PR filterbank No strict orthogonality for transf. filters so energy is not preserved But could be close to orthogonal filters’ performance Advantage Covers a much broader class of filters including symmetric filters that eliminate coefficient expansion Commonly used filters for compression 9/7 biorthogonal symmetric filter Efficient implementation: Lifting (see Swelden’s tuotial) UMCP ENEE631 Slides (created by M.Wu © 2001) ENEE631 Digital Image Processing (Spring'04)

Summary of Compression Tools Lossless compression tools Entropy coding Huffman, Arithmetic, Lempel-Ziv run-length Predictive coding reduce the dynamic range to code Transform enhance energy compaction Lossy compression tools Discarding and thresholding Quantization ( previous lectures ) Scalar quantizer Quantize more than one component => vector quantization ENEE631 Digital Image Processing (Spring'04)

Question for Today: Bring in Motion  Video Capturing video Frame by frame => image sequence Image sequence: A 3-D signal 2 spatial dimensions & time dimension continuous I( x, y, t ) => discrete I( m, n, tk ) Encode digital video Simplest way ~ compress each frame image individually e.g., “motion-JPEG” only spatial redundancy is explored and reduced How about temporal redundancy? Will differential coding work well? UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) ENEE631 Digital Image Processing (Spring'04)