M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering.

M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park   bb.eng.umd.edu (select ENEE631 S’09)   minwu@eng.umd.edu ENEE631 Spring’09 Lecture 13 (3/11/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [2] Overview and Logistics Last Time: –JPEG Compression –Subband and Wavelet based compression u Subband decomposition; basic ideas of EZW encoding Today –Continue on Subband and Wavelet based compression u Exploit the structures between coefficients for removing redundancy u More on wavelet transform, and consideration on filters Assignment#3 is Due Friday noon March 13 –JPEG part may be turned in after spring break Project#1 Posted. Due Monday April 6 UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [3] UMCP ENEE631 Slides (created by M.Wu © 2004) Review: 2-D Subband/DWT Decomposition From Usevitch’s article in IEEE Sig.Proc. Mag. 9/01 Separable transform by successively applying 1-D DWT to rows and columns

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [4] Review: Key Concepts in EZW Exploit multi-resolution and self-similarity Parent-children relation among coeff. –Each parent coeff at level k spatially correlates with 4 coeff at level (k-1) of same orientation –A coeff at lowest band correlates with 3 coeff. Coding significance map via zero-tree –Encode “insignificance map” w/ zero-trees u Avoid large overhead by coding only significant coefficients Successive approximation quantization –Send most-significant-bits first and gradually refine coefficient value –“Embedded” nature of coded bit-stream u get higher fidelity image by adding extra refining bits => Quality Scalability with fine granularity From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [5] Recall: EZW Algorithm and Example Initial threshold ~ 2 ^ floor(log 2 x max ) –Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands) –Assign symbol to each coeff. and entropy encode symbols u ps – positive significance u ns – negative significance u iz – isolated zero u ztr – zero-tree root –Significant coefficients u Move to subordinate list u Set them to zero in dominant list Subordinate Pass –Output one bit for subordinate list u According to position in up/low half of quantization interval Repeat with half threshold, until bit budget achieved –E.g. first look for coeff. above 32; then focus on coeff. between 16 and 32 Fig. From Usevitch (IEEE Sig. Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [6] After 1 st Pass From Usevitch (IEEE Sig.Proc. Mag. 9/01) Divide quantization interval of [32,64) by half: [32,48) => 40 and [48, 64) => 56 UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [7] After 2 nd Pass From Usevitch (IEEE Sig.Proc. Mag. 9/01) [16,24) => 20, [24, 32) => 28; [32,40) => 36, [40,48) => 44; [48, 56) => 52, [56, 64) => 60 2 nd Subordinate Pass: orig. 53  symbol 0  reconstruct 52 34  0  36 -22  0  -20 21  0  20 UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [8] EZW and Beyond Can apply DWT to entire images or larger blocks than 8x8 Symbol sequence can be entropy encoded –e.g. arithmetic coding Cons of EZW –Poor error resilience; Difficult for selective spatial decoding SPIHT (Set Partitioning in Hierarchal Trees) –Further improvement over EZW to remove redundancy –Achieve equal/better performance than EZW w/o arithmetic coding EBCOT (Embedded Block Coding with Optimal Truncation) –Used in JPEG 2000 –Address the shortcomings of EZW (random access, error resilience, …) –Embedded wavelet coding in each block + bit-allocations among blocks UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [9] A Close Look at Wavelet Transform Haar Transform – unitary Orthonormal Wavelet Filters Biorthogonal Wavelet Filters UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [10] Construction of Haar Functions Unique decomposition of integer k  (p, q) – k = 0, …, N-1 with N = 2 n, 0 <= p <= n-1 – q = 0, 1 (for p=0); 1 0) e.g., k=0 k=1 k=2 k=3 k=4 … (0,0) (0,1) (1,1) (1,2) (2,1) … h k (x) = h p,q (x) for x  [0,1] k = 2 p + q – 1 “remainder” power of 2 1 x UMCP ENEE631 Slides (created by M.Wu © 2001/2004) (0,0) (0,1) (1,1) (1,2) (2,1)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [11] Haar Transform Haar transform H –Sample h k (x) at {m/N} for m = 0, …, N-1 –Real and orthogonal –Transition at each scale p is localized according to q Basis images of 2-D (separable) Haar transform –Outer product of two basis vectors UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [13] Time-Freq (or Space-Freq) Interpretations –Standard Basis (for data samples); Fourier Basis; Wavelet Basis

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [14] Recap on Haar Transform Two major sub-operations –Scaling captures info. at different frequencies –Translation captures info. at different locations Can be represented by filtering and downsampling Relatively poor energy compaction –Equiv. filter response doesn’t have good cutoff & stopband attenuation 1 x UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [15] Orthonormal Filters Equiv. to projecting input signal to orthonormal basis Energy preservation property –Convenient for quantizer design u MSE by transform domain quantizer is same as reconstruction MSE in image domain Shortcomings: “coefficient expansion” –Linear filtering with N-element input & M-element filter  (N+M-1)-element output  (N+M)/2 after downsample –Length of output per stage grows ~ undesirable for compression Solutions to coefficient expansion –Symmetrically extended input (circular convolution) & Symmetric filter UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [16] Solutions to Coefficient Expansion Circular convolution in place of linear convolution –Periodic extension of input signal –Problem: artifacts by large discontinuity at borders Symmetric extension of input –Reduce border artifacts (note the signal length doubled with symmetry) –Problem: output at each stage may not be symmetric From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [17] Solutions to Coefficient Expansion (cont’d) Symmetric extension + symmetric filters –No coefficient expansion and little artifacts –Symmetric filter (or asymmetric filter) => “linear phase filters” (no phase distortion except by delays) Problem –Only one set of linear phase filters for real FIR orthogonal wavelets  Haar filters: (1, 1) & (1,-1) do not give good energy compaction Ref: review ENEE630 discussions on FIR perfect reconstruction Qudrature Mirror Filters (QMF) for 2-channel filter banks. UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [18] Review: Quadrature Mirror Filter (QMF) Bank

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [21] Biorthogonal Wavelets “Biorthogonal” –Basis in forward and inverse transf. are not the same but give overall perfect reconstruction (PR) u recall EE630 PR filterbank –No strict orthogonality for transf. filters so energy is not preserved u But could be close to orthogonal filters’ performance Advantage –Covers a much broader class of filters u including symmetric filters that eliminate coefficient expansion Commonly used filters for compression –9/7 biorthogonal symmetric filter –Efficient implementation: Lifting approach (ref: Swelden’s tutorial) UMCP ENEE631 Slides (created by M.Wu © 2001) From Usevitch (IEEE Sig.Proc. Mag. 9/01)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [22] UMCP ENEE631 Slides (created by M.Wu © 2004) Smoothness Conditions on Wavelet Filter –Ensure the low band coefficients obtained by recursive filtering can provide a smooth approximation of the original signal From M. Vetterli’s wavelet/filter-bank paper

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [23] More on Wavelets (1) UMCP ENEE631 Slides (created by M.Wu © 2004) Linear expansion of a function via an expansion set –Form basis functions if the expansion is unique Orthogonal basis Non-orthogonal basis –Coefficients are computed with a set of dual-basis Discrete Wavelet Transform –Wavelet expansion gives a set of 2-parameter basis functions and expansion coefficients: scale and translation

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [24] More on Wavelets (2) UMCP ENEE631 Slides (created by M.Wu © 2004) 1 st generation wavelet systems: –Scaling and translation of a generating wavelet (“mother wavelet”) Multiresolution conditions: –Use a set of basic expansion signals with half width and translated in half step size to represent a larger class of signals than the original expansion set (the “scaling function”) Represent a signal by combining scaling functions and wavelets

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [27] JPEG 2000: A Wavelet-Based New Standard Targets and features –Excellent low bit rate performance without sacrifice performance at higher bit rate –Progressive decoding to allow from lossy to lossless –Region-of-interest (ROI) coding –Error resilience For details –JPEG2000 Tutorial by Skrodras @ IEEE Sig. Proc Magazine 9/2001 –David Taubman: “High Performance Scalable Image Compression with EBCOT”, IEEE Trans. On Image Proc, vol.9(7), 7/2000. –Taubman’s book on JPEG 2000 (on library reserve) –Links and tutorials @ http://www.jpeg.org/JPEG2000.htmhttp://www.jpeg.org/JPEG2000.htm UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [29] DCT vs. Wavelet: Which is Better? 3dB improvement? –Wavelet compression was claimed to have 3dB improvement over DCT-based compression –Comparison is done on JPEG Baseline Improvement not all due to transforms –Main contribution from better rate allocation, advanced entropy coding, & smarter redundancy reduction via zero-tree –Difference due to DCT vs. Wavelet transform is less than 1dB –DCT coder can be improved to decrease the gap [Ref] "A comparative study of DCT- and wavelet-based image coding", Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang, IEEE Trans. on Circuits and Systems for Video Technology, v.9, no.5, 8/99, pp692-695. UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [30] Non-Dyadic Decomposition – Wavelet Packets Figures are from slides at Gonzalez/ Woods DIP book 2/e website (Chapter 7)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [31] Bit Allocation in Image Coding Bit Allocation in Image Coding Focus on MSE-based optimization; Can further adjust based on HVS UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [32] Here  k 2 : variance of k-th coeff. v(k); r k is # bits allocated for v(k) f(-): a func. relating bit rate to distortion based on a coeff’s p.d.f. Bit Allocation How many bits to be allocated for each coefficient? –Related to each coeff’s variance (and probability distribution) –More bits for high variance  k 2 to keep total MSE small “Reverse Water-filling” – –Try to keep same amount of error in each frequency band UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [33] Rate/Distortion Allocation via Reverse Water-filling UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Optimal solution for Gaussian distributed coefficients Idea: Try to keep same amount of error in each freq. band and no need to spend bits to represent coeff. w/ smaller variance than water level –Results based on R-D function and via Lagrange-multiplier optimization –Note the convex shape of R-D function for Gaussian => slope becomes milder; require more bits to further reduce distortion => “Equal slope” idea can be extended to other convex (operational) R-D functions D RDRD 22 Given D, determine  and then R D ; or vice versa for given total rate

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [34] Rate/Distortion Allocation via Reverse Water-filling UMCP ENEE631 Slides (created by M.Wu © 2001/2004) Optimal solution for Gaussian distributed coefficients Idea: try to keep same amount of error in each freq. band; no need to spend bits to represent coeff. w/ smaller variance than water level –Results based on R-D function and via Lagrange-multiplier optimization –Note the convex shape of R-D function for Gaussian => slope becomes milder; require more bits to further reduce distortion D RDRD 22 Given total rate R, determine  and then D; or vice versa given D, determine R Optimal Not Optimal

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [35] Details on Reverse Water-filling Solution Construct func. using Lagrange multiplier Necessary condition  Keep the same marginal gain Equal Slope Condition for bit allocation D RDRD 22 Resulting in equal distortions on Gaussian r.v’s as interpreted in “reverse water filling”

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [36] Key Result in Rate Allocation: Equal R-D Slope Keep the slope in R-D curve the same (not just for Gaussian r.v) –Otherwise some bits can be applied to other r.v. for better “return” in reducing the overall distortion If all r.v. are Gaussian, the same slope in R-D curves for these r.v. correspond to an identical amount of distortion. R(D) = ½ [ log  2 – log D ] => dR/dD = – 1 / (2D) Use operational/measured rate-distortion information for general r.v. UMCP ENEE631 Slides (created by M.Wu © 2004) Rate allocation results: => Equal slope (o.w. exist a better allocation strategy)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [37] Summary of Today’s Lecture Wavelet transform and coding –Haar basis/transform; considerations for wavelet filters; JPEG 2000 Optimal bit allocation: “equal slope” in R-D curves –Lead to reverse water filling for Gaussian coefficients Next lecture –Optimal transform Readings: see wavelet coding references in Project #1 –Gonzalez’s 3/e book Section 8.2.10; 7.1, 7.4-7.5; (7.2-7.3 on wavelet) –Transform coding articles in IEEE Sig. Proc. Magazine Sept. 2001 (special issue on transform coding) –To explore more: Tutorial on rate-distortion by Ortega-Ramchandran in IEEE Sig. Proc. Magazine, Nov. 98, and Jain’s book Section 2.13; “Wavelets and the lifting scheme: A 5 minute tour” by Wim Sweldens [Zeitschrift für Angewandte Mathematik und Mechanik, 76 (2), pp. 41-44, 1996] UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [38] Review: Correlation After a Linear Transform Consider an Nx1 zero-mean random vector x – Covariance (or autocorrelation) matrix R x = E[ x x H ] u give us ideas of the correlation between elements u R x is a diagonal matrix for if all N r.v.’s are uncorrelated Apply a linear transform to x: y = A x What is the autocorrelation matrix for y ? R y = E[ y y H ] = E[ (Ax) (Ax) H ] = E[ A x x H A H ] = A E[ x x H ] A H = A R x A H Decorrelation: try to search for A that can produce a decorrelated y (equiv. a diagonal correlation matrix R y ) UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [39]

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [40] Optimal Transform Recall: Why use transform in coding/compression? –Decorrelate the correlated data –Pack energy into a small number of coefficients –Interested in unitary/orthogonal or approximate orthogonal transforms u Energy preservation s.t. quantization effects can be better understood and controlled Unitary transforms we’ve dealt so far are data independent –Transform basis/filters are not depending on the signals we are processing u Can hard-wire them in encoder and decoder What unitary transform gives the best energy compaction and decorrelation? –“Optimal” in a statistical sense to allow the codec works well with many images=> Signal statistics would play an important role UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [41] Review: Correlation After a Linear Transform Consider an Nx1 zero-mean random vector x – Covariance (or autocorrelation) matrix R x = E[ x x H ] u give ideas of correlation between elements u R x is a diagonal matrix for if all N r.v.’s are uncorrelated Apply a linear transform to x: y = A x What is the correlation matrix for y ? R y = E[ y y H ] = E[ (Ax) (Ax) H ] = E[ A x x H A H ] = A E[ x x H ] A H = A R x A H Decorrelation: try to search for A that can produce a decorrelated y (equiv. a diagonal correlation matrix R y ) UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [42] K-L Transform (Principal Component Analysis) Eigen decomposition of R x : R x u k = k u k –Recall the properties of R x u Hermitian (conjugate symmetric R H = R); u Nonnegative definite (real non-negative eigen values) Karhunen-Loeve Transform (KLT) y = U H x  x = U y with U = [ u 1, … u N ] –KLT is a unitary transform with basis vectors in U being the orthonormalized eigenvectors of R x –R x U = [ 1 u 1, … N u N ] = U diag{ 1, 2, …, N } => U H R x U = diag{ 1, 2, …, N } i.e. KLT performs decorrelation u Often order {u i } so that 1  2  …  N –Also known as Hotelling transform or Principle Component Analysis (PCA) UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [43] K-L Transform (Principal Component Analysis) Eigen decomposition of R x : R x u k = k u k –Recall the properties of R x u Hermitian (conjugate symmetric R H = R); u Nonnegative definite (real non-negative eigen values) Karhunen-Loeve Transform (KLT) y = U H x  x = U y with U = [ u 1, … u N ] –KLT is a unitary transform with basis vectors in U being the orthonormalized eigenvectors of R x –R x U = U diag{ 1, 2, …, N } –E[ y y H ]= E[ (U H x) (U H x) H ]= U H E[ x x H ] U = U H R x U = diag{ 1, 2, …, N } i.e. KLT performs decorrelation u Often order {u i } so that 1  2  …  N –Minimizing MSE under basis restriction u If only to keep m coefficients (1  m  N), the best way to minimize reconstruction error is to keep the coeff. w.r.t. the eigenvectors of the first m largest eigenvalues UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [44] Properties of K-L Transform Decorrelation –E[ y y H ]= E[ (U H x) (U H x) H ]= U H E[ x x H ] U = diag{ 1, 2, …, N } –Note: Other matrices (unitary or nonunitary) may also decorrelate the transformed sequence [ Jain’s e.g.5.7 pp166] Minimizing MSE under basis restriction –If only allow to keep m coefficients for any 1  m  N, what’s the best way to minimize reconstruction error?  Keep the coefficients w.r.t. the eigenvectors of the first m largest eigen values u Reference: Theorem 5.1 and Proof in Jain’s Book (pp166) UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [45] KLT Basis Restriction Basis restriction –Keep only a subset of m transform coefficients and then perform inverse transform (1  m  N) –Basis restriction error: MSE between original & new sequences Goal: to find the forward and backward transform matrices to minimize the restriction error for each and every m –The minimum is achieved by KLT arranged according to the decreasing order of the eigenvalues of R UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [46] K-L Transform for Images Work with 2-D autocorrelation function –R(m,n; m’,n’)= E[ x(m, n) x(m’, n’) ] for all 0  m, m’, n, n’  N-1 –K-L Basis images is the orthonormalized eigen functions of R Rewrite images into vector form (N 2 x1) –Need solve the eigen problem for N 2 xN 2 matrix! ~ O(N 6 ) Reduced computation for separable R –R(m,n; m’,n’)= r 1 (m,m’) r 2 (n,n’) –Only need solve the eigen problem for two NxN matrices ~ O(N 3 ) –KLT can now be performed separably on rows and columns u Reducing the transform complexity from O(N 4 ) to O(N 3 ) UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [47] Pros and Cons of K-L Transform Optimality –Decorrelation and MMSE for the same# of partial transf coeff. Data dependent –Have to estimate the 2 nd -order statistics to determine the transform –Can we get data-independent transform with similar performance? u DCT Applications –(non-universal) compression –Pattern recognition: e.g., eigen faces –Analyze the principal (“dominating”) components and reduce feature dimensions UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [48] Energy Compaction of DCT vs. KLT DCT has excellent energy compaction for highly correlated data DCT is a good replacement for K-L –Close to optimal for highly correlated data –Not depend on specific data like K-L does –Fast algorithm available [ref and statistics: Jain’s pp153, 168-175] UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [49] Energy Compaction of DCT vs. KLT (cont’d) Preliminaries –The matrices R, R -1, and  R -1 share the same set of eigen vectors –DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrix Q c –Covariance matrix R of 1 st -order stationary Markov sequence with has an inverse in the form of symmetric tri-diagonal matrix DCT is close to KLT on 1 st -order stationary Markov –For highly correlated sequence, a scaled version of R -1 approx. Q c UMCP ENEE631 Slides (created by M.Wu © 2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [51] Hadamard Transform From Mani Thomas U.Del CISC489/689 2D FT http://vims.cis.udel.edu/~mani/TACourses/Spring06/cv/index.html

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [52] Summary and Review on Unitary Transform Representation with orthonormal basis  Unitary transform –Preserve energy Common unitary transforms –KLT; DFT, DCT, Haar … Which transform to choose? –Depend on need in particular task/application –DFT ~ reflect physical meaning of frequency or spatial frequency –KLT ~ optimal in energy compaction –DCT ~ real-to-real, and close to KLT’s energy compaction => A comparison table in Jain’s book Table 5.3 UMCP ENEE631 Slides (created by M.Wu © 2001)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [54] Bit Allocation Among Indep. r.v. of Different Var. Problem formulation –To encode a set of independent Gaussian r.v. {X1, …, Xn}, X k ~ N( 0,  k 2 ) –Allocate R k bits to represent each r.v. X k, incurring distortion D k –Total bit cost is R = R1+…+Rn –Total MSE distortion D = D1+…+Dn  What is the best bit allocation {R1, …, Rn} such that R is minimized and total distortion D  D (req) ?  What is the best bit allocation {R1, …, Rn} such that D is minimized and total rate R  R (req) ? –Recall R k = max( ½ * log(  k 2 / D k ), 0 ) –Solving the constrained optimization problem using Lagrange multiplier  0 UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [55] Rate/Distortion Allocation via Reverse Water-filling How many bits to be allocated for each coefficient? –Related to the variance of the coefficients –More bits for high variance  k 2 to keep total MSE small Optimal solution for Gaussian: “Reverse Water-filling” – –Idea: Try to keep same amount of error in each freq. band and no need to spend bit resource to represent coeff. w/ small variance than water level – –Results based on R-D function & via Lagrange-multiplier optimization given D, determine  and then R D ; or vice versa for given R D => “Equal slope” idea can be extended to other convex (operational) R-D functions UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [56] Revisiting Quantizer Quantizer achieves compression in a lossy way –Lloyd-Max quantizer minimize MSE distortion with a given rate Need at least how many # bits for certain amount of error? => Rate distortion function of a r.v. –Minimum average rate R D bits/sample required to represent this r.v. while allowing a fixed distortion D –For Gaussian r.v. and MSE u Convex shape (slope is becoming milder) i.e. more bits required to further reduce distortion (See info. theory course for details on R-D theory) D RDRD 22 UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [57] Basic Steps in R-D Optimization Determine source’s statistics –Simplified models: Gaussian, Laplacian, Generalized Gaussian –Obtain from training samples Obtain operational R-D points/functions/curves –E.g., compute distortion for each candidate quantizer Determine objective function and constraints Search for optimal operational R-D points –Lagrange multiplier approach u select only operating points on the convex hull of overall R-D characteristics (Fig.19 of Ortega’s tutorial) u may handle different coding unit independently –Dynamic programming approach u not constrained by convex hull but has larger search space ~ higher complexity => See Ortega’s SPM’98 tutorial for details

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [58] Bridging the Theory and the Practice Operational R-D curves –Directly achievable R-D pairs with practical implementation Tradeoff! –How close are the R-D operational points to info. theory bounds? –Practical considerations: cost of memory, computation, delay, etc.  Often narrowing down to an optimization problem: ~ optimize an objective function subject to a set of constraints Problem of model mismatch –How good are our assumptions on source modeling? –How well does a coding algorithm tolerate model mismatch?

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [59] Ref: convex hull Formally –Smallest convex set containing all points u For R-D problem, we concerned of the lower boundary of this set that close to the info. theoretical bound (the upper/right end are open) –Convex set u if X1,..,Xn are in the set, then the linear combinations of them (with non-negative combination factors) are still in the set Intuitively –Use a rubber band to wrap the outermost points so that the band encloses all points

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [60] where  k 2 -- variance of k-th coeff. v(k); n k -- # bits allocated for v(k) f(-) – quantization distortion func. Bit Allocation How many bits to be allocated for each coeff.? –Determined by the variance of coeff. –More bits for high variance  k 2 to keep total MSE small “Inverse Water-filling” (from info. theory) – –Try to keep same amount of error in each freq. band  See Jain’s pp501 for detailed bit-allocation algorithm

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [61] Details on Reverse Water-filling Solution (cont’d) Construct func. using Lagrange multiplier Necessary condition  Keep the same marginal gain Necessary condition for choosing

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [62] General Constrained Optimization Problem Define Necessary condition: “Kuhn-Tucker conditions”

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [63] Lagrangian Opt. for Indep. Budget Constraint Previous: fix distortion, minimize total rate Alternative: fix total rate (bit budget), minimize distortion (Discrete) Lagrangian optimization on general source –“Constant slope optimization” (e.g. in Box 6 Fig. 17 of Ortega’s tutorial) –Need to determine the quantizer q(i) for each coding unit i –Lagrangian cost for each coding unit u use a line with slope - to intersect with each operating point (Fig.14) –For a given operating quality, the minimum can be computed independently for each coding unit => Find operating quality satisfying the rate constraint

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [64] (from Box 6 Fig. 17 of Ortega’s tutorial) (from Fig. 14 of Ortega’s tutorial)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [65] Bridging the Theory and Ad-hoc Practice Operational R-D curves –Directly achievable with practical implementation Tradeoff! Tradeoff! Tradeoff! –Rate vs. Distortion u how close are the operational points to info. theory bounds? –Also tradeoff among practical considerations u Cost of memory, computation, delay, etc.  Often narrowing down to an optimization problem: ~ optimize an objective func. subj. to a set of constraints Model mismatch problem –How good are our assumptions on source modeling? –How well does a coding algorithm tolerate model mismatch?

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [66] Basic Steps in R-D Optimization Determine source’s statistics –Simplified models: Gaussian, Laplacian, Generalized Gaussian –Obtain from training samples Obtain operational R-D points/functions/curves –E.g., compute distortion for each candidate quantizer Determine objective func. and constraints Search for optimal operational R-D points –Lagrange multiplier approach u select only operating points on the convex hull of overall R-D characteristics (Fig.19 of Ortega’s tutorial) u may handle different coding unit independently –Dynamic programming approach u not constrained by convex hull but has larger search space (higher complexity)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [67] Ref: convex hull Formally –Smallest convex set containing all points u For R-D problem, we concerned of the lower boundary of this set that close to the info. theoretical bound (the upper/right end are open) –Convex set u if X1,..,Xn are in the set, then the linear combinations of them (with non-negative combination factors) are still in the set Informally –Use a rubber band to wrap the outermost points so that the band encloses all points

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [68] (from Fig. 19 of Ortega’s tutorial)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering.

Similar presentations

Presentation on theme: "M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering.

Similar presentations

Presentation on theme: "M. Wu: ENEE631 Digital Image Processing (Spring'09) Wavelet Coding and Related Issues Spring ’09 Instructor: Min Wu Electrical and Computer Engineering."— Presentation transcript:

Similar presentations

About project

Feedback