EE565 Advanced Image Processing Copyright Xin Li Application of Wavelets (I): Denoising Problem formulation Frequency-domain solution: linear Wiener filtering The need for a better basis Wavelet-domain solution: nonlinear thresholding Advanced denoising in wavelet space Go beyond wavelets Focus will be on ideas instead of technical details

EE565 Advanced Image Processing Copyright Xin Li Problem Formulation Noisy measurements signalN(0,σ w 2 ) Toy Example: how to measure Yao Ming’s height? Assume independent measurement errors: {y 1, y 2,…,y N } y k =x+w k

EE565 Advanced Image Processing Copyright Xin Li Real Challenges X is not a scalar but a vector or matrix (curse of dimensionality) X is not a single function but a class of functions (even multiple classes of functions) Uncertainty in W: Y=X+W=(X+z)+(W-z) We are taking a deterministic view in this course

EE565 Advanced Image Processing Copyright Xin Li Sample Function Classes Original data set Noisy data set

EE565 Advanced Image Processing Copyright Xin Li Easier Problem to Solve What if the signal x(t) is assumed to be piecewise constant (blocks)? Nearly optimal solution does exist x’(t) contains impulses in time domain Nonlinear diffusion (e.g., minimize the total variation) Graduated Non-Convexity (GNC) Algorithm (e.g., detect transition locations) Averaging N noisy data of the same X reduces the noise variance by a factor of N

EE565 Advanced Image Processing Copyright Xin Li Model-Data Mismatch What happened if we apply the techniques designed for blocks to bumps? Fails miserably – no transition point exists Class of piecewise constant functions is too small “All models are wrong, only some are useful”. – George Box

EE565 Advanced Image Processing Copyright Xin Li Pursuit of Better Models Sobolev space functions Google it if you want to learn about the mathematical definition Characterize the class of smooth functions Fourier transform offers nearly optimal approximation (the first K FT coefficients are the principal components) Wiener filtering has a linear frequency weighting interpretation (the larger spectral magnitude, the higher weight)

EE565 Advanced Image Processing Copyright Xin Li Besov-Space Function Again, we skip its mathematical definition Intuitively, it is a function that is smooth almost everywhere (allow finite exceptions) More suitable for modeling edges in images

EE565 Advanced Image Processing Copyright Xin Li Failure of Linear Frequency Weighting Why Fourier transform is suboptimal for Besov-space function? Every exception causes infinitely many large coefficients (spread across the whole spectrum) Difficult to tell signal (e.g. edges) apart from noise due to the overlapping in positions At the mercy of Gibbs phenomenon

EE565 Advanced Image Processing Copyright Xin Li Wavelets Save the Day Now think of the class of compactly-supported wavelets They would produce finite (localized) large coefficients around exceptions Easier to tell signal (e.g. edges) apart from noise due to better separation in positions Reduced Gibbs Phenomenon (depends on the length of wavelet filters)

EE565 Advanced Image Processing Copyright Xin Li Wavelet Shrinkage WTIWT Nonlinear Thresholding Noisy signal denoised signal Hard thresholding Soft thresholding

EE565 Advanced Image Processing Copyright Xin Li Why Thresholding Works? We need to distinguish spatially- localized events (edges) from noise components Think about noise components Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain As long as it does not generate exceptions Additive White Gaussian Noise after WT is still AWGN

EE565 Advanced Image Processing Copyright Xin Li Tails of Distribution noise signal

EE565 Advanced Image Processing Copyright Xin Li Choice of Threshold Donoho and Johnstone’1994 Gives denoising performance close to the “ideal weighting” Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp )

EE565 Advanced Image Processing Copyright Xin Li Soft vs. Hard thresholding ● It can be also viewed as a computationally efficient approximation of ideal weighting soft ideal ● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better. ● Shrinking the amplitude by T guarantees with a high probability that.

EE565 Advanced Image Processing Copyright Xin Li Edges are kept, but the noise wasn’t fully suppressed Edges aren’t kept. However, the noise was almost fully suppressed

EE565 Advanced Image Processing Copyright Xin Li Denoising Example noisy image (σ 2 =100) Wiener-filtering (ISNR=2.48dB) Wavelet-thresholding (ISNR=2.98dB) X: original, Y: noisy, X: denoised ~ Improved SNR

EE565 Advanced Image Processing Copyright Xin Li Advanced Wavelet Denoising Translation Invariant Denoising Redundancy helps Spatially adaptive Wiener filtering of wavelet high-frequency band coefficients Extending thresholding to weighting Statistical modeling of wavelet high-frequency band coefficients Hidden Markov Model (Crouse et al.’1998) Gaussian Scalar Mixture (Portilla et al.’2003)

EE565 Advanced Image Processing Copyright Xin Li What do We Buy from Redundancy? 0 1 N-1 … x(n) H1H1 T -T

EE565 Advanced Image Processing Copyright Xin Li Translation Invariance (TI) Denoising T oe T oe -1 Thresholding T ce T ce -1 Thresholding T ce T ce -1 Thresholding z + x(n) Implementation based on overcomplete expansion Implementation based on complete expansion z -1

EE565 Advanced Image Processing Copyright Xin Li Gain Brought by Redundancy

EE565 Advanced Image Processing Copyright Xin Li From Thresholding to Weighting Challenges with wavelet thresholding Determination of a global optimal threshold Spatially adjusting threshold based on local statistics How to go beyond thresholding? We need an accurate modeling of wavelet coefficients – nonlinear thresholding is a computationally efficient yet suboptimal solution

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

EE565 Advanced Image Processing Copyright Xin Li Practical Implementation T T N=T 2 Recall Conceptually very similar to its counterpart in the spatial domain (ML estimation of signal variance) Better solution than ML is MAP (refer to GSM paper by Portilla et al.)

EE565 Advanced Image Processing Copyright Xin Li Example Translation-Invariant thresholding (ISNR=3.51dB) Spatially-adaptive wiener filtering (ISNR=4.53dB)

EE565 Advanced Image Processing Copyright Xin Li Beyond Wavelets Transient events are not everything in the natural world What are non-transient events? Pitch-related periodicity in speech Texture patterns in image Long-term dependency in stock market Need a better understanding of CHANGE

EE565 Advanced Image Processing Copyright Xin Li Patch-based Denoising (BM3D) WD T T -1 ThresholdingWD = Noisy patches Denoised patches

EE565 Advanced Image Processing Copyright Xin Li State-of-the-Art Image Denoising PSNR(DB) PERFORMANCE COMPARISON AMONG DIFFERENT SCHEMES FOR 12 TEST IMAGES ATσw = 100

EE565 Advanced Image Processing Copyright Xin Li Duality with Image Coding* DWT IWTThresholding DWT IWT Q Q -1 Channel Image denoising system Image coding system

EE565 Advanced Image Processing Copyright Xin Li Difference from Image Coding G0G0 G1G1 x(n) H0H0 H1H1 y 0 (n) y 1 (n) x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) complete expansion (non-redundant) - suitable for image coding overcomplete expansion (redundant) - suitable for image denoising T ce T ce -1 T oe T oe -1