EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Latest advances Patch-based image denoising Learning-based image denoising

EE565 Advanced Image Processing Copyright Xin Li From Scalar to Vector Case Suppose X is a Gaussian process whose covariance matrix is a diagonalized matrix R X =diag{η m }(m=0,…,N-1), the linear MMSE estimator is given by (A) and the minimal MSE is given by

EE565 Advanced Image Processing Copyright Xin Li Decorrelating Q: What if X={X[0],…,X[N-1]} is correlated (i.e., R x is not diagonalized)? A: We need to transform X into a set of uncorrelated basis and then apply the above result. The celebrated Karhunen-Loeve Transform does this job! Diagonal matrix Karhunen-Loeve Transform

EE565 Advanced Image Processing Copyright Xin Li Transform-Domain Denoising Forward Transform Inverse Transform Denoising operation e.g., KLT DCT WT e.g., Linear Wiener filtering Nonlinear Thresholding Noisy signal denoised signal The performance of such transform-domain denoising is determined by how well the assumed probability model in the transform domain matches the true statistics of source signal (optimality can only be established for the Gaussian source so far).

EE565 Advanced Image Processing Copyright Xin Li Why Wavelet Denoising? We need to distinguish spatially- localized events (edges) from noise components More about noise components Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain As long as it does not generate exceptions Additive White Gaussian Noise is just one of them

EE565 Advanced Image Processing Copyright Xin Li Tails of Distribution noise signal

EE565 Advanced Image Processing Copyright Xin Li Wavelet Thresholding DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

EE565 Advanced Image Processing Copyright Xin Li Choice of Threshold Donoho and Johnstone’1994 Gives denoising performance close to the “ideal weighting” Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp )

EE565 Advanced Image Processing Copyright Xin Li Soft vs. Hard thresholding ● It can be also viewed as a computationally efficient approximation of ideal weighting soft ideal ● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better. ● Shrinking the amplitude by T guarantees with a high probability that.

EE565 Advanced Image Processing Copyright Xin Li Denoising Example noisy image (σ 2 =100) Wiener-filtering (ISNR=2.48dB) Wavelet-thresholding (ISNR=2.98dB) X: original, Y: noisy, X: denoised ~ Improved SNR

EE565 Advanced Image Processing Copyright Xin Li Duality with Image Coding* DWT IWTThresholding DWT IWT Q Q -1 Channel Image denoising system Image coding system

EE565 Advanced Image Processing Copyright Xin Li Difference from Image Coding G0G0 G1G1 x(n) H0H0 H1H1 y 0 (n) y 1 (n) x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) complete expansion (non-redundant) - suitable for image coding overcomplete expansion (redundant) - suitable for image denoising T ce T ce -1 T oe T oe -1

EE565 Advanced Image Processing Copyright Xin Li What do We Buy from Redundancy? 0 1 N-1 … x(n) H1H1 T -T

EE565 Advanced Image Processing Copyright Xin Li Translation Invariance (TI) Denoising T oe T oe -1 Thresholding T ce T ce -1 Thresholding T ce T ce -1 Thresholding z + x(n) Implementation based on overcomplete expansion Implementation based on complete expansion z -1

EE565 Advanced Image Processing Copyright Xin Li D Extension Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

EE565 Advanced Image Processing Copyright Xin Li Example Wavelet-thresholding (ISNR=2.98dB) Translation-Invariant thresholding (ISNR=3.51dB)

EE565 Advanced Image Processing Copyright Xin Li Go Beyond Thresholding Challenges with wavelet thresholding Determination of a global optimal threshold Spatially adjusting threshold based on local statistics How to go beyond thresholding? We need an accurate modeling of wavelet coefficients – nonlinear thresholding is a computationally efficient yet suboptimal solution

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

EE565 Advanced Image Processing Copyright Xin Li Practical Implementation T T N=T 2 Recall Conceptually very similar to its counterpart in the spatial domain In demo3.zip, you can find a C-coded example (de_noise.c) (ML estimation of signal variance)

EE565 Advanced Image Processing Copyright Xin Li Example Translation-Invariant thresholding (ISNR=3.51dB) Spatially-adaptive wiener filtering (ISNR=4.53dB)