EE565 Advanced Image Processing Copyright Xin Li Image Denoising: a Statistical Approach Linear estimation theory summary Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

EE565 Advanced Image Processing Copyright Xin Li Denoising Problem Noisy measurements N(0,σ w 2 ) MMSE estimator Wiener’s idea To simplify estimation by computing the best estimator that is a linear scaling of Y Difficulty: we need to know conditional pdf N(0,σ x 2 )

Orthogonality Principle EE565 Advanced Image Processing Copyright Xin Li A linear estimator X of a random variable X ^ Minimizes E{(X-X) 2 } if and only if ^ Geometric Interpretation X Y X-X ^ X ^

EE565 Advanced Image Processing Copyright Xin Li Linear MMSE Estimation For Gaussian signal The optimal LMMSE estimation is given by And it achieves Note: it can be shown such linear estimator is indeed E[X|Y] for Gaussian signal

EE565 Advanced Image Processing Copyright Xin Li What if Signal Variance is Unknown? Maximum-likelihood estimation ofis given by Since variance is nonnegative, we modify it When multiple observations y i ’s are available, we have

EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

Conventional Wiener Filtering Basic assumption: image source is modeled by a stationary Gaussian process Signal variance is estimated from the noisy observation data Can be interpreted as a linear frequency weighting EE565 Advanced Image Processing Copyright Xin Li

8 Linear Frequency Weighting FT Power spectrum |X| 2

Image Example EE565 Advanced Image Processing Copyright Xin Li Noisy,  =50 (MSE=2500) denoised (MSE=1130)

Image Example (Con’d) EE565 Advanced Image Processing Copyright Xin Li Noisy,  =10 (MSE=100) denoised (MSE=437)

Conclusions from the Experiments Why did it Fail? Nonstationary NonGaussian Poor modeling How to improve? Achieve spatial adaptation Use linear transform Putting them together EE565 Advanced Image Processing Copyright Xin Li

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering Basic assumption: image source is modeled by a nonstationary Gaussian process Signal variance is locally estimated from the windowed noisy observation data T T N=T 2 Recall

Image Example EE565 Advanced Image Processing Copyright Xin Li Noisy,  =10 (MSE=100) denoised (T=3,MSE=56)

Image Example (Con’d) EE565 Advanced Image Processing Copyright Xin Li Noisy,  =50 (MSE=2500) denoised (MSE=354)

EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

From Scalar to Vector Case EE565 Advanced Image Processing Copyright Xin Li Suppose X is a Gaussian process whose covariance matrix is a diagonalized matrix R X =diag{η m }(m=0,…,N-1), the linear MMSE estimator is given by (A) and the minimal MSE is given by

EE565 Advanced Image Processing Copyright Xin Li Decorrelating Q: What if X={X[0],…,X[N-1]} is correlated (i.e., R x is not diagonalized)? A: We need to transform X into a set of uncorrelated basis and then apply the above result. The celebrated Karhunen-Loeve Transform does this job! Diagonal matrix Karhunen-Loeve Transform

Transform-Domain Denoising EE565 Advanced Image Processing Copyright Xin Li Forward Transform Inverse Transform Denoising operation e.g., KLT DCT WT e.g., Linear Wiener filtering Nonlinear Thresholding Noisy signal denoised signal The performance of such transform-domain denoising is determined by how well the assumed probability model in the transform domain matches the true statistics of source signal (optimality can only be established for the Gaussian source so far).

One-Minute Tour of Wavelets EE565 Advanced Image Processing Copyright Xin Li G0G0 G1G1 x(n) H0H0 H1H1 y 0 (n) y 1 (n) x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) complete expansion (with decimation) overcomplete expansion (without decimation) T ce T ce -1 T oe T oe -1

Why Wavelet Denoising? We need to distinguish spatially-localized events (edges) from noise components More about noise components EE565 Advanced Image Processing Copyright Xin Li Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain As long as it does not generate exceptions Additive White Gaussian Noise is just one of them

Wavelet Thresholding EE565 Advanced Image Processing Copyright Xin Li DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

EE565 Advanced Image Processing Copyright Xin Li Choice of Threshold Donoho and Johnstone’1994 Gives denoising performance close to the “ideal weighting” Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp )

EE565 Advanced Image Processing Copyright Xin Li Soft vs. Hard thresholding ● It can be also viewed as a computationally efficient approximation of ideal weighting soft ideal ● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better. ● Shrinking the amplitude by T guarantees with a high probability that.

EE565 Advanced Image Processing Copyright Xin Li Denoising Example noisy image (σ 2 =100) Wiener-filtering (ISNR=2.48dB) Wavelet-thresholding (ISNR=2.98dB) X: original, Y: noisy, X: denoised ~ Improved SNR

What is Wrong with Wavelets? EE565 Advanced Image Processing Copyright Xin Li N-1 … x(n) H1H1 T -T

EE565 Advanced Image Processing Copyright Xin Li Translation Invariance (TI) Denoising T oe T oe -1 Thresholding T ce T ce -1 Thresholding T ce T ce -1 Thresholding z + x(n) Implementation based on overcomplete expansion Implementation based on complete expansion z -1

EE565 Advanced Image Processing Copyright Xin Li D Extension Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

EE565 Advanced Image Processing Copyright Xin Li Example Wavelet-thresholding (ISNR=2.98dB) Translation-Invariant thresholding (ISNR=3.51dB)

Challenges with wavelet thresholding Determination of a global optimal threshold Spatially adjusting threshold based on local statistics How to go beyond thresholding? We need an accurate modeling of wavelet coefficients – nonlinear thresholding is a computationally efficient yet suboptimal solution EE565 Advanced Image Processing Copyright Xin Li Go Beyond Thresholding

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

EE565 Advanced Image Processing Copyright Xin Li Practical Implementation T T N=T 2 Recall Conceptually very similar to its counterpart in the spatial domain In demo3.zip, you can find a C-coded example (de_noise.c) (ML estimation of signal variance)

Example EE565 Advanced Image Processing Copyright Xin Li Translation-Invariant thresholding (ISNR=3.51dB) Spatially-adaptive wiener filtering (ISNR=4.53dB)

Further Improvements* Gaussian scalar mixture (GSM) based denoising (Portilla et al.’ 2003) Instead of estimating the variance, it explicitly addresses the issue of uncertainty with variance estimation Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band coefficients (refer to the posted paper) EE565 Advanced Image Processing Copyright Xin Li

EE565 Advanced Image Processing Copyright Xin Li Gaussian Scalar Mixture (GSM)* Model definition:u~N(0,1) Noisy observation model Gaussian pdf scale (variance) parameter

EE565 Advanced Image Processing Copyright Xin Li Basic Idea* In spatially adaptive Wiener filtering, we estimate the variance from the data of a local window. The uncertainty with such variance estimation is ignored. In GSM model, such uncertainty is addressed through the scalar z (it decides the variance of GSM). Instead of using a single z (estimated variance), we build a probability model over z, i.e., E{x|y}=E z {E{x|y,z}}

EE565 Advanced Image Processing Copyright Xin Li Posterior Distribution* where Due to is so-called Jeffery’s prior Question: What is E{x c |y,z}? Bayesian formula (proof left as exercise)

GSM Denoising Algorithm* EE565 Advanced Image Processing Copyright Xin Li codes available at:

Image Examples EE565 Advanced Image Processing Copyright Xin Li Noisy,  =50 (MSE=2500) denoised (MSE=201)

Image Examples (Con’d) EE565 Advanced Image Processing Copyright Xin Li Noisy,  =10 (MSE=100) denoised (MSE=31.7)