EE565 Advanced Image Processing Copyright Xin Li Further Improvements Gaussian scalar mixture (GSM) based denoising* (Portilla et al.’ 2003) Instead of estimating the variance, it explicitly addresses the issue of uncertainty with variance estimation Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band coefficients (refer to the posted paper)

EE565 Advanced Image Processing Copyright Xin Li Gaussian Scalar Mixture (GSM) Model definition: u~N(0,1) Noisy observation model Gaussian pdf scale (variance) parameter

EE565 Advanced Image Processing Copyright Xin Li Basic Idea In spatially adaptive Wiener filtering, we estimate the variance from the data of a local window. The uncertainty with such variance estimation is ignored. In GSM model, such uncertainty is addressed through the scalar z (it determines the variance of GSM). Instead of using a single z (estimated variance), we build a probability model over z, i.e., E{x|y}=E z {E{x|y,z}}

EE565 Advanced Image Processing Copyright Xin Li Posterior Distribution where Due to is so-called Jeffery’s prior Question: What is E{x c |y,z}? Bayesian formula (proof left as exercise)

EE565 Advanced Image Processing Copyright Xin Li GSM Denoising Algorithm codes available at:

EE565 Advanced Image Processing Copyright Xin Li Image Examples Noisy,  =50 (MSE=2500) denoised (MSE=201)

EE565 Advanced Image Processing Copyright Xin Li Image Examples (Con’d) Noisy,  =10 (MSE=100) denoised (MSE=31.7)

EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Latest advances Patch-based image denoising Learning-based image denoising

EE565 Advanced Image Processing Copyright Xin Li Similar Patches Self-similarity is a fundamental property of nature

EE565 Advanced Image Processing Copyright Xin Li Patch-based Texture Synthesis Self-similarity allows us to synthesize “new” texture patterns from a small-size sample

EE565 Advanced Image Processing Copyright Xin Li Patch-based Denoising (NL- mean) Image denoising via nonlocal mean (CVPR’2005 Best Paper Honorable Mention) Noisy patches w1w1 + wNwN w2w2 Linear combination v(N i ) v(N j )

EE565 Advanced Image Processing Copyright Xin Li Patch-based Denoising (BM3D) WD T T -1 ThresholdingWD = Noisy patches Denoised patches

EE565 Advanced Image Processing Copyright Xin Li State-of-the-art Result MSE=100 MSE=17

EE565 Advanced Image Processing Copyright Xin Li Learning-based Denoising Training Data ? cleannoisydenoised

EE565 Advanced Image Processing Copyright Xin Li Denoising Summary How improved image models improve the denoising performance Spatial to transform, complete to overcomplete, Wiener filtering to GSM There is a trend of from local to nonlocal, however, the pursuit has just started You are encouraged to take patch-based nonlocal denoisng as your final project topic Nobody can claim that he/she has solved the denoising problem

EE565 Advanced Image Processing Copyright Xin Li A Modeler’s View on Denoising Spatial-domain models Transform-domain models Stationary Gaussian Non-Stationary Gaussian Conventional Wiener filtering Stationary GGD Non-Stationary Gaussian Mixture Model Spatial-domain Spatially adaptive Wiener filtering Wavelet thresholding Wavelet-domain Spatially adaptive Wiener filtering BLS-GSM Algorithm Nonparametric (patch-based) Latest advances