1. School of Electrical & Computer Engineering Image Denoising Using Steerable Pyramids Alex Cunningham Ben Clarke Dy narath Eang ECE6258 20 November 2008

2. ECE6258 Digital Image Processing, Group3 2 School of Electrical & Computer Engineering Agenda Background Steerable Pyramids (SPyr) Image denoising with Spyrs (algorithm) Matlab design Portilla’s results Our results Improvements in the algorithm

3. ECE6258 Digital Image Processing, Group3 3 School of Electrical & Computer Engineering Background Terms Gaussian scale mixture (GSM) – The product of a Gaussian random vector, and an independent hidden random scalar multiplier. (p. 1338)  A hidden scalar multiplier is also known as a GSM. GSM distributions represent an important subset of the elliptically symmetric distributions, which are those that can be defined as functions of a quadratic norm of the random vector. (p. 1339)  A random vector x that can be expressed as the product of a zero-mean Gaussian vector and an independent positive scalar random variable √z. (p. 1340) Empirical Bayes estimator – One first estimates the local signal variance from a neighborhood of observed pixels, and then (proceeding as if this were the true variance) applies the standard linear least squares (LLS) solution. A parameter of the local model is estimated from the data, and this estimate is used to estimate the signal. (p. 1340) Steerable pyramid ML-estimated nonparametric prior – Produces the best results for denoising the pyramid coefficients. (p. 1341) Noninformative prior – typically leads to better denoising performance in the image domain (roughly +0.15dB, on average). References Use matlabPyrTools toolbox:  http://www.cns.nyu.edu/~eero/software.php http://www.cns.nyu.edu/~eero/software.php Reference:  J. Portilla, V. Strela, M. Wainwright, and E. P. Simoncelli, “Image denoising using a scale mixture of Gaussians in the wavelet domain,” IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1338–1351, November 2003.

4. ECE6258 Digital Image Processing, Group3 4 School of Electrical & Computer Engineering Steerable Pyramids Steerable pyramid – A multiscale linear decomposition whose basis functions are spatially localized, oriented, and span roughly one octave in bandwidth. They are polar-separable in the Fourier domain, and are related by translation, dilation, and rotation. (p. 1340) http://www.cns.nyu.edu/~eero/STEERPYR/

5. ECE6258 Digital Image Processing, Group3 5 School of Electrical & Computer Engineering Image Denoising Algorithm 3-step process: 1) decompose the image into pyramid subbands at different scales and orientations; 2) denoise each subband, except for the lowpass residual band; 3) invert the pyramid transform, obtaining the denoised image. (p. 1342) Full algorithm: 1) Decompose the image into subbands. 2) For each subband (except the lowpass residual):  a) Compute neighborhood noise covariance, Cw, from the image-domain noise covariance.  b) Estimate noisy neighborhood covariance, Cy.  c) Estimate Cu from Cw and Cy using (7).  d) Compute A and M (Section III-B).  e) For each neighborhood: i) For each value z in the integration range:  A) Compute E{xc,y,z} using (12).  B) Compute p(y z) using (14). ii) Compute p(z y) using (13) and(4). iii) Compute E{xc|y} numerically using (8). 3) Reconstruct the denoised image from the processed subbands and the lowpass residual. (p. 1343)

6. ECE6258 Digital Image Processing, Group3 6 School of Electrical & Computer Engineering (1) The random vector x is the product of the zero-mean Gaussian vector u and the positive scalar RV √z. (2) The density of “the infinite mixture of Gaussian vectors”, determined by the covariance C u and the mixing density p z (z). (3) ML approach to estimate a nonparametric p z (z) from an observed set of neighborhood vectors. (A) Taking the Expectation of this gives the Fisher information matrix for the GSM model. (B) General rule? Used to obtain Jeffrey’s prior. This noninformative prior means that it does not require the fitting of any parameters to the noisy observation. Equations1

7. ECE6258 Digital Image Processing, Group3 7 School of Electrical & Computer Engineering (5) The vector y corresponds to a neighborhood of N observed coefficients of the pyramid representation. (6) Density of the observed neighborhood vector y on conditional on z. (C) Covariance of neighborhood vector y. (7) Covariance of zero-mean Gaussian vector u. (8) The Bayes least squares (BLS) estimate is just the conditional mean. Value inside integral is just a local linear (Wiener) estimate. Equations2

8. ECE6258 Digital Image Processing, Group3 8 School of Electrical & Computer Engineering (9) Local Wiener estimate? (10) Where {Q,Λ} is the eigenvector/eigenvalues expansion of the matrix S -1 C u S -T. (11) Simplification of (9) using result of (10). (12) Restriction on estimate to reference coefficient. Equations3

9. ECE6258 Digital Image Processing, Group3 9 School of Electrical & Computer Engineering Image Denoising Algorithm2 denoise_demo.m Load image of interest Make copy of image and add Gaussian noise of known variance Define number of levels/bands for steerable pyramid (SPyr) decomposition Calculate need bound extensions and PSDs Decompose image and noisy image into SPyr structure Perform denoising (gsm_denoise.m) Reconstruct image from denoised SPyr samples

10. ECE6258 Digital Image Processing, Group3 10 School of Electrical & Computer Engineering Image Denoising Algorithm3 gsm_demo.m Extract particular subband (both original and noisy image) Calculate Gaussian vectors and covariance matrices (covariance_calculator.m) Perform (local_weiner_estimator.m) from covariance matrices to get eigenvalues and eigenvectors Use eigenvalues/eigenvectors to find expectation values for each pixel in the neighborhood (denoiser.m)

11. ECE6258 Digital Image Processing, Group3 11 School of Electrical & Computer Engineering Matlab Algorithm Tree blank denoise_demo.m pyr_decomp.m gsm_denoise.m pyr_reconstruct.m pyrBand.m local_wiener_estimator.m denoiser.m covariance_calculator.m bound_extension.m

12. ECE6258 Digital Image Processing, Group3 12 School of Electrical & Computer Engineering Portilla Results

13. ECE6258 Digital Image Processing, Group3 13 School of Electrical & Computer Engineering Our Results

14. ECE6258 Digital Image Processing, Group3 14 School of Electrical & Computer Engineering Remarks

15. ECE6258 Digital Image Processing, Group3 15 School of Electrical & Computer Engineering Matlab Algorithm Tree (old) blank denoise_demo.m pyr_decomp.m gsm_denoise.m pyr_reconstruct.m pyr_extract.m band_denoise.m pyr_insert.m covariance.m