Discriminative Approach for Wavelet Denoising Yacov Hel-Or and Doron Shaked I.D.C.- Herzliya HPL-Israel.

Slides:



Advertisements
Similar presentations
Object Specific Compressed Sensing by minimizing a weighted L2-norm A. Mahalanobis.
Advertisements

Compressive Sensing IT530, Lecture Notes.
Joint work with Irad Yavneh
Pixel Recovery via Minimization in the Wavelet Domain Ivan W. Selesnick, Richard Van Slyke, and Onur G. Guleryuz *: Polytechnic University, Brooklyn, NY.
Prediction with Regression
Signal Denoising with Wavelets. Wavelet Threholding Assume an additive model for a noisy signal, y=f+n K is the covariance of the noise Different options.
Patch-based Image Deconvolution via Joint Modeling of Sparse Priors Chao Jia and Brian L. Evans The University of Texas at Austin 12 Sep
Discriminative Approach for Transform Based Image Restoration
Learning sparse representations to restore, classify, and sense images and videos Guillermo Sapiro University of Minnesota Supported by NSF, NGA, NIH,
* * Joint work with Michal Aharon Freddy Bruckstein Michael Elad
Rob Fergus Courant Institute of Mathematical Sciences New York University A Variational Approach to Blind Image Deconvolution.
ECE 472/572 - Digital Image Processing Lecture 8 - Image Restoration – Linear, Position-Invariant Degradations 10/10/11.
Sparse & Redundant Signal Representation, and its Role in Image Processing Michael Elad The CS Department The Technion – Israel Institute of technology.
Oriented Wavelet 國立交通大學電子工程學系 陳奕安 Outline Background Background Beyond Wavelet Beyond Wavelet Simulation Result Simulation Result Conclusion.
Sparse and Overcomplete Data Representation
SRINKAGE FOR REDUNDANT REPRESENTATIONS ? Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000, Israel.
Mathematics and Image Analysis, MIA'06
Image Denoising via Learned Dictionaries and Sparse Representations
ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa.
New Results in Image Processing based on Sparse and Redundant Representations Michael Elad The Computer Science Department The Technion – Israel Institute.
* Joint work with Michal Aharon Guillermo Sapiro
Recent Trends in Signal Representations and Their Role in Image Processing Michael Elad The CS Department The Technion – Israel Institute of technology.
EE565 Advanced Image Processing Copyright Xin Li Statistical Modeling of Natural Images in the Wavelet Space Parametric models of wavelet coefficients.
Understanding and evaluating blind deconvolution algorithms
Multiscale transforms : wavelets, ridgelets, curvelets, etc.
Sparse and Redundant Representation Modeling for Image Processing Michael Elad The Computer Science Department The Technion – Israel Institute of technology.
Empirical Bayes approaches to thresholding Bernard Silverman, University of Bristol (joint work with Iain Johnstone, Stanford) IMS meeting 30 July 2002.
(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.
Noise Estimation from a Single Image Ce Liu William T. FreemanRichard Szeliski Sing Bing Kang.
Linear Algebra and Image Processing
Image Representation Gaussian pyramids Laplacian Pyramids
School of Electrical & Computer Engineering Image Denoising Using Gaussian Scale Mixtures in the Wavelet domain Alex Cunningham Ben Clarke Dy narath Eang.
Image Denoising using Wavelet Thresholding Techniques Submitted by Yang
Predicting Wavelet Coefficients Over Edges Using Estimates Based on Nonlinear Approximants Onur G. Guleryuz Epson Palo Alto Laboratory.
WEIGHTED OVERCOMPLETE DENOISING Onur G. Guleryuz Epson Palo Alto Laboratory Palo Alto, CA (Please view in full screen mode to see.
Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography.
Computer Vision - Restoration Hanyang University Jong-Il Park.
1 Wavelets, Ridgelets, and Curvelets for Poisson Noise Removal 國立交通大學電子研究所 張瑞男
Iterated Denoising for Image Recovery Onur G. Guleryuz To see the animations and movies please use full-screen mode. Clicking on.
Supervised Learning of Edges and Object Boundaries Piotr Dollár Zhuowen Tu Serge Belongie.
CS654: Digital Image Analysis
Basis Expansions and Regularization Part II. Outline Review of Splines Wavelet Smoothing Reproducing Kernel Hilbert Spaces.
School of Electrical & Computer Engineering Image Denoising Using Steerable Pyramids Alex Cunningham Ben Clarke Dy narath Eang ECE November 2008.
Learning to Perceive Transparency from the Statistics of Natural Scenes Anat Levin School of Computer Science and Engineering The Hebrew University of.
Image Denoising Using Wavelets
EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener.
EE565 Advanced Image Processing Copyright Xin Li Image Denoising: a Statistical Approach Linear estimation theory summary Spatial domain denoising.
Image Decomposition, Inpainting, and Impulse Noise Removal by Sparse & Redundant Representations Michael Elad The Computer Science Department The Technion.
Adaptive Wavelet Packet Models for Texture Description and Segmentation. Karen Brady, Ian Jermyn, Josiane Zerubia Projet Ariana - INRIA/I3S/UNSA June 5,
EE565 Advanced Image Processing Copyright Xin Li Further Improvements Gaussian scalar mixture (GSM) based denoising* (Portilla et al.’ 2003) Instead.
Imola K. Fodor, Chandrika Kamath Center for Applied Scientific Computing Lawrence Livermore National Laboratory IPAM Workshop January, 2002 Exploring the.
Single Image Interpolation via Adaptive Non-Local Sparsity-Based Modeling The research leading to these results has received funding from the European.
EE565 Advanced Image Processing Copyright Xin Li Application of Wavelets (I): Denoising Problem formulation Frequency-domain solution: linear Wiener.
EE565 Advanced Image Processing Copyright Xin Li Why do we Need Image Model in the first place? Any image processing algorithm has to work on a collection.
Spread Spectrum and Image Adaptive Watermarking A Compare/Contrast summary of: “Secure Spread Spectrum Watermarking for Multimedia” [Cox ‘97] and “Image-Adaptive.
Jianchao Yang, John Wright, Thomas Huang, Yi Ma CVPR 2008 Image Super-Resolution as Sparse Representation of Raw Image Patches.
Bayesian fMRI analysis with Spatial Basis Function Priors
Biointelligence Laboratory, Seoul National University
Sparsity Based Poisson Denoising and Inpainting
Degradation/Restoration Model
Ch3: Model Building through Regression
Wavelets : Introduction and Examples
Directional Multiscale Modeling of Images
Image Denoising in the Wavelet Domain Using Wiener Filtering
Hidden Markov Tree Model of the Uniform Discrete Curvelet Transform Image for Denoising Yothin Rakvongthai.
Image Restoration and Denoising
* * Joint work with Michal Aharon Freddy Bruckstein Michael Elad
Image restoration, noise models, detection, deconvolution
Lecture 7 Patch based methods: nonlocal means, BM3D, K- SVD, data-driven (tight) frame.
Presentation transcript:

Discriminative Approach for Wavelet Denoising Yacov Hel-Or and Doron Shaked I.D.C.- Herzliya HPL-Israel

- Can we clean Lena? Motivation – Image denoising

Some reconstruction problems Images of Venus taken by the Russian lander Ventra-10 in Can we “see” through the missing pixels? Sapiro et. al.

Image Inpainting Sapiro et.al.

Image De-mosaicing - Can we reconstruct the color image?

Image De-blurring Can we sharpen Barbara?

All the above deal with degraded images. Their reconstruction requires solving an inverse problem Inpainting De-blurring De-noising De-mosaicing

Typical Degradation Sources Low Illumination Atmospheric attenuation (haze, turbulence, …) Optical distortions (geometric, blurring) Sensor distortion (quantization, sampling, sensor noise, spectral sensitivity, de-mosaicing)

Reconstruction as an Inverse Problem Distortion H noise measurements Original image Reconstruction Algorithm Years of extensive study Thousands of research papers

Typically: –The distortion H is singular or ill-posed. –The noise n is unknown, only its statistical properties can be learnt.

Key point: Stat. Prior of Natural Images

The Image Prior Px(x)Px(x) Image space 1 0

measurements Most probable solution From amongst all possible solutions, choose the one that maximizes the a-posteriori probability P X (x|y) Bayesian Reconstruction (MAP) P(x|y) P X (x)

Unfortunately not! The p.d.f. P x defines a prior dist. over natural images: –Defined over a huge dim. space (1E6 for 1Kx1K grayscale image) –Sparsely sampled. –Known to be non Gaussian. –Complicated to model. So, are we set?

Example: 3D prior of 2x2 image neighborhoods form Mumford & Huang, 2000

Marginalization of Image Prior Observation1: The Wavelet transform tends to de- correlate pixel dependencies of natural images. W.T.

Observation2: The statistics of natural images are homogeneous. Share the same statistics How Many Mapping Functions

Wavelet Shrinkage Denoising Donoho & Johnston 94 (unitary case) Degradation Model: The MAP estimator:

The MAP estimator gives:

The MAP estimator diagonalizes the system: This leads to a very useful property: Scalar mapping functions:

Wavelet Shrinkage Pipe-line Transform W Transform W Mapping functions M i (y i w ) Mapping functions M i (y i w ) Inverse Transform W T Inverse Transform W T yiwyiw xiwxiw Non linear operation

Due to the fact that: N mapping functions are needed for N sub- bands. How Many Mapping Functions?

Subband Decomposition Wavelet transform: Shrinkage: where

Wavelet Shrinkage Pipe-line B1B1 B1B1 Wavelet transform B1B1 B1B1 B1B1 B1B1 BiBi BiBi Shrinkage functions Inverse transform B1B1 B1B1 B1B1 B1B1 B1B1 B1B1 BTiBTi BTiBTi xiBxiB yiByiB +

The shape of the mapping function M j depends solely on P j and the noise variance . Designing The Mapping Function ywyw Modeling marginal p.d.f. of band j (noise variance) MAP objective MAP objective

Commonly P j (y w ) are approximated by GGD: for p<1 from: Simoncelli 99

Hard Thresholding Soft Thresholding Linear Wiener Filtering MAP estimators for GGD model with three different exponents. The noise is additive Gaussian, with variance one third that of the signal.

Due to its simplicity Wavelet Shrinkage became extremely popular: –Thousands of applications. –Hundreds of related papers (984 citations of D&J paper in Google Scholar). What about efficiency? –Denoising performance of the original Wavelet Shrinkage technique is far from the state-of-the-art results. Why? –Wavelet coefficients are not really independent.

Recent Developments Since the original approach suggested by D&J significant improvements were achieved: Original Shrinkage Over-complete Joint (Local) Coefficient Modeling Overcomplete transform Scalar MFs Simple Not considered state-of-the-art Multivariate MFs Complicated Superior results

Joint (Local) Coefficient Modeling HMM Crouse et. al. Joint Bayesian Pizurika et. al Context Modeling Portilla et. al. Context Modeling Chang, et. al. HMM Fan-Xia Sparsity Mallat, Zhang Joint Bayesian Simoncelli Bivariate Sendur, Selesnick Co-occurence Shan, Aviyente Adaptive Thresh. Li, Orchad

Shrinkage in Over-complete Transforms 94 Shrinkage D.J Steerable Simoncelli, Adelson Undecimated wavelet Coifman, Donoho Ridgelets Candes Ridgelets Carre, Helbert Ridgelets Nezamoddini et. al. Contourlets Matalon, et. al. Contourlets Do, Vetterli Curvelets Starck et. al. K-SVD Aharon, Elad

Over-Complete Shrinkage Denoising Over-complete transform: Shrinkage: Mapping Functions: Naively borrowed from the Unitary case. where

What’s wrong with existing MFs? 1.Map criterion: –Solution is biased towards the most probable case. 2.Independent assumption: –In the overcomplete case, the wavelet coefficients are inherently dependent. 3.Minimization domain: –For the unitary case MFs optimality is expressed in the transform domain. This is incorrect in the overcomplete case. 4.White noise assumption: –Image noise is not necessarily white i.i.d.

Why unitary based MFs are being used? Non-marginal statistics. Multivariate minimization. Multivariate MFs. Non-white noise.

Suggested Approach: Maintain simplicity –Use scalar LUTs. Improve Efficiency –Use Over-complete Transforms. –Design optimal MFs with respect to a given set of images. –Express optimality in the spatial domain. –Attain optimality with respect to MSE.

Optimal Mapping Function: Traditional approach: Descriptive Suggested approach: Discriminative Modeling wavelet p.d.f. MAP objective MAP objective Optimality criteria

The optimality Criteria Design the MFs with respect to a given set of examples: {x e i } and {y e i } Critical problem: How to optimize the non-linear MFs

The Spline Transform Let x  R be a real value in a bounded interval [a,b). We divide [a,b) into M segments q=[q 0,q 1,...,q M ] w.l.o.g. assume x  [q j-1,q j ) Define residue r(x)=(x-q j-1 )/(q j -q j-1 ) a b x q0q0 q1q1 qMqM q j-1 qjqj r(x) x= r(x) q j +(1- r(x) ) q j-1 x =[ 0, , 0, 1-r(x), r(x), 0,  ] q = S q ( x )q

The Spline Transform-Cont. We define a vectorial extension: We call this the Spline Transform (SLT) of x.  i th row

The SLTProperties The SLT Properties Substitution property: Substituting the boundary vector q with a different vector p forms a piecewise linear mapping. =S q (x) x q0q0 q1q1 q2q2 q3q3 q4q4 q1q1 q2q2 q3q3 q4q4 q p p0p0 p1p1 p2p2 p3p3 p4p4 x x’ x x

Back to the MFs Design We approximate the non-linear {M k } with piece-wise linear functions: Finding {p k } is a standard LS problem with a closed form solution!

B1B1 B1B1 B1B1 B1B1 B1B1 B1B1 BkBk BkBk B1B1 B1B1 B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk + Designing the MFs closed form solution: M k (y; p k ) (B T B) -1 Undecimated wavelet: 2D convolutions

Results

Training Images

Tested Images

Simulation setup Transform used: Undecimated DCT Noise: Additive i.i.d. Gaussian Number of bins: 15 Number of bands: 3x3.. 10x10

B1B1 B1B1 B1B1 B1B1 BkBk BkBk Option 1: Transform domain – independent bands M k (y; p k ) (B T B) -1 B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk B1B1 B1B1 B1B1 B1B1 BkBk BkBk B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk

B1B1 B1B1 B1B1 B1B1 BkBk BkBk B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk B1B1 B1B1 B1B1 B1B1 BkBk BkBk B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk Option 2: Spatial domain – independent bands M k (y; p k )

B1B1 B1B1 B1B1 B1B1 BkBk BkBk (B T B) -1 B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk B1B1 B1B1 B1B1 B1B1 BkBk BkBk B1B1 B1B1 B1B1 B1B1 BTkBTk BTkBTk Option 3: Spatial domain – joint bands M k (y; p k )

MFs for UDCT 8x8 (i,i) bands, i=1..4,  =20 Option Option 1 Option Option 2 Option Option 3

Comparing psnr results for 8x8 undecimated DCT, sigma=20.

8x8 UDCT  =10

8x8 UDCT  =20

8x8 UDCT  =10

The Role of Quantization Bins 8x8 UDCT  =10

The Role of Transform Used  =10

The Role of Training Image

MFs for UDCT 8x8 (i,i) bands, i=2..6.  =5  =10  =15  =20 The Role of noise variance

Observation: The obtained MFs for different noise variances are similar up to scaling: The role of noise variance

Comparison between M 20 (v) and 0.5M 10 (2v) for basis [2:4]X[2:4]

Comparison with BLS-GSM

Other Degradation Models

JPEG Artifact Removal

Image Sharpening

Conclusions New and simple scheme for over-complete transform based denoising. MFs are optimized in a discriminative manner. Linear formulation of non-linear minimization. Eliminating the need for modeling complex statistical prior in high-dim. space. Seamlessly applied to other degradation problems as long as scalar MFs are used for reconstruction.

Conclusions – cont. Extensions: –Filter-cascade based denoising. –Multivariate MFs (activity level). –Non-homogeneous noise characteristics. Open problems: –What is the best transform for a given image? –How to choose training images that form faithful representation?

Thank You The End

MSE for MF scaling from  =10 to  =20

MSE for MF scaling from  =15 to  =20

MSE for MF scaling from  =25 to  =20

Image Sharpening

Wavelet Shrinkage Pipe-line B1B1 B1B1 Wavelet transform B1B1 B1B1 B1B1 B1B1 BiBi BiBi Shrinkage functions Inverse transform B1B1 B1B1 B1B1 B1B1 B1B1 B1B1 BTiBTi BTiBTi xiBxiB yiByiB + (B T B) -1

MFs for UDCT 8x8 (i,i) bands, i=1..4,  =20 Option Option 1

MFs for UDCT 8x8 (i,i) bands, i=1..4,  =20 Option Option 2

MFs for UDCT 8x8 (i,i) bands, i=1..4,  =20 Option Option 3

Comparing psnr results for 8x8 undecimated DCT, sigma=20.

Comparing psnr results for 8x8 undecimated DCT, sigma=10.