Stopping Criteria Image Restoration Alfonso Limon Claremont Graduate University

Outline Image Restoration Noise and Image Details Observations using Wavelets Some Preliminary Results Future Work CGU IPAM 2003: Inverse Problems

Image Restoration The goal of image restoration is to improve a degrade image in some predefined sense. Schematically this process can be visualized as where f is the original image, g is a degraded/noisy version of the original image and f ̃ is a restored version. CGU IPAM 2003: Inverse Problems

Image Degradation / Restoration Image restoration removes a known degradation. If the degradation is linear and spatially-invariant where F - original image, H - degradation, N - additive noise and G - recorded image. Given H, an estimate of the original image is Notice that if H ~ 0, the noise will be amplified. CGU IPAM 2003: Inverse Problems

Degradation Models Image degradation can occur for many reasons, some typical degradation models are Motion Blur: due to camera panning or subject moving quickly. Atmospheric Blur: long exposure Uniform 2D Blur Out-of-Focus Blur CGU IPAM 2003: Inverse Problems

Noise Models Most noise models assume the noise is some known probability density function. The density function is chosen based on the underlining physics. Gaussian: poor illumination. Rayleigh: range image Salt and Pepper: faulty switch during imaging Gamma or Exp: laser imaging CGU IPAM 2003: Inverse Problems

Restoration Model Many restoration filters can be applied to recover the degraded image: Rudin-Osher-Fatemi, Wiener, Inverse FFT with threshold, etc. But all require a stopping criteria based on a noise measure. CGU IPAM 2003: Inverse Problems

Stopping Criteria As was illustrated by S. Osher (on Monday) and M. Bertero (in the tutorials) whether the restoration filter is direct or iterative, it must be stopped before the image begins to degrade.

Stopping the Restoration Noise in the data is amplified as the number of iterations increase, while the blur decreases as the number of iteration increase. This is the prototypical behavior which was illustrated by Professor Heinz Engl during the inverse problem tutorials. CGU IPAM 2003: Inverse Problems

Discrepancy Principle The discrepancy principle gives a natural stopping criteria, namely to stop the restoration process when the residual is of the same order as the noise. The drawback to this approach for image processing is that the discrepancy principle tends to recover an image that is still too blurred. Question: How many more times can we iterate passed the threshold given by the discrepancy principle and still get an image that is not predominately noisy. CGU IPAM 2003: Inverse Problems

Objective The objective is to highlight fine details in the image which were suppressed by the blur. The problem is that enhancement of fine detail (or edges) is equivalent to enhancement of noise. The challenge is to enhance details as much as possible before the noise overtakes the image details. CGU IPAM 2003: Inverse Problems

Wavelets and Details Wavelets provide a nature way to separate signal scales, especially for 1D signals. Taking the fine scale wavelet coefficients of the diagonal of the image provides a measure of the noise and image details. The next set of sides show the fine scale coefficients for a reconstructed version of the quarter image (to the right) at various iterations steps. CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (1) CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (2) CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (3) CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (4) CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (5) CGU IPAM 2003: Inverse Problems

Fine Wavelet Coefficients (6) CGU IPAM 2003: Inverse Problems

Observations The discrepancy principle gives a good starting guess for the image details as the noise level is low. Noise will eventually overtakes details, therefore tracking changes in the fine wavelet coefficient gives a possible way to differential between image details and noise. A natural stopping criteria is to stop iterating when the fine wavelet coefficients that have been identified as details are of the same order as the fine wavelet coefficients that have been identified as noise, i.e., the discrepancy principle applied to fine wavelet coefficients. CGU IPAM 2003: Inverse Problems

Stopping Algorithm 1.Choose your initial image according to the discrepancy principle. 2.Calculate fine wavelet coefficients corresponding to the image’s diagonal. 3.Calculate the mean of the fine wavelet coefficients. 4.Assign the fine wavelet coefficients which are larger than the mean to the set: details. 5.If there is a previous details set, take the intersection of it with the new details set. 6.Repeat steps 2 through 5 until the mean of the details set is equal to the mean of the noise set; noise is the set of wavelet coefficients not in details.

Stopping Criteria Results: eight

Stopping Criteria Results: autumn

Stopping Criteria Results: rice

Stopping Criteria Results: alumgrns

Stopping Criteria Results: bacteria

Stopping Criteria Results: cameraman

Stopping Criteria Results: mri

Stopping Criteria Results: ic

Stopping Criteria Results: moon

Preliminary Results CGU IPAM 2003: Inverse Problems

Preliminary Results CGU IPAM 2003: Inverse Problems

Preliminary Results CGU IPAM 2003: Inverse Problems

Future Work Define an energy measure to compare wavelet coefficients instead of using the mean value. Apply a moving class to the wavelet coefficients to take into account the difference in behavior between details and noise. Apply different deblurring/noise models to test that the stopping criteria is not sensitive to the deblurring/noise process. Expend the method from a global to local stopping criteria. CGU IPAM 2003: Inverse Problems

Special Thanks to the Organizing Committee and IPAM Staff.

References M. Bertero, Image Deconvolution, Inverse Problems: Computational Methods and Emerging Applications, IPAM ( P. Blomgren and T. F. Chan, Modular Solver for Constraint Image Restoration Problems Using the Discrepancy Principle, Numerical Linear Algebra with Applications,Vol 9, issue 5, pp David Donoho, De-Noising by Soft-Thresholding ( H. W. Engl, Inverse Problems 1, Inverse Problems: Computational Methods and Emerging Applications Tutorials, IPAM ( R. Gonzalez and R. Woods, Digital Image Processing, second edition, Prentice Hall, Inc., K. Lee, J. Nagy and L. Perrone, Iterative Methods for Image Restoration: A Matlab Object Oriented Approach. Iterative Methods for Image Restoration, May 15, R. Ramlau, Morozov’s Discrepancy Principle for Tikhonov regularization of nonlinear operators, Report 01-08, Berichte aus der Technomathematik, July 2001.