1. Stopping Criteria Image Restoration Alfonso Limon Claremont Graduate University

2. Outline Image Restoration Noise and Image Details Observations using Wavelets Some Preliminary Results Future Work CGU IPAM 2003: Inverse Problems

3. Image Restoration The goal of image restoration is to improve a degrade image in some predefined sense. Schematically this process can be visualized as where f is the original image, g is a degraded/noisy version of the original image and f ̃ is a restored version. CGU IPAM 2003: Inverse Problems

4. Image Degradation / Restoration Image restoration removes a known degradation. If the degradation is linear and spatially-invariant where F - original image, H - degradation, N - additive noise and G - recorded image. Given H, an estimate of the original image is Notice that if H ~ 0, the noise will be amplified. CGU IPAM 2003: Inverse Problems

5. Degradation Models Image degradation can occur for many reasons, some typical degradation models are Motion Blur: due to camera panning or subject moving quickly. Atmospheric Blur: long exposure Uniform 2D Blur Out-of-Focus Blur CGU IPAM 2003: Inverse Problems

6. Noise Models Most noise models assume the noise is some known probability density function. The density function is chosen based on the underlining physics. Gaussian: poor illumination. Rayleigh: range image Salt and Pepper: faulty switch during imaging Gamma or Exp: laser imaging CGU IPAM 2003: Inverse Problems

7. Restoration Model Many restoration filters can be applied to recover the degraded image: Rudin-Osher-Fatemi, Wiener, Inverse FFT with threshold, etc. But all require a stopping criteria based on a noise measure. CGU IPAM 2003: Inverse Problems

8. Stopping Criteria As was illustrated by S. Osher (on Monday) and M. Bertero (in the tutorials) whether the restoration filter is direct or iterative, it must be stopped before the image begins to degrade.

9. Stopping the Restoration Noise in the data is amplified as the number of iterations increase, while the blur decreases as the number of iteration increase. This is the prototypical behavior which was illustrated by Professor Heinz Engl during the inverse problem tutorials. CGU IPAM 2003: Inverse Problems

10. Discrepancy Principle The discrepancy principle gives a natural stopping criteria, namely to stop the restoration process when the residual is of the same order as the noise. The drawback to this approach for image processing is that the discrepancy principle tends to recover an image that is still too blurred. Question: How many more times can we iterate passed the threshold given by the discrepancy principle and still get an image that is not predominately noisy. CGU IPAM 2003: Inverse Problems

11. Objective The objective is to highlight fine details in the image which were suppressed by the blur. The problem is that enhancement of fine detail (or edges) is equivalent to enhancement of noise. The challenge is to enhance details as much as possible before the noise overtakes the image details. CGU IPAM 2003: Inverse Problems

12. Wavelets and Details Wavelets provide a nature way to separate signal scales, especially for 1D signals. Taking the fine scale wavelet coefficients of the diagonal of the image provides a measure of the noise and image details. The next set of sides show the fine scale coefficients for a reconstructed version of the quarter image (to the right) at various iterations steps. CGU IPAM 2003: Inverse Problems

13. Fine Wavelet Coefficients (1) CGU IPAM 2003: Inverse Problems

14. Fine Wavelet Coefficients (2) CGU IPAM 2003: Inverse Problems

15. Fine Wavelet Coefficients (3) CGU IPAM 2003: Inverse Problems

16. Fine Wavelet Coefficients (4) CGU IPAM 2003: Inverse Problems

17. Fine Wavelet Coefficients (5) CGU IPAM 2003: Inverse Problems

18. Fine Wavelet Coefficients (6) CGU IPAM 2003: Inverse Problems

19. Observations The discrepancy principle gives a good starting guess for the image details as the noise level is low. Noise will eventually overtakes details, therefore tracking changes in the fine wavelet coefficient gives a possible way to differential between image details and noise. A natural stopping criteria is to stop iterating when the fine wavelet coefficients that have been identified as details are of the same order as the fine wavelet coefficients that have been identified as noise, i.e., the discrepancy principle applied to fine wavelet coefficients. CGU IPAM 2003: Inverse Problems

20. Stopping Algorithm 1.Choose your initial image according to the discrepancy principle. 2.Calculate fine wavelet coefficients corresponding to the image’s diagonal. 3.Calculate the mean of the fine wavelet coefficients. 4.Assign the fine wavelet coefficients which are larger than the mean to the set: details. 5.If there is a previous details set, take the intersection of it with the new details set. 6.Repeat steps 2 through 5 until the mean of the details set is equal to the mean of the noise set; noise is the set of wavelet coefficients not in details.

22. Stopping Criteria Results: eight

24. Stopping Criteria Results: autumn

26. Stopping Criteria Results: rice

28. Stopping Criteria Results: alumgrns

30. Stopping Criteria Results: bacteria

32. Stopping Criteria Results: cameraman

34. Stopping Criteria Results: mri

36. Stopping Criteria Results: ic

38. Stopping Criteria Results: moon

39. Preliminary Results CGU IPAM 2003: Inverse Problems

40. Preliminary Results CGU IPAM 2003: Inverse Problems

41. Preliminary Results CGU IPAM 2003: Inverse Problems

42. Future Work Define an energy measure to compare wavelet coefficients instead of using the mean value. Apply a moving class to the wavelet coefficients to take into account the difference in behavior between details and noise. Apply different deblurring/noise models to test that the stopping criteria is not sensitive to the deblurring/noise process. Expend the method from a global to local stopping criteria. CGU IPAM 2003: Inverse Problems

43. Special Thanks to the Organizing Committee and IPAM Staff.

44. References M. Bertero, Image Deconvolution, Inverse Problems: Computational Methods and Emerging Applications, IPAM 2003. (http://www.ipam.ucla.edu/publications/invws1/invws1_3804.pdf) P. Blomgren and T. F. Chan, Modular Solver for Constraint Image Restoration Problems Using the Discrepancy Principle, Numerical Linear Algebra with Applications,Vol 9, issue 5, pp 347-358. David Donoho, De-Noising by Soft-Thresholding (http://www-stat.stanford.edu/~donoho/reports.html) H. W. Engl, Inverse Problems 1, Inverse Problems: Computational Methods and Emerging Applications Tutorials, IPAM 2003. (http://www.ipam.ucla.edu/publications/invtut/invtut_hengl1.pdf) R. Gonzalez and R. Woods, Digital Image Processing, second edition, Prentice Hall, Inc., 2002. K. Lee, J. Nagy and L. Perrone, Iterative Methods for Image Restoration: A Matlab Object Oriented Approach. Iterative Methods for Image Restoration, May 15, 2002. R. Ramlau, Morozov’s Discrepancy Principle for Tikhonov regularization of nonlinear operators, Report 01-08, Berichte aus der Technomathematik, July 2001.