1. Some Blind Deconvolution Techniques in Image Processing
Tony Chan
Math Dept., UCLA
Joint work with Frederick Park and Andy M. Yip
Astronomical Data Analysis Software & Systems
Conference Series 2004
Pasadena, CA, October 24-27, 2004

2. Outline Part I: Total Variation Blind Deconvolution Part II:
Simultaneous TV Image Inpainting and Blind Deconvolution
Part III:
Automatic Parameter Selection for TV Blind Deconvolution
Caution: Our work not developed specifically for Astronomical images

3. Blind Deconvolution Problem=  +
Observed image
Unknown true image
Unknown point spread function
Unknown noise
Goal: Given uobs, recover both uorig and k

4. Typical PSFs PSFs w/ sharp edges: PSFs w/ smooth transitions
Motion blur: length of the line linearly proportion to the speed of the motion
Scatter blur: f(x) = (\beta^2 + ||x||^2)^(-3/2)

5. Total Variation Regularization
Deconvolution ill-posed: need regularization
Total variation Regularization:
The characteristic function of D with height h (jump):
TV = Length(∂D)h
TV doesn’t penalize jumps
Co-area Formula: D h

6. TV Blind Deconvolution Model
(C. and Wong (IEEE TIP, 1998))
Objective:
Subject to:
1 determined by signal-to-noise ratio
2 parameterizes a family of solutions, corresponds to different spread of the reconstructed PSF
Alternating Minimization Algorithm:
Globally convergent with H1 regularization.

7. Blind v.s. non-Blind Deconvolution
Clean image
Blind v.s. non-Blind Deconvolution
Recovered Image
PSF
Blind
1 = 2106, 2 = 1.5105
Observed Image noise-free
non-Blind
True PSF
An out-of-focus blur is recovered automatically
Recovered blind deconvolution images almost as good as non-blind
Edges well-recovered in image and PSF

8. Blind v.s. non-Blind Deconvolution: High Noise
Clean image
Blind v.s. non-Blind Deconvolution: High Noise
Blind
Observed Image SNR=5 dB
non-Blind
True PSF
1 = 2105, 2 = 1.5105
An out-of-focus blur is recovered automatically
Even in the presence of high noise level, recovered images from blind deconvolution are almost as good as those recovered with the exact PSF

9. Controlling Focal-Length
Recovered Images are 1-parameter family w.r.t. 2
1107
1105
1104
Recovered Blurring Functions
(1 = 2106)
2:
The parameter 2 controls the focal-length

10. Generalizations to Multi-Channel Images
Inter-Channel Blur Model
Color image (Katsaggelos et al, SPIE 1994):
k1: within channel blur
k2: between channel blur
m-channel TV-norm (Color-TV)
(C. & Blomgren, IEEE TIP ‘98)

11. Examples of Multi-Channel Blind Deconvolution
(C. and Wong (SPIE, 1997))
Original image
Out-of-focus blurred blind non-blind
Gaussian blurred blind non-blind
Blind is as good as non-blind
Gaussian blur is harder to recover (zero-crossings in frequency domain)

12. TV Blind Deconvolution Patented!

13. Outline Part I: Total Variation Blind Deconvolution Part II:
Simultaneous TV Image Inpainting and Blind Deconvolution
Part III:
Automatic Parameter Selection for TV Blind Deconvolution

14. TV Inpainting Model (C. & Shen SIAP 2001)
Scratch Removal
Graffiti Removal

15. Images Degraded by Blurring and Missing Regions
Calibration errors of devices
Atmospheric turbulence
Motion of objects/camera
Missing regions
Scratches
Occlusion
Defects in films/sensors +

16. Problems with Inpaint then Deblur
Original Signal
Blurring func. 
Blurred Signal =
Blurred + Occluded  =
Inpaint first  reduce plausible solutions
Should pick the solution using more information

17. Problems with Deblur then Inpaint
Original
Occluded
Support of PSF
Dirichlet
Neumann
Inpainting
Different BC’s correspond to different image intensities in inpaint region.
Most local BC’s do not respect global geometric structures

18. The Joint Model A natural combination of TV deblur + TV inpaint
Inpainting take place
Coupling of inpainting & deblur
Do --- the region where the image is observed
Di --- the region to be inpainted
A natural combination of TV deblur + TV inpaint
No BC’s needed for inpaint regions
2 parameters (can incorporate automatic parameter selection techniques)

19. Simulation Results (1) The vertical strip is completed Not completed
Degraded
Restored
Zoom-in
The vertical strip is completed
Not completed
Use higher order inpainting methods
E.g. Euler’s elastica, curvature driven diffusion

20. Simulation Results (2) Original Observed Restored Deblur then inpaint
(many artifacts)
Inpaint then deblur
(many ringings)

21. Boundary Conditions for Regular Deblurring
Dirichlet B.C.
Periodic B.C.
Neumann B.C.
Original image domain and artificial boundary outside the scene
Inpainting B.C.

23. Outline Part I: Total Variation Blind Deconvolution Part II:
Simultaneous TV Image Inpainting and Blind Deconvolution
Part III:
Automatic Parameter Selection for TV Blind Deconvolution
(Ongoing Research)

24. Automatic Blind Deblurring (ongoing research)
observed image
Clean image
SNR = 15 dB
Problem: Find 2 automatically to recover best u & k
Recovered images: 1-parameter family wrt 2
Consider external info like sharpness to choose optimal 2

25. Motivation for Sharpness & Support
Support of
Sharpest image has large gradients
Preference for gradients with small support

26. Proposed Sharpness Evaluator
Support of
F(u) small => sharp image with small support
F(u)=0 for piecewise constant images
F(u) penalizes smeared edges

27. Planets Example 1=0.02 (optimal)
Rel. errors in u (blue) and k (red) v.s. 2
1=0.02 (optimal)
Optimal Restored Image
Auto-focused Image
Proposed Objective v.s. 2
(minimizer of rel. error in u)
(minimizer of sharpness func.)
The minimum of the sharpness function agrees with that of the rel. errors of u and k

28. Satellite Example 1=0.3 (optimal)
Rel. errors in u (blue) and k (red) v.s. 2
1=0.3 (optimal)
Optimal Restored Image
Auto-focused Image
Proposed Objective v.s. 2
(minimizer of rel. error in u)
(minimizer of sharpness func.)
The minimum of the sharpness function agrees with that of the rel. errors of u and k

29. Potential Applications to Astronomical Imaging
TV Blind Deconvolution
TV/Sharp edges useful?
Auto-focus: appropriate objective function?
How to incorporate a priori domain knowledge?
TV Blind Deconvolution + Inpainting
Other noise models: e.g. salt-and-pepper noise

30. References
C. and C. K. Wong, Total Variation Blind Deconvolution, IEEE Transactions on Image Processing, 7(3): , 1998.
C. and C. K. Wong, Multichannel Image Deconvolution by Total Variation Regularization, Proc. to the SPIE Symposium on Advanced Signal Processing: Algorithms, Architectures, and Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F. Luk.
C. and C. K. Wong, Convergence of the Alternating Minimization Algorithm for Blind Deconvolution, UCLA Mathematics Department CAM Report
R. H. Chan, C. and C. K. Wong, Cosine Transform Based Preconditioners for Total Variation Deblurring, IEEE Trans. Image Proc., 8 (1999), pp
C., A. Yip and F. Park, Simultaneous Total Variation Image Inpainting and Blind Deconvolution, UCLA Mathematics Department CAM Report