Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Joint work with Frederick Park and Andy M. Yip Astronomical Data Analysis Software & Systems Conference Series 2004 Pasadena, CA, October 24-27, 2004

Outline Part I: Total Variation Blind Deconvolution Part II: Simultaneous TV Image Inpainting and Blind Deconvolution Part III: Automatic Parameter Selection for TV Blind Deconvolution Caution: Our work not developed specifically for Astronomical images

Blind Deconvolution Problem =  + Observed image Unknown true image Unknown point spread function Unknown noise Goal: Given uobs, recover both uorig and k

Typical PSFs PSFs w/ sharp edges: PSFs w/ smooth transitions Motion blur: length of the line linearly proportion to the speed of the motion Scatter blur: f(x) = (\beta^2 + ||x||^2)^(-3/2)

Total Variation Regularization Deconvolution ill-posed: need regularization Total variation Regularization: The characteristic function of D with height h (jump): TV = Length(∂D)h TV doesn’t penalize jumps Co-area Formula: D h

TV Blind Deconvolution Model (C. and Wong (IEEE TIP, 1998)) Objective: Subject to: 1 determined by signal-to-noise ratio 2 parameterizes a family of solutions, corresponds to different spread of the reconstructed PSF Alternating Minimization Algorithm: Globally convergent with H1 regularization.

Blind v.s. non-Blind Deconvolution Clean image Blind v.s. non-Blind Deconvolution Recovered Image PSF Blind 1 = 2106, 2 = 1.5105 Observed Image noise-free non-Blind True PSF An out-of-focus blur is recovered automatically Recovered blind deconvolution images almost as good as non-blind Edges well-recovered in image and PSF

Blind v.s. non-Blind Deconvolution: High Noise Clean image Blind v.s. non-Blind Deconvolution: High Noise Blind Observed Image SNR=5 dB non-Blind True PSF 1 = 2105, 2 = 1.5105 An out-of-focus blur is recovered automatically Even in the presence of high noise level, recovered images from blind deconvolution are almost as good as those recovered with the exact PSF

Controlling Focal-Length Recovered Images are 1-parameter family w.r.t. 2 1107 1105 1104 Recovered Blurring Functions (1 = 2106) 2: The parameter 2 controls the focal-length

Generalizations to Multi-Channel Images Inter-Channel Blur Model Color image (Katsaggelos et al, SPIE 1994): k1: within channel blur k2: between channel blur m-channel TV-norm (Color-TV) (C. & Blomgren, IEEE TIP ‘98)

Examples of Multi-Channel Blind Deconvolution (C. and Wong (SPIE, 1997)) Original image Out-of-focus blurred blind non-blind Gaussian blurred blind non-blind Blind is as good as non-blind Gaussian blur is harder to recover (zero-crossings in frequency domain)

TV Blind Deconvolution Patented!

Outline Part I: Total Variation Blind Deconvolution Part II: Simultaneous TV Image Inpainting and Blind Deconvolution Part III: Automatic Parameter Selection for TV Blind Deconvolution

TV Inpainting Model (C. & Shen SIAP 2001) Scratch Removal Graffiti Removal

Images Degraded by Blurring and Missing Regions Calibration errors of devices Atmospheric turbulence Motion of objects/camera Missing regions Scratches Occlusion Defects in films/sensors +

Problems with Inpaint then Deblur Original Signal Blurring func.  Blurred Signal = Blurred + Occluded  = Inpaint first  reduce plausible solutions Should pick the solution using more information

Problems with Deblur then Inpaint Original Occluded Support of PSF Dirichlet Neumann Inpainting Different BC’s correspond to different image intensities in inpaint region. Most local BC’s do not respect global geometric structures

The Joint Model A natural combination of TV deblur + TV inpaint Inpainting take place Coupling of inpainting & deblur Do --- the region where the image is observed Di --- the region to be inpainted A natural combination of TV deblur + TV inpaint No BC’s needed for inpaint regions 2 parameters (can incorporate automatic parameter selection techniques)

Simulation Results (1) The vertical strip is completed Not completed Degraded Restored Zoom-in The vertical strip is completed Not completed Use higher order inpainting methods E.g. Euler’s elastica, curvature driven diffusion

Simulation Results (2) Original Observed Restored Deblur then inpaint (many artifacts) Inpaint then deblur (many ringings)

Boundary Conditions for Regular Deblurring Dirichlet B.C. Periodic B.C. Neumann B.C. Original image domain and artificial boundary outside the scene Inpainting B.C.

Outline Part I: Total Variation Blind Deconvolution Part II: Simultaneous TV Image Inpainting and Blind Deconvolution Part III: Automatic Parameter Selection for TV Blind Deconvolution (Ongoing Research)

Automatic Blind Deblurring (ongoing research) observed image Clean image SNR = 15 dB Problem: Find 2 automatically to recover best u & k Recovered images: 1-parameter family wrt 2 Consider external info like sharpness to choose optimal 2

Motivation for Sharpness & Support Support of Sharpest image has large gradients Preference for gradients with small support

Proposed Sharpness Evaluator Support of F(u) small => sharp image with small support F(u)=0 for piecewise constant images F(u) penalizes smeared edges

Planets Example 1=0.02 (optimal) Rel. errors in u (blue) and k (red) v.s. 2 1=0.02 (optimal) Optimal Restored Image Auto-focused Image Proposed Objective v.s. 2 (minimizer of rel. error in u) (minimizer of sharpness func.) The minimum of the sharpness function agrees with that of the rel. errors of u and k

Satellite Example 1=0.3 (optimal) Rel. errors in u (blue) and k (red) v.s. 2 1=0.3 (optimal) Optimal Restored Image Auto-focused Image Proposed Objective v.s. 2 (minimizer of rel. error in u) (minimizer of sharpness func.) The minimum of the sharpness function agrees with that of the rel. errors of u and k

Potential Applications to Astronomical Imaging TV Blind Deconvolution TV/Sharp edges useful? Auto-focus: appropriate objective function? How to incorporate a priori domain knowledge? TV Blind Deconvolution + Inpainting Other noise models: e.g. salt-and-pepper noise

References C. and C. K. Wong, Total Variation Blind Deconvolution, IEEE Transactions on Image Processing, 7(3):370-375, 1998. C. and C. K. Wong, Multichannel Image Deconvolution by Total Variation Regularization, Proc. to the SPIE Symposium on Advanced Signal Processing: Algorithms, Architectures, and Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F. Luk. C. and C. K. Wong, Convergence of the Alternating Minimization Algorithm for Blind Deconvolution, UCLA Mathematics Department CAM Report 99-19. R. H. Chan, C. and C. K. Wong, Cosine Transform Based Preconditioners for Total Variation Deblurring, IEEE Trans. Image Proc., 8 (1999), pp. 1472-1478 C., A. Yip and F. Park, Simultaneous Total Variation Image Inpainting and Blind Deconvolution, UCLA Mathematics Department CAM Report 04-45.