Image Deblurring with Optimizations Qi Shan Leo Jiaya Jia Aseem Agarwala University of Washington The Chinese University of Hong Kong Adobe Systems, Inc.

2 The Problem

An Example

4 Previous Work (1) Hardware solutions: [Raskar et al. 2006] [Ben-Ezra and Nayar 2004] [Levin et al. 2008]

5 Previous Work (2) Multi-frame solutions: [Petschnigg et al. 2004] [Jia et al. 2004] [Rav-Acha and Peleg 2005] [Yuan et al. 2007]

6 Previous Work (3) Single image solutions: [Jia 2007] [Fergus et al. 2006] [Levin et al. 2007]

Most recent work on Single Image Deblurring Qi Shan, Jiaya Jia, and Aseem Agarwala High-Quality Motion Deblurring From a Single Image. SIGGRAPH 2008 Lu Yuan, Jian Sun, Long Quan and Heung-Yeung Shum Progressive Inter-scale and intra-scale Non-blind Image Deconvolution. SIGGRAPH Joshi, N., Szeliski, R. and Kriegman, D. PSF Estimation using Sharp Edge Prediction, CVPR A. Levin, Y. Weiss, F. Durand, W. T. Freeman Understanding and evaluating blind deconvolution algorithms. CVPR 2009 Sunghyun Cho and Seungyong Lee, Fast Motion Deblurring. SIGGRAPH ASIA 2009 And many more...

Some take home ideas 1. Using hierarchical approaches to estimate kernel in different scales 2. Realize the importance of strong edges 3. Bilateral filtering to suppress ringing artifacts 4. RL deconvolution is good, but we've got better chioces 5. Stronger prior does a better job 6. Deblurring by assuming spatially variant kernel is a good way to go

Today's topic How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process Short discussion on 1. the stability of a non-blind deconvolution process 2. noise resistant non-blind deconvolution and denoising

10 Image Global Statistics …

11 … Image Global Statistics

12 Image Global Statistics

13 Image Local Constraint

14 Image Local Constraint

15 Image Local Constraint

16 Image Local Constraint

17 exponentially distributed Kernel Statistics

18 Combining All constraints Lfn Two-step iterative optimization Optimize L Optimize f

19 Idea: separate convolution Optimize L Optimization Process replace with

20 Idea: separate convolution Optimize L Optimization Process replace with

21 Adding a new constraint to make Removing terms that are not relevant to Updating L An easy quadratic optimization problem with a closed form solution in the frequency domain

22 Updating Removing terms that are not relevant to

23 each only contains a single variable Ψ i It is then a set of easy single variable optimization problems

24 Iteration 0 (initialization)

25 Time: about 30 seconds for an 800x600 image Iteration 8 (converge)

26 A comparison RL deconvolution

27 A comparison Our deconvolution

28 Two-step iterative optimization Optimize L Optimize f Optimization with a total variation regularization

29 Results

30 Results

31

32

33 More results

34 More results

Today's topic How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process Short discussion on 1. the stability of a non-blind deconvolution process 2. noise resistant non-blind deconvolution and denoising

Stability Considering the simplest case: Wiener Filtering How about if And

Stability Thus where is the frequency domain representation of is the variance of the noise Observation: the noise in the blur image is magnified in the deconvolved image. And the Noise Magnification Factor (NMF) is solely determined by the filter

Some examples

Dense kernels are less stable for deconvolution than sparse ones

40 Noise resistant deconvolution and denoising With Jiaya Jia, Singbing Kang and Zenlu Qin In CVPR 2010 Blind and non-blind image deconvolution software is available online and will be updated soon! See you in San Francisco!