1. Sparse Approximation by Wavelet Frames and Applications
Bin Dong
Department of Mathematics
The University of Arizona
2012 International Workshop on
Signal Processing , Optimization, and Control June 30- July 3, 2012
USTC, Hefei, Anhui, China

2. Outlines
Wavelet Frame Based Models for Linear Inverse Problems (Image Restoration)
Applications in CT Reconstruction
1-norm based models
Connections to variational model
0-norm based model
Comparisons: 1-norm v.s. 0-norm
Quick Intro of Conventional CT Reconstruction
CT Reconstruction with Radon Domain Inpainting

3. Tight Frames in Orthonormal basis Riesz basis
Tight frame: Mercedes-Benz frame
Expansions:
Unique
Not unique

4. Tight Frames General tight frame systems Tight wavelet frames
Construction of tight frame: unitary extension principles [Ron and Shen, 1997]
They are redundant systems satisfying Parseval’s identity
Or equivalently
where and

5. Tight Frames Example: Fast transforms
Lecture notes: [Dong and Shen, MRA-Based Wavelet Frames and Applications, IAS Lecture Notes Series,2011]
Decomposition
Reconstruction
Perfect Reconstruction
Redundancy
Lecture notes: self-contained theoretical review of MRA-based wavelet frames, as well as recent applications in image restorations and analysis.

6. Image Restoration Model
Image Restoration Problems
Challenges: large-scale & ill-posed
Denoising, when is identity operator
Deblurring, when is some blurring operator
Inpainting, when is some restriction operator
CT/MR Imaging, when is partial Radon/Fourier
transform

7. Frame Based Models Image restoration model:
Balanced model for image restoration [Chan, Chan, Shen and Shen, 2003], [Cai, Chan and Shen, 2008]
When , we have synthesis based model [Daubechies, Defrise and De Mol, 2004; Daubechies, Teschke and Vese, 2007]
When , we have analysis based model [Stark, Elad and Donoho, 2005; Cai, Osher and Shen, 2009]
Synthesis: sharper edges but more artifact. Analysis: less artifact but edges are a bit blurry. Balance in between. Which one to choose depends on applications
Key: It works because images are piecewise smooth functions and have sparse representation under tight frame transforms
Resembles
Variational Models

8. Connections: Wavelet Transform and Differential Operators
Nonlinear diffusion and iterative wavelet and wavelet frame shrinkage
2nd-order diffusion and Haar wavelet: [Mrazek, Weickert and Steidl, 2003&2005]
High-order diffusion and tight wavelet frames in 1D: [Jiang, 2011]
Difference operators in wavelet frame transform:
True for general wavelet frames with various vanishing moments [Weickert et al., 2006; Shen and Xu, 2011]
Filters
Transform
Approximation

9. Connections: Analysis Based Model and Variational Model
[Cai, Dong, Osher and Shen, Journal of the AMS, 2012]:
The connections give us
Leads to new applications of wavelet frames:
Converges
For any differential operator when proper parameter is chosen.
Geometric interpretations of the wavelet frame transform (WFT)
WFT provides flexible and good discretization for differential operators
Different discretizations affect reconstruction results
Good regularization should contain differential operators with varied orders (e.g., total generalized variation [Bredies, Kunisch, and Pock, 2010])
Synthesis: sharper edges but more artifact. Analysis: less artifact but edges are a bit blurry. Balance in between. Which one to choose depends on applications
Key: It works because images are piecewise smooth functions and have sparse representation under tight frame transforms
Image segmentation: [Dong, Chien and Shen, 2010]
Surface reconstruction from point clouds: [Dong and Shen, 2011]
Standard Discretization
Piecewise Linear WFT

10. Frame Based Models: 0-Norm
Nonconvex analysis based model [Zhang, Dong and Lu, 2011]
Motivations:
Related work:
Restricted isometry property (RIP) is not
satisfied for many applications
Penalizing “norm” of frame coefficients
better balances sparsity and smoothnes
Synthesis: sharper edges but more artifact. Analysis: less artifact but edges are a bit blurry. Balance in between. Which one to choose depends on applications
Key: It works because images are piecewise smooth functions and have sparse representation under tight frame transforms
“norm” with : [Blumensath and Davies, 2008&2009]
quasi-norm with : [Chartrand, 2007&2008]

11. Fast Algorithm: 0-Norm
Penalty decomposition (PD) method [Lu and Zhang, 2010]
Algorithm:
Change of variables
Quadratic penalty
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

12. Fast Algorithm: 0-Norm Step 1: Subproblem 1a): quadratic
Subproblem 1b): hard-thresholding
Convergence Analysis [Zhang, Dong and Lu, 2011] :
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

13. Numerical Results Comparisons (Deblurring)
Most important: relationship between minimizers.
PFBS/FPC: [Combettes and Wajs, 2006] /[Hale, Yin and Zhang, 2010]
Balanced
Split Bregman: [Goldstein and Osher, 2008] & [Cai, Osher and Shen, 2009]
Analysis
0-Norm
PD Method: [Zhang, Dong and Lu, 2011]

14. Numerical Results Comparisons Portrait Couple Balanced Analysis
Most important: relationship between minimizers.
Couple
Balanced
Analysis

15. Faster Algorithm: 0-Norm
Start with some fast optimization method for nonsmooth and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].
Given the problem:
The DAL method:
where
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.
We solve the joint optimization problem of the DAL method using an inexact alternative optimization scheme

16. Faster Algorithm: 0-Norm
Start with some fast optimization method for nonsmooth and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].
The inexact DAL method:
Given the problem:
The DAL method:
where
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.
Hard thresholding

17. Faster Algorithm: 0-Norm
However, the inexact DAL method does not seem to converge!! Nonetheless, the sequence oscillates and is bounded.
The mean doubly augmented Lagrangian method (MDAL) [Dong and Zhang, 2011] solve the convergence issue by using arithmetic means of the solution sequence as outputs instead:
MDAL:
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

18. Comparisons: Deblurring
Comparisons of best PSNR values v.s. various noise level
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

19. Comparisons: Deblurring
Comparisons of computation time v.s. various noise level
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

20. Comparisons: Deblurring
What makes “lena” so special?
Decay of the magnitudes of the wavelet frame coefficients is very fast, which is what 0-norm prefers.
Similar observation was made earlier by [Wang and Yin, 2010].
1-norm
0-norm: PD
0-norm: MDAL
There are other algorithms also, but split Bregman was proved to be efficient for frame based models.

21. Applications in CT Reconstruction
With the Center for Advanced Radiotherapy and Technology (CART), UCSD
Applications in CT Reconstruction

22. 3D Cone Beam CT
Cone Beam CT

23. 3D Cone Beam CT
Discrete =
Animation created by Dr. Xun Jia

24. Cone Beam CT Image Reconstruction
Goal: solve
Difficulties:
Related work:
Unknown Image
Projected Image
In order to reduce dose, the system is highly underdetermined. Hence the solution is not unique.
Projected image is noisy.
Total Variation (TV): [Sidkey, Kao and Pan 2006], [Sidkey and Pan, 2008], [Cho et al. 2009], [Jia et al. 2010];
EM-TV: [Yan et al. 2011]; [Chen et al. 2011];
Wavelet Frames: [Jia, Dong, Lou and Jiang, 2011];
Dynamical CT/4D CT: [Chen, Tang and Leng, 2008],
[Jia et al. 2010], [Tian et al., 2011]; [Gao et al. 2011];

25. CT Image Reconstruction with Radon Domain Inpainting
Idea: start with
Benefits:
Instead of solving
We find both and such that:
is close to but with better quality
Prior knowledge of them should be used
Safely increase imaging dose
Utilizing prior knowledge we have for both CT
images and the projected images

26. CT Image Reconstruction with Radon Domain Inpainting
Model [Dong, Li and Shen, 2011]
Algorithm: alternative optimization & split Bregman.
where
p=1, anisotropic
p=2, isotropic

27. CT Image Reconstruction with Radon Domain Inpainting
Algorithm [Dong, Li and Shen, 2011]: block coordinate descend method [Tseng, 2001]
Convergence Analysis
Problem:
Algorithm:
Note: If each subproblem is solved exactly, then the convergence analysis was given by [Tseng, 2001], even for nonconvex problems.

28. CT Image Reconstruction with Radon Domain Inpainting
Results: N denoting number of projections
N=15
N=20

29. CT Image Reconstruction with Radon Domain Inpainting
Results: N denoting number of projections
N=15
N=20
W/O Inpainting
With Inpainting

30. Thank You Collaborators: Mathematics Stanley Osher, UCLA
Zuowei Shen, NUS
Jia Li, NUS
Jianfeng Cai, University of Iowa
Yifei Lou, UCLA/UCSD
Yong Zhang, Simon Fraser University, Canada
Zhaosong Lu, Simon Fraser University, Canada
Medical School
Steve B. Jiang, Radiation Oncology, UCSD
Xun Jia, Radiation Oncology, UCSD
Aichi Chien, Radiology, UCLA