1. Unitary Extension Principle: Ten Years After Zuowei Shen Department of Mathematics National University of Singapore

2. Outline Unitary Extension Principle (UEP) Applications in Image Processing New Development

3. Wavelet Tight Frame Let be countable. is a tight frame if It is equivalent to A wavelet system is the collection of the dilations and the shifts of a finite set

4. Unitary Extension Principle Function is refinable with mask if Let, where, with wavelet masks Define. Unitary Extension Principle: (Ron and Shen, J. Funct. Anal., 1997), is a tight frame provided

5. Why the Unitary Extension Principle?

6. Constructions of wavelets become painless Symmetric spline wavelets with short support; Wavelets for practical problems.

7. Lead to Pseudo-splines Provide a better approximation order for truncated wavelet series. Splines, orthonormal and interpolatory refinable functions are special cases of pseudo-splines. First Introduced in: I. Daubechies, B, Han, A. Ron and Z. Shen, Framelets: MRA-based constructions of wavelet frames, Applied and Computation Harmonic Analysis, 14, 1—46, 2003. Regularity analysis and … B. Dong and Z. Shen Pseudo-splines, wavelets and framelets, Applied and Computation Harmonic Analysis, 22 (1), 78—104, 2007.

8. I. Daubechies, B, Han, A. Ron and Z. Shen, Framelets: MRA- based constructions of wavelet frames, Applied and Computation Harmonic Analysis, 14, 1—46, 2003 C. K. Chui, W. He, J. St ö ckler, Compactly supported tight and sibling frames with maximum vanishing moments, Applied and Computation Harmonic Analysis, 13, 224 — 262, 2002 Lead to Oblique Extension Principle

9. Nonstationary tight frames Nonstationary tight frames have been studied extensively by C. Chui, W. He and J. Stockler. Compactly supported, symmetric tight frames with infinite order of smoothness and vanishing moment by using (nonstationary pseudo-splines). B. Han, Z. Shen, Compactly Supported Symmetric Wavelets With Spectral Approximation Order, (2006).

10. Characterization of spaces Characterization of various space norms by wavelet frame coefficients has been studied by L. Borup, R. Grinbonval, and M. Nieslsen; Y. Hur and A. Ron Link the characterizations to frames in Sobolev spaces with their duals in dual Sobolev spaces. B. Han and Z. Shen, Dual Wavelet Frames and Riesz Bases in Sobolev Spaces, preprint (2007)

11. Application I: Filling missing data

12. Inpainting

13. Given 64 x 64 imageFirst approximated 128 x 128image

14. Given 64 X 64 imageFirst approximated 128 X 128 imageResult 128 X 128 image

15. Given 64 X 64 image Result 128 X 128 imageGiven 128 X 128 image

16. Matrix Representations Let rows of be frame, i.e. Decomposition: Reconstruction: can be generated by tight frame filters obtained via UEP

17. Algorithm DecompositionThreshold Reconstruction Replace the data on by the known data g B-Spline tight frame derived by UEP is used.

18. Convergence and minimization the sequence converges to a solution of the minimization problem

19. Convergence and minimization the sequence converges to a solution of the minimization J. Cai, R. Chan, Z. Shen, A Framelet-based Image Inpainting Algorithm, Preprint (2006)

20. Numerical Experiments Observed ImageFramelet-Based Method PSNR=33.83dB PDE Method PSNR=32.91dB

21. Numerical Results Observed Image Framelet-Based Method PSNR=33.10dB Minimizing the functional without penalty term PSNR=30.70dB

22. Application II: Deconvolution

23. Setting Question: Given How to find ?  Regularization Methods: Solving a system of linear equations;

24. Designing a tight (or bi) frame system with being one of the masks using UEP;Designing a tight (or bi) frame system with being one of the masks using UEP; Reducing to the ``problem of recovering wavelet coefficients’’;Reducing to the ``problem of recovering wavelet coefficients’’; Deriving an algorithm from the tight frame system designed;Deriving an algorithm from the tight frame system designed; Proving convergence of the algorithm;Proving convergence of the algorithm; Analyzing the minimization properties of the solution.Analyzing the minimization properties of the solution. A. Chai and Z. Shen, Deconvolution by tight framelets, Numerische Mathematik to appear. Ideas

25. Algorithm Decomposition Replace the known data g Threshold Reconstruction Project onto the set of non-negative vectors

26. R. Chan, T. Chan, L. Shen, Z. Shen: Wavelet algorithms for high R. Chan, T. Chan, L. Shen, Z. Shen: Wavelet algorithms for high resolution image reconstruction, SIAM Journal on Scientific resolution image reconstruction, SIAM Journal on Scientific Computing, 24 (2003) 1408-1432. Computing, 24 (2003) 1408-1432. Ideas started in Using bi-frames derived from biorthogonal wavelets, it performs better than the regularization method.

27. Other’s work Wavelet-Vaguelette decomposition by Donoho Mirror wavelet method by Mallat et al. Wavelet Galerkin method, inverse truncated operator under wavelet basis by Cohen et al. Iterative threshold method, sparse representation of solution under wavelet basis given by Daubechies et al.

28. High-Resolution Image Reconstruction Resolution = 64  64Resolution = 256  256

29. Four low resolution images (64  64) of the same scene. Each shifted by sub-pixel length. Construct a high-resolution image (128  128) from them.

30. #2 #4 #1 taking lens CCD sensor array relay lenses partially silvered mirrors

31. Modeling: High-resolution pixels LR image: the down samples of observed image at different sub- pixel position. Observed image: HR image passing through a low-pass filter a. Reducing to a deconvolution problem

32. Reconstruction high resolution image Original LR FrameObserved HR Regularization Wavelets

33. Infrared Astronomy Imaging: Chopped and Nodded Process

34. Numerical Results: 1D Signals K=37, N=128 Original Projected Landweber Framelet Method Ex 1 Ex 2

35. Numerical Results: Real Images Observed Image from United Kingdom Infra-Red Telescope Projected Landweber’s Iteration Framelet-Based Method

36. J. Cai, R. Chan, L. Shen, Z. Shen, Restoration of Chopped and Nodded Images by Framelet, preprint (2006). Restoring chopped and nodded images by tight frames, Proc. SPIE Symposium on Advanced Signal Processing: Algorithms, Architectures, and Implementations, Vol. 5205, 310-319, San Diego CA, August, 2003.

37. Another Example One of the frame in a video Before enhancement After enhancement R. Chan, Z. Shen, T. Xia A framelet algorithm for enchancing video stills, Applied and Computational Harmonic Analysis

38. Reference frame t Displacement error   Improving resolution of reference frame

39. 704-by-578 image of f 100 by bilinear interpolation

40. 704-by-578 image of f 100 by tight frame method using 20 frames from the movie

41. Bilinear methodTight frame method Video Enhancement

42. New Development Two systems: one represents piecewise smooth function sparsely, the other represents the texture sparsely Two systems: one is a frame (or Riesz basis) in one space and the other in its dual space. It is better to have such two systems satisfying some `dual’ relations

43. New Development Let be a compactly supported refinable function with some smoothness. Define Can the corresponding wavelet system be a Riesz basis for some space?

44. New Development Let be a compactly supported refinable function with some smoothness. Will form a frame in some space?

45. New Development B. Han and Z. Shen, Dual Wavelet Frames and Riesz Bases in Sobolev Spaces, preprint (2007) This paper takes a new approach to handle all the questions raised before. Most of questions are solved, many new interesting directions are opened.

46. Thanks!