1. Edge-Directed Image Interpolation Nickolaus Mueller, Yue Lu, and Minh N. Do “In theory, there is no difference between theory and practice; In practice, there is.” -Chuck Reid

2. I. Description of Problem A. Examples B. One-Dimensional Signals C. Two-Dimensional Images II. State of the Art A. Description of Methods B. Results III. Wavelet Algorithms A. Regularity Preserving Image Interpolation B. Proposed Method using Contourlets Outline of the Talk

3. Basic Image Interpolation Given a low-resolution image, increase resolution by a factor of 2 or larger

4. Description of Problem Problem: Basic interpolation techniques cause “jagged” or “blurred” edges Goal: Reduce artifacts using edge information Simple image model: continuous, smooth objects piecewise continuous, smooth edges

5. Examples of Edge Artifacts Original Bilinear Bicubic Original

6. One-Dimensional Problem

7. Images: A More Difficult Task 2-D Edges - Magnitude and directional component Edges have “Geometric Regularity” Challenge: Estimate orientation so that edges are both sharp and free from artifacts.

8. State of the Art Methods Sub-pixel Edge Localization Kris Jensen and Dimitris Anastassiou, 1995 New Edge Directed Interpolation Xin Li and Michael T. Orchard, 2001 Canny Edge Based Interpolation Hongjian Shi and Rabab Ward, 2002 Data-Dependent Triangulation Dan Su and Phillip Willis, 2004 Edge-Guided Interpolation Lei Zhang and Xiaolin Wu, 2006

9. Sub-pixel Edge Localization Explicitly calculate edges in 3 x 3 window of image Ideal step edge assumption Calculating the parameters: Develop continuous space theory - projections onto an orthonormal basis Use discrete approximations to inner products. A B

10. New Edge-Directed Interpolation Classical Wiener theory to develop MMSE weighting scheme for interpolation Estimate high resolution covariances from low resolution image. y is the data vector, C is a matrix used to estimate the high resolution covariance matrix Dark Pixels: Low Resolution Lattice Red Pixel: Pixel to be Interpolated in Step 1 Green Pixels: Pixels Interpolated in Step 2

11. Canny Edge Based Expansion First, expand image using bilinear or bicubic interpolation Run Canny edge detector on expanded image Determine if magnitude of gradient is larger vertically or horizontally at each edge pixel Modify pixels on either side of edge in vertical or horizontal direction

12. Data-Dependent Triangulation For each set of four low resolution pixels, estimate edge as dividing pixels into two triangles Create an image mesh which stores the direction of each edge Use linear interpolation within triangles Image Mesh

13. Edge Guided Image Interpolation More general triangulation technique Use directional variances to produce weighting scheme Perform interpolation using both triangles, fuse with weighting scheme

14. Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

15. Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

16. Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

17. Factor of Four Interpolation OriginalBilinear NEDICanny Edge BasedDDT

18. Algorithm Comparison Lena Gaussian Disc Bilinear32.4239.58 SEL33.0946.04 NEDI37.3742.76 Canny37.2940.37 DDT37.4241.68 Edge Guided37.3741.68 PSNR Lena Gaussian Disc Bilinear0.2870.314 SEL2.0473.026 NEDI42.536.1 Canny1.3861.299 DDT0.9450.982 Edge Guided 1.1241.230 Speed in Seconds

19. Regularity Preserving Image Interpolation High similarity between different wavelet scales in regions of low regularity Convergence of series of features across scales for edge detection Goal: Synthesize a new sub-band by extrapolating from rate of decay of features across known sub-bands Apply algorithm separably along rows and columns

20. Regularity Preserving Image Interpolation

21. Take Home Message Higher cost methods can result in significant improvement Still room for improvement using low-cost algorithms Current wavelet techniques still have room for improvement Proposed Method: Edge- Directed Interpolation using Multiscale Geometric Representations Questions?