Edge-Directed Image Interpolation Nickolaus Mueller, Yue Lu, and Minh N. Do “In theory, there is no difference between theory and practice; In practice, there is.” -Chuck Reid

I. Description of Problem A. Examples B. One-Dimensional Signals C. Two-Dimensional Images II. State of the Art A. Description of Methods B. Results III. Wavelet Algorithms A. Regularity Preserving Image Interpolation B. Proposed Method using Contourlets Outline of the Talk

Basic Image Interpolation Given a low-resolution image, increase resolution by a factor of 2 or larger

Description of Problem Problem: Basic interpolation techniques cause “jagged” or “blurred” edges Goal: Reduce artifacts using edge information Simple image model: continuous, smooth objects piecewise continuous, smooth edges

Examples of Edge Artifacts Original Bilinear Bicubic Original

One-Dimensional Problem

Images: A More Difficult Task 2-D Edges - Magnitude and directional component Edges have “Geometric Regularity” Challenge: Estimate orientation so that edges are both sharp and free from artifacts.

State of the Art Methods Sub-pixel Edge Localization Kris Jensen and Dimitris Anastassiou, 1995 New Edge Directed Interpolation Xin Li and Michael T. Orchard, 2001 Canny Edge Based Interpolation Hongjian Shi and Rabab Ward, 2002 Data-Dependent Triangulation Dan Su and Phillip Willis, 2004 Edge-Guided Interpolation Lei Zhang and Xiaolin Wu, 2006

Sub-pixel Edge Localization Explicitly calculate edges in 3 x 3 window of image Ideal step edge assumption Calculating the parameters: Develop continuous space theory - projections onto an orthonormal basis Use discrete approximations to inner products. A B

New Edge-Directed Interpolation Classical Wiener theory to develop MMSE weighting scheme for interpolation Estimate high resolution covariances from low resolution image. y is the data vector, C is a matrix used to estimate the high resolution covariance matrix Dark Pixels: Low Resolution Lattice Red Pixel: Pixel to be Interpolated in Step 1 Green Pixels: Pixels Interpolated in Step 2

Canny Edge Based Expansion First, expand image using bilinear or bicubic interpolation Run Canny edge detector on expanded image Determine if magnitude of gradient is larger vertically or horizontally at each edge pixel Modify pixels on either side of edge in vertical or horizontal direction

Data-Dependent Triangulation For each set of four low resolution pixels, estimate edge as dividing pixels into two triangles Create an image mesh which stores the direction of each edge Use linear interpolation within triangles Image Mesh

Edge Guided Image Interpolation More general triangulation technique Use directional variances to produce weighting scheme Perform interpolation using both triangles, fuse with weighting scheme

Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

Comparison of Methods OriginalBilinearSub-pixel Edge Loc. NEDICanny Edge BasedDDT

Factor of Four Interpolation OriginalBilinear NEDICanny Edge BasedDDT

Algorithm Comparison Lena Gaussian Disc Bilinear SEL NEDI Canny DDT Edge Guided PSNR Lena Gaussian Disc Bilinear SEL NEDI Canny DDT Edge Guided Speed in Seconds

Regularity Preserving Image Interpolation High similarity between different wavelet scales in regions of low regularity Convergence of series of features across scales for edge detection Goal: Synthesize a new sub-band by extrapolating from rate of decay of features across known sub-bands Apply algorithm separably along rows and columns

Regularity Preserving Image Interpolation

Take Home Message Higher cost methods can result in significant improvement Still room for improvement using low-cost algorithms Current wavelet techniques still have room for improvement Proposed Method: Edge- Directed Interpolation using Multiscale Geometric Representations Questions?