Image Processing and Analysis (ImagePandA) 5 – Image Restoration and Reconstruction Christoph Lampert / Chris Wojtan Based on slides by Selim Aksoy, Bilkent University TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.:

Image interpolation

Nearest neighbor interpolation

Image interpolation

Nearest neighbor interpolation First, find the x and y values of the nearest pixels int[] px = new int[4]; int[] py = new int[4]; px[0] = Math.floor(x);py[0] = Math.floor(y); px[1] = Math.ceil(x);py[1] = Math.floor(y); px[2] = Math.floor(x);py[2] = Math.ceil(y); px[3] = Math.ceil(x);py[3] = Math.ceil(y);

Nearest neighbor interpolation Next, find the nearest pixel int nx, ny;// x,y coords of nearest neighbor double minDist = Double.MAX_VALUE; for(int i=0; i<4; i++) { double dist = Math.abs(x-px[i])+Math.abs(y-py[i]); if(dist<minDist) { minDist = dist; nx=px[i]; ny=py[i]; } Finally, assign the value of the nearest pixel pixelVal = image.get(nx,ny);

Nearest neighbor interpolation

Piecewise linear interpolation

Piecewise Linear interpolation 0 1

Piecewise linear interpolation

Piecewise cubic interpolation

Piecewise Cubic (Hermite) interpolation

2D interpolation Nearest neighborBi-linear Bi-cubic

17 Resizing images How can we generate a half-sized version of a large image? Adapted from Steve Seitz, U of Washington

18 Resizing images Throw away every other row and column to create a 1/2 size image (also called sub-sampling). 1/4 1/8 Adapted from Steve Seitz, U of Washington

19 Resizing images Does this look nice? 1/4 (2x zoom)1/8 (4x zoom)1/2 Adapted from Steve Seitz, U of Washington

20 Sampling and aliasing Adapted from Steve Seitz, U of Washington

Sampling and aliasing

22 Sampling and aliasing Errors appear if we do not sample properly. Common phenomenon: High spatial frequency components of the image appear as low spatial frequency components. Examples: Wagon wheels rolling the wrong way in movies. Checkerboards misrepresented in ray tracing. Striped shirts look funny on color television.

23 Resizing images Throw away every other row and column… … regular sampling of a high-frequency function! 1/4 1/8 Adapted from Steve Seitz, U of Washington

24 Resizing images Does this look nice? 1/4 (2x zoom)1/8 (4x zoom)1/2 Adapted from Steve Seitz, U of Washington

25 Resizing images Regular sampling of a high-frequency image causes aliasing! Solution: smooth the image (remove high frequencies) first! Gaussian 1/4 Gaussian 1/8 Gaussian 1/2 Adapted from Steve Seitz, U of Washington

26 Resizing images Gaussian 1/4 (2x zoom) Gaussian 1/8 (4x zoom) Gaussian 1/2 Adapted from Steve Seitz, U of Washington

27 Gaussian pyramids Adapted from Gonzales and Woods

28 Gaussian pyramids Adapted from Michael Black, Brown University

29 Gaussian pyramids Adapted from Michael Black, Brown University

De-noising 30

Removing Noise 31

Inverting the Degradation Function 32

Inverting the Degradation Function 33 Original motion-blurred image PSF Clean image of The Beehive Nebula

Inverting the Degradation Function 34

Inverting the Degradation Function 35

Texture Synthesis 36 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels ?

Texture Synthesis 37 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

Texture Synthesis 38 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems

Texture Synthesis 39 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems Can use this to fill holes in an image Instead of some example, use the image itself

Super-resolution 40 How do we increase image size? Interpolation? Need to fill in missing details EMBIGGEN INTERPOLATION

Super-resolution 41 How do we increase image size? Interpolation? Need to fill in missing details Combine multiple images at sub-pixel off-sets Synthesize details from examples Steal details from other images Copy details from same image (at different scales)