1. Image Processing and Analysis (ImagePandA) 5 – Image Restoration and Reconstruction Christoph Lampert / Chris Wojtan Based on slides by Selim Aksoy, Bilkent University TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.:

2. Image interpolation

5. Nearest neighbor interpolation

6. Image interpolation

8. Nearest neighbor interpolation First, find the x and y values of the nearest pixels int[] px = new int[4]; int[] py = new int[4]; px[0] = Math.floor(x);py[0] = Math.floor(y); px[1] = Math.ceil(x);py[1] = Math.floor(y); px[2] = Math.floor(x);py[2] = Math.ceil(y); px[3] = Math.ceil(x);py[3] = Math.ceil(y);

9. Nearest neighbor interpolation Next, find the nearest pixel int nx, ny;// x,y coords of nearest neighbor double minDist = Double.MAX_VALUE; for(int i=0; i<4; i++) { double dist = Math.abs(x-px[i])+Math.abs(y-py[i]); if(dist<minDist) { minDist = dist; nx=px[i]; ny=py[i]; } Finally, assign the value of the nearest pixel pixelVal = image.get(nx,ny);

10. Nearest neighbor interpolation

11. Piecewise linear interpolation

12. Piecewise Linear interpolation 0 1

13. Piecewise linear interpolation

14. Piecewise cubic interpolation

15. Piecewise Cubic (Hermite) interpolation

16. 2D interpolation Nearest neighborBi-linear Bi-cubic

17. 17 Resizing images How can we generate a half-sized version of a large image? Adapted from Steve Seitz, U of Washington

18. 18 Resizing images Throw away every other row and column to create a 1/2 size image (also called sub-sampling). 1/4 1/8 Adapted from Steve Seitz, U of Washington

19. 19 Resizing images Does this look nice? 1/4 (2x zoom)1/8 (4x zoom)1/2 Adapted from Steve Seitz, U of Washington

20. 20 Sampling and aliasing Adapted from Steve Seitz, U of Washington

21. Sampling and aliasing http://www.youtube.com/watch?v=ckmtG8CZjDA

22. 22 Sampling and aliasing Errors appear if we do not sample properly. Common phenomenon: High spatial frequency components of the image appear as low spatial frequency components. Examples: Wagon wheels rolling the wrong way in movies. Checkerboards misrepresented in ray tracing. Striped shirts look funny on color television.

23. 23 Resizing images Throw away every other row and column… … regular sampling of a high-frequency function! 1/4 1/8 Adapted from Steve Seitz, U of Washington

24. 24 Resizing images Does this look nice? 1/4 (2x zoom)1/8 (4x zoom)1/2 Adapted from Steve Seitz, U of Washington

25. 25 Resizing images Regular sampling of a high-frequency image causes aliasing! Solution: smooth the image (remove high frequencies) first! Gaussian 1/4 Gaussian 1/8 Gaussian 1/2 Adapted from Steve Seitz, U of Washington

26. 26 Resizing images Gaussian 1/4 (2x zoom) Gaussian 1/8 (4x zoom) Gaussian 1/2 Adapted from Steve Seitz, U of Washington

27. 27 Gaussian pyramids Adapted from Gonzales and Woods

28. 28 Gaussian pyramids Adapted from Michael Black, Brown University

29. 29 Gaussian pyramids Adapted from Michael Black, Brown University

30. De-noising 30

31. Removing Noise 31

32. Inverting the Degradation Function 32

33. Inverting the Degradation Function 33 http://www.sweaglesw.com/cs448/ Original motion-blurred image PSF Clean image of The Beehive Nebula

34. Inverting the Degradation Function 34

35. Inverting the Degradation Function 35

36. Texture Synthesis 36 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels ?

37. Texture Synthesis 37 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

38. Texture Synthesis 38 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems

39. Texture Synthesis 39 Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems Can use this to fill holes in an image Instead of some example, use the image itself http://www.youtube.com/watch?v=NH0aEp1oDOI

40. Super-resolution 40 How do we increase image size? Interpolation? Need to fill in missing details EMBIGGEN INTERPOLATION

41. Super-resolution 41 How do we increase image size? Interpolation? Need to fill in missing details Combine multiple images at sub-pixel off-sets Synthesize details from examples Steal details from other images Copy details from same image (at different scales) http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html