1. Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am

2. Image Resampling and Pyramids Lecture #8

3. Image Scaling This image is too big to fit on the screen. How can we reduce it? How to generate a half- sized version?

4. Image Sub-Sampling Throw away every other row and column to create a 1/2 size image - called image sub-sampling 1/4 1/8

5. Image Sub-Sampling 1/4 (2x zoom) 1/8 (4x zoom) 1/2

6. Good and Bad Sampling Good sampling: Sample often or, Sample wisely Bad sampling: Aliasing!

7. Aliasing

8. Alias: n., an assumed name Picket fence receding into the distance will produce aliasing… Input signal: x = 0:.05:5; imagesc(sin((2.^x).*x)) Matlab output: WHY? Alias! Not enough samples

9. Really bad in video Wagon-wheel effect

10. Stroboscopic effect

11. Sampling Theorem Continuous signal: Shah function (Impulse train): Sampled function:

12. Sampling Theorem Sampled function: Only if Sampling frequency

13. Nyquist Frequency If Aliasing When can we recover from ? Only if (Nyquist Frequency) We can use Thenand Sampling frequency must be greater than

14. Sub-Sampling with Gaussian Pre-Filtering G 1/4 G 1/8 Gaussian 1/2 Solution: filter the image, then subsample –Filter size should double for each ½ size reduction. Why?

15. G 1/4G 1/8Gaussian 1/2 Sub-Sampling with Gaussian Pre-Filtering

16. Compare with... 1/4 (2x zoom) 1/8 (4x zoom) 1/2

17. Aliasing

18. Canon D60 (w/ anti-alias filter)Sigma SD9 (w/o anti-alias filter) From Rick Matthews website, images by Dave Etchells

19. Image Resampling What about arbitrary scale reduction? How can we increase the size of the image? Recall how a digital image is formed –It is a discrete point-sampling of a continuous function –If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale 12345

20. Image Resampling So what to do if we don’t know 123452.5 1 –Answer: guess an approximation –Can be done in a principled way: filtering

21. Resampling filters What does the 2D version of this hat function look like? Better filters give better resampled images –Bicubic is common choice performs linear interpolation performs bilinear interpolation

22. Bilinear interpolation A common method for resampling images

23. Image Rotation

24. Multi-Resolution Image Representation Fourier domain tells us “what” (frequencies, sharpness, texture properties), but not “where”. Spatial domain tells us “where” (pixel location) but not “what”. We want a image representation that gives a local description of image “events” – what is happening where. Naturally, think about representing images across varying scales.

25. Figure from David Forsyth

26. Multi-resolution Image Pyramids High resolution Low resolution

27. Space Required for Pyramids

29. Constructing a pyramid by taking every second pixel leads to layers that badly misrepresent the top layer

30. Even worse for synthetic images

31. Decimation

32. Expansion

33. Interpolation Results

34. The Gaussian Pyramid Smooth with Gaussians because –a Gaussian*Gaussian=another Gaussian Synthesis –smooth and downsample Gaussians are low pass filters, so repetition is redundant Kernel width doubles with each level

35. Smoothing as low-pass filtering High frequencies lead to trouble with sampling. Suppress high frequencies before sampling ! –truncate high frequencies in FT –or convolve with a low-pass filter Common solution: use a Gaussian –multiplying FT by Gaussian is equivalent to convolving image with Gaussian.

36. The Gaussian Pyramid High resolution Low resolution blur down-sample blur down-sample

39. Pyramids at Same Resolution

40. Difference of Gaussians (DoG) Laplacian of Gaussian can be approximated by the difference between two different Gaussians

41. Gaussian – Image filter Gaussian delta function Fourier Transform

42. expand Gaussian Pyramid Laplacian Pyramid The Laplacian Pyramid - = - = - =

44. Applications of Image Pyramids Coarse-to-Fine strategies for computational efficiency. Search for correspondence –look at coarse scales, then refine with finer scales Edge tracking –a “good” edge at a fine scale has parents at a coarser scale Control of detail and computational cost in matching –e.g. finding stripes –very important in texture representation Image Blending and Mosaicing Data compression (laplacian pyramid)

45. Fast Template Matching

52. Blending Apples and Oranges

53. Image Blending and Mosaicing

54. Pyramid blending of Regions

55. Horror Photo © prof. dmartin

56. Image Fusion Multi-scale Transform (MST) = Obtain Pyramid from Image Inverse Multi-scale Transform (IMST) = Obtain Image from Pyramid

57. Multi-Sensor Fusion

58. Image Compression

59. Next Class Edge Detection