1. Applications of Image Filters Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/04/10

2. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0 0000000000 0000000000 000 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Credit: S. Seitz Review: Image filtering 111 111 111

3. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 010 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

4. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 01020 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz

5. Filtering in spatial domain 01 -202 01 * =

6. Filtering in frequency domain FFT Inverse FFT =

7. Sharpening revisited What does blurring take away? original smoothed (5x5) – detail = sharpened = Let’s add it back: originaldetail + α

8. Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006 “Hybrid Images,”

9. Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006 “Hybrid Images,”

10. Today’s class How to use filters for – Matching – Denoising – Anti-aliasing Image representation with pyramids Texture – What is it? – How to represent it?

11. Matching with filters Goal: find in image

12. Matching with filters Goal: find in image Method 1: SSD Input 1- sqrt(SSD) Threshold at 0.8

13. Matching with filters Goal: find in image Method 1: SSD Method 2: Normalized cross-correlation Input Normalized X-Correlation Threshold at 0.5

14. Noise Gaussian Additive Noise + = Noisy Image

15. Gaussian noise Mathematical model: sum of many independent factors Assumption: independent, zero-mean noise Source: M. Hebert

16. Noise Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution Source: S. Seitz

17. Denoising Additive Gaussian Noise Gaussian Filter

18. Smoothing with larger standard deviations suppresses noise, but also blurs the image Reducing Gaussian noise Source: S. Lazebnik

19. Reducing salt-and-pepper noise by Gaussian smoothing 3x35x57x7

20. Alternative idea: Median filtering A median filter operates over a window by selecting the median intensity in the window Is median filtering linear? Source: K. Grauman

21. Median filter What advantage does median filtering have over Gaussian filtering? – Robustness to outliers Source: K. Grauman

22. Median filter Salt-and-pepper noise Median filtered Source: M. Hebert MATLAB: medfilt2(image, [h w])

23. Median vs. Gaussian filtering 3x35x57x7 Gaussian Median

24. Throw away every other row and column to create a 1/2 size image Subsampling by a factor of 2

25. 1D example (sinewave): Source: S. Marschner Aliasing problem

26. Source: S. Marschner 1D example (sinewave): Aliasing problem

27. Sub-sampling may be dangerous…. Characteristic errors may appear: – “Wagon wheels rolling the wrong way in movies” – “Checkerboards disintegrate in ray tracing” – “Striped shirts look funny on color television” Source: D. Forsyth Aliasing problem

28. Aliasing in video Slide by Steve Seitz

29. Source: A. Efros Aliasing in graphics

30. Sampling and aliasing

31. When sampling a signal at discrete intervals, the sampling frequency must be  2  f max ; f max = max frequency of the input signal. This will allows to reconstruct the original perfectly from the sampled version good bad vvv Nyquist-Shannon Sampling Theorem

32. Anti-aliasing Solutions: Sample more often Get rid of all frequencies that are greater than half the new sampling frequency – Will lose information – But it’s better than aliasing – Apply a smoothing filter

33. Algorithm for downsampling by factor of 2 1.Start with image(h, w) 2.Apply low-pass filter im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1)) 3.Sample every other pixel im_small = im_blur(1:2:end, 1:2:end);

34. Anti-aliasing Forsyth and Ponce 2002

35. Subsampling without pre-filtering 1/4 (2x zoom) 1/8 (4x zoom) 1/2 Slide by Steve Seitz

36. Subsampling with Gaussian pre-filtering G 1/4G 1/8Gaussian 1/2 Slide by Steve Seitz

37. Gaussian pyramid Source: Forsyth

38. Laplacian filter Gaussian unit impulse Laplacian of Gaussian Source: Lazebnik

39. Laplacian pyramid Source: Forsyth

40. Computing Gaussian/Laplacian Pyramid http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html Can we reconstruct the original from the laplacian pyramid?

41. Related idea: 2d wavelets

42. 2d Wavelets Matlab: wavemenu

43. Image representation Pixels: great for spatial resolution, poor access to frequency Fourier transform: great for frequency, not for spatial info Pyramids/wavelets: balance between spatial and frequency information

44. Major uses of image pyramids Compression Object detection – Scale search – Features Detecting stable interest points Registration – Course-to-fine

45. Texture Source: Forsyth

46. Texture and Material http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

47. Texture and Orientation http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

48. Texture and Scale http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

49. What is texture? Regular or stochastic patterns caused by bumps, grooves, and/or markings

50. How can we represent texture? Measure frequencies at various orientations and scales

51. Overcomplete representation: filter banks LM Filter Bank Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html

52. Filter banks Process image with each filter and keep responses (or squared/abs responses)

53. Representing texture Idea 1: take simple statistics (e.g., mean, std) of various absolute filter responses

54. Can you match the texture to the response? Mean abs responses Filters A B C 1 2 3

55. Representing texture by mean abs response Mean abs responses Filters

56. Representing texture Idea 2: take vectors of filter responses at each pixel and cluster them, then take histograms (more on in coming weeks)

57. Things to remember When matching using a filter, normalized cross correlation is preferred Use Gaussian or median filter for denoising Beware of aliasing – use lowpass filter to downsample Laplacian pyramids and wavelets provide spatial/frequency information Filter banks provide overcomplete representation, good for modeling/recognizing texture

58. Next class Edges and lines

59. Questions

60. Denoising “Salt and Pepper” Noise

61. Denoising 3x3 Median Filter “Salt and Pepper” Noise

62. Denoising Gaussian Additive Noise 3x3 Median Filter