1. CSE 185 Introduction to Computer Vision Image Filtering: Spatial Domain

2. Image filtering Spatial domain Frequency domain Reading: Chapters 3

3. 3D world from 2D image

4. Analysis from local evidence

5. Image filter Spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing, sharpening, measuring texture Frequency domain –Filtering is a way to modify the frequencies of images –Denoising, sampling, image compression Templates and image pyramids –Filtering is a way to match a template to the image –Detection, coarse-to-fine registration

6. Image filtering Image filtering: compute function of local neighborhood at each position Important tools –Enhance images Denoise, resize, increase contrast, etc. –Extract information from images Texture, edges, distinctive points, etc. –Detect patterns Template matching

7. 111 111 111 Box filter

8. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0 0000000000 0000000000 000 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 111 111 111 Image filtering

9. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 010 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 111 111 111 Image filtering

10. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 01020 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 111 111 111 Image filtering

11. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 111 111 111 Image filtering

12. 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 111 111 111 Image filtering

13. 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 ? 111 111 111 Image filtering

14. 0102030 50 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 ? 111 111 111 Image filtering

15. 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 2010 0204060 4020 0306090 6030 0 5080 906030 0 5080 906030 0203050 604020 102030 2010 00000 111 111 111 Image filtering

16. What does it do? Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) 111 111 111 Box filter

17. Smoothing with box filter

18. Practice with linear filters 000 010 000 Original ?

19. Practice with linear filters 000 010 000 Original Filtered (no change)

20. Practice with linear filters 000 100 000 Original ?

21. Practice with linear filters 000 100 000 Original Shifted left By 1 pixel

22. Practice with linear filters Original 111 111 111 000 020 000 - ? (Note that filter sums to 1)

23. Practice with linear filters Original 111 111 111 000 020 000 - Sharpening filter - Accentuates differences with local average

24. Sharpening

25. Sobel filter 01 -202 01 Vertical Edge (absolute value) Sobel

26. Sobel filter -2 000 121 Horizontal Edge (absolute value) Sobel

27. Synthesize motion blur I = imread('cameraman.tif'); subplot(2,2,1);imshow(I);title('Original Image'); H = fspecial('motion',20,45); MotionBlur = imfilter(I,H,'replicate'); subplot(2,2,2);imshow(MotionBlur);title('Mot ion Blurred Image'); H = fspecial('disk',10); blurred = imfilter(I,H,'replicate'); suplot(2,2,3);imshow(blurred);title('Blurred Image');

29. Other filters I=imread('cameraman.tif'); Hpr=fspecial('prewitt'); Hso=fspecial('sobel'); fHpr=imfilter(I,Hpr); fHso=imfilter(I,Hso); fHso2=imfilter(I,Hso'); subplot(2,2,1);imshow(I); title('Original Image') subplot(2,2,2); imshow(fHpr); title('Prewitt filter'); subplot(2,2,3); imshow(fHso); title('Sobel filter - horizontal'); subplot(2,2,4); imshow(fHso2); title('Sobel filter - vertical'); Prewitt filter Sobel filter

31. Filtering vs. convolution 2d filtering –h=filter2(g,f); or h=imfilter(f,g); 2d convolution –h=conv2(g,f); f=imageg=filter

32. Key properties of linear filters Linearity: filter(f 1 + f 2 ) = filter(f 1 ) + filter(f 2 ) Shift invariance: same behavior regardless of pixel location filter(shift(f)) = shift(filter(f)) Any linear, shift-invariant operator can be represented as a convolution

33. More properties Commutative: a * b = b * a –Conceptually no difference between filter and signal –But particular filtering implementations might break this equality Associative: a * (b * c) = (a * b) * c –Often apply several filters one after another: (((a * b 1 ) * b 2 ) * b 3 ) –This is equivalent to applying one filter: a * (b 1 * b 2 * b 3 ) Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [0, 0, 1, 0, 0], a * e = a

34. Gaussian filter Weight contributions of neighboring pixels by nearness 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5,  = 1

35. Smoothing with Gaussian filter

36. Smoothing with box filter

37. Gaussian filters Remove “high-frequency” components from the image (low-pass filter) –Images become more smooth Convolution with self is another Gaussian –So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have –Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ√2 Separable kernel –Factors into product of two 1D Gaussia ns

38. Separability of Gaussian filter

39. Separability example * * = = 2D convolution (center location only) The filter factors into a product of 1D filters: Perform convolution along rows: Followed by convolution along the remaining column:

40. Separability Why is separability useful in practice?

41. Practical matters How big should the filter be? Values at edges should be near zero Rule of thumb for Gaussian: set filter half-width to about 2.4 (or 3) σ

42. Practical matters What about near the edge? –the filter window falls off the edge of the image –need to extrapolate –methods: clip filter (black) wrap around copy edge reflect across edge

43. Practical matters –methods (MATLAB): clip filter (black): imfilter(f, g, 0) wrap around:imfilter(f, g, ‘circular’) copy edge: imfilter(f, g, ‘replicate’) reflect across edge: imfilter(f, g, ‘symmetric’) Q?

44. Practical matters What is the size of the output? MATLAB: filter2(g, f, shape) –shape = ‘full’: output size is sum of sizes of f and g –shape = ‘same’: output size is same as f –shape = ‘valid’: output size is difference of sizes of f and g f gg gg f gg g g f gg gg full samevalid

45. Median filters A Median Filter operates over a window by selecting the median intensity in the window. What advantage does a median filter have over a mean filter? Is a median filter a kind of convolution?

46. Comparison: salt and pepper noise

47. Sharpening revisited What does blurring take away ? original smoothed (5x5) – detail = sharpened = Let’s add it back: originaldetail + α

48. Take-home messages Linear filtering is sum of dot product at each position –Can smooth, sharpen, translate (among many other uses) Be aware of details for filter size, extrapolation, cropping 111 111 111

49. Practice questions 1. Write down a 3x3 filter that returns a positive value if the average value of the 4-adjacent neighbors is less than the center and a negative value otherwise 2. Write down a filter that will compute the gradient in the x-direction: gradx(y,x) = im(y,x+1)-im(y,x) for each x, y