1. Filtering CSE P 576 Larry Zitnick (larryz@microsoft.com)

2. 243 239240225206185188218211206216225 242239218110673134152213206208221 24324212358948213277108208 215 23521711521224323624713991209208211 23320813122221922619611474208213214 23221713111677150695652201228223 232 18218618417915912393232235 23223620115421613312981175252241240 2352382301281721386563234249241245 23723624714359781094255248247251 2342372451935533115144213255253251 2482451611281491091386547156239255 1901073910294731145817751137 233233148168203179432717128 172612160255 1092226193524 What do computers see? Images can be viewed as a 2D function.

3. 000000 000000 00200000 000000 00010000 000000 Image filtering Linear filtering = Applying a local function to the image using a sum of weighted neighboring pixels. Such as blurring: 00000 00.11 0 0 0 0 0 00000 * = 000000 022 00 0 00 0 33 110 00 0 00 0 Input image Kernel Output image

4. Image filtering 000000 000000 00200000 000000 00010000 000000 00000 00.11 0 0 0 0 0 00000 * = 000000 022 00 0 00 0 33 110 00 0 00 0 Input image f Filter h Output image g Mean filter

5. Image filtering Linear filters can have arbitrary weights. Typically they sum to 0 or 1, but not always. Weights may be positive or negative. Many filters aren’t linear (median filter.) 00000 00000 00010 00000 00000 What does this filter do?

6. Gaussian filter * = Input image f Filter h Output image g Compute empirically

7. Gaussian vs. mean filters What does real blur look like?

8. First and second derivatives * =

9. Original First Derivative xSecond Derivative x, y What are these good for?

10. Subtracting filters OriginalSecond Derivative Sharpened

11. for some Combining filters * * 00000 00000 0010 00000 00000 00000 00 00 0 4 0 00 00 00000 = = It’s also true:

12. Combining Gaussian filters More blur than either individually (but less than ) * = ?

13. Separable filters * = Compute Gaussian in horizontal direction, followed by the vertical direction. Not all filters are separable. Freeman and Adelson, 1991 Much faster!

14. Sums of rectangular regions 243 239240225206185188218211206216225 242239218110673134152213206208221 24324212358948213277108208 215 23521711521224323624713991209208211 23320813122221922619611474208213214 23221713111677150695652201228223 232 18218618417915912393232235 23223620115421613312981175252241240 2352382301281721386563234249241245 23723624714359781094255248247251 2342372451935533115144213255253251 2482451611281491091386547156239255 1901073910294731145817751137 233233148168203179432717128 172612160255 1092226193524 How do we compute the sum of the pixels in the red box? After some pre-computation, this can be done in constant time for any box. This “trick” is commonly used for computing Haar wavelets (a fundemental building block of many object recognition approaches.)

15. Sums of rectangular regions The trick is to compute an “integral image.” Every pixel is the sum of its neighbors to the upper left. Sequentially compute using:

16. Sums of rectangular regions AB CD Solution is found using: A + D – B - C

17. Spatially varying filters Some filters vary spatially. Useful for deblurring. Durand, 02

18. * * * input output Same Gaussian kernel everywhere. Slides courtesy of Sylvian Paris Constant blur

19. * * * input output The kernel shape depends on the image content. Slides courtesy of Sylvian Paris Bilateral filter Maintains edges when blurring!

20. Borders What to do about image borders: blackfixed periodic reflected

21. Sampling CSE P 576 Larry Zitnick (larryz@microsoft.com)

22. Up-sampling How do we compute the values of pixels at fractional positions?

23. Up-sampling f (x,y)f (x+1,y) f (x+1,y+1) f (x,y+1) f (x+0.8,y+0.3) f (x + a, y + b) = (1 - a)(1 - b) f (x, y) + a(1 - b) f (x + 1, y) + (1 - a)b f (x,y + 1) + ab f (x + 1, y + 1) Bilinear sampling: Bicubic sampling fits a higher order function using a larger area of support. How do we compute the values of pixels at fractional positions?

24. Up-sampling Nearest neighborBilinear Bicubic

25. Down-sampling If you do it incorrectly your images could look like this: Check out Moire patterns on the web.

26. Down-sampling Aliasing can arise when you sample a continuous signal or image – occurs when your sampling rate is not high enough to capture the amount of detail in your image – Can give you the wrong signal/image—an alias – formally, the image contains structure at different scales called “frequencies” in the Fourier domain – the sampling rate must be high enough to capture the highest frequency in the image

27. Solution Filter before sampling, i.e. blur the image first. With blur Without blur