1. CS 691B Computational Photography
Instructor: Gianfranco Doretto
Image Filtering

2. What is an image?
We can think of an image as a function, f, from R2 to R:
f( x, y ) gives the intensity at position ( x, y )
Realistically, we expect the image only to be defined over a rectangle, with a finite range:
f: [a,b]x[c,d]  [0,1]
A color image is just three functions pasted together. We can write this as a “vector-valued” function:
As opposed to [0..255]

3. Images as functions
Render with scanalyze????

4. Image Processing image warping: change domain of image g(x) = f(h(x))
image filtering: change range of image
g(x) = h(f(x)) f x f x h
image warping: change domain of image
g(x) = f(h(x)) f x f x h

5. Image Processing image warping: change domain of image g(x) = f(h(x))
image filtering: change range of image
g(x) = h(f(x)) h
image warping: change domain of image
g(x) = f(h(x)) h

6. Point Processing
The simplest kind of range transformations are these independent of position x,y:
g = t(f)
This is called point processing.
Important: every pixel for himself – spatial information completely lost!

7. Negative

8. Contrast Stretching

9. Image Histograms
Cumulative Histograms
s = T(r)

10. Histogram Equalization

11. Image filtering
Image filtering: compute function of local neighborhood at each position
Really important!
Enhance images
Denoise, resize, increase contrast, etc.
Extract information from images
Texture, edges, distinctive points, etc.
Detect patterns
Template matching

12. 1D Smoothing examples Pixel offset coefficient original 8 impulse 2.4
Pixel offset
coefficient
original 8
impulse
2.4
0.3
filtered

13. 1D Smoothing examples Pixel offset coefficient 8 8 edge 4 4 0.3
Pixel offset
coefficient 8 8
edge 4 4
0.3
original
filtered

14. Example: Box filter 1

15. Image filtering 1 90 90
ones, divide by 9

16. Image filtering 1 90 90 10
ones, divide by 9

17. Image filtering 1 90 90 10 20
ones, divide by 9

18. Image filtering 1 90 90 10 20 30
ones, divide by 9

19. Image filtering 1 90 10 20 30
ones, divide by 9

20. Image filtering 1 90 10 20 30 ?
ones, divide by 9

21. Image filtering 1 90 10 20 30 50 ?
ones, divide by 9

22. Image filtering 1 90 10 20 30 40 60 90 50 80

23. Box Filter What does it do?
Replaces each pixel with an average of its neighborhood
Achieve smoothing effect (remove sharp features) 1

24. Smoothing with box filter

25. Cross-correlation filtering
Let’s write the box filter down as an equation. Assume the averaging window is (2k+1)x(2k+1):
We can generalize this idea by allowing different weights for different neighboring pixels:
This is called a cross-correlation operation and written:
H is called the “filter,” “kernel,” or “mask.”

26. Convolution Cross-correlation:
A convolution operation is a cross-correlation where the filter is flipped both horizontally and vertically before being applied to the image:
It is written:
Suppose h is the mean kernel (or box filter). How does convolution differ from cross-correlation?

27. Convolution is nice! Notation:
Convolution is a multiplication-like operation
commutative
associative
distributes over addition
scalars factor out
identity: unit impulse e = […, 0, 0, 1, 0, 0, …]
Conceptually no distinction between filter and signal
Usefulness of associativity
often apply several filters one after another: (((a * b1) * b2) * b3)
this is equivalent to applying one filter: a * (b1 * b2 * b3)

28. Practice with linear filters1 ?
Original

29. Practice with linear filters1
Original
Filtered
(no change)

30. Practice with linear filters1 ?
Original

31. Practice with linear filters1
Original
Shifted left
By 1 pixel

32. Practice with linear filters- 2 1 ?
(Note that filter sums to 1)
Original

33. Practice with linear filters- 2 1
Original
Sharpening filter
Accentuates differences with local average

34. 1D sharpening example 1.7 11.2 8 8 coefficient -0.25 -0.3 original
Sharpened
(differences are
accentuated; constant
areas are left untouched).

35. Sharpening

36. Other filters -1 1 -2 2
Sobel
Vertical Edge
(absolute value)

37. Other filters -1 -2 1 2 Horizontal Edge (absolute value) Q? Sobel1 2
Questions at this point?
Sobel
Horizontal Edge
(absolute value)

38. Important filter: Gaussian
Weight contributions of neighboring pixels by nearness
5 x 5

39. Smoothing with Gaussian filter

40. Smoothing with box filter

41. Gaussian filters
Remove “high-frequency” components from the image (low-pass filter)
Images become more smooth
Convolution with self is another Gaussian
So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have
Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ√2
Separable kernel
Factors into product of two 1D Gaussians
Linear vs. quadratic in mask size

42. Separability of the Gaussian filter
The 2D Gaussian can be expressed as the product of two functions, one a function of x and the other a function of y
In this case, the two functions are the (identical) 1D Gaussian

43. Separability example * 2D convolution (center location only)
The filter factors into a product of 1D filters: * =
Perform convolution along rows:
Followed by convolution along the remaining column:

44. Separability
Why is separability useful in practice?

45. Some practical matters

46. Practical matters How big should the filter be?
Values at edges should be near zero
Rule of thumb for Gaussian: set filter half-width to about 3 σ

47. Practical matters What is the size of the output?
MATLAB: filter2(h, f, shape)
shape = ‘full’: output size is sum of sizes of f and h
shape = ‘same’: output size is same as f
shape = ‘valid’: output size is difference of sizes of f and h
full
same
valid h h h h f f f h h h h h h h h

48. Practical matters What about near the edge?
the filter window falls off the edge of the image
need to extrapolate
methods:
clip filter (black)
wrap around
copy edge
reflect across edge

49. Practical matters methods (MATLAB):
clip filter (black): imfilter(f, h, 0)
wrap around: imfilter(f, h, ‘circular’)
copy edge: imfilter(f, h, ‘replicate’)
reflect across edge: imfilter(f, h, ‘symmetric’)

50. Questions?

51. Assignment 1: Hybrid Images
A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Gaussian Filter!
Gaussian
unit impulse
Laplacian of Gaussian
Laplacian Filter!
Assignment Instructions:

52. Slide Credits
This set of sides also contains contributions kindly made available by the following authors
Alexei Efros
Svetlana Lazebnik
Frédo Durand
Steve Seitz
Derek Hoiem
David Lowe
Steve Marschner