1. Linear Filters T-11 Computer Vision University of Houston
Christophoros Nikou
Images and slides from:
James Hayes, Brown University, Computer Vision course
Svetlana Lazebnik, University of North Carolina at Chapel Hill, Computer Vision course
D. Forsyth and J. Ponce. Computer Vision: A Modern Approach, Prentice Hall, 2011.
R. Gonzalez and R. Woods. Digital Image Processing, Prentice Hall, 2008.

2. Linear Filtering
Highlight the characteristic appearance of small groups of pixels (zebra strips, Dalmatian dog spots).
Reduce the effect of noise.
Find edges and other patterns (e.g. corner features).

3. Linear Filtering

4. Motivation: Noise Reduction
Given a camera and a still scene, how can you reduce noise?
Take lots of images and average them!
What’s the next best thing?
Source: S. Seitz

5. Moving Average
Let’s replace each pixel with a weighted average of its neighborhood.
The weights are called the filter kernel.
What are the weights for a 3x3 moving average? 1
“box filter”
Source: D. Lowe

6. Convolution 1 90 90
Credit: S. Seitz
ones, divide by 9

7. Image filtering 1 90 90 10
Credit: S. Seitz
ones, divide by 9

8. Image filtering 1 90 90 10 20
Credit: S. Seitz
ones, divide by 9

9. Image filtering 1 90 90 10 20 30
Credit: S. Seitz
ones, divide by 9

10. Image filtering 1 90 10 20 30
Credit: S. Seitz
ones, divide by 9

11. Image filtering 1 90 10 20 30 ?
Credit: S. Seitz
ones, divide by 9

12. Image filtering 1 90 10 20 30 50 ?
Credit: S. Seitz
ones, divide by 9

13. Image filtering 1 90 10 20 30 40 60 90 50 80
Credit: S. Seitz

14. Key Properties Linearity: filter(f1 + f2 ) = filter(f1) + filter(f2)
Shift invariance: same behavior regardless of pixel location: filter(shift(f)) = shift(filter(f)).
Theoretical result: any linear shift-invariant operator can be represented as a convolution.
MATLAB: conv2 vs. filter2 (also imfilter)

15. Key Properties (cont.) Commutative: a * b = b * a
Conceptually no difference between filter and signal
Associative: a * (b * c) = (a * b) * c
Often apply several filters one after another:
(((a * b1) * b2) * b3).
This is equivalent to applying one filter: a * (b1 * b2 * b3).
Distributes over addition: a * (b + c) = (a * b) + (a * c)
Scalars factor out: ka * b = a * kb = k (a * b)
Identity: unit impulse e = […, 0, 0, 1, 0, 0, …], a * e = a

16. Implementation Details
full
same
valid g g g g f f g f g g g g g g g

17. Implementation Details (cont.)
What about image borders?
the filter window falls off the edge of the image
need to extrapolate
methods:
clip filter (black)
wrap around
copy edge
reflect across edge
Source: S. Marschner

18. Implementation Details (cont.)
What about near the edge?
the filter window falls off the edge of the image.
need to extrapolate.
methods (MATLAB):
clip filter (black): imfilter(f, g, 0)
wrap around: imfilter(f, g, ‘circular’)
copy edge: imfilter(f, g, ‘replicate’)
reflect across edge: imfilter(f, g, ‘symmetric’)
Source: S. Marschner

19. Practice with Linear Filters1 ?
Original
Source: D. Lowe

20. Practice with Linear Filters1
Original
Filtered
(no change)
Source: D. Lowe

21. Practice with Linear Filters1 ?
Original
Attention to the coordinates flip
Source: D. Lowe

22. Practice with Linear Filters1
Original
Shifted left
By 1 pixel
Source: D. Lowe

23. Practice with Linear Filters? 1
Original
Source: D. Lowe

24. Practice with Linear Filters1
Original
Blur (with a
box filter)
Source: D. Lowe

25. Practice with Linear Filters- 2 1 ?
(Note that filter sums to 1)
Original
Source: D. Lowe

26. Practice with Linear Filters- 2 1
Original
Sharpening filter
Accentuates differences with local average.
Source: D. Lowe

27. Sharpening
Source: D. Lowe

28. Smoothing with Box Filter Revisited
It doesn’t compare well with a defocused lens.
A single point of light viewed in a defocused lens looks like a fuzzy blob.
This is the point spread function (PSF) of acquisition systems.
Averaging gives a square.
This is a drawback yielding a block effect in the resulting image.
Source: D. Forsyth

29. Smoothing with Box Filter Revisited
Source: D. Forsyth

30. Smoothing with Box Filter Revisited
Better idea: to eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center.
“fuzzy blob”

31. Gaussian Kernel
5 x 5,  = 1
The constant factor is for normalization to 1 (it can be ignored, as we should re-normalize weights to sum to 1 in any case).
Source: C. Rasmussen

32. Selecting the Kernel Width
Gaussian filters have infinite support, but discrete filters use finite kernels.
Source: K. Grauman

33. Selecting the Kernel Width
Rule of thumb: set filter half-width at about 3σ.
Little effect if the standard deviation of the Gaussian is small
For larger standard deviation the average is biased to a consensus of the neighbors.
The noise disappears at the cost of some blurring.
Large standard deviations cause image detail to disappear along with the noise.

34. Example: Smoothing with a Gaussian

35. Mean vs Gaussian Filtering

36. Gaussian Filters
Remove “high-frequency” components from the image (low-pass filter).
Convolution with itself is also Gaussian
Convolving twice with a Gaussian kernel of width σ is the same as convolving once with a kernel of width σ√2.
Separable kernel
Factorization into a product of two 1D Gaussians.
Source: K. Grauman

37. Separability of the Gaussian Filter
The computational complexity becomes linear from square (per pixel).
Source: D. Lowe

38. Noise
Salt and pepper : contains random occurrences of black and white pixels.
Impulse: contains random occurrences of white pixels.
Gaussian : variations in intensity drawn from a Gaussian normal distribution
Source: S. Seitz

39. Gaussian Noise Mathematical model: sum of many independent factors.
Good model for small standard deviations.
Assumption: independent, i.i.d, zero-mean.
Source: M. Hebert

40. Reducing Gaussian Noise
The effects of smoothing
Each row shows smoothing
with Gaussians of different
width; each column shows
different realizations of
an image of Gaussian noise.
Notice the overloading notation: Gaussian noise and Gaussian filter which both have standard deviations.
Gaussian filters are good at suppressing Gaussian noise.

41. Reducing Salt-and-Pepper Noise
3x3
5x5
7x7
What’s wrong with the results?

42. Alternative Idea: Median Filtering
A median filter operates over a window by selecting the median intensity in the window.
Is median filtering linear?
Source: K. Grauman

43. Median Filter Advantage over Gaussian filtering.
Robustness to outliers.
Source: K. Grauman

44. Salt-and-pepper noise
Median Filter (cont.)
Salt-and-pepper noise
Median filtered
Source: M. Hebert

45. Median vs Gaussian Filtering
3x3
5x5
7x7
Gaussian
Median

46. What does blurring take away?
Sharpening Revisited
What does blurring take away?
original
smoothed (5x5)
detail = –
Let’s add it back:
original
detail
+ α
sharpened =

47. unit impulse (identity)
Unsharp Mask Filter
Gaussian
unit impulse
Laplacian of Gaussian
image
blurred image
unit impulse (identity)

48. Other non-linear filters
Weighted median (pixels further from center count less)
Clipped mean (ignoring few brightest and darkest pixels)
Bilateral filtering (weight by spatial distance and intensity difference)
Bilateral filtering

49. Anisotropic Scaling
The symmetric Gaussian filter tends to blur out edges and consequently it merges structures near one another.
This suggests that smoothing should be differently at edge points (we will discuss edge detection in the next lecture in detail).
The idea is to estimate the gradient and orientation of the gradient and smooth in an oriented manner at large gradient amplitudes (edge-preserving smoothing).
P. Perona and J. Malik [ICCV 1990] noticed that this is equivalent to apply the diffusion equation to the image.

50. Anisotropic Scaling (cont.)
The symmetric Gaussian filter tends to blur out edges and consequently it merges structures near one another.
With the initial condition:
Applying this equation iteratively smooths the image by diffusing the intensity of a pixel to its neighboring pixels and edges are not preserved.

51. Anisotropic Scaling (cont.)
To introduce selective smoothing, the diffusion equation is modified:
If c(x,y)=0, there is no smoothing. If c(x,y)=1, we have the diffusion equation. We must construct an appropriate c(x,y) that has a value of 0 at edges and a value of 1 at uniform regions.

52. Anisotropic Scaling (cont.)
For example:
Smoothing is performed by discretizing the diffusion equation over scale. This is equivalent to considering that scale is a parameter associated with time (σ ~ t):

53. Anisotropic Scaling (cont.)
Substituting this approximation to the anisotropic diffusion equation yields:
The time step parameter dt and the convergence criterion should be defined by the user.

54. Anisotropic Scaling (cont.)

55. Anisotropic Scaling (cont.)

56. Differentiation and Convolution
We could approximate this as
It is obviously a convolution; however, it is not a very good way to do things, as we shall see.
Recall
This is linear and shift invariant, so it must be the result of a convolution.

57. Differentiation and Convolution (cont.)
The kernel may be interpreted as a template:
A large positive response to a configuration which is positive on one side and negative on the other side.
A large negative response to the mirror image. -1 1
Kernel

58. Finite Differences
Which derivative is this? Is it horizontal or vertical?
In the derivative image, mid-gray is zero, dark gray is negative and light gray is positive.

59. Finite Differences (cont.)
Finite differences respond to noise
Noise free
σ=0.03
σ=0.09
Some sort of smoothing is needed.