1. CSCE 643 Computer Vision: Image Sampling and Filtering
Jinxiang Chai

2. Review: 1D Fourier Transform
A function f(x) can be represented as a weighted combination of phase-shifted sine waves
How to compute F(u)?
Inverse Fourier Transform
Fourier Transform

3. Review: Cosine  -1 1
If f(x) is even, so is F(u)

4. Review: Gaussian
If f(x) is gaussian, F(u) is also guassian.

5. Review: Properties Linearity: Time shift: Derivative: Integration:
Convolution:

6. Outline 2D Fourier Transform Nyquist sampling theory Antialiasing
Filtering

7. Extension to 2D
Fourier Transform:
Inverse Fourier transform:

8. Building Block for 2D Transform
Frequency:
Orientation:
Oriented wave fields

9. Building Block for 2D Transform
Frequency:
Orientation:
Oriented wave fields
Higher frequency

10. Some 2D Transforms
From Lehar

11. Some 2D Transforms
Why we have a DC component?
From Lehar

12. Some 2D Transforms
Why we have a DC component?
From Lehar

13. Some 2D Transforms
Why we have a DC component?
From Lehar

14. Some 2D Transforms
Why we have a DC component?
From Lehar

15. Some 2D Transforms Why we have a DC component?
- the sum of all pixel values
From Lehar

16. Some 2D Transforms Why we have a DC component?
- the sum of all pixel values
Oriented stripe in spatial domain = an oriented line in spatial domain
From Lehar

17. Online Java Applet

18. 2D Image Filtering
Fourier transform
Inverse transform
From Lehar

19. 2D Image Filtering Fourier transform Inverse transform Low-pass filter
From Lehar

20. 2D Image Filtering Fourier transform Inverse transform Low-pass filter
high-pass filter
From Lehar

21. 2D Image Filtering Fourier transform Inverse transform Low-pass filter
high-pass filter
band-pass filter
From Lehar

22. Aliasing
Why does this happen?

23. Aliasing
How to reduce it?

24. Sampling Analysis T 2T …
-2T -T
fs(x) x
f(x) x
Sampling

25. Sampling Analysis T 2T …
-2T -T
fs(x) x
f(x) x
Sampling
Reconstruction

26. Sampling Analysis T 2T …
-2T -T
fs(x) x
f(x) x
Sampling
Reconstruction
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?

27. Sampling Theory
How many samples are required to represent a given signal without loss of information?
What signals can be reconstructed without loss for a given sampling rate?

28. Sampling Analysis: Spatial Domain
f(x) X … … x
-2T -T T 2T x ? T 2T …
-2T -T
fs(x) x

29. Sampling Analysis: Spatial Domain
f(x) X … … x
-2T -T T 2T x ?
What happens in Frequency domain? T 2T …
-2T -T
fs(x) x

30. Fourier Transform of Dirac Comb

31. Review: Dirac Delta and its Transform
f(x) x
|F(u)| 1 u
Fourier transform and inverse Fourier transform are
qualitatively the same, so knowing one direction gives you the other

32. Fourier Transform of Dirac Comb
Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!

33. Sampling Analysis: Spatial Domain
f(x) X … … x
-2T -T T 2T x ?
What happens in Frequency domain? T 2T …
-2T -T
fs(x) x

34. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u

35. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u
How does the convolution result look like?

36. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u

37. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u

38. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u
G(0)?
G(fmax)?
G(u)?

39. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u
G(0) = F(0)
G(fmax) = F(fmax)
G(u) = F(u)

40. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax
How about …
-1/T
1/T … u
Fs(u)
-fmax
fmax u
-1/T
1/T

41. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax
How about …
-1/T
1/T … u
Fs(u)
-fmax
fmax u
-1/T
1/T

42. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u
Fs(u)
-fmax
fmax u
-1/T
1/T

43. Sampling Theory
How many samples are required to represent a given signal without loss of information?
What signals can be reconstructed without loss for a given sampling rate?

44. Sampling Analysis: Freq. Domain
F(u) u
-fmax
fmax …
-1/T
1/T … u
How can we reconstruct the original signal?
Fs(u)
-fmax
fmax u
-1/T
1/T

45. Reconstruction in Freq. Domain
Fs(u) u
-fmax
fmax
box(u)
-fmax
fmax u
-fmax
fmax u

46. Signal Reconstruction in Freq. Domain
Fs(u) u
-fmax
fmax T 2T …
-2T -T x
Fourier transform
fs(x)
f(x) x
F(u) u
-fmax
fmax
Inverse Fourier transform

47. Signal Reconstruction in Spatial Domain
sinc(x)
fs(x) x … …
-2T -T T 2T x

48. Aliasing
Why does this happen?

49. Sampling Analysis When does aliasing happen? f(x) F(u) -fmax fmax
fs(x)
Fs(u) u
-fmax
fmax x … … -T T

50. Sampling Analysis When does aliasing happen? f(x) F(u) -fmax fmax
fs(x)
Fs(u) u
-fmax
fmax x … … -T T

51. Sampling Analysis When does aliasing happen? f(x) F(u) -fmax fmax
fs(x)
Fs(u) u
-fmax
fmax x … … -T T

52. Sampling Analysis When does aliasing happen? f(x) F(u) -fmax fmax
fs(x)
Fs(u) u
-fmax
fmax x … … -T T

53. Sampling Analysis T 2T …
-2T -T
fs(x) x
f(x) x
Sampling
Reconstruction
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?

54. Sampling Analysis T 2T …
-2T -T
fs(x) x
f(x) x
Sampling
Reconstruction
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
Sampling Rate ≥ 2 * max frequency in the signal
this is known as the Nyquist Rate

55. Antialiasing

56. Antialiasing
Increase the sampling rate to above twice the highest frequency u
-fmax
fmax

57. Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax u
-fmax
fmax
Fs(u) u
-fmax
fmax

58. Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax u
-fmax
fmax
Fs(u)
-fmax
fmax u

59. Sinc Filter Is this a good filter? F(u) -fmax fmax u
They don't make sense when filtering a bitmap, which has only positive values of RGB (we might get, for instance, a negative value for the red component of some pixel!). In addition, the sync function extends to infinity (although its value quickly decreases like 1/x). It means that for each sample of the output we theoretically have to average out an infinite number of input samples.

60. Inverse Fourier transform
Sinc Filter
Is this a good filter?
F(u)
Inverse Fourier transform x
-fmax
fmax u

61. Sinc Filter Is this a good filter? F(u) Inverse Fourier transform x
-fmax
fmax u
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain

62. Sinc Filter Is this a good filter? F(u) Inverse Fourier transform x
-fmax
fmax u
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain
Good: removes all frequency components above a given bandwidth
Bad:
- an infinite support in spatial domain and hard to do in the spatial domain
- Fluctuation causes ripples
- Negative weights (not good for filtering pixles)

63. Good Pre-filtering filters
Finite support in frequency domain
- cut-off frequency
Finite support in spatial domain
- implementation in spatial domain
Positive weights
- good for filtering pixels

64. Other Filter Choices Mean filter Triangular filter Gaussian filterx
f(x)
|F(u)|
sinc(u)
Mean filter
Triangular filter
Gaussian filter u
Sin2c(u)
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains

65. Other Filter Choices Mean filter Triangular filter Gaussian filterx
f(x)
|F(u)|
sinc(u)
Mean filter
Triangular filter
Gaussian filter u
Sin2c(u)
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains

66. Gaussian Filter
Fs(u) u
-fmax
fmax

67. Inverse Fourier transform
Gaussian Filter
-0.02
0.03
0.08
0.13
0.18
Fs(u) u
-fmax
fmax
Inverse Fourier transform

68. Gaussian Filter Fs(u) -fmax fmax Inverse Fourier transform
-0.02
0.03
0.08
0.13
0.18
Fs(u) u
-fmax
fmax
Inverse Fourier transform
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain

69. Gaussian Filter Fs(u) -fmax fmax Inverse Fourier transform
-0.02
0.03
0.08
0.13
0.18
Fs(u) u
-fmax
fmax
Inverse Fourier transform
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain
Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window
Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency
Easy to implement it in the spatial domain

70. Filtering in Spatial Domain=
Filtered image
Input image
Filter function

71. Convolution
Compute the integral of the product between f and a reversed and translated version of g
t=2.5 71

72. 2D Image Convolution
Compute the sum of the product between f and a reversed and translated version of g
Correlation is the “unflipped kernel” version of convolution that
we saw earlier as weighted neighborhood mask 72

73. 2D Image Convolution
Compute the sum of the product between f and a reversed and translated version of g
Correlation is the “unflipped kernel” version of convolution that
we saw earlier as weighted neighborhood mask 73

74. 2D Image Convolution
Compute the sum of the product between f and a reversed and translated version of g
By discretization, we have
Correlation is the “unflipped kernel” version of convolution that
we saw earlier as weighted neighborhood mask 74

75. 2D Image Convolution
Compute the sum of the product between f and a reversed and translated version of g
By discretization, we have
Correlation is the “unflipped kernel” version of convolution that
we saw earlier as weighted neighborhood mask
More generally, 2D image filtering is often defined as a ``correlation” process, which is the “unflipped kernel” version of convolution. 75

76. 2D Image Filtering
More generally, 2D image filtering is defined as correlation, which is the “unflipped kernel” version of convolution.
This is a discretized version of the “unflipped kernel” version of convolution. 76

77. 2D Image Filtering
More generally, 2D image filtering is defined as correlation, which is the “unflipped kernel” version of convolution.
This is a discretized version of the “unflipped kernel” version of convolution.
This is true when filtering functions are even! 77

78. Example: box filter 1

79. Image filtering 1 90 90
ones, divide by 9 79

80. Image filtering 1 90 90 10
ones, divide by 9 80

81. Image filtering 1 90 90 10 20
ones, divide by 9 81

82. Image filtering 1 90 90 10 20 30
ones, divide by 9 82

83. Image filtering 1 90 10 20 30
ones, divide by 9 83

84. Image filtering 1 90 10 20 30 ?
ones, divide by 9 84

85. Image filtering 1 90 10 20 30 50 ?
ones, divide by 9 85

86. Image filtering 1 90 10 20 30 40 60 90 50 80 86

87. Gaussian Filtering in Spatial Domain
Weight contributions of neighboring pixels by nearness
5 x 5,  = 1 87

88. Gaussian Filtering in Spatial Domain
Weight contributions of neighboring pixels by nearness
5 x 5,  = 1
Sum = 0.987;
Need to normalize the weights so that the sum becomes 1! 88

89. Gaussian Pre-filtering
Solution: filter the image, then subsample

90. Gaussian Pre-filtering
Solution: filter the image, then subsample

91. Without Prefiltering
1/2
1/4 (2x zoom)
1/8 (4x zoom)

92. Assignment #1: Hybrid Images
A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Gaussian Filter!
Gaussian
unit impulse
Laplacian of Gaussian
Laplacian Filter! 92

93. Summary 2D Fourier Transform Nyquist sampling theory
- Sampling Rate ≥ 2 * max frequency in the signal
Antialiasing
- prefiltering
2D Image Filtering