1. CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai

2. Review: 1D Fourier Transform A function f(x) can be represented as a weighted combination of phase-shifted sine waves How to compute F(u)? Inverse Fourier Transform Fourier Transform

3. Review: Box Function x u f(x) |F(u)| If f(x) is bounded, F(u) is unbounded

4. Review: Cosine 1  If f(x) is even, so is F(u)

5. Review: Gaussian If f(x) is gaussian, F(u) is also guassian.

6. Review: Properties Linearity: Time shift: Derivative: Integration: Convolution:

7. Outline 2D Fourier Transform Nyquist sampling theory Antialiasing Gaussian pyramid

8. Extension to 2D Fourier Transform: Inverse Fourier transform:

9. Building Block for 2D Transform Building block: Frequency: Orientation: Oriented wave fields

10. Building Block for 2D Transform Higher frequency Building block: Frequency: Orientation: Oriented wave fields

11. Some 2D Transforms From Lehar

12. Some 2D Transforms Why we have a DC component? From Lehar

13. Some 2D Transforms Why we have a DC component? From Lehar

14. Some 2D Transforms Why we have a DC component? From Lehar

15. Some 2D Transforms Why we have a DC component? From Lehar

16. Some 2D Transforms Why we have a DC component? - the sum of all pixel values From Lehar

17. Some 2D Transforms Why we have a DC component? - the sum of all pixel values Oriented stripe in spatial domain = an oriented line in spatial domain From Lehar

18. 2D Fourier Transform Why? - Any relationship between two slopes?

19. 2D Fourier Transform Linearity Why? - Any relationship between two slopes?

20. 2D Fourier Transform Why is the spectrum bounded? Linearity Why? - Any relationship between two slopes?

21. Online Java Applet http://www.brainflux.org/java/classes/FFT2D Applet.html

22. 2D Fourier Transform Pairs Gaussian

23. 2D Image Filtering Inverse transform Fourier transform From Lehar

24. 2D Image Filtering Inverse transform Fourier transform Low-pass filter From Lehar

25. 2D Image Filtering Inverse transform Fourier transform Low-pass filter high-pass filter From Lehar

26. 2D Image Filtering Inverse transform Fourier transform Low-pass filter high-pass filter band-pass filter From Lehar

27. Aliasing Why does this happen?

28. Aliasing How to reduce it?

29. Sampling Analysis f(x) x T2T … -2T-T … 0 f s (x) x Sampling

30. Sampling Analysis f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

31. Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

32. Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

33. Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ?

34. Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ?What happens in Frequency domain?

35. Fourier Transform of Dirac Comb T

36. Review: Dirac Delta and its Transform x 1 u f(x) |F(u)| Fourier transform and inverse Fourier transform are qualitatively the same, so knowing one direction gives you the other

37. Review: Fourier Transform Properties Linearity: Time shift: Derivative: Integration: Convolution:

38. Fourier Transform of Dirac Comb T

40. T1/T1/T Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!

41. Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ?What happens in Frequency domain?

42. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

43. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max How does the convolution result look like?

44. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

45. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

46. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max G(0)? G(f max )? G(u)?

47. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max G(0) = F(0) G(f max ) = F(f max ) G(u) = F(u)

48. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How about

49. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How about

50. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T

51. Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

52. Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How can we reconstruct the original signal?

53. Reconstruction in Freq. Domain u -f max f max F s (u) u -f max f max box(u) u -f max f max

54. Signal Reconstruction in Freq. Domain T2T … -2T-T … 0 x f s (x) f(x) x F s (u) u -f max f max F(u) u -f max f max Inverse Fourier transform Fourier transform

55. Signal Reconstruction in Spatial Domain T2T … -2T-T … 0 x x f s (x) sinc(x)

56. Aliasing Why does this happen?

57. Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

58. Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

59. Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

60. Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

61. Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

62. Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction Sampling Rate ≥ 2 * max frequency in the signal this is known as the Nyquist Rate

63. Antialiasing

64. Increase the sampling rate to above twice the highest frequency u -f max f max

65. Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce f max u -f max f max F s (u) u -f max f max

66. Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce f max u -f max f max F s (u) u -f max f max

67. Sinc Filter F(u) u -f max f max Is this a good filter?

68. Sinc Filter F(u) u -f max f max x Inverse Fourier transform Is this a good filter?

69. Sinc Filter Is this a good filter? F(u) u -f max f max x Multiplying with a box function in frequency domain Convolution with a sinc function in spatial domain Inverse Fourier transform

70. Sinc Filter Is this a good filter? F(u) u -f max f max x Multiplying with a box function in frequency domain Convolution with a sinc function in spatial domain Good: removes all frequency components above a given bandwidth Bad: - an infinite support in spatial domain and hard to do in the spatial domain - Fluctuation causes ripples - Negative weights (not good for filtering pixles) Inverse Fourier transform

71. Good Pre-filtering filters Finite support in frequency domain - cut-off frequency Finite support in spatial domain - implementation in spatial domain Positive weights - good for filtering pixels

72. Other Filter Choices Mean filter Triangular filter Gaussian filter x f(x) u |F(u)| sinc(u) Sin 2 c(u) Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

73. Other Filter Choices Mean filter Triangular filter Gaussian filter x f(x) u |F(u)| sinc(u) Sin 2 c(u) Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

74. Gaussian Filter F s (u) u -f max f max

75. Gaussian Filter F s (u) u -f max f max -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

76. Gaussian Filter F s (u) u -f max f max Multiplying with gaussian function in frequency domain Convolution with a gaussian function in spatial domain -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

77. Gaussian Filter F s (u) u -f max f max Multiplying with gaussian function in frequency domain Convolution with a gaussian function in spatial domain - Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window - Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency - Easy to implement it in the spatial domain -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

78. Filtering in Spatial Domain = Filter functionInput imageFiltered image

79. Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

80. Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

81. Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

82. Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

83. Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 Filtered_I 45 = X

84. Gaussian Pre-filtering G 1/4 G 1/8 Gaussian 1/2 Solution: filter the image, then subsample –Filter size should double for each ½ size reduction. Why?

85. Gaussian Pre-filtering G 1/4G 1/8Gaussian 1/2 Solution: filter the image, then subsample –Filter size should double for each ½ size reduction. Why?

86. Without Prefiltering 1/4 (2x zoom) 1/8 (4x zoom) 1/2

87. Known as a Gaussian Pyramid –MipMap (Williams, 1983) Image Pyramids

88. MipMap Mip (latin phase): many things in a small place Used for texture mapping Small-scale texture used for rendering distant objects

89. Gaussian Pyramid filter mask Repeat –Filter –Subsample Until minimum resolution reached –can specify desired number of levels (e.g., 3-level pyramid ) The whole pyramid is only 4/3 the size of the original image!

90. Laplacian Pyramid Laplacian Pyramid (subband images) - Created from Gaussian pyramid by subtraction - Encode fine to course structure in a different level

91. Laplacian Pyramid What happens in frequency domain?

92. Where Are They Used? Texture mapping - Mipmap Image processing - Image composition/blending - Image compression Computer vision - Image matching - Motion tracking - Optical flow estimation

93. Summary 2D Fourier Transform Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal Antialiasing - prefiltering Gaussian pyramid - Mipmap

94. Next Lecture Image Processing - Filtering noise - Bilateral filter - Feature extraction