1. Image Enhancement in the Frequency Domain Spring 2005, Jen-Chang Liu

2. Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

3. Background 1807, French math. Fourier Any function that periodically repeats itself can be expressed as the sum of of sines and/or cosines of different frequencies, each multiplied by a different coefficient (Fourier series)

4. Periodic function f(t) = f(t+T), T: period (sec.) 1/T: frequency (cycles/sec.)

5. Periodic function f Frequency Weight f 1 w 1 f 2 w 2 f 3 w 3 f 4 w 4

7. How to measure weights? Assume f 1, f 2, f 3, f 4 are known How to measure w 1, w 2, w 3, w 4 ? min Minimize squared error

8. Minimize MSE calculation min

9. Orthogonal condition f 1 and f 2 are orthogonal if f 1, f 2, f 3, f 4 are orthogonal to each other 正交

10. Minimization calculation To satisfy min We have => Recall in linear algebra: projection

11. Weight = Projection magnitude Represent input f(x) with another basis functions v Vector space (1,0) projection Functional space f f1f1

12. Summary 1 A function f can be written as sum of f 1, f 2, f 3, … If f1, f2, f3, … are orthogonal to each other Weight (magnitude)

13. Summary 1: sine, cosine bases Let f 1, f 2, f 3, … carry frequency information Let them be sines and cosines n, k:integers => They all satisfy orthogonal conditions

14. Summary 1: orthogonal

15. Fourier series For (Assume periodic outside) DC 頻率 =1 頻率 =2 頻率 =3

16. Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

17. Correlation with different phase Weight calculation 相關係數 f1f1 f 相位

18. Correlation with different phase (cont.) Weight calculation 相關係數 ? f1f1 f

19. Deal with phase: method 1 For example, expand f(t) over the cos(wt) basis function Consider different phases  0 22  Corr(  Problem: weight(w,  )

20. Deal with phase: method 2 Complex exponential as basis j 1 real With frequency w: Advantage: Derive magnitude and phase  simultaneously

21. Deal with phase 2: example Input magnitude phase

22. Fourier series with phase For (Assume periodic outside) DC 頻率 k=1 k=2 k=3 Complex weight

23. Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

24. Fourier transform Functions that are not periodic can be expressed as the integral of sines and/or cosines multiplied by a weighting functions Frequency up to infinity Perfect reconstruction Functions  --  Fourier transform Operation in frequency domain without loss of information

25. 1-D Fourier Transform Fourier transform F(u) of a continuous function f(x) is: Inverse transform: Forward Fourier transform:

26. 2-D Fourier Transform Fourier transform F(u,v) of a continuous function f(x,y) is: Inverse transform: x y u v F

27. Future development 1950, fast Fourier transform (FFT) Revolution in the signal processing Discrete Fourier transform (DFT) For digital computation

28. 1-D Discrete Fourier Transform f(x), x=0,1, …,M-1. discrete function F(u), u=0,1, …,M-1. DFT of f(x) Inverse transform: Forward discrete Fourier transform:

29. 0123 … M-1 f(x) x Assume periodic outside 0123M-1 x 頻率 =0 0123 M-1 x 頻率 =1/M ( 基頻 ) … … 0123 M-1 x 頻率 =2/M … … 0123 M-1 x 頻率 =(M-1)/M …

30. Frequency Domain 頻率域 Where is the frequency domain? j 1 Euler ’ s formula: frequency u F(u)

31. Fourier transform

33. Physical analogy Mathematical frequency splitting Fourier transform Physical device Galss prism 三稜鏡 Split light into frequency components

34. F(u) Complex quantity? Polar coordinate real imaginary m magnitude phase Power spectrum

35. Some notes about sampling in time and frequency axis Time index Frequency index Also follow reciprocal property

36. Extend to 2-D DFT from 1-D 2-D: x-axis then y-axis

37. Complex Quantities to Real Quantities Useful representation magnitude phase Power spectrum

38. Some notes about 2-D Fourier transform Frequency axis x y u v u v Fshift 0

39. DFT: example log(F)

40. Properties in the frequency domain Fourier transform works globally No direct relationship between a specific components in an image and frequencies Intuition about frequency Frequency content Rate of change of gray levels in an image

41. +45,-45 degree artifacts

42. Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

43. Filtering in the frequency domain Filter (mask) in the spatial domain Apply a filter transfer function in the frequency domain Output(u,v)=Input(u,v) H(u,v) Suppress certain frequencies while leaving others unchanged 乘法

44. Filtering in the frequency domain even dim. gray-level scaling … complex real real part

45. Basic filters Notch filter Make a hole(notch) in the frequency domain Ex. DC

46. Notch filter: result

47. Basic filters (cont.) Low-pass filter Attenuates high frequencies while passing low frequencies smoothing High-pass filter Attenuates low frequencies while passing high frequencies sharpening

48. Low-pass High-pass

49. Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

50. Periodic function f(t) = f(t+T), T: period 1/T: frequency How to measure?

51. Auto-correlation function Recall correlation function 相關係數 f1f1 f

52. Auto-correlation function (cont.) 自相關係數  Corr(  )   T 

53. Example: audio file phone = wavread('phone.wav'); wavplay(phone);

54. Example: autocorrelation corr=autocorr(phone);

55. Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

56. Function scaling and frequency P=2  P=  P=1 sin  t)

57. P=1 P=8 P=4 P=8/3? P=3

58. P=2 P=8/5 ? If t is continuous in time, then has period =8/5 However, now t ={0,1,2, …,} is discrete, sampling period 1>4/5 P=8/3? P=3

59. Half of the DFT frequencies is redundant x=0,1, …,M-1, u=0,1, …,M-1 u<2/M u |F(u)| 01M2M2 … M-1 … M2M2 …

60. For real signal Forward DFT: That is,Magnitude: |F(M-u)| = |F(u)|, u<2/M Phase: F(M-u) = - F(u)

61. DFT example for real signal f(x) |F(u)| angle(F(u))

63. DFT implementation m = 0:1:M-1; F = 1/M*f*exp(-j*2*pi*1/M*m'*m); Matlab code:

64. Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

65. 2-D Translation

66. 2-D Translation (cont.) M N M/2 N/2

67. Periodicity and conjugate symmetry Periodicity F(u,v)=F(u+M,v)=F(u,v+N)=F(u+M,v+N) Conjugate symmetry

68. M N M/2 N/2 0

69. Separability of 2-D DFT

70. Distribution and Scaling Distribution over addition Scaling linear Reciprocal property

71. Rotation invariant f(x,y) -> f(r,  )

72. Inverse Fourier transform using Forward Fourier transform forward inverse F(u) F * (u) Inverse transform (1/M)f * (x) xM f * (x) f(x)

73. Periodicity and zero padding What is the equivalent spatial-domain convolution to the frequency-domain multiplication? ?

74. correct conv. erroneous conv. 400

75. 800 Implicit periodicity of DFT zero padding Correct convolution after zero-padding

76. Ideal low-pass filter with zero-padding Spatial convolution

77. Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

78. Connection between spatial and frequency filters Convolution theorem

79. Convolution - 1-D case convolution h(m) m m h(-m) 轉 m h(x-m) x m f(m) 乘 x m f(m)h(x-m) x 積

80. Convolution – 2-D case 2d convolution Mask operation

81. mask coefficients underlying neighborhood X (product) output

82. Convolution theorem f Fourier transform F h Fourier transform H Time domain Frequency domain convolution multiplication

83. Gaussian filters (low pass) mean=0 variance Frequency domain Time domain Low-pass

84. Difference of Gaussian (high pass)

85. Why frequency domain operation? It ’ s more intuitive to design frequency filters Computation complexity Frequency domain: Fourier transform + multiplication Spatial domain: convolution Size of mask

86. Outline Introduction to the Fourier Transform and Frequency Domain Filtering in the Frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

87. Practical applications 1028x732

88. Filtering in the frequency domain

89. Smoothing frequency-domain filters Design issue G(u,v)=F(u,v) H(u,v) Remove high freq. component (details, noise, …) Ideal low-pass filter Butterworth filter Gaussian filter More smooth in the edge of cut-off frequency

90. Ideal low-pass filter Sharp cut-off frequency where D(u,v) is the distance to the center freq.

91. Ideal low-pass filter (cont.) Cut-off freq.

92. Ideal low-pass filter (con.t) ILPF can not be realized in electronic components, but can be implemented in a computer Decision of cut-off freq.? Measure the percentage of image power within the low freq. Total image power

93. ILPF: example originalFreq. 99.5 98 96.4 94.6 92

94. original  =92 D 0 =5  =94.6 D 0 =15  =96.4 D 0 =30  =98 D 0 =80  =99.5 D 0 =230 Ideal low-pass filtering

95. Ringing effect

96. Effects of ideal low-pass filtering Blurring and ringing ILPF: Freq. F -1 blurring ringing ILPF: spatial

97. Effects of ideal low-pass filtering (cont.) spatial impulse ILPF spatial

98. Butterworth low-pass filters H=0.5 when D(u,v)=D 0

99. Order of butterworth filters n=1n=2n=5n=20

100. Butterworth filters Order = 2 original D 0 =5 D 0 =15D 0 =30 D 0 =80D 0 =230

101. Gaussian low-pass filters Variance or cut-off freq. D(u,v)=D 0 H = 0.607

102. Gaussian smoothing original D 0 =5 D 0 =15D 0 =30 D 0 =80D 0 =230

103. Practical applications: 1 444x508 GLPF, D 0 =80

104. Practical applications: 2 GLPF, D 0 =100 GLPF, D 0 =80 1028x732

105. Practical applications: 3 588x600 GLPF, D 0 =30 GLPF, D 0 =10 Scan line

106. Outline Introduction to the Fourier Transform and Frequency Domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

107. Sharpening frequency-domain filters Image details corresponds to high- frequency Sharpening: high-pass filters H hp (u,v)=1-H lp (u,v) Ideal high-pass filters Butterworth high-pass filters Gaussian high-pass filters Difference filters

108. Ideal HPF Butterworth HPF Gaussian HPF

109. Spatial-domain HPF ideal Butterworth Gaussian negative

110. Ideal high-pass filters D 0 =15D 0 =30D 0 =80 ringing original

111. Butterworth high-pass filters n=2,D 0 =15D 0 =30D 0 =80

112. Gaussian high-pass filters D 0 =15D 0 =30D 0 =80

113. Laplacian frequency-domain filters Spatial-domain Laplacian Fourier transform

114. Laplacian frequency-domain filters Input f(x,y) Laplacian F(u,v) F F -(u 2 +v 2 )F(u,v) ? -(u 2 +v 2 ) The Laplacian filter in the frequency domain is H(u,v) = -(u 2 +v 2 )

115. 0 frequency spatial

116. original Laplacian Scaled Laplacian original+ Laplacian

117. High-boost filtering and High- frequency emphasis Ideal: tune the ratio of original image and the high-passed image High-boost filter High-frequency emphasis H hb (u,v)= (A-1) + H hp (u,v), A >= 1 Equivalent to (A-1)f(x,y) in the spatial-domain H hfe (u,v)= a + b H hp (u,v), a>=0, b>a

118. High-boost filtering original Laplacian A=2 A=2.7

119. High-frequency emphasis original Butterworth a=0.5 b=2.0 Histogram Equalization

120. Outline Introduction to the Fourier Transform and Frequency Domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

121. Homomorphic filtering Homomorphism: Image formation model f(x,y)=i(x,y) r(x,y) illumination: reflectance: Slow spatial variations vary abruptly, particularly at the junctions of dissimilar objects

122. Image Formation Model Illumination source scene reflection

123. Homomorphic filtering Product term Log of product z(x,y)=ln f(x,y)=ln i(x,y)+ ln r(x,y) Separation of signal source:

124. Homomorphic filtering approach ln i(x,y) ln r(x,y)

125. How to identify the illumination and reflection Illumination -> low frequency Reflection -> high frequency Radius from the origin sharpening

126. Homomophic filtering: example originalHomomorphic filtering