Digital Image Processing Image Enhancement Frequency domain methods 12/31/2018 Duong Anh Duc - Digital Image Processing

Image Enhancement: Frequency domain methods The concept of filtering is easier to visualize in the frequency domain. Therefore, enhancement of image f(m,n) can be done in the frequency domain, based on its DFT F(u,v) . This is particularly useful, if the spatial extent of the point-spread sequence h(m,n) is large. In this case, the convolution g(m,n) = h(m,n)*f(m,n) may be computationally unattractive. PSS Enhanced Image Given Image 12/31/2018 Duong Anh Duc - Digital Image Processing

Frequency domain methods We can therefore directly design a transfer function H(u,v) and implement the enhancement in the frequency domain as follows: G(u,v) = H(u,v)*F(u,v) Transfer Function Enhanced Image Given Image 12/31/2018 Duong Anh Duc - Digital Image Processing

1-d Fourier transform of a sequence Given a 1-d sequence s[k], k={…,-1,0,1,2,…,} Fourier transform Fourier transform is periodic with 2 Inverse Fourier transform 12/31/2018 Duong Anh Duc - Digital Image Processing

1-d Fourier transform of a sequence How is the Fourier transform of a sequence s[k] related to the Fourier transform of the continuous signal Continuous-time Fourier transform 12/31/2018 Duong Anh Duc - Digital Image Processing

2-d Fourier transform of a digital image Given a 2-d matrix of image samples s[m,n], m,n  Z2 Fourier transform Fourier transform is 2-periodic both in x and y Inverse Fourier transform 12/31/2018 Duong Anh Duc - Digital Image Processing

2-d Fourier transform of a digital image How is the Fourier transform of a sequence s[m,n] related to the Fourier transform of the continuous signal Continuous-space 2D Fourier transform 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform Example f(x,y) |F(u,v)| displayed as image 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform Example |F(u,v)| displayed in 3-D 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform ExampleImage Magnitude Spectrum 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform ExampleImage Magnitude Spectrum 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform ExampleImage Magnitude Spectrum 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform ExampleImage As the size of the box increases in spatial domain, the corresponding “size” in the frequency domain decreases. 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform of “Rice” Image f(x,y) |F(u,v)| 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform of “Rice” Image F(u,v) 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform of “Camera Man” Image g(x,y) |G(u,v)| 12/31/2018 Duong Anh Duc - Digital Image Processing

Fourier Transform of “Camera Man” Image G(u,v) 12/31/2018 Duong Anh Duc - Digital Image Processing

Importance of Phase Information in Images Image formed from magnitude spectrum of Rice and phase spectrum of Camera man 12/31/2018 Duong Anh Duc - Digital Image Processing

Importance of Phase Information in Images Image formed from magnitude spectrum of Camera man and phase spectrum of Rice 12/31/2018 Duong Anh Duc - Digital Image Processing

1-D Discrete Fourier Transform (DFT) For discrete images of finite extent, the analogous Fourier transform is the DFT. We will first study this for the 1-D case, which is easier to visualize. Suppose { f(0), f(1), …, f(N-1)} is a sequence/ vector/1-D image of length N. Its N-point DFT is defined as Inverse DFT (note the normalization): 12/31/2018 Duong Anh Duc - Digital Image Processing

1-D Discrete Fourier Transform (DFT) Example: Let f(n) = {1,-1,2,3 } (Note that N=4) 12/31/2018 Duong Anh Duc - Digital Image Processing

1-D Discrete Fourier Transform (DFT) F(u) is complex even though f(n) is real. This is typical. Implementing the DFT directly requires O(N2) computations, where N is the length of the sequence. There is a much more efficient implementation of the DFT using the Fast Fourier Transform (FFT) algorithm. This is not a new transform (as the name suggests) but just an efficient algorithm to compute the DFT. 12/31/2018 Duong Anh Duc - Digital Image Processing

1-D Discrete Fourier Transform (DFT) The FFT works best when N = 2m (or is the power of some integer base/radix). The radix-2 algorithm is most commonly used. The computational complexity of the radix-2 FFT algorithm is Nlog(N) adds and ½Nlog(N) multiplies. So it is an Nlog(N) algorithm. In MATLAB, the command fft implements this algorithm (for 1-D case). See section 3.4 of the text for more details about the FFT algorithm. 12/31/2018 Duong Anh Duc - Digital Image Processing

2-D Discrete Fourier Transform (DFT) The Fourier transform is suitable for continuous-domain images, which maybe of infinite extent. For discrete images of finite extent, the analogous Fourier transform is the 2-D DFT. 12/31/2018 Duong Anh Duc - Digital Image Processing

2-D Discrete Fourier Transform (DFT) Suppose f(m,n), m = 0,1,2,…M-1, n = 0,1,2,…N-1, is a discrete NM image. Its 2-D DFT F(u,v) is defined as: Inverse DFT is defined as: 12/31/2018 Duong Anh Duc - Digital Image Processing

2-D Discrete Fourier Transform (DFT) For discrete images of finite extent, the analogous Fourier transform is the 2-D DFT. Note about normalization: The normalization by MN is different than that in text. We will use the one above since it is more widely used. The Matlab function fft2 implements the DFT as defined above. 12/31/2018 Duong Anh Duc - Digital Image Processing

2-D Discrete Fourier Transform (DFT) Most often we have M=N (square image) and in that case, we define a unitary DFT as follows: We will refer to the above as just DFT (drop unitary) for simplicity. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Convolution Example 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Convolution Example In matlab, if f and h are matrices representing two images, conv2(f, h) gives the 2D-convolution of images f and h. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Linearity (Distributivity and Scaling): This holds inboth discrete and continuous-domains. DFT of the sum of two images is the sum of their individual DFTs. DFT of a scaled image is the DFT of the original image scaled by the same factor. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Spatial scaling (only for continuous-domain): If a, b > 1, image “shrinks” and the spectrum“expands.” 12/31/2018 Duong Anh Duc - Digital Image Processing

Properties of DFT Periodicity (only for discrete case): The DFT and its inverse are periodic (in both the dimensions), with period N. F(u,v) = F(u+N,v) = F(u,v+N) = F(u+N,v+N) Similarly, is also N-periodic in m and n. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Separability (both continuous and discrete): Decomposition of 2D DFT into 1D DFTs 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Similarly, 12/31/2018 Duong Anh Duc - Digital Image Processing

Properties of DFT Convolution: In continuous-space, Fourier transform of the convolution is the product of the Four transforms. F[f(x,y)*h(x,y)] = F(u,v) H(u,v) So if g(x,y) = f(x,y)*h(x,y) is the output of an LTI transformation with PSF h(x,y) to an input image f(x,y), then G(u,v) = F(u,v)*H(u,v) 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT In other words, output spectrum G(u,v) is the product of the input spectrum F(u,v) and the transfer function H(u,v). So the FT can be used as a computational tool to simplify the convolution operation. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Correlation: In continuous-space, correlation between two images f(x,y) and h(x,y) is defined as: Therefore, 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT rff(x,y) is usually called the auto-correlation of image f(x,y) (with itself) and rfh(x,y) is called the crosscorrelation between f(x,y) and h(x,y). Roughly speaking, rfh(x,y) measures the degree of similarity between images f(x,y) and h(x,y). Large values of rfh(x,y) would indicate that the images are very similar. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT This is usually used in template matching, where h(x,y) is a template shape whose presence we want to detect in the image f(x,y). Locations where rfh(x,y) is high (peaks of the crosscorrelation function) are most likely to be the location of shape h(x,y) in image f(x,y). 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Convolution property for discrete images: Suppose f(m,n), m = 0,1,2,…M-1, n = 0,1,2,…N-1 is an NM image and h(m,n), m = 0,1,2,…K-1, n = 0,1,2,…L-1 is an NM image. Then g(m,n) = f(m,n)*h(m,n) is a (M+K-1)(N+L-1) image. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT So if we want a convolution property for discrete images --- something like g(m,n) = f(m,n)*h(m,n) we need to have G(u, v) to be of size (M+K-1)(N+L-1) (since g(m, n) has that dimension). Therefore, we should require that F(u, v) and H(u, v)also have the same dimension, i.e. (M+K-1)(N+L-1) 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT So we zero-pad the images f(m, n), h(m, n), so that they are of size (M+K-1)(N+L-1). Let fe(m,n) and he(m,n) be the zero-padded (or extended images). Take their 2D-DFTs to obtain F(u, v) and H(u, v), each of size (M+K-1)(N+L-1). Then Similar comments hold for correlation of discrete images as well. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Translation: (discrete and continuous case): Note that so f(m, n) and f(m-m0, n-n0) have the same magnitude spectrum but different phase spectrum. Similarly, 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Conjugate Symmetry: If f(m, n) is real, then F(u, v) is conjugate symmetric, i.e. Therefore, we usually display F(u-N/2,v-N/2), instead of F(u, v), since it is easier to visualize the symmetry of the spectrum in this case. This is done in Matlab using the fftshift command. 12/31/2018 Duong Anh Duc - Digital Image Processing

Properties of DFT Multiplication: (In continuous-domain) This is the dual of the convolution property. Multiplication of two images corresponds to convolving their spectra. F[f(x,y)h(x,y)] = F(u,v) H(u,v) 12/31/2018 Duong Anh Duc - Digital Image Processing

“Centered” Magnitude Spectrum f(m,n) 12/31/2018 Duong Anh Duc - Digital Image Processing

“Centered” Magnitude Spectrum |F(u,v)| |F(u-N/2,v-N/2)| 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Average value: The average pixel value in an image: Notice that (substitute u = v = 0 in the definition): 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT Differentiation: (Only in continuous-domain): Derivatives are normally used for detecting edged in an image. An edge is the boundary of an object and denotes an abrupt change in grayvalue. Hence it is a region with high value of derivative. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Properties of DFT 12/31/2018 Duong Anh Duc - Digital Image Processing

Lowpass and Hipass Filters 12/31/2018 Duong Anh Duc - Digital Image Processing

Frequency domain Filters vs Spatial Filters 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Lowpass filtering Edges and sharp transitions in grayvalues in an image contribute significantly to high-frequency content of its Fourier transform. Regions of relatively uniform grayvalues in an image contribute to low-frequency content of its Fourier transform. Hence, an image can be smoothed in the Frequency domain by attenuating the high-frequency content of its Fourier transform. This would be a lowpass filter! 12/31/2018 Duong Anh Duc - Digital Image Processing

Ideal Lowpass filtering 12/31/2018 Duong Anh Duc - Digital Image Processing

Ideal Lowpass filtering For simplicity, we will consider only those filters that are real and radially symmetric. An ideal lowpass filter with cutoff frequency r0: 12/31/2018 Duong Anh Duc - Digital Image Processing

Ideal Lowpass filtering Note that the origin (0, 0) is at the center and not the corner of the image (recall the “fftshift” operation). The abrupt transition from 1 to 0 of the transfer function H(u,v) cannot be realized in practice, using electronic components. However, it can be simulated on a computer. Ideal LPF with r0= 57 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal LPF examples Original Image Ideal LPF with r0= 57 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal LPF examples Ideal LPF with r0= 36 Ideal LPF with r0= 26 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal LPF examples Notice the severe ringing effect in the blurred images, which is a characteristic of ideal filters. It is due to the discontinuity in the filter transfer function. 12/31/2018 Duong Anh Duc - Digital Image Processing

Choice of cutoff frequency in ideal LPF The cutoff frequency r0 of the ideal LPF determines the amount of frequency components passed by the filter. Smaller the value of r0, more the number of image components eliminated by the filter. In general, the value of r0 is chosen such that most components of interest are passed through, while most components not of interest are eliminated. Usually, this is a set of conflicting requirements. We will see some details of this is image restoration A useful way to establish a set of standard cut-off frequencies is to compute circles which enclose a specified fraction of the total image power. 12/31/2018 Duong Anh Duc - Digital Image Processing

Choice of cutoff frequency in ideal LPF Suppose where is the total image power. Consider a circle of radius =r0(a) as a cutoff frequency with respect to a threshold a such that We can then fix a threshold a and obtain an appropriate cutoff frequency r0(a). 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth lowpass filter A two-dimensional Butterworth lowpass filter has transfer function: n: filter order, r0: cutoff frequency 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth lowpass filter 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth lowpass filter 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth lowpass filter Frequency response does not have a sharp transition as in the ideal LPF. This is more appropriate for image smoothing than the ideal LPF, since this not introduce ringing. 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth LPF examples Original Image LPF with r0= 18 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth LPF examples LPF with r0= 13 LPF with r0= 10 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth LPF example: False contouring Image with false contouring due to insufficient bits used for quantization Lowpass filtered version of previous image 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth LPF example: Noise filtering Original Image Noisy Image 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth LPF example: Noise filtering LPF Image 12/31/2018 Duong Anh Duc - Digital Image Processing

Gaussian Low pass filters The form of a Gaussian lowpass filter in two-dimensions is given by where is the distance from the origin in the frequency plane. The parameter s measures the spread or dispersion of the Gaussian curve. Larger the value of s, larger the cutoff frequency and milder the filtering. When s = D(u,v) , the filter is down to 0.607 of its maximum value of 1. 12/31/2018 Duong Anh Duc - Digital Image Processing

Gaussian Low pass filters 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Highpass filtering Edges and sharp transitions in grayvalues in an image contribute significantly to high-frequency content of its Fourier transform. Regions of relatively uniform grayvalues in an image contribute to low-frequency content of its Fourier transform. Hence, image sharpening in the Frequency domain can be done by attenuating the low-frequency content of its Fourier transform. This would be a highpass filter! 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Highpass filtering For simplicity, we will consider only those filters that are real and radially symmetric. An ideal highpass filter with cutoff frequency r0: 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Highpass filtering Note that the origin (0, 0) is at the center and not the corner of the image (recall the “fftshift” operation). The abrupt transition from 1 to 0 of the transfer function H(u,v) cannot be realized in practice, using electronic components. However, it can be simulated on a computer. Ideal HPF with r0= 36 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal HPF examples Original Image Ideal HPF with r0= 18 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal HPF examples Ideal HPF with r0= 36 Ideal HPF with r0= 26 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing Ideal HPF examples Notice the severe ringing effect in the output images, which is a characteristic of ideal filters. It is due to the discontinuity in the filter transfer function. 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth highpass filter A two-dimensional Butterworth highpass filter has transfer function: n: filter order, r0: cutoff frequency 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth HPF with r0 = 47 and 2 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth highpass filter Frequency response does not have a sharp transition as in the ideal HPF. This is more appropriate for image sharpening than the ideal HPF, since this not introduce ringing 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth HPF examples Original Image HPF with r0= 47 12/31/2018 Duong Anh Duc - Digital Image Processing

Butterworth HPF examples HPF with r0= 36 HPF with r0= 81 12/31/2018 Duong Anh Duc - Digital Image Processing

Gaussian High pass filters The form of a Gaussian lowpass filter in two-dimensions is given by where is the distance from the origin in the frequency plane. The parameter s measures the spread or dispersion of the Gaussian curve. Larger the value of s, larger the cutoff frequency and more severe the filtering. 12/31/2018 Duong Anh Duc - Digital Image Processing

Duong Anh Duc - Digital Image Processing 12/31/2018 Duong Anh Duc - Digital Image Processing

Spatial representations 12/31/2018 Duong Anh Duc - Digital Image Processing