1. ENEE631 Digital Image Processing (Spring'04) Fourier Transform and Spatial Filtering Spring ’04 Instructor: Min Wu ECE Department, Univ. of Maryland, College Park   www.ajconline.umd.edu (select ENEE631 S’04)   minwu@eng.umd.edu Based on ENEE631 Spring’04 Section 5

2. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [2] Overview Last Time: –Assignment-1 –Dithering and Halftoning –2-D signals and systems Today –2-D Fourier Transform –Image Enhancement via Spatial Filtering Additional reference on halftoning: –Special Issue on “Digital Halftoning”, IEEE Signal Processing Magazine, July 2003 (five articles) UMCP ENEE631 Slides (created by M.Wu © 2004)

3. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [3] Review of 1-D Fourier Transform UMCP ENEE624 Slides (created by M.Wu © 2003)

4. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [4] 2-D Fourier Transform FT for a 2-D continuous function u Horizontal and vertical spatial frequencies (cycles per degree of viewing angle) –Separability: u 2-D transform can be realized by a succession of 1-D transform along each spatial coordinate –Many other properties can be extended from 1-D FT u convolution in one domain  multiplication in another domain u inner product preservation (Parseval energy conservation theorem) UMCP ENEE408G Slides (created by M.Wu © 2002)

5. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [5] 2-D Fourier Transform FT for a 2-D continuous function u Horizontal and vertical spatial frequencies (cycles per degree of viewing angle) –Separability: u 2-D transform can be realized by a succession of 1-D transform along each spatial coordinate –Many other properties can be extended from 1-D FT u convolution in one domain  multiplication in another domain u inner product preservation (Parseval energy conservation theorem) UMCP ENEE408G Slides (created by M.Wu © 2002)

6. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [6] Freq. Response & Eigen functions for LSI System Eigen function of a system –Defined as an input function that is reproduced at the output with a possible change only in its amplitude Fundamental property of a Linear Shift Invariant System –Its eigen functions are complex exponentials exp[j2  (x  x + y  y )] (recall similar property for 1-D LTI system) Frequency response H(  x,  y ) for a 2-D continuous LSI system is the Fourier Transform of its impulse response u represents the (complex) amplitude of the system response for an complex exponential input at spatial frequency (  x,  y ) exp[j2  (x  x + y  y )] H(  x,  y ) exp[j2  (x  x + y  y )] H UMCP ENEE631 Slides (created by M.Wu © 2004)

7. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [7] Freq. Response & Eigen functions for LSI System Eigen function of a system –Defined as an input function that is reproduced at the output with a possible change only in its amplitude Fundamental property of a Linear Shift Invariant System –Its eigen functions are complex exponentials exp[j2  (x  x + y  y )] (recall similar property for 1-D LTI system) Frequency response H(  x,  y ) for a 2-D continuous LSI system is the Fourier Transform of its impulse response u represents the (complex) amplitude of the system response for an complex exponential input at spatial frequency (  x,  y ) exp[j2  (x  x + y  y )] H(  x,  y ) exp[j2  (x  x + y  y )] H UMCP ENEE631 Slides (created by M.Wu © 2004)

8. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [8] 2-D FT on Discrete/Sampled 2-D Function Note: (1) X(  1,  2 ) is periodic with period 2  in each argument (2) exp{j(m  1 + n  2 )} are eigen functions of 2-D discrete LSI system UMCP ENEE408G Slides (created by M.Wu © 2002)

9. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [9] 2-D FT on Discrete/Sampled 2-D Function Note: (1) X(  1,  2 ) is periodic with period 2  in each argument (2) exp{j(m  1 + n  2 )} are eigen functions of 2-D discrete LSI system UMCP ENEE408G Slides (created by M.Wu © 2002)

10. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [10] 2-D DFT (on Discrete Periodic 2-D Function) –W N = exp{ - j2  / N } complex conjugate of primitive N th root of unity –Separability: realize 2-D DFT by succession of 1-D DFTs –Circular convolution in one domain ~ multiplication in another domain UMCP ENEE408G Slides (created by M.Wu © 2002)

11. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [11] Examples of 2-D DFT Image examples are from Gonzalez-Woods 2/e online slides. UMCP ENEE408G Slides (created by M.Wu © 2002)

12. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [12] Frequency Domain View of Linear Spatial Filtering Image examples are from Gonzalez-Woods 2/e online slides Fig.4.4 & 4.7. UMCP ENEE408G Slides (created by M.Wu © 2002)

13. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [13] Optical and Modulation Transfer Function Optical Transfer Function (OTF) for a LSI imaging system –Defined as its normalized frequency response Modulation Transfer Function (MTF) –Defined as the magnitude of the OTF UMCP ENEE631 Slides (created by M.Wu © 2004)

14. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [14] MTF of the Visual System Direct MTF measurement of Human Visual System (HVS) –Use sinusoidal grating of varying contrast and spatial frequency u The contrast is specified by the ratio of maximum to minimum intensity u Observation of this grating shows the visibility thresholds at various spatial frequencies Typical MTF has band-pass shape –Suggest HVS is most sensitive to mid-freq. and least to high freq. –Some variations with viewer & viewing angle From Jain’s book Figure 3.7 UMCP ENEE631 Slides (created by M.Wu © 2004)

15. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [15] Image Enhancement via Spatial Filtering UMCP ENEE631 Slides (created by M.Wu © 2004)

16. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [16] Spatial Operations with Spatial Mask Spatial mask is 2-D finite impulse response (FIR) filter –Usually has small support 2x2, 3x3, 5x5, 7x7 –Convolve this mask with the image u v(m,n) =  u(m-k, n-l) h(k,l) … mirror w.r.t. origin, then shift & sum up –Frequency domain interpretation u multiplying DFT(image) with DFT(mask) UMCP ENEE631 Slides (created by M.Wu © 2004)

17. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [17] Spatial Averaging Masks For softening, noise smoothing, LPF before subsampling(anti-aliasing), etc. –“isotropic” (i.e. circularly symmetric / rotation invariant) filters: with response independent of directions 1/4 0 1 0101 1/9 -1 0 1 0 1 1/9 01/8 1/2 -1 0 1 0 1 0 1/8 0 0 UMCP ENEE408G/631 Slides (created by M.Wu & R.Liu © 2002/2004)

18. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [18] Frequency Response of Averaging Mask Recall: averaging mask is a FIR filter with a square support => take FT to get its frequency response: it is a low pass filter Image examples are from Gonzalez-Woods 2/e online slides. UMCP ENEE408G Slides (created by M.Wu © 2002)

19. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [19] Suppressing Noise via Spatial Averaging Image with iid noise y(m,n) = x(m,n) + N(m,n) Averaged version v(m,n) = (1/N w )  x(m-k, n-l) + (1/N w )  N(m-k, n-l) Noise variance reduced by a factor of N w –N w ~ # of pixels in the averaging window SNR improved by a factor of Nw if x(m,n) is constant in local window Window size is limited to avoid blurring UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

20. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [20] Directional Smoothing Problems with simple spatial averaging mask –Edges get blurred Improvement –Restrict smoothing to along edge direction –Avoid filtering across edges Directional smoothing –Compute spatial average along several directions –Take the result from the direction giving the smallest changes before & after filtering Other solutions –Use more explicit edge detection and adapt filtering accordingly  WW UMCP ENEE631 Slides (created by M.Wu © 2001)

21. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [21] Coping with Salt-and-Pepper Noise ( From Matlab Image Toolbox Guide Fig.10-12 & 10-13 ) UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

22. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [22] Median Filtering Salt-and-Pepper noise –Isolated white/black pixels spread randomly over the image –Spatial averaging filter may incur blurred output Median filtering –Take median value over a small window as output ~ nonlinear u Median{ x(m) + y(m) }  Median{x(m)} + Median{y(m)} –Odd window size is commonly used u 3x3, 5x5, 7x7 u 5-pixel “ + ”-shaped window –Even-sized window ~ take the average of two middle values as output UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

23. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [23] Image Sharpening Use LPF to generate HPF –Subtract a low pass filtered result from the original signal –HPF extracts edges and transitions Enhance edges I 0  I LP  I HP = I 0 – I LP  I 1 = I 0 + a*I HP UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

24. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [24] Example of Image Sharpening –v(m,n) = u(m,n) + a * g(m,n) –Often use Laplacian operator to obtain g(m,n) –Laplacian operator is a discrete form of 2 nd -order derivatives 0-¼ 1 -1 0 1 0 1 0 -¼ 0 0 UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

25. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [25] Example of Image Sharpening Original moon image is from Matlab Image Toolbox. UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

26. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [26] Other Variations of Image Sharpening High boost filter (Gonzalez-Woods 2/e pp132 & pp188) I 0  I LP  I HP = I 0 – I LP  I 1 = (b-1) I 0 + I HP –Equiv. to high pass filtering for b=1 –Amplify or suppress original image pixel values when b  2 Combine sharpening with histogram equalization Image example is from Gonzalez- Woods 2/e online slides. UMCP ENEE408G Slides (created by M.Wu © 2002)

27. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [27] Spatial LPF, HPF, & BPF HPF and BPF can be constructed from LPF Low-pass filter –Useful in noise smoothing and downsampling/upsampling High-pass filter –h HP (m,n) =  (m,n) – h LP (m,n) –Useful in edge extraction and image sharpening Band-pass filter –h BP (m,n) = h L2 (m,n) – h L1 (m,n) –Useful in edge enhancement –Also good for high-pass tasks in the presence of noise u avoid amplifying high-frequency noise UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

28. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [28] Gradient, 1 st -order Derivatives, and Edges Edge: pixel locations of abrupt luminance change For binary image –Take black pixels with immediate white neighbors as edge pixel u Detectable by XOR operations For continuous-tone image –Spatial luminance gradient vector of an edge pixel  edge u a vector consists of partial derivatives along two orthogonal directions u gradient gives the direction with highest rate of luminance changes –How to represent edge? u by intensity + direction => Edge map ~ edge intensity + directions –Detection Method-1: prepare edge examples (templates) of different intensities and directions, then find the best match –Detection Method-2: measure transitions along 2 orthogonal directions UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

29. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [29] Common Gradient Operators for Edge Detection –Move the operators across the image and take the inner products u Magnitude of gradient vector g(m,n) 2 = g x (m,n) 2 + g y (m,n) 2 u Direction of gradient vector tan –1 [ g y (m,n) / g x (m,n) ] –Gradient operator is HPF in nature ~ could amplify noise u Prewitt and Sobel operators compute horizontal and vertical differences of local sum to reduce the effect of noise 01 0 0 0 1 1 01 0 -20 1 2 01 10 0 00 0 111 -2 00 0 121 Roberts H 1 (m,n) H 2 (m,n) PrewittSobel UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

30. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [30] Examples of Edge Detectors –Quantize edge intensity to 0/1: u set a threshold u white pixel denotes strong edge RobertsPrewittSobel UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

31. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [31] Examples of Edge Detectors –Quantize edge intensity to 0/1: u set a threshold u white pixel denotes strong edge RobertsPrewittSobel UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

32. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [32] Edge Detection: Summary Measure gradient vector –Along two orthogonal directions ~ usually horizontal and vertical u g x =  L /  x u g y =  L /  y –Magnitude of gradient vector u g(m,n) 2 = g x (m,n) 2 + g y (m,n) 2 u g(m,n) = |g x (m,n) | + |g y (m,n)| (preferred in hardware implement.) –Direction of gradient vector u tan –1 [ g y (m,n) / g x (m,n) ] Characterizing edges in an image –(binary) Edge map: specify “edge point” locations with g(m,n) > thresh. –Edge intensity map: specify gradient magnitude at each pixel –Edge direction map: specify directions UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

33. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [33] Summary of Today’s Lecture 2-D Fourier Transform Image enhancement via spatial filtering –Filtering with a small FIR filter Next time –Edge detection –2-D random signals and image restoration Readings –Jain’s book Section 2.4-2.6, 3.3, 7.4 Gonzalez’s book Section 4.1-4.2, 3.5-3.7 UMCP ENEE631 Slides (created by M.Wu © 2004)

34. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [34]

35. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [35] Extra: For Future Lectures UMCP ENEE631 Slides (created by M.Wu © 2004)

36. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [36] 2-D Z-Transform The 2-D Z-transform is defined by –The space represented by the complex variable pair (z 1, z 2 ) is 4-D Unit surface –If ROC include unit surface Transfer function of discrete LSI system UMCP ENEE631 Slides (created by M.Wu © 2004)

37. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [37] Example of 2-D ZT and Region of Convergence See Lim’s book UMCP ENEE631 Slides (created by M.Wu © 2004)

38. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [38] Stability Recall for 1-D LTI system –Stability condition in bounded-input bounded-output sense (BIBO) is that the impulse response h[n] is absolutely summable u i.e. ROC of H(z) includes the unit circle –The ROC of H(z) for a causal and stable system should have all poles inside the unit circle 2-D Stable LSI system –Requires the 2-D impulse response is absolutely summable –i.e. ROC of H(z 1, z 2 ) must include the unit surface |z 1 |=1, |z 2 |=1 UMCP ENEE631 Slides (created by M.Wu © 2004)

39. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [39] Stability Recall for 1-D LTI system –Stability condition in bounded-input bounded-output sense (BIBO) is that the impulse response h[n] is absolutely summable u i.e. ROC of H(z) includes the unit circle –The ROC of H(z) for a causal and stable system should have all poles inside the unit circle 2-D Stable LSI system –Requires the 2-D impulse response is absolutely summable –i.e. ROC of H(z 1, z 2 ) must include the unit surface |z 1 |=1, |z 2 |=1 UMCP ENEE631 Slides (created by M.Wu © 2004)

40. ENEE631 Digital Image Processing (Spring'04) Lec5 – Spatial Filtering [40] Frequency Domain View of LPF / HPF UMCP ENEE631 Slides (created by M.Wu © 2004) Image example is from Gonzalez-Woods 2/e online slides.