1. University of Ioannina - Department of Computer Science Filtering in the Frequency Domain (Fundamentals) Digital Image Processing Christophoros Nikou cnikou@cs.uoi.gr

2. 2 C. Nikou – Digital Image Processing (E12) Filtering in the Frequency Domain Filter: A device or material for suppressing or minimizing waves or oscillations of certain frequencies. Frequency: The number of times that a periodic function repeats the same sequence of values during a unit variation of the independent variable. Webster’s New Collegiate Dictionary

3. 3 C. Nikou – Digital Image Processing (E12) Jean Baptiste Joseph Fourier Fourier was born in Auxerre, France in 1768. –Most famous for his work “La Théorie Analitique de la Chaleur” published in 1822. –Translated into English in 1878: “The Analytic Theory of Heat”. Nobody paid much attention when the work was first published. One of the most important mathematical theories in modern engineering.

4. 4 C. Nikou – Digital Image Processing (E12) The Big Idea = Any function that periodically repeats itself can be expressed as a sum of sines and cosines of different frequencies each multiplied by a different coefficient – a Fourier series Images taken from Gonzalez & Woods, Digital Image Processing (2002)

5. 5 C. Nikou – Digital Image Processing (E12) 1D continuous signals Images taken from Gonzalez & Woods, Digital Image Processing (2002) It may be considered both as continuous and discrete. Useful for the representation of discrete signals through sampling of continuous signals.

6. 6 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Impulse train function

7. 7 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002)

8. 8 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) The Fourier series expansion of a periodic signal f (t).

9. 9 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) The Fourier transform of a continuous signal f (t). Attention: the variable is the frequency ( Hz ) and not the radial frequency ( Ω=2πμ ) as in the Signals and Systems course.

10. 10 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002)

11. 11 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Convolution property of the FT.

12. 12 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Intermediate result −The Fourier transform of the impulse train. It is also an impulse train in the frequency domain. Impulses are equally spaced every 1/ΔΤ.

13. 13 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Sampling

14. 14 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Sampling −The spectrum of the discrete signal consists of repetitions of the spectrum of the continuous signal every 1/ΔΤ. −The Nyquist criterion should be satisfied.

15. 15 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Nyquist theorem

16. 16 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Oversampling Undersampling Aliasing appears Critical sampling with the Nyquist frequency FT of a continuous signal

17. 17 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Reconstruction (under correct sampling).

18. 18 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Reconstruction −Provided a correct sampling, the continuous signal may be perfectly reconstructed by its samples.

19. 19 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Under aliasing, the reconstruction of the continuous signal not correct.

20. 20 C. Nikou – Digital Image Processing (E12) 1D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Aliased signal

21. 21 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform Images taken from Gonzalez & Woods, Digital Image Processing (2002) The Fourier transform of a sampled (discrete) signal is a continuous function of the frequency. For a N -length discrete signal, taking N samples of its Fourier transform at frequencies: provides the discrete Fourier transform (DFT) of the signal.

22. 22 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) DFT pair of signal f [n] of length N.

23. 23 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Property –The DFT of a N -length f [n] signal is periodic with period N. –This is due to the periodicity of the complex exponential:

24. 24 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Property: sum of complex exponentials The proof is left as an exercise.

25. 25 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) DFT pair of signal f [n] of length N may be expressed in matrix-vector form.

26. 26 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.)

27. 27 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) Example for N=4

28. 28 C. Nikou – Digital Image Processing (E12) The Discrete Fourier Transform (cont.) The inverse DFT is then expressed by: This is derived by the complex exponential sum property.

29. 29 C. Nikou – Digital Image Processing (E12) Linear convolution is of length N=N 1 +N 2 -1=4

30. 30 C. Nikou – Digital Image Processing (E12) Linear convolution (cont.)

31. 31 C. Nikou – Digital Image Processing (E12) Circular shift Signal x[n] of length N. A circular shift ensures that the resulting signal will keep its length N. It is a shift modulo N denoted by Example: x[n] is of length N =8.

32. 32 C. Nikou – Digital Image Processing (E12) Circular convolution The result is of length g[n]=f [n]  h[n] Circular shift modulo N

33. 33 C. Nikou – Digital Image Processing (E12) Circular convolution (cont.) g[n]=f [n]  h[n]

34. 34 C. Nikou – Digital Image Processing (E12) DFT and convolution g[n]=f [n]  h[n] The property holds for the circular convolution. In signal processing we are interested in linear convolution. Is there a similar property for the linear convolution?

35. 35 C. Nikou – Digital Image Processing (E12) DFT and convolution (cont.) g[n]=f [n]  h[n] Let f [n] be of length N 1 and h[n] be of length N 2. Then g[n]=f [n]*h[n] is of length N 1 +N 2 -1. If the signals are zero-padded to length N=N 1 +N 2 -1 then their circular convolution will be the same as their linear convolution: Zero-padded signals

36. 36 C. Nikou – Digital Image Processing (E12) DFT and convolution (cont.) Zero-padding to length N=N 1 +N 2 -1 =4 The result is the same as the linear convolution.

37. 37 C. Nikou – Digital Image Processing (E12) DFT and convolution (cont.) Verification using DFT

38. 38 C. Nikou – Digital Image Processing (E12) DFT and convolution (cont.) Element-wise multiplication

39. 39 C. Nikou – Digital Image Processing (E12) DFT and convolution (cont.) Inverse DFT of the result The same result as their linear convolution.

40. 40 C. Nikou – Digital Image Processing (E12) 2D continuous signals Images taken from Gonzalez & Woods, Digital Image Processing (2002) Separable:

41. 41 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) The 2D impulse train is also separable:

42. 42 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) The Fourier transform of a continuous 2D signal f (x,y).

43. 43 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Example: FT of f (x,y)=δ(x) x y f (x,y)=δ(x) μ ν F(μ,ν)=δ(ν)

44. 44 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) Example: FT of f (x,y)=δ(x-y) x y f (x,y)=δ(x-y) μ ν F(μ,ν)=δ(μ+ν)

45. 45 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002)

46. 46 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) 2D continuous convolution We will examine the discrete convolution in more detail. Convolution property

47. 47 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) 2D sampling is accomplished by The FT of the sampled 2D signal consists of repetitions of the spectrum of the 1D continuous signal.

48. 48 C. Nikou – Digital Image Processing (E12) 2D continuous signals (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) The Nyquist theorem involves both the horizontal and vertical frequencies. Over-sampled Under-sampled

49. 49 C. Nikou – Digital Image Processing (E12) Aliasing Images taken from Gonzalez & Woods, Digital Image Processing (2002)

50. 50 C. Nikou – Digital Image Processing (E12) Aliasing - Moiré Patterns Effect of sampling a scene with periodic or nearly periodic components (e.g. overlapping grids, TV raster lines and stripped materials). In image processing the problem arises when scanning media prints (e.g. magazines, newspapers). The problem is more general than sampling artifacts.

51. 51 C. Nikou – Digital Image Processing (E12) Aliasing - Moiré Patterns (cont.) Superimposed grid drawings (not digitized) produce the effect of new frequencies not existing in the original components.

52. 52 C. Nikou – Digital Image Processing (E12) Aliasing - Moiré Patterns (cont.) In printing industry the problem comes when scanning photographs from the superposition of: The sampling lattice (usually horizontal and vertical). Dot patterns on the newspaper image.

53. 53 C. Nikou – Digital Image Processing (E12) Aliasing - Moiré Patterns (cont.)

54. 54 C. Nikou – Digital Image Processing (E12) Aliasing - Moiré Patterns (cont.) The printing industry uses halftoning to cope with the problem. The dot size is inversely proportional to image intensity.

55. 55 C. Nikou – Digital Image Processing (E12) 2D discrete convolution Images taken from Gonzalez & Woods, Digital Image Processing (2002) m n 32 f [m,n] m n 1 1 1 h [m,n] Take the symmetric of one of the signals with respect to the origin. Shift it and compute the sum at every position [m,n].

56. 56 C. Nikou – Digital Image Processing (E12) 2D discrete convolution (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) n m 32 f [m,n] n m 1 1 1 h [m,n] l k h [1-k,1-l] 1 11 32 l k h [-k,-l] 1 11 32 g [0,0]=0 g [1,1]=0

57. 57 C. Nikou – Digital Image Processing (E12) 2D discrete convolution (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) n m 32 f [m,n] n m 1 1 1 h [m,n] l k h [3-k,2-l] 1 11 3 2 l k h [2-k,2-l] 1 11 3 2 g [2,2]=3 g [3,2]=-1

58. 58 C. Nikou – Digital Image Processing (E12) 2D discrete convolution (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) n m 32 f [m,n] n m 1 1 1 h [m,n] n m g[m,n] 3 5 3 2 -3 M 1 +M 2 -1=3 N 1 +N 2 -1=2

59. 59 C. Nikou – Digital Image Processing (E12) The 2D DFT Images taken from Gonzalez & Woods, Digital Image Processing (2002) 2D DFT pair of image f [m,n] of size MxN.

60. 60 C. Nikou – Digital Image Processing (E12) The 2D DFT (cont.) Images taken from Gonzalez & Woods, Digital Image Processing (2002) All of the properties of 1D DFT hold. Particularly: –Let f [m,n] be of size M 1 xN 1 and h[m,n] of size M 2 xN 2. –If the signals are zero-padded to size ( M 1 +M 2 - 1)x(N 1 +N 2 -1) then their circular convolution will be the same as their linear convolution and: