1. Fourier Transform in Image Processing

2. Review - Image as a Function
We can think of an image as a function, f,
f: R2  R
f (x, y) gives the intensity at position (x, y)
Realistically, we expect the image only to be defined over a rectangle, with a finite range:
f: [a,b]x[c,d]  [0,1]
A color image is just three functions pasted together.
We can write this as a “vector-valued” function:

3. Image as a Function function, f, f: R2  R
f (x, y) gives the intensity at position (x, y)

4. Linear Transformations on Images

5. Transformations on Images
Define a new image g in terms of an existing image f
We can transform:
either the domain
or the range of f
Range transformation:
What kinds of operations on the image can this transformation perform?
What t transformations are good for what?

6. Operations that change the domain of image
Some operations preserve the range but change the domain of f
Here is an example that changes the domain :
What kinds of operations can this perform?
Still other operations operate on both the domain and the range of f .
In general we are interested in all these transformations
Specifically we want them to have some nice (like linear) properties

7. Linear Shift Invariant Systems (LSIS)
Linearity:
Shift invariance:

8. Example of LSIS: ideal lens
Linear Shift Invariant Systems (LSIS)
Defocused image ( g ) is a processed version of the focused image ( f )
Ideal lens is a LSIS
LSIS
Linearity: Brightness variation
Shift invariance: Scene movement
(this is not valid for lenses with non-linear distortions)

9. A Reminder of a Convolution concept

10. LSIS is Doing a Convolution
Linear Shift Invariant Systems (LSIS)
convolution is linear and shift invariant
Continuous convolution defined by integral
Discrete convolution from previous lectures defined by finite sums
We negate (mirror) the kernel of convolution h
kernel h

11. Convolution – two more Examples
Here we show two more examples of convolution of functions f and g
We use two non-idea approximation of Dirac’s Delta function as a signal model
From Eric Weinstein’s Math World

12. Convolution – Another Example-1 1 -1 1
Here we convolve signals a and b into convolved signal c 1 -2 -1 1 2

13. Convolution Kernel – Impulse Response
We ask question “What h will give us g = f ?”
Dirac Delta Function (Unit Impulse)
Sifting property:
We convolve f with Dirac Delta:
And we get this
Therefore g=f when we convolve with delta

14. Point Spread Function scene image PSF stands for Point Spread Function
Now we apply information from last slide to optical system
Optical
System
scene
image
Ideally, the optical system should be a Dirac delta function. (no distortion)
Optical
System
point source
point spread function
However, optical systems are never ideal.
Point spread function of Human Eyes
PSF stands for Point Spread Function
If the PSF of human eye is different (not a good approximation of delta) then we have eye troubles, related to this PSF type

15. Point Spread Function PSF stands for Point Spread Function
normal vision
myopia
hyperopia
Tell me what your PSF is and I will tell you what is your eye problem.
astigmatism
Images by Richmond Eye Associates

16. Properties of Convolution
Convolution operator
Commutative
Associative
This is unlike matrix multiplication and Kronecker (tensor) products
Cascade system
We used in cascading filters in previous lectures. Now we have a theoretical base
These are general properties of convolution
, not related to signal representation

17. So now we know what is convolution and why it is so important.
However signals for convolution can be represented in various ways.
So the question is, “how to represent signals?” 1D 2D
3D?

18. How to Represent Signals?
Option 1: Taylor series represents any function using polynomials.
Polynomials are not the best - unstable and not very physically meaningful. (they still have some applications)
Easier to talk about “signals” in terms of its “frequencies”
(how fast/often signals change, etc).

19. Mr. Jean Baptiste Fourier and Fourier Transform idea

20. Jean Baptiste Joseph Fourier (1768-1830)
Had crazy idea (1807):
Any periodic function can be rewritten as a weighted sum of Sines and Cosines of different frequencies.
Don’t believe it?
Neither did Lagrange, Laplace, Poisson and other big wigs
Not translated into English until 1878!
But it’s true!
called Fourier Series
Possibly the greatest tool
used in Engineering

21. A Sum of Sinusoids Our building block:
Add enough of them to get any signal f(x) you want!
How many degrees of freedom?
What does each control?
Which one encodes the coarse vs. fine structure of the signal?

22. Math Review - Complex numbers
Real numbers: 1
-5.2 
Complex numbers i
– 6.7i
( i)
We often denote in EE i by j

23. Math Review - Complex numbers
Amplitude
A = | Z | = √(a2 + b2)
Phase
 =  Z = tan-1(b/a)
Complex numbers i
– 6.7i
-5.2 ( i)
General Form
Z = a + bi
Re(Z) = a
Im(Z) = b
Real and imaginary parts

24. Math Review – Complex Numbers
Polar Coordinate
Z = a + bi
Amplitude
A = √(a2 + b2)
Phase
 = tan-1(b/a)  a b A  

25. Math Review – Complex Numbers and Cosine Waves
Cosine wave has three properties
Frequency
Amplitude
Phase
Complex number has two properties
Wave
Complex numbers to represent cosine waves at varying frequency
Frequency 1: Z1 = 5 +2i
Frequency 2: Z2 = i
Frequency 3: Z3 = 1.3 – 1.6i
Simple but great idea !!

26. Fourier Transform Idea
We want to understand the frequency w of our signal. So, let’s reparameterize the signal by w instead of x:
f(x)
F(w)
Fourier
Transform
For every w from 0 to infinity, F(w) holds the amplitude A and phase f of the corresponding sine
How can F hold both? Complex number trick!
F(w)
f(x)
Inverse Fourier
Transform
All info preserved

27. Math Review - Periodic Functions – more illustrations to our formalism

28. Math Review - Periodic Functions
If there is some a, for a function f(x), such that
f(x) = f(x + na)
then function is periodic with the period a a 2a 3a

29. Math Review - Attributes of cosine wave
Amplitude
Phase
f(x) = cos (x)
f(x) = 5 cos (x)
f(x) = 5 cos (x )

30. Math Review - Attributes of cosine wave
Amplitude
Phase
Frequency
f(x) = 5 cos (x)
f(x) = 5 cos (x )
f(x) = 5 cos (3 x )

31. Math Review - Attributes of cosine wave
f(x) = cos (x)
Amplitude, Frequency, Phase
f(x) = A cos (kx + )
This math is base of Fourier Analysis and Fourier Synthesis

32. Time and Frequency example : g(t) = sin(2 f t) + (1/3)sin(2(3f) t)
Our periodic signal is a mixture of two frequencies
Before we go to using complex numbers and formulas let us get intuition about time and frequency

33. Time and Frequency example : g(t) = sin(2 f t) + (1/3)sin(2(3f) t) =

34. Frequency Spectra example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t) = +
In time domain
In frequency domain

35. Frequency Spectra
Usually, frequency is more interesting than the phase
We will decompose it to frequencies,
we will add one more coefficient based base function in each slide

36. Frequency Spectra = + = First base function second base function
Sum of first and second base functions is already not bad approximation

37. Frequency Spectra = + =
Sum of first , second and third base functions is even better approximation

38. Frequency Spectra = + =

39. Frequency Spectra = + =

40. Frequency Spectra = + =

41. Frequency Spectra =
In time domain
In frequency domain

42. Frequency Spectra
Observe the negative values of domain that appear in spectrum

43. Fourier Transform is Just a change of basis
Important concept

44. Fourier Transform is Just a change of basis
M * f(x) = F(w)
M can be a matrix, a decision diagram, a butterfly, a system (physical or simulated) * = .

45. IFT: Just a change of basis
Inverse Fourier Transform is similarly only a change of basis
M-1 * F(w) = f(x) = * .

46. Fourier Transform is an invertible operator
Image
Fourier Transform FT
v2 will display image or its transform, Matlab examples.

47. Fourier Transform is an invertible operator
Image
Fourier Transform
{f(x,y)} = F(kx,ky)
Nx ⁄ 2
Ny ⁄ 2 x y Nx Ny
{F(kx,ky)} = f(x,y)}
f(x,y)
F(kx,ky)
Similarly as for Hadamard-Walsh or Reed-Muller Transforms we can talk about transform pairs.

48. Continuous Fourier Transform

49. Continuous Fourier Transform
f(x)
= F(s)
Euler’s Formula
Various normalizations of pairs

50. Some Conventions for Fourier Transforms
Image Domain
Forward Transform
f(x,y,z)
g(x) F
Fourier Domain
Reciprocal space
Fourier Space
K-space
Frequency Space
Spectral domain
Reverse Transform, Inverse Transform
F(kx,ky,kz)
G(s) F

51. Fourier Analysis
Decompose f(x) into a series of cosine waves that when summed reconstruct f(x)

52. Fourier Analysis in 1D. Audio signals
Amplitude Only 5 10 15
(Hz) 5 10 15
(Hz)

53. Fourier Analysis in 1D. Audio signals5 10 15
(Hz)
Your ear performs fourier analysis.

54. Fourier Analysis in 1D. Spectrum Analyzer.
iTunes performs fourier analysis.
We know it from our Hi-Fi equipment
The same can be done with standard images or radar images or any other images, in any number of dimensions

55. Summing cosine waves reconstructs the original function
Fourier Synthesis
Summing cosine waves reconstructs the original function

56. Example: Fourier Synthesis of Boxcar Function
Periodic Boxcar
Can this function be reproduced with cosine waves?

57.  k=1. One cycle per period A1·cos(2kx + 1) k=1 Ak·cos(2kx + k) 1

58. k=2. Two cycles per period
A2·cos(2kx + 2)
k=2 2 
Ak·cos(2kx + k)
k=1

59. k=3. Three cycles per period
A3·cos(2kx + 3)
k=3 3 
Ak·cos(2kx + k)
k=1

60. Fourier Synthesis. N Cycles
A3·cos(2kx + 3)
k=3  N
Ak·cos(2kx + k)
k=1

61. Example: Fourier Synthesis of a 2D Function
An image is two dimensional data.
Intensities as a function of x,y
White pixels represent the highest intensities.
Greyscale image of iris
128x128 pixels

62. Fourier Synthesis of a 2D Function
We select coefficients, multiply them by weights and add
We can create artificial images

63. Fourier Filters
Fourier Filters in 1D data (sound) change the image by changing which frequencies of cosine waves go into the image
Represented by 1D spectral profile
2D Specteral Profile is rotationally symmetrized 1D profile

64. Low and High Frequency Image Filters
Low frequency terms
Close to origin in Fourier Space
Changes with great spatial extent (like ice gradient), or particle size
High frequency terms
Closer to edge in Fourier Space
Necessary to represent edges or high-resolution features

65. Frequency-based Filters
Low-pass Filter (blurs)
Restricts data to low-frequency components
High-pass Filter (sharpens)
Restricts data to high-frequency-components
Band-pass Filter
Restrict data to a band of frequencies
Band-stop Filter
Suppress a certain band of frequencies
We will show many practical examples soon

66. Cutoff Low-pass Filter
Image is blurred
Sharp features are lost
Ringing artifacts
Filter characteristics

67. Butterworth Low-pass Filter
Flat in the pass-band
Zero in the stop-band
No ringing

68. Gaussian Low-pass Filter

69. Butterworth High-pass Filter
Note the loss of solid densities

70. How the filter looks in 2D
bandpass
unprocessed
spectra
lowpass
highpass

71. Convolution again
Our goal is now to approach the most important results of spectral processing

72. Convolution Convolution of some function f(x) with some kernel g(x)
Continuous
Discrete * =

73. Convolution in 2D result  =  = kernel image x x x x x x x x x x x x

74. Microscope Point-Spread-Function is Convolution

75. Convolution Theorem f  g = {FG} f = FG G
Convolution in image domain
Is equivalent to multiplication in fourier domain
f  g = {FG}
f = FG G
Can be derived from above because FG/G = F and f = F-1 (F)

76. Lowpass Filtering by Convolution
f  g = {FG}
Camera shake
We showed examples of this using standard convolution, but it is done in modern equipment through digitally calculated FFT and IFFT
2D gaussian is example of kernel
2D gaussian as a matrix of a kernel

77. Lens Performs Fourier Transform

78. PS = Contrast Theory F2(s) CTF2(s) Env2(s) + N2(s)
Power spectrum
PS =
F2(s) CTF2(s) Env2(s) + N2(s)
obs(x) = f(x)  psf(x)  env(x) + n(x)
Incoherant average of transform
observed image
f(x) for true particle
point-spread function
envelope function
noise

79. Gibbs Ringing 5 waves 25 waves 125 waves
Even with 125 waves we still have some small Gibbs Ringing

80. Fourier Transform Math Properties

81. Fourier Transform – more formally
Represent the signal as an infinite weighted sum
of an infinite number of sinusoids
Note:
Arbitrary function
Single Analytic Expression
Spatial Domain (x)
Frequency Domain (u)
(Frequency Spectrum F(u))
Inverse Fourier Transform (IFT)

82. Fourier Transform Also, defined as: Note:
Inverse Fourier Transform (IFT)

83. Fourier Transform Pairs (I)
Note that these are derived using
angular frequency ( )

84. Fourier Transform Pairs (I)
Note that these are derived using
angular frequency ( )

85. Fourier Transform Pairs (I)
Note that these are derived using
angular
frequency ( )

86. Fourier Transform and Convolution

87. Fourier Transform and Convolution
Let
Here we derive the most important formula of image processing and other applications of Fourier Transform
Then
Add and subtract tau
From definition
Separate domains
Convolution in spatial domain
Multiplication in frequency domain

88. MAIN THEOREM OF MODERN IMAGE PROCESSING:
Convolution in spatial domain
Multiplication in frequency domain

89. Fourier Transform and Convolution
Spatial Domain (x)
Frequency Domain (u)
So, we can find g(x) by Fourier transform FT
IFT

90. Properties of Fourier Transform

91. Properties of Fourier Transform 1
Spatial Domain (x)
Frequency Domain (u)
Linearity
Scaling
Shifting
Symmetry
Conjugation
Convolution
Differentiation
Note that these are derived using
frequency ( )

92. Properties of Fourier Transform 2
Star will be used for conjugation in complex plane
Substitute f instead of g in this formula to get the upper formula
For instance, when f is R then F is RE, IO,. etc
RO – real odd
IE – imaginary even
IO – imaginary odd
RE – Real even
CE – Complex even
CO – Complex odd
We explained even-odd properties for Walsh, we will explain again for other transforms

93. Example use of Fourier: Smoothing/Blurring
We want a smoothed function of f(x)
But we do this in spectral domain
2. Let us use a Gaussian kernel
3. Then
H(u) attenuates high frequencies in F(u) (Low-pass Filter)!

94. Our Images are Discrete so discrete Fourier applies

95. Image as a Discrete Function

96. Digital Images The scene is projected on a 2D plane,
sampled on a regular grid, and each sample is
quantized (rounded to the nearest integer)
Image as a matrix

97. Review Fourier Filters Convolution Theorem
Fourier Transform is invertible operator
Math Review
Periodic functions
Amplitude, Phase and Frequency
Complex number
Amplitude and Phase
Fourier Analysis (Forward Transform) Decomposition of periodic signal into cosine waves
Fourier Synthesis (Inverse Transform) Summation of cosine waves into multi-frequency waveform
Fourier Transforms in 1D, 2D, 3D, ND
Image Analysis
Image (real-valued)
Transform (complex-valued, amplitude plot)
Fourier Filters
Low-pass
High-pass
Band-pass
Band-stop
Convolution Theorem
Deconvolute by Division in Fourier Space
All Fourier Filters can be expressed as real-space Convolution Kernels
Lens does Fourier transforms
Diffraction Microscopy

98. Where we are in this class?
We defined continuous Fourier Transform
We defined discrete Fourier Transform DFT
DFT is calculated by matrix multiplication
It is slow
We will need factorizations, Kroneckers, recursions and butterflies to calculate it faster.
Only then FT will become a really useful tool, FFT.

99. Further Reading Wikipedia Mathworld
The Fourier Transform and its Applications. Ronald Bracewell

101. Filtering with EMAN2 LowPass Filters HighPass Filters BandPass Filters
filtered=image.process(‘filter.lowpass.guass’, {‘sigma’:0.10})
filtered=image.process(‘filter.lowpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})
filtered=image.process(‘filter.lowpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})
HighPass Filters
filtered=image.process(‘filter.highpass.guass’, {‘sigma’:0.10})
filtered=image.process(‘filter.highpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})
filtered=image.process(‘filter.highpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})
BandPass Filters
filtered=image.process(‘filter.bandpass.guass’, {‘center’:0.2,‘sigma’:0.10})
filtered=image.process(‘filter.bandpass.butterworth’, {‘low_cutoff_frequency’:0.10, ‘high_cutoff_frequency’:0.35})
filtered=image.process(‘filter.bandpass.tanh’, {‘cutoff_frequency’:0.10, ‘falloff’:0.2})

102. Sources of slides S. Narasimhan Mike Marsh
National Center for Macromolecular Imaging
Baylor College of Medicine