Fourier Analysis ENGI 4559 Signal Processing for Software Engineers Dr. Richard Khoury Fall 2009

Fourier Analysis: Overview ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Analysis: Overview Continuous Fourier Transform Discrete-Time Fourier Transform Sampling Theory Decimation and Interpolation Discrete Fourier Transform Fast Fourier Transform

Fourier Jean Baptiste Joseph Fourier (1768-1830) ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Jean Baptiste Joseph Fourier (1768-1830) Orphan in pre-revolutionary France Raised by Benedictine monks Kept out of scientific corps (reserved for aristocracy) Schooled in a military academy, where he discovered his talent and love of math Career as mathematician 1789 (21 years old) he became a math celebrity for his work solving numerical equations 1794: made chair of mathematics at the École Normale Supérieure 1795: made chair of mathematics at the much more prestigious École Polytechnique

Fourier Political career “Théorie analytique de la chaleur” (1822) ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Political career Local district organizer in the French Revolution (1789) Part of Napoleon’s Egypt Expedition (1798) Made governor of Lower Egypt and secretary of the Institut d’Égypte Pushed for French-sponsored scientific and mathematical work in Egypt, to limit British cultural influence 1801: French forces surrender to the British in Egypt, Fourier returns to France “Théorie analytique de la chaleur” (1822) Showed that Newton’s Law of Cooling could be reduced to a mathematical analysis problem Required him to prove that any periodic function could be represented as a sum of sinusoids Led to the postulation of the global greenhouse effect (1824)

ENGI 4559 © Dr. Richard Khoury, 2009 Mathematical History 1753: Daniel Bernoulli postulates (but does not prove) that a vibrating string can be represented as a weighted sum of sinusoids 1788: Joseph Lagrange (Fourier’s doctoral advisor) writes “Méchanique Analytique”, a comprehensive book on analytical mechanics 1805: Carl Friedrich Gauss develops a recursive divide-and-conquer mathematical computation algorithm; forgotten 1822: Fourier proves that any periodic function can be represented as a weighted sum of sinusoids (later known as the Fourier Series) 1823: Siméon Poisson (Fourier’s doctoral student) wrote the summation formula linking Fourier Series to the Fourier Transform, generalizing it to non-periodic functions 1829: Johann Dirichlet (Fourier and Poisson’s doctoral student) developed the conditions under which Fourier’s Series doesn’t lose information from the original function 1965: Rediscovery of Fast Fourier Transform computer algorithm

Review of Complex Numbers ENGI 4559 © Dr. Richard Khoury, 2009 Review of Complex Numbers A complex number Cartesian form: Polar form: Exponential form: Conversion formulae: Other operations: Conjugate: Complex norm:

Fourier Series Any signal can be represented as a sum of sinusoids ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Any signal can be represented as a sum of sinusoids

ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Any signal can be transformed losslessly into a sum of sinusoids The original signal might be in the time or space domain The Fourier Series is in the frequency domain The sinusoids are often easier to model, analyse, and work with than the signal The transformation can be done very efficiently since the Fast Fourier Transform algorithm After processing, the resulting series of sinusoids can be losslessly transformed back into a signal

Continuous-Time Fourier Series ENGI 4559 © Dr. Richard Khoury, 2009 Continuous-Time Fourier Series Fourier series of a continuous periodic signal with period T0 Where:

Continuous-Time Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Continuous-Time Fourier Transform A continuous periodic signal is a sum of sinusoids that are integer multiples of Ω0 i.e. defined at the harmonics A continuous non-periodic signal s(t) is also a combination of sinusoids, but defined everywhere, not just at harmonics So it's an integral

Fourier Series Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Exercise Compute the Fourier Series of this signal

Fourier Series Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Exercise

Fourier Series Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Exercise

Fourier Series Exercise 14 ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Series Exercise

Fourier Transform Example ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Example Compute the Fourier Transform of a rectangular signal

Fourier Transform Example ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Example

Discrete-Time Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Discrete-Time Fourier Transform Continuous-Time Discrete-Time

Discrete-Time Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Discrete-Time Fourier Transform Continuous Fourier Series and Continuous Fourier Transform convert a periodic and non-periodic continuous-time input signal into a continuous frequency function Discrete-Time Fourier Transform converts a periodic or non-periodic discrete-time input signal into a continuous frequency function

Fourier Transform Exercise 19 ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Exercise Compute the CTFT of the signal And the DTFT of the signal

Fourier Transform Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Exercise

Fourier Transform Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Exercise

Fourier Transform Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Exercise

Fourier Transform Exercise ENGI 4559 © Dr. Richard Khoury, 2009 Fourier Transform Exercise

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Let’s study what happens to the Fourier Transform of a continuous signal when it is sampled Recall sampling from Lecture 1: where sc is the original continuous-time signal, ss is the sampled discrete-time signal, Ts is the sampling period and n is an integer

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Starting with the continuous-time signal and its Fourier transform: We perform the sampling:

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory We can break the integral in segments, compute the integral of each segment, and sum them This is getting messy… let’s offset the frequency and adjust the limits to simplify:

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Notice that the exponent contains , which is always equal to 1! We know from the last lecture that

Sampling Theory Finally, swap the summation and integral ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Finally, swap the summation and integral Compare with the discrete-time Fourier Transform we know:

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Where Ωs is the sampling frequency The frequency spectrum of the sampled function is the frequency spectrum of the original function, shifted by the sampling frequency, and scaled by the sampling period

Sampling Theory Given the frequency spectrum of an original function ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theory Given the frequency spectrum of an original function The frequency spectrum of the sampled function is the same, shifted by the sampling frequency, and scaled by the sampling period

Sampling Theorem Recall the sampling theorem: ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theorem Recall the sampling theorem: If the maximum frequency contained in an analog signal is Ωmax = B, then it can be perfectly reconstructed from samples taken at the sampling frequency Ωs = 2B (or more). Now we see why!

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theorem We can reconstruct the original signal from a sample using a low-pass filter

Sampling Theorem What if we set Ωs < 2B ? ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theorem What if we set Ωs < 2B ?

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Theorem We no longer can reconstruct the original signal because of aliasing

ENGI 4559 © Dr. Richard Khoury, 2009 Sampling Rate Sometimes we need to change the rate at which samples come into the system Downsampling Slower sampling rate Prefiltering + downsampling = decimation Upsampling Faster sampling rate Upsampling + postfiltering = interpolation

Downsampling Reduce the sampling of s(n) by a factor of M ENGI 4559 © Dr. Richard Khoury, 2009 Downsampling Reduce the sampling of s(n) by a factor of M Simply keep every Mth sample and discard the others Example: downsampling by half:

Downsampling What is the impact on the frequency spectrum? ENGI 4559 © Dr. Richard Khoury, 2009 Downsampling What is the impact on the frequency spectrum? The frequency spectrum of the downsampled function is the sum of the original scaled by the sampling rate and shifted by a factor of 

Downsampling ENGI 4559 © Dr. Richard Khoury, 2009

Downsampling Aliasing can happen in downsampling too! ENGI 4559 © Dr. Richard Khoury, 2009 Downsampling Aliasing can happen in downsampling too! Aliasing in sampling if: Aliasing in downsampling if Recall:

Downsampling Recall: aliasing occurs when neighbouring signals overlap ENGI 4559 © Dr. Richard Khoury, 2009 Downsampling Recall: aliasing occurs when neighbouring signals overlap Avoid aliasing by preventing overlap Can be done easily by putting a lowpass filter with width of /M before downsampler This is decimation s(n) f(n)

Upsampling Increase the sampling of s(n) by a factor of M ENGI 4559 © Dr. Richard Khoury, 2009 Upsampling Increase the sampling of s(n) by a factor of M Simply add M-1 zeros between each sample Example: upsampling by three:

Upsampling What is the impact on the frequency spectrum? ENGI 4559 © Dr. Richard Khoury, 2009 Upsampling What is the impact on the frequency spectrum? The frequency spectrum of the upsampled function is the original spectrum stretched by the sampling rate

ENGI 4559 © Dr. Richard Khoury, 2009 Upsampling

Upsampling Aliasing cannot happen with upsampling ENGI 4559 © Dr. Richard Khoury, 2009 Upsampling Aliasing cannot happen with upsampling It compressed the frequency components However, there are images to eliminate Notice the interval [-π, π] went from one component to three Can be done easily by putting a lowpass filter with width of /M after upsampler This is interpolation s(n) f(n)

Discrete Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Discrete Fourier Transform Continuous-Time Discrete-Time Discrete

DFT Exercise Compute the Fourier Transform of the discrete signal ENGI 4559 © Dr. Richard Khoury, 2009 DFT Exercise Compute the Fourier Transform of the discrete signal

DFT Exercise Given the previous values of S2(ω), compute s2 (n) ENGI 4559 © Dr. Richard Khoury, 2009 DFT Exercise Given the previous values of S2(ω), compute s2 (n)

Fast Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Fast Fourier Transform The Fourier Transform is a very useful technique But too slow to compute: O(N²) FFT are a family of algorithms that allow for fast computation: O(Nlog2N) Divide-and-conquer algorithm Given a signal s1 = s2s3, compute the transform of s2 and s3 separately and sum them We’ll look at two popular FFT algorithms Decimation in time FFT Decimation in frequency FFT

Fast Fourier Transform ENGI 4559 © Dr. Richard Khoury, 2009 Fast Fourier Transform For simplicity of notation, define the following: This leads to two equivalences: The DFT now is:

Decimation in Time Assumes that N is a power of two: N = 2a ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Assumes that N is a power of two: N = 2a N-point DFT computed as a set of 2-point DFTs A 2-point DFT is computed simply as:

Decimation in Time Now separate the even and odd DFTs ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Now separate the even and odd DFTs Compute two N/2-point DFTs

ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time

Decimation in Time Recall the equivalence: ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Recall the equivalence: We only need to compute half the multiplications!

ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time

ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time So we divided a DFT over N points into two DFTs over N/2 points N = 2a Each one can be recursively divided 1 DFT over N points 2 DFTs over N/2 points computed with P and Q 4 DFTs over N/4 points to compute P and Q … 2a-1 DFTs over 2 points

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example Example: 8-point DFT

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example 8-point DFT

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example 4-point DFT 4-point DFT

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example 2-point DFT 2-point DFT 2-point DFT 2-point DFT

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example

Decimation in Time Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Time Example

Decimation in Frequency ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Again, assumes that N is a power of two: N = 2a Decimation in time divided s(n) into subsets to make the DFT computation easier Decimation in frequency divides S(k) into subsets instead

Decimation in Frequency ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Divide the DFT formula into two sets of frequencies

Decimation in Frequency ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Next, separate the odd- and even-numbered frequencies

Decimation in Frequency ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency We see a new but familiar graph appear

Decimation in Frequency Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Example Example: 8-point DFT

Decimation in Frequency Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Example 8-point DFT

Decimation in Frequency Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Example 4-point DFT 4-point DFT

Decimation in Frequency Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Example 2-point DFT 2-point DFT 2-point DFT 2-point DFT

Decimation in Frequency Example ENGI 4559 © Dr. Richard Khoury, 2009 Decimation in Frequency Example

ENGI 4559 © Dr. Richard Khoury, 2009 Summary Fourier transform allows us to convert a signal from the time or space domain into a summation of sinusoids in the frequency domain that represent its frequency spectrum Analysis in the frequency domain gives us a different perspective on the signals, and a deeper understanding of sampling and related phenomena The Fourier transform are mathematical operations, with interesting properties that you should read about in the textbook

Summary Continuous-time Fourier transform ENGI 4559 © Dr. Richard Khoury, 2009 Summary Continuous-time Fourier transform Discrete-time Fourier transform Discrete Fourier transform

ENGI 4559 © Dr. Richard Khoury, 2009 Summary Two of many possible ways of computing a fast Fourier transform Decimation in time Decimation in frequency

Textbook Material Readings Recommended problems Chapter 2 ENGI 4559 © Dr. Richard Khoury, 2009 Textbook Material Readings Chapter 2 Recommended problems 2, 3, 4, 5, 6, 10, 11, 12, 13, 14, 15, 19, 20, 21, 25, 26, 29, 30, 32, 33, 34, 35, 36, 37, 39, 50, 51, 52, 60