Fourier series, Discrete Time Fourier Transform and Characteristic functions

Fourier proposed in 1807 A periodic waveform f(t) could be broken down into an infinite series of simple sinusoids which, when added together, would construct the exact form of the original waveform. Consider the periodic function T = Period, the smallest value of T that satisfies the above equation. Fourier series

To be described by the Fourier Series the waveform f(t) must satisfy the following mathematical properties: 1. f(t) is a single-value function except at possibly a finite number of points. 2. The integral for any t f(t) has a finite number of discontinuities within the period T. 4. f(t) has a finite number of maxima and minima within the period T. Fourier series: conditions

T2T3T t f(t)f(t) DC Part Even Part Odd Part T is a period of all the above signals Let ω 0 =2π/T. Fourier series: synthesis

A Fourier Series is an accurate representation of a periodic signal (when N  ∞) and consists of the sum of sinusoids at the fundamental and harmonic frequencies. The waveform f(t) depends on the amplitude and phase of every harmonic components, and we can generate any non-sinusoidal waveform by an appropriate combination of sinusoidal functions. Fourier series: definition

Call a set of functions {ϕ k } orthogonal on an interval a < t < b if it satisfies Example 0 m =1 n = 2 -π π Orthogonal functions

Define ω 0 =2π/T. We now prove this one Orthogonal set of sinusoidal functions

0 Case 1: m ≠ n 0 Proof or orthogonality

0 Case 2: m = n

Define ω 0 =2π/T. an orthonormal set. Orthogonal set of sinusoidal functions

Decomposition

Example (Square Wave) π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π f(t)f(t) 1

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π f(t)f(t) 1 Example (Square Wave)

π 2π 3π 4π 5π-π-2π-3π-4π-5π-6π f(t)f(t) 1 Example (Square Wave) When series is truncated

Harmonics DC Part Even Part Odd Part T is a period of all the above signals

Harmonics Define, called the fundamental angular frequency. Define, called the n-th harmonic of the periodic function.

Harmonics

Amplitudes and Phase Angles harmonic amplitudephase angle

Complex Form of the Fourier Series

Complex Exponentials

Complex Form of the Fourier Series

If f(t) is real,

Complex Frequency Spectra |cn||cn| n amplitude spectrum ϕnϕn n phase spectrum

Example t f(t)f(t) A

40π80π120π-40π π-80π A/5 5ω 0 10ω 0 15ω 0 -5ω 0 -10ω 0 -15ω 0

Example 40π80π120π-40π π-80π A/10 10ω 0 20ω 0 30ω 0 -10ω 0 -20ω 0 -30ω 0

Example t f(t)f(t) A 0

Discrete-time Fourier transform Until this moment we were talking continuous periodic functions. However, probability mass function is a discrete aperiodic function. One method to find the bridge is to start with a spectral representation for periodic discrete function and let the period become infinitely long. T2T3T t f(t)f(t) 0 continuous discrete

Discrete-time Fourier transform We will take a shorter but less direct approach. Recall Fourier series A spectral representation for the continuous periodic function f(t) Consider now, a spectral representation for the sequence c n, -∞ < n < ∞ We are effectively interchanging the time and frequency domains. We want to express an arbitrary function f(t) in terms of complex exponents.

Discrete-time Fourier transform To obtain this, we make the following substitutions in This is the inverse transform discrete continuous

Discrete-time Fourier transform To obtain the forward transform, we make the same substitution in

Discrete-time Fourier transform Putting everything together Sufficient conditions of existence

Properties of the Discrete-time Fourier Transform Initial value Homework: Prove it

Characteristic functions Determining the moments E[X n ] of a RV can be difficult. An alternative method that can be easier is based on characteristic function ϕ X (ω). There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables.

Characteristic functions The function g(X) = exp(jωX) is complex but by defining E[g(X)] = E[cos(ωX) + jsin(ωX)] = E[cos(ωX)] + jE[jsin(ωX)], we can apply formula for transform RV and obtain 37 for those integers not included in S X.

Characteristic functions The definition is slightly different than the usual Fourier transform(the discrete time Fourier transform), which uses the function exp(-jωk) in its definition. As a Fourier transform it has all the usual properties. The Fourier transform of a sequence is periodic with period of 2π.

Finding moments using CF To find moments, let’s differentiate the sum “term by term” Carrying out the differentiation So that Repeated differentiation produces the formula for the nth moment as

Moments of geometric RV: example Since the PMF for a geometric RV is given by p X [k] = (1 - p) k-1 p for k = 1,2,…, we have that but since |(1-p) exp(jω)| < 1, we can use the result For z a complex number with |z| < 1 to yield the CF Note that CF is periodic with period 2π.

Moments of geometric RV: example Let’s find the mean(first moment) using CF. Let’s find the second moment using CF and then variance. Where D = exp(-jω) - (1-p). Since D| ω=0 = p, we have that

Expected value of binomial PMF Binomial theoremab By finding second moment we can find variance

Properties of characteristic functions Property 1. CF always exists since Proof Property 2. CF is periodic with period 2π. Proof: For m an integer since exp(j2πmk) = 1 for mk an integer.

Properties of characteristic functions Property 3. The PMF may be recovered from the CF. Given the CF, we may determine the PMF using Proof: Since the CF is the Fourier transform of a sequence (although its definition uses a +j instead of the usual -j), it has an inverse Fourier transform. Although any interval of length 2π may be used to perform the integration in the inverse Fourier transform, it is customary to use [-π, π]. Fourier transform

Properties of characteristic functions Property 4. Convergence of characteristic functions guarantees convergence of PMFs (Continuity theorem of probability). If we have a sequence of CFs φ n X (ω) converge to a given CF, say φ X (ω), then the corresponding sequence of PMF, say p n X [k], must converge to a given PMF say p X [k]. The theorem allows us to approximate PMFs by simpler ones if we can show that the CFs are approximately equal.

Application example of property 4 Recall the approximation of the binomial PMF by Poisson PMF under the conditions that p  0 and M  ∞ with Mp = λ fixed. To show this using the CF approach we let X b denote a binomial RV. And replacing p by λ/M we have as M  ∞.

Application example of property 4 For Poisson RV X P we have that Since φ Xb (ω)  φ Xp (ω) as M  ∞, by property 4 we must have that p Xb [k]  p Xp [k] for all k. Thus, under the stated conditions the binomial PMF becomes the Possion PMF as M  ∞.

Practice problems 1. Prove that the transformed RV has an expected value of 0 and a variance of If Y = aX + b, what is the variance of Y in terms of the variance of X? 3. Find the characteristic function for the PMF p X [k] = 1/5, for k = -2,-1,0,1,2. 4. A central moment of a discrete RV is defined as E[(X – E[X]) n ], for n positive integer. Derive a formula that relates the central moment to the usual (raw) moments. 5. Determine the variance of a binomial RV by using the properties of the CF. Assume knowledge of CF for binomial RV.

Homework 1. Apply Fourier series to the following functions on (0; 2π) a. b. c. 2. Find the second moment for a Poisson random variable by using the characteristic function exp[λ(exp(jω)-1)]. 3. A symmetric PMF satisfies the relationship p X [­-k] = p X [k] for k = …,- 1,0,1,…. Prove that all the odd order moments, E[X n ] for n odd, are zero.