1. Extends Euclidean space algebra to higher dimensions
<0,0,1>
Linear Algebra
Extends Euclidean space algebra to higher dimensions
Definitions
Dot product of vectors A=<a1,a2,…an> and B=<b1,b2, …, bn> A  B = a1*b1 + a2*b2 + … + an * bn
The length of a vector <a1, a2, … , an> is (a12 + a22 + … + an2)½
A set of vectors are linearly independent if none of them can be expressed as a linear combination of the others
A set of n vectors is a basis for an n-dimensional space if they are all linearly independent
Vectors are orthogonal if their dot product = 0 (ex: <1/5½,2/5½ >, <-2/5½,1/5½ >)
Two orthogonal vectors are orthonormal if their lengths are all unity
<0,1,0>
<1,0,0>
Interpretation
We can represent any vector by a linear combination of an orthogonal set <1/5½,2/5½ > • <4,7> = 18/5½ and <-2/5½,1/5½>• <4,7> = -1/5½
(18/5½)<1/5½,2/5½ > + (-1/5½)<-2/5½,1/5½> = <20/5, 35/5> = <4,7>
Orthogonal vectors extends the concept of: “perpendicular”

2. Extend Linear Algebra to Calculus
Two functions, f(x) and g(x) and are orthogonal over the interval with weighting function w(x) if
If, in addition, f(x) ≠ g(x) ≠ 0 and w(x) ≠ 0
= 0 , and = 0
the functions f(x) and g(x) are said to be orthornormal
Interpretation: Sets of periodic functions are orthornormal

3. Fourier Decomposition
Fourier showed us that we can decompose any time domain signal into a (possibly infinite) series of waves described by a set of orthonormal functions
Each wave has a well-defined frequency and can vary in amplitude

4. Fourier Series
An infinite sum of sinusoids modeling a continuous signal is an example of a Fourier Series
Each point on the Fourier number line represents a particular sinusoid
Fourier transform: Decompose a signal into Fourier components, perform processing, and recombine the results to solve an original problem
We use transforms to alter a problem that cannot be solved in one domain into another, where the solution is possible. Many DSP algorithms are able to compute useful results by analyzing a signals component frequencies

5. Fourier Square Wave Synthesis

6. Fourier Decomposition
Example: decompose a signal into 9 cosine and sine waves
The decomposition models the signal’s phase and amplitude

7. The Fourier Transform Family
Fourier Series A possibly infinite decomposed weighted sum of sinusoidal functions that models a periodic signal
Fourier Transform A linear operation that maps an arbitrary function with into a spectrum of its frequency components
Discrete Fourier Transform (DFT) A Fourier Transform applied to a discrete periodic series of measurements
Discrete Time Fourier Transform (DTFT) A Fourier Transform applied to a an aperiodic discrete series of measurements
Fast Fourier Transform: Fast way to calculate DFT

8. Orthonormal Sinusoids

9. Orthonormal Attenuating/Amplifying Sinusoids
Parameters
Frequency of attenuation or amplification
Speed of attenuation or amplification

10. Alternative Basis Functions

11. Choosing a Basis Function Set
Any orthonormal function set works, but all choices are not equal
A good choice:
Has desirable mathematical characteristics. Those with sharp edges fail
Represents frequencies, amplitudes, phases. Pure sines and cosines fail
Integral: Cos (πx/3) * cos (2 πx/3)
cos (πx/3) and cos (2 πx/3)

12. Orthonormal Basis Sets (cont.)
The winners:
Continuous version: e-j2πft = cos(2πft) – j sin(2πft) where t is point in time, f is a frequency, polar notation models both phase angle and magnitude
Discrete version: e-j2πfn/N = cos(2πfn/N) – j sin(2π fn/N) where n is a signal measurement, f is one of the N orthonormal functions
Note: The angular frequency, ωm, measured in radians per second is defined as follows: ωm = 2πfF0 = 2πf/T0 measures how fast a particular basis function oscillates with respect to the fundamental frequency, F0, having period, T0. For every one time F0 rotates about the unit circle, 2πfF0 rotates f times faster.

13. The Fourier Transform

14. The Fourier Algorithm Fourier analysis This Signal has
Signal: x(t) = sin(2π*1000t) sin(2*2000πt+3π/4)
This Signal has
Two component frequencies: 1Khz, and 2Khz
The second signal has a 1350 phase difference from the first
We measure the signal at N (8) measurements per cycle (fs=8kHz)
Fourier analysis
Multiply each measurement by the corresponding sin and cosine values of e-2πfn/N and sum the results
The resulting frequency array will have non zero elements at indices X[1] and X[2] as we would expect
The phase angles are relative to the cosine of f0
X[1] phase is -900 (-π/2) because sin(π/2) = cos(0) and cos(f0) leads by 900
X[2] phase is +45 (π/4) because cos(2*f0) trails by that amount
Negative frequencies X[6] and X[7] equal -X[2] and -X[1] respectively

15. Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

16. Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

17. Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

18. Second Example Comparisons with the first example
Same signal: x(t) = sin(2π*1000t) sin(2*2000πt+3π/4)
Sampling starts three time units later (see above figure)
Fourier results (We apply the same Fourier algorithm)
The same frequency components have non zero magnitudes
The cosine and sine sums are different, but the magnitudes are the same
The phase of X[1] from -90 and +45; the third sample moves forward.
The phase of X[2] goes from +45 to -45 because the third sample, because each sample is π /2 apart, and starting with the 3π/4, the third sample starts with 3 * (π /2) + 3π/4 = 9 π /4, π /2 ahead of the basis function

19. Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

20. Example Results Notes The results are scaled by N/2 (4 + 2 + 2 +4 = 12f
First Example
Second Example M ϕ
Real
Imag 1 4
-90
0.000
-4.000
+45
2.8284 2
1.414
-45
1.4142 3 5 6
-1.414 7
+90
4.000
Notes
The results are scaled by N/2 ( = 12
The frequency domain is symmetrical about point N/2
X[0] = X[4] in this example, but not generally

21. DFT Properties
The DFT is a linear transformation: c1DFT(x1[n])+c2DFT(x2[n]) = DFT(c1x[n] + c2x2[n])
The resulting magnitudes are amplified by N/2 (8/2 in this case). This means the 4 should be 1 and the 2 should be ½.
A shift in the measurements correspond to a shift in the DFT phase calculations (as seen in the examples)
The frequency domain is symmetric.

22. DFT Symmetry The resulting frequency domain is symmetric
X[n-k] = X[n+k] when k ≠N/2 and k ≠0,
The upper frequencies represent negative frequencies (the extension of the signal if moving back in time)
X[0] represents zero DC bias, a non oscillating constant
X[N/2] represents the component fs/2
Negative Frequencies
Positive Frequencies

23. DFT and its Inverse
Definition: Continuous Fourier Transform and Inverse
Transform: X(w) = ∫-∞, ∞ x(t)e-iwtdt
Inverse: x(t) = (1/2π)∫ -∞, ∞ X(w)eiwtdw
Discrete Fourier Transform and Inverse
Transform: X[k] = ∑n=0,N-1 x[n] e-i2πkn/N
Inverse: x[k] = (1/N)∑n=0,N-1 X[k] ei2 πkn/N
Note: We can change the scale factor if we want the transform and its inverse to be completely symmetric
Conclusion: We can use DFT to go back and forth between the time and frequency domains

24. DFT Correlation Algorithm
public double[] DFT( double[] time)
{ int N = time.length;
double fourier[] = new double[2*N], real, imag;
for (int k=0; k<N; k++)
{ for (int i=0; i<N; i++)
{ real = Math.cos(2*Math.PI*k*i/N);
imag = -Math.sin(2*Math.PI*k*i/N);
fourier[2*k] += time[i]*real;
fourier[2*k+1]+= time[i]*imag;
} }
return fourier; }
Complexity: O(N2) because of the double loop of N each
Example: For 512 samples, loops times
Evaluation: Too slow, but FFT is O(N lg N)
Note: even indices = real part, odd indices = imaginary part

25. Relation to Speech
For analysis we breakup signal into overlapping windows
Why?
Speech is quasi-periodic, not periodic
Vocal musculature is always changing
Within a small window of time, we assume constancy, and process as if the signal were periodic
Typical Characteristics
10-30 ms length
1/3 to 1/2 overlap

26. DFT Issues Implementation difficulties
Impact: Algorithm runs too slow, especially for real time processing
Solution: FFT algorithm
Spectral Leakage: Occurs when a signal has frequencies that fall between those of the DFT basis functions
Result: Magnitude leaks over the entire spectrum
Solutions: Windowing and DFT padding
Inadequate frequency resolution
Impact: DFT magnitudes represent wide ranges of frequencies
Solution: DFT zero padding
Signal rough edges:
Impact: Inaccuracies because windows don’t have smooth transistions
Solution: Windowing

27. DFT Zero Padding
Each frequency domain bin represents a range of frequencies
Assume sampling rate fs = 16000, N = 512
Frequency domain 3, contains magnitudes for frequencies between 16000/512*3 (93.75) to 16000/512*4 (125)
Each frequency bin models a range of frequencies
Padding the time domain frame with zeroes adds high frequency components, but does not negatively impact the result, or impact the lower frequencies of the spectrum

28. DFT Zero Padding

29. FFT zero padding can increase SNR
Definition:
SNR = 10log( Psignal/Pnoise
Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

30. Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.