Signal: sin(2 π * 3f s ) Three periods per sampling time period –When f=3, DFT correlation squares each value; result >0 –Result is N/2 times the time domain amplitude The four periods per time period –Positives and negatives cancel each other –FFT spectral value for f=4 totals zero No Spectral Leakage Time Domain Frequency Domain

Signal: sin(2 π * 3.5f s ) Three periods per sampling time period –When f=3, DFT correlation attenuates the total –Some of the amplitude spills across the spectrum Spectral Leakage Time Domain Frequency Domain

DFT Window-sync Distribution Window sync function: sin(x)/x –Period is the same as sin(x) –Amplitude degrades as x increases –Sin( π(k-m))/{π(k-m)}; k is signal frequency, m is DFT basis function Integer k points compute to zero because sin( π k) = 0 Non integer k points ≠ 0 because sin( π(k-m) ) is not a multiple of π

Spectral Leakage Examples

Leakage Symmetry 3.4 cycles per period 64 point DFT Circular view of the frequency spectrum

Leakage Symmetry (cont.) Linear view of the frequency spectrum

More Examples

Leakage means power loss in non-center frequencies Loss amount greatest for frequencies at the intersections Loss can be significant (0.637 in this example) Result is the scalloped pattern of frequency amplitudes Solutions: –Zero DFT creates finer frequency resolution, and therefor causes the scalloping low points to be closer together –Applying a windowing function alters the scalloping curve shape Scalloping Loss

Windowing Issue: In practice, we apply DFT to finite signal pieces (frames) Impact: – The start of the truncated signal does not typically line up with the end (rough edge) – The frequency domain experiences distortion Using a window – First Create the window: Create an array of frame length with values calculated with a windowing formula – Apply the window: multiply these values by each time domain frame amplitude

Speech Application Goal: Extract short-term signal features from speech signal frames Approach –Block the speech into short overlapping segments –Duration of each block varies from 10 to 30 ms –Assumption: Speech within a single frame is unchanging –Apply a window function to smooth the signal –Apply DFT after applying the window –Perform frequency analysis to extract speech characteristics Speech signal characteristics change as we speak

Window Function Application Application of windows to sinusoid signal 1.rectangular (no) window 2.Trangular window 3.Hanning window

Popular Window Types Rectangular: w k = 1 where k = 0 … M – Advantage: Easy to calculate, array elements unchanged – Disadvantage: Messes up the frequency domain Ideal (window-sync): sin( 2 π f i) / (πi) – Disadvantage: Must be infinitely long – Truncation causes ripples and overshoots Hamming: w k = 0.54 – 0.46 cos(2kπ/M) – Advantage: Fast roll-off in frequency domain – Disadvantage: worse attenuation Blackman: w k = 0.42 – 0.5 cos(2kπ/M) cos(4kπ/M) – Advantage: better attenuation – Disadvantage: slower roll-off Multiply the window, point by point, to the audio signal

Other Windowing Formulae Hanning: w[n] = cos(2πn/(N-1)) – Advantage: Adding 50% overlapped frames produces no gain – Advantage: Excellent for pitch manipulation algorithms Bartlett: w[n] = 2/(N-1). (N-1)/2 - |(n–(N-1)/2|) Triangle: w[n] = 2/N. (N/2 - |(n–(N-1)/2|) Blackman: w[n ] = a 0 – a 1 cos(2πn/(N-1)) – a 2 cos(4πn/(N-1)) where: a 0 = (1-α)/2 ; a 1 = ½ ; a 2 = α /2; α =0.16 Blackman-Harris: w[n] = a 0 –a 1 cos(2πn/(N-1))–a 2 cos(4πn/(N-1)) - a 3 cos(6πn/(N-1)) where: a 0 = ; a 1 = ; a 2 = ; a 3 = Note : Small changes can cause large frequency domain impact

Code: Hamming Window Create Window double[] window = new double[windowSize]; double c = 2*Math.PI / (windowSize - 1); for (int h=0; h<windowSize; h++) window[h] = *Math.cos(c*h); Apply window for(int i=0; i<window.length; i++) frame[i]=frame[i]*window[i];

Rectangular Window Frequency Response Time Domain Filter Top Right: Amplitude linear scale, Bottom Right: Power logarithmic scale

Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

Windowing Frequency Response Main Lobe: Narrow implies better frequency resolution. As the window length grows, the main lobe width narrows (less initial spectral leakage). Side lobe: Higher for abrupt time domain window transitions from 1 to 0. Roll-off rate: Slower for abrupt window discontinuities. Main Lobe Side Lobe Roll-off Rate

Rectangular Window Narrow main lobe width: 4π/M, Side lobe width: 2 π /M Spectral leakage directed to places that hurt analysis Poor roll off and high initial lobe

Convoluting Rectangular Window with a particular frequency Note: Windows with more points narrow the central lobe

Triangular (Bartlett Window)

Compare: Hamming to Hanning HanningHamming Hanning: Faster roll of; Hamming: lower first lobe

Blackman & Hamming Frequency Response

Comparing Windows

Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, Spectral Leakage using Hanning Window

Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, Spectral Leakage using Hamming Window

Efficiency Issues Calculation of Sin(x), Cos(x), e x –Sin(x) = x – x 3 /3! + x 5 /5! – x 7 /7! … –Cos(x) = x – x 2 /2! + x 4 /4! – x 6 /6! … –e x = x + x 2 /2! + x 3 /3! + x 3 /4! … Suggestions – Use table look up –x 3 = x * x * x –Consider the use of the cache when using loops

Fourier Analysis The Naïve DFT requires a double loop –Outer loop is T long (length of the time domain frame) –Inner loop is N long (number of FFT frequency bins) –Total calculations: O(N*T), or O(N 2 ) when N = T Fast Fourier Transform (FFT) –N must be an even power of 2, if not, pad with zeros –Total calculations O(N lgN) calculations Compare efficiencies NN2N2 lg NN lg NImprovement 12816, ,0241,048, , ,19267,108, , Real-time DSP Algorithms must be as efficient as possible

Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, 2011.

Optimization of DFT 1.Replace the double loop by a recursive approach 2.Eliminate the recursion for further efficiency 3.Perform obscure optimizations to extract even more speed How do we know if the optimized version works? 1.Observe the frequency domain results a.Value at index N/2-1 = value at index N/2+1 b.Value at index N/2-k = value at index N/2+k c.The 0 index contains a constant multiplier that when removed causes the signal’s mean to be zero d.Value at N/2 is unrelated to the rest of the spectrum 2.Perform an inverse FFT to restore the original signal 3.Compare the results to the slower DFT algorithm 4.Apply algorithm to simple sinusoid signals and check results

Why is FFT fast Evaluate polynomial at n roots of unity Why is this important? –Dealing with exponents is easy e a e b =e a+b, (e a ) b = e ab –The i th root of unity ω i n = e 2πi/n. –For even numbers, w n n/2 =w 2 1 =-1 –For even numbers, squares of the n complex roots of unity are the same as the n/2 complex roots of unity –Summing all n roots of unity equals zero –The calculation of e 4πi/n =e 2 π i/n because of the periodicity We can avoid many calculations, and use a divide and conquer recursive solution –Solve smaller parts of the linear equations and easily combine results to solve the larger equation

The FFT is an efficient divide and conquer algorithm For polynomials, If n is even, we can divide into two smaller polynomials and we can write the equation as follows This suggests recursion of lg(n) steps. The above equation is the recursive relationship between levels The Fast Fourier Transform P(x) = p even (x) + x * p odd (x) P even (x) = a 0 + a 2 x 2 + a 4 x 4 + … + a n-2 x n-2 P odd (x) = a 1 + a 3 x 2 + … + a n-1 x n-1

Theory for Optimization Base Case: x[0] Recursive Relationship F k = ∑ t=0  N-1 x[t] e -j2πkt/N = ∑ t=0  N/2-1 x[2t] e -j2πk(2t)/N + ∑ t=0  N/2-1 x[2t+1] e -j2πk(2t+1)/N = ∑ t=0  N/2-1 x[2t] e -j2πkt/(N/2) + ∑ t=0  N/2-1 x[2t+1] e -j2πk(2t+1)/N = ∑ t=0  N/2-1 x[2t] e -j2πkt/(N/2) + e -j2πk/N ∑ t=0  N/2-1 x[2t+1] e -j2πkt/(N/2) = F k even + e -j2πk/N * F k odd (2t)/N = t/(N/2)

The FFT Algorithm The recursive step has O(N) computations The recursion requires O(lg(N)) levels Total calculations are O(N lg(N))

FFT Recursive Efficiency Figure to the right –FFT of 16 points –O(N) work at each level –lg N + 1 = 5 levels –Total calculations: O(N lg(N)

The FFT Algorithm Overview Initial Recursive Call (FFT(time) time contains signal amplitudes, and is a power of 2 length (N) Base Case: n=1, then return the time array Divide Step separate time into even (time even ) and odd (time odd ) sub-arrays Recursive Step: y even = FFT(time even ), y odd = FFT(time odd ) Combine Step For loop of complex number multiplications

Simple Recursive Solution public static Complex[] FFT(Complex[] time) {int N = time.length; Complex[] fourier = new Complex[N]; if (N==1) { fourier[0] = time[0]; return fourier; } Complex[] even = new Complex[N/2], odd = new Complex[N/2]; for (int m=0; m<N/2; m++) { even[m] = time[2*m]; odd[m] = time[2*m+1]; } Complex[] q = FFT(even), r = FFT(odd); for (int k=0; k<N/2; k++) {double exp = -2*k* Math.PI /N; Complex wk = new Complex(Math.cos(exp), Math.sin(exp)); fourier[k] = q[k].plus(wk.times(r[k])); fourier[k+N/2] = q[k].minus(wk.times(r[k])); } return fourier; } Note: e -2kπ/N = -e -2kπ/N+N/2

Inefficiencies Repetitive calculations of sines and cosines are extremely slow Recursive activation record overhead is huge Declaring and copying arrays at every step slows things down at least by half N>>1 is ten times faster than N/2 The Complex class methods cause machine code jumps that put pressure on the hardware cache Computations still are an order of magnitude slower than necessary

Eliminating the Recursion The numbers in the rectangles are the array indices Notice how the indices change during the recursion Can you see a pattern ? Butterfly algorithm

Bit flipping implementation Determine if a number is an even power of two: n & (n-1) Determine the upper most bit number, B int B=1, x = n-1; while (x&(x-1)) B++; Result = 0 FOR each bit b from 0 to B IF bit set, shift B – b and OR into result

Butterfly Code int j = N>>1, k; for (int i=1;i<N-1;i++) { if (i < j) { swap (x[i],x[j]);} k = N>>1; while (k>=2 & j>=k) { j -= k; k >>= 1; } j += k; } Most Significant Bit SwapBit ( x, x + lgN) Second most significant bit SwapBit(x, x + lg(N/2) Third most significant bit SwapBit(x, x + lg(N/4) kth most significant bit SwapBit(x, x + lg(N/2 k )) Flip bits from left to right

Sin and Cosine Table Look Up e i2πk/N = cos(2πk/N) + i sin(2πk/N) We can store in an array (sinX[]) sin(2π0/N), sin(2π1/N), sin(2π2/N) sin(2π3/N), … sin(2π(N-1)/N) cos(2πk/N) = cosX[(k+(N>>2))%N] Compute the values ahead of time and save repetitive calculations

Understanding Digital Signal Processing, Third Edition, Richard Lyons ( ) © Pearson Education, W k N = e 2 πk/N This relationship allows table lookups for precomputed sines and cosines

Optimized FFT // Perform the fft calculations. for (int stage=1; stage<=M; stage++) // M = lg N { // Remember that complex numbers require pairs of doubles fftSubGroupGap = 2<<stage; // 4, 8, 16,... – subgroup distance gap = fftSubGroupGap>>1; // 2, 4, 8,... – odd/even distance kInc = N>>1; // Number of 2PIki/N steps for odd/even entries. // Outer loop: each sub-fft group; inner loop: combine group elements for (int even=0; even<complex.length; even+=fftSubGroupGap) { k = 0; // Index into the trigonometric lookup table. for (int element=even; element<(even+gap); element+=2) { // ***** See Next Slide ***** k += kInc; // position for next look up. } } kInc >>= 1; }

Multiplication Portion // Look up e^2PIki/N avoiding trig calculations here. realW = sines[(k+(N>>2))%N]; // cos(2PIk/N); imagW = -sines[k%N]; // -sin(2PIk/N); // Complex multiplication of the odd entry of the subgroup // with (e^2PIi/N)^k = (cos(2PI/N) - i * sin(2*PI/N)^k j = (element + gap); tempReal = realW * complex[j] - imagW * complex[j+1]; tempImag = realW * complex[j+1] + imagW * complex[j]; // Adjust the odd entry (subtract: the fft is periodic). complex[j] = complex[element] - tempReal; complex[j+1] = complex[element+1] -tempImag; //Adjust the even entry. complex[element] += tempReal; complex[element+1] += tempImag;

Final Notes The above analysis –Assumes that N is a power of 2 –If not, pad the array with zeroes –For speech recognition, we always use power of 2 frames What if not a power of 2 –The same analysis is possible with other radices –When hit a prime, fall back to the slower O(n 2 ) algorithm

Final Notes Standard Fast Fourier Transform – requires N to be a power of 2 for recursion to work – Can pad the array with zeroes to extend frequency domain Can it work if N is not a power of 2? – Yes, but special slower processing is needed How do we know if it works? – Point N/2-1 = Point N/2+1, Point N/2-2 = Point N/2+2, Point N/2-k = Point N/2 + k, etc. – Note: Points 0 and N/2 don't match, so don’t check these – The FFT Inverse should restore the time domain signal – Compare to the slower correlation DFT calculation – Try some simple impulses and check the results