Concise guide on numerical methods Grzegorz Pytel, Warsaw University of Technology Fourier Transform and its applications Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Fourier Transform and its application Talk scheme 1 Theoritcal introduction 2 Numerical methods 3 Applications Fourier transform and its application in review 17/11/2018 Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Theoretical introduction 1 Theoretical introduction t Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Fourier Transform and its application Fourier analysis Symmetry Periodic Phenomena Linear Numerical methods Group theory 17/11/2018 Fourier Transform and its application

Periodicity Periodicity sin⁡(2Π𝑡) Period 1 Freq 1 In space In time Number of repetitions of a patttern in a second Uses period. Measurement of how big the pattern is that repeats Periodicity arises from symmetry sin⁡(6Π𝑡) Period 1 3 Freq 3 t? You’re fixed in a period. Period of sum=1 17/11/2018 Fourier Transform and its application

Fourier Transform and its application 𝐿 𝑏𝑐 1 = 𝐹:𝑅 𝐶,𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑜𝑢𝑠,𝑏𝑜𝑢𝑛𝑑𝑒𝑑, 𝑓 1 = −∞ ∞ 𝑓(𝑥) 𝑑𝑥<∞ Theorem: 𝑓,𝑔∈ 𝐿 𝑏𝑐 1 𝑡ℎ𝑒𝑛 𝑐𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑓∗𝑔 𝑥 = −∞ ∞ 𝑓 𝑦 𝑔 𝑥−𝑦 𝑑𝑦 𝑎𝑛𝑑 𝑒𝑥𝑖𝑠𝑡𝑠 𝑓𝑜𝑟 ∀𝑥∈𝑅: 𝑓∗𝑔∈ 𝐿 𝑏𝑐 1 𝑅 f*g=g*f Iff ℎ∈ 𝐿 𝑏𝑐 1 𝑅 𝑡ℎ𝑒𝑛 𝑓∗ 𝑔∗ℎ = 𝑓∗𝑔 ∗ℎ f*(g+h)=f*g+f*h Definition If 𝑓∈ 𝐿 𝑏𝑐 1 𝑅 then we define Fourier Transform as: 𝑓 𝑦 = −∞ ∞ 𝑓(𝑥) 𝑒 −2Π𝑖𝑦𝑥 𝑑𝑥 Time Frequency More, including properties: http://en.wikipedia.org/wiki/Fourier_transform 17/11/2018 Fourier Transform and its application

Discrete Fourier Transform Let 𝑎 𝑖 ∈𝑅,𝑖=0,..,𝑁−1 be discretized values of signal DFT: 𝑎 0 , 𝑎 1 ,.., 𝑎 𝑁−1 𝐴 0 , 𝐴 1 ,.., 𝐴 𝑁−1 𝑤ℎ𝑒𝑟𝑒 𝐴 𝑖 ∈𝐶 𝑎𝑛𝑑 𝐴 𝑘 = 𝑛=0 𝑁−1 𝑎 𝑛 𝜔 𝑁 −𝑘𝑛 ,𝑘=0,..,𝑁−1 Where 𝜔 𝑁 = 𝑒 2Π𝑖 𝑁 𝐴 𝑘 = 𝑛=0 𝑁−1 𝑎 𝑛 𝑒 − 2Π𝑖𝑘𝑛 𝑁 ,𝑘=0,..,𝑁−1 Inverse Discrete Fourier Transform: 𝑎 𝑛 = 1 𝑁 𝑘=0 𝑁−1 𝐴 𝑘 𝜔 𝑁 𝑘𝑛 ,𝑛=0,..,𝑁−1 Complexity: 𝑂( 𝑁 2 ) 17/11/2018 Fourier Transform and its application

Fast Fourier Transform Wolfgang Bangerth, Guido Kanschat, and Ralf Hartmann, a software library for computational solution of partial differential equations using adaptive finite elements A fast Fourier transform (FFT) is an efficient algorithm to compute the discrete Fourier transform (DFT) and its inverse. FFTs was constructed by J. W. Cooley of IBM and John W. Tukey of Princeton published a paper in 1965 reinventing the algorithm and describing how to perform it conveniently on a computer http://www.fftw.org/ 2007 Jonathan Shewchuk for Triangle, a two-dimensional mesh generator and Delaunay Triangulator 2003 1999 Chris Bischof and Alan Carle for ADIFOR 2.0, an automatic differentiation tool for Fortran 77 programs 1995 On Thursday, 8 July 1999, the third Wilkinson Prize for Numerical Software was awarded to Matteo Frigo and Steven Johnson of Massachusetts Institute of Technology. The winning entry, FFTW (the "Fastest Fourier Transform in the West"), is a library of C routines for the efficient computation of the discrete Fourier transform of real and complex data. The unparalleled efficiency over a wide range of computer platforms is by automatically determining the best computational strategy for the particular hardware. Linda Petzold for DASSL, a differential algebraic equation solver 1991 Complexity: 𝑂(𝑁𝑙𝑜𝑔𝑁) Wilkinson Prize axis 17/11/2018 Fourier Transform and its application

Fast Fourier Transform – Danielson Lenczos Procedure FFT(N,ω,a(x)) If N=1 then 𝐴 0 𝑎 0 Else { 𝑏 𝑥 = 𝑖=0 𝑁 2 −1 𝑎 2𝑖 𝑥 𝑖 ;𝑐 𝑥 𝑖=0 𝑁 2 −1 𝑎 2𝑖+1 𝑥 𝑖 𝐵 𝐹𝐹𝑇 𝑁 2 , 𝜔 2 ,𝑏 𝑥 ;𝐶 𝐹𝐹𝑇 𝑁 2 , 𝜔 2 ,𝑐 𝑥 for i from 0 to 𝑁 2 −1 do { 𝐴 𝑖 𝐵 𝑖 + 𝜔 𝑖 𝐶 𝑖 𝐴 𝑁 2 +𝑖 𝐵 𝑖 + 𝜔 𝑖 𝐶 𝑖 } } Return 𝐴 0 , 𝐴 1 ,…, 𝐴 𝑁−1 end 17/11/2018 Fourier Transform and its application

Fourier Transform and its application 2 Numerical methods World’s major methods used for computing FFT Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Fast Fourier Transform Cooley–Tukey Prime-factor Bruun's Rader's Bluestein's Winogard 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Cooley-Tukley Radix 2-Decimation in Time, 1965 𝑋 𝑘=0..𝑁−1 = 𝑚=0 𝑁 2 −1 𝑥 2𝑚 𝑒 − 2Π𝑖 𝑁 2 2𝑚 𝑘 + 𝑒 − 2Π𝑖 𝑁 𝑘 𝑚=0 𝑁 2 −1 𝑥 2𝑚+1 𝑒 − 2Π𝑖 𝑁 2 𝑚𝑘 Danielson-Lanczos Lemma, 1942 C-T C source code: http://en.literateprograms.org/Cooley-Tukey_FFT_algorithm_(C) Twiddle factor The discrete Fourier transform of length (where is even) can be rewritten as the sum of two discrete Fourier transforms, each of length N/2. One is formed from the even-numbered points; the other from the odd-numbered points. Denote the th point of the discrete Fourier transform by . Then 𝐹 𝑛 = 𝑘=0 𝑁−1 𝑓 𝑘 𝑒 − 2Π𝑖𝑛 𝑁 = 𝑘=0 𝑁 2 𝑒 − 2Π𝑖𝑘𝑛 𝑁 2 𝑓 2𝑘 + 𝑊 𝑛 𝑘=0 𝑁 2 −1 𝑒 − 2Π𝑖𝑘𝑛 𝑁 2 𝑓 2𝑘+1 = 𝐹 𝑛 𝑒 + 𝑊 𝑛 𝐹 𝑛 0 More: http://mathworld.wolfram.com/Danielson-LanczosLemma.html Danielson-Lanczos Cooley-Tukley Time [min] 140 0.02 Task’s size 64 DFT 3–5 significant digits, real numbers 2048 DFT 8 digits 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Cooley-Tukley Can only be used to speed the calculation of DFTs of a size that is a power of two Requires padding with 0 extra multiplications by roots of unity called twiddle factors NlogN complexity More inc. sourcecode at: http://numericalrecipes.blogspot.com/2009/04/cooley-turkey-fft-algorithm.html 17/11/2018 Fourier Transform and its application

Cooley-Tukley general factorizations DCT DFT 1 𝑁 2 2 Transpose Multiply by N twiddle factors 𝑁 1 More info and Python sourcecode: http://numericalrecipes.wordpress.com/2009/05/29/the-cooley-tukey-fft-algorithm-for-general-factorizations/ 17/11/2018 Fourier Transform and its application

Prime factor (Good-Thomas algorithm) DFT: Is a FFT that re-expresses the DFT of a 𝑁= 𝑁 1 𝑁 2 as two-dimensional 𝑁 1 𝑥 𝑁 2 DFT, but only for the case where 𝑁 1 and 𝑁 2 are relatively prime 𝑥 𝑘 = 𝑛=0 𝑁−1 𝑥 𝑛 𝑒 − 2Π𝑖 𝑁 𝑛𝑘 ,𝑘=0,…,𝑁−1 𝑛= 𝑛 1 𝑁 2 + 𝑛 2 𝑁 1 𝑚𝑜𝑑 𝑁 𝑘= 𝑘 1 𝑁 2 −1 𝑁 2 + 𝑘 2 𝑁 1 −1 𝑁 1 𝑋 𝑘 1 𝑁 2 −1 𝑁 2 + 𝑘 2 𝑁 1 −1 𝑁 1 = 𝑛 1 =0 𝑁 1 −1 𝑛 2 =0 𝑁 2 −1 𝑥 𝑛 1 𝑁 2 + 𝑛 2 𝑁 1 𝑒 − 2Π𝑖 𝑁 2 𝑛 2 𝑘 2 𝑒 − 2Π𝑖 𝑁 1 𝑛 1 𝑘 1 More at: http://neohumanism.org/p/pr/prime_factor_fft_algorithm.html Example at: http://cnx.org/content/m12033/latest/ 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Bruun’s Only real coefficients until the last computation stage May be less accurate than Cooley–Tukey in the face of finite numerical precision (Storn, 1993) 𝜔 𝑁 𝑛 = 𝑛 𝑘 𝑒 − 2Π𝑖 𝑁 𝑛 𝑥 𝑧 = 𝑛=0 𝑁−1 𝑥 𝑛 𝑧 𝑛 x(z) – polynomial whose coefficients are 𝑥 𝑛 𝑋 𝑘 =𝑥 𝜔 𝑁 𝑘 =𝑥 𝑧 𝑚𝑜𝑑 𝑧− 𝜔 𝑁 𝑘 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Rader’s Computes DFT of prime sizes by re-expressing the DFT as a cyclic convolution Is a one-dimensional index-mapping scheme that turns a length-N DFT (N prime) into a length-(N−1) convolution and a few additions. Rader's conversion works only for prime-length N. Most efficient for prime-length N More at: http://cnx.org/content/m12023/latest/ 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Rader’s Mapping 𝑘= 𝑟 𝑚 𝑚𝑜𝑑𝑁,𝑚=0,..,𝑁−2, N-prime, uniquely generates all elements k=[1,..,N-1] N=5,r=2 2 0 𝑚𝑜𝑑5=1 2 1 𝑚𝑜𝑑5=2 2 2 𝑚𝑜𝑑5=4 2 3 𝑚𝑜𝑑5=3 N-prime, the inverse of r 𝑟 −1 𝑟𝑚𝑜𝑑𝑁=1 N=5,r=2, 𝑟 −1 =3 2𝑥3𝑚𝑜𝑑5=1 3 0 𝑚𝑜𝑑5=1 3 1 𝑚𝑜𝑑5=3 3 2 𝑚𝑜𝑑5=4 3 3 𝑚𝑜𝑑5=2 𝑚𝑛 𝑚=[0,..,𝑁−2 ]⋀𝑛=[1,..,𝑁−1])(𝑛= 𝑟 −𝑚 𝑚𝑜𝑑𝑁) 𝑝𝑘 𝑝=[0,..,𝑁−2 ]⋀𝑘=[1,..,𝑁−1])(𝑘= 𝑟 −𝑝 𝑚𝑜𝑑𝑁) 𝑋 𝑘 = 𝑛=0 𝑁−1 𝑥 𝑛 𝑊 𝑁 𝑛𝑘 = 𝑥 0 + 𝑛=1 𝑁−1 𝑥 𝑛 𝑊 𝑁 𝑛𝑘 𝑖𝑓 𝑘≢0 𝑛=0 𝑁−1 𝑥 𝑛 𝑖𝑓 𝑘=0 𝑊 𝑁 𝑛𝑘 = 𝑒 − 𝑖 2Π𝑖𝑛𝑘 𝑁 For 𝑘≢0: 𝑋 𝑟 𝑝 𝑚𝑜𝑑𝑁 = 𝑚=0 𝑁−2 𝑥 𝑟 −𝑚 𝑚𝑜𝑑𝑁 𝑊 𝑟𝑃𝑟−𝑚 +𝑥 0 = 𝑥 0 +𝑥 𝑟 −𝑙 𝑚𝑜𝑑𝑁 ∗ 𝑊 𝑟 𝑙 ,𝑙=0,1,..,𝑁−2 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Bluestein’s (1968) Computes FFT at arbitrary sizes by re-expressing the DFT as a convolution can be performed with a pair of FFTs (plus the pre-computed FFT of bn) via the convolution theorem. The key point is that these FFTs are not of the same length N: such a convolution can be computed exactly from FFTs only by zero-padding it to a length greater than or equal to 2N–1 O(N log N) 𝑋 𝑘 = 𝑛=0 𝑁−1 𝑥 𝑛 𝑒 − 2Π𝑖 𝑁 𝑛𝑘 ,𝑘=0,..,𝑁−1 𝑛𝑘= − 𝑘−𝑛 2 + 𝑛 2 + 𝑘 2 2 𝑋 𝑘 = 𝑒 − Π𝑖 𝑁 𝑘 2 𝑛=0 𝑁−1 𝑥 𝑛 𝑒 − Π𝑖 𝑁 𝑛 2 𝑒 Π𝑖 𝑁 (𝑘−𝑛) 2 ,𝑘=0,..,𝑁−1 𝑎 𝑛 = 𝑥 𝑛 𝑒 − Π𝑖 𝑁 𝑛 2 𝑏 𝑛 = 𝑒 Π𝑖 𝑁 𝑛 2 𝑋 𝑘 = 𝑏 𝑘 𝑛=0 𝑁−1 𝑎 𝑛 𝑏 𝑘−𝑛 ,𝑘=0,..,𝑁−1 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Winogard 𝑋= 𝐹 1 𝐹 2 𝑥 𝐹= 𝐴 𝑇 𝐷𝐴 𝑋= 𝐴 2 𝑇 𝐷 2 𝐴 2 𝐴 1 𝑇 𝐷 1 𝐴 1 𝑥 X= 𝐴 2 𝑇 𝐴 1 𝑇 𝐷 2 𝐷 1 𝐴 2 𝐴 1 𝑥 𝐷= 𝐷 2 𝐷 1 𝑋= 𝐴 1 𝑇 𝐴 2 𝑇 𝐷 𝐴 2 𝐴 1 𝑥 DFT’s row DFT’s column Winogard need O(N) multiplications to compute the FFT. However, at the cost of asymptotically many more additions, and thus they represent a tradeoff that is typically unfavorable on modern CPUs with hardware multipliers. Fastest, smallest number of operations Extremely efficient for small N (N<20) The number of adds becomes huge for large N Most difficult More at: http://cnx.org/content/m16333/latest/#uid24 17/11/2018 Fourier Transform and its application

Industrial applications 3 Industrial applications Examples of industrial applications of FFT and DFT Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Industrial products/companies reliable on FFT 17/11/2018 Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Fourier Transform and its application Voice power in matlab DEMO 17/11/2018 Fourier Transform and its application

Fourier Transform and its application DEMO Source code function [n,fs,waveData] = waveRecordFFT() n=20000; fs=11025; waveData=wavrecord(n,fs); wavplay(waveData,fs); frequency(waveData,n,fs); End function[] = frequency(waveData,n,fs) s1=waveData(:,1); p=fft(s1); nUniquePts=ceil((n+1)/2); p=p(1:nUniquePts); p=abs(p); p=p/n; p=p.^2; if rem(n,2) p(2:end) = p(2:end)*2; else p(2:end -1) = p(2:end - 1)*2; end freqArray=(0:nUniquePts-1)*(fs/n); plot(freqArray/1000,10*log10(p),'k') xlabel('Frequency (kHz)') ylabel('Power (dB)') 17/11/2018 Fourier Transform and its application

Fourier Transform and its application DIGITIZING NATURE 17/11/2018 Fourier Transform and its application Fourier Transform and its applications. Grzegorz Pytel. Warsaw University of Technology

Fourier Transform and its application End remarks FFT length Multiplies Adds Mults+Adds Radix 2 1024 10248 30728 40976 Split radix 7172 27652 34842 Prime factor 1008 5804 29100 34904 Winograd 3548 34416 37964 The most commonly used FFT algorithms by far are the power-of-two-length FFT algorithms Prime Factor and Winograd require somewhat fewer multiplies, but the overall difference usually isn't sufficient to warrant the extra difficulty Changing radix can speed up, up to 30-40 % 17/11/2018 Fourier Transform and its application

Fourier Transform and its application Bibliography http://see.stanford.edu/see/courseinfo.aspx?coll=84d174c2-d74f-493d-92ae-c3f45c0ee091 17/11/2018 Fourier Transform and its application