1. p. 1 DSP-II Digital Signal Processing II Lecture 2: FIR & IIR Filter Design Marc Moonen Dept. E.E./ESAT, K.U.Leuven marc.moonen@esat.kuleuven.be homes.esat.kuleuven.be/~moonen/

2. DSP-II p. 2 Version 2005-2006 Chapter-2 FIR/IIR Filter Design PART-I : Filter Design/Realization Step-1 : define filter specs (pass-band, stop-band, optimization criterion,…) Step-2 : derive optimal transfer function FIR or IIR design Step-3 : filter realization (block scheme/flow graph) direct form realizations, lattice realizations,… Step-4 : filter implementation (software/hardware) accuracy issues, … question: implemented filter = designed filter ? Lecture-2 Lecture-3 Lecture-4

3. DSP-II p. 3 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Lecture-2 : FIR & IIR Filter Design FIR filters –Linear-phase FIR filters –FIR design by optimization Weighted least-squares design, Minimax design –FIR design in practice `Windows’, Equiripple design, Software (Matlab,…) IIR filters –Poles and Zeros –IIR design by optimization Weighted least-squares design, Minimax design –IIR design in practice Analog IIR design : Butterworth/Chebyshev/elliptic Analog->digital : impulse invariant, bilinear transform,… Software (Matlab)

4. DSP-II p. 4 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filters FIR filter = finite impulse response filter Also known as `moving average filters’ (MA) N poles at the origin z=0 (hence guaranteed stability) N zeros (zeros of B(z)), `all zero’ filters corresponds to difference equation impulse response

5. DSP-II p. 5 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Linear Phase FIR Filters Non-causal zero-phase filters : example: symmetric impulse response h[-L],….h[-1],h[0],h[1],...,h[L] h[k]=h[-k], k=1..L frequency response is i.e. real-valued (=zero-phase) transfer function k L

6. DSP-II p. 6 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Linear Phase FIR Filters Causal linear-phase filters = non-causal zero-phase + delay example: symmetric impulse response & N even h[0],h[1],….,h[N] N=2L (even) h[k]=h[N-k], k=0..L frequency response is = i.e. causal implementation of zero-phase filter, by introducing (group) delay k N0

7. DSP-II p. 7 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Linear Phase FIR Filters Type-1 Type-2 Type-3 Type-4 N=2L=even N=2L+1=odd symmetric symmetric anti-symmetric anti-symmetric h[k]=h[N-k] h[k]=h[N-k] h[k]=-h[N-k] h[k]=-h[N-k] zero at zero at zero at LP/HP/BP LP/BP BP HP PS: `modulating’ Type-2 with 1,-1,1,-1,.. gives Type-4 (LP->HP) PS: `modulating’ Type-4 with 1,-1,1,-1,.. gives Type-2 (HP->LP) PS: `modulating’ Type-1 with 1,-1,1,-1,.. gives Type-1 (LP HP) PS: `modulating’ Type-3 with 1,-1,1,-1,.. gives Type-3 (BP BP) PS: IIR filters can NEVER have linear-phase property ! (proof see literature)

8. DSP-II p. 8 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Filter Specification Ex: Low-pass

9. DSP-II p. 9 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization (I) Weighted Least Squares Design : select one of the basic forms that yield linear phase e.g. Type-1 specify desired frequency response (LP,HP,BP,…) optimization criterion is where is a weighting function

10. DSP-II p. 10 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization …this is equivalent to = `Quadratic Optimization’ problem (see course HG94)

11. DSP-II p. 11 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization Example: Low-pass design optimization function is i.e.

12. DSP-II p. 12 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization a simpler problem is obtained by replacing the F(..) by… where the wi’s are a set of (n) selected sample frequencies This leads to an equivalent (`discretized’) quadratic optimization problem: +++ : simple - - - : unpredictable behavior in between sample freqs. Compare to p.27

13. DSP-II p. 13 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization …then all this is often supplemented with additional constraints Example: Low-pass (LP) design (continued) pass-band ripple control : stop-band ripple control :

14. DSP-II p. 14 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization Example: Low-pass (LP) design (continued) a realistic way to implement these constraints, is to impose the constraints (only) on a set of sample frequencies, e.g. in the pass-band and in the stop-band The resulting optimization problem is : minimize : subject to (pass-band constraints) (stop-band constraints) = `Quadratic Programming’ problem (see HG94)

15. DSP-II p. 15 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization (II) `Minimax’ Design : select one of the basic forms that yield linear phase e.g. Type-1 specify desired frequency response (LP,HP,BP,…) optimization criterion is where is a weighting function

16. DSP-II p. 16 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization this is equivalent to the constraint is equivalent to a so-called `semi-definiteness’ constraint where D>=0 denotes that the matrix is positive semi-definite Ignore this slide

17. DSP-II p. 17 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Filter Design by Optimization a realistic way to implement these constraints, is to impose the constraints (only) on a set of sample frequencies : i.e. a `Semi-Definite Programming’ (SDP) problem, for which efficient interior-point algorithms and software are available. Ignore this slide

18. DSP-II p. 18 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design by Optimization Conclusion: (I) weighted least squares design (II) minimax design provide general `framework’, procedures to translate filter design problems into standard optimization problems In practice (and in textbooks): emphasis on specific (ad-hoc) procedures : - filter design based on `windows’ - equiripple design

19. DSP-II p. 19 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design using `Windows’ Example : Low-pass filter design ideal low-pass filter is hence ideal time-domain impulse response is truncate hd[k] to N+1 samples : add (group) delay to turn into causal filter

20. DSP-II p. 20 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design using `Windows’ Example : Low-pass filter design (continued) ps : it can be shown that time-domain truncation corresponds to solving a weighted least-squares optimization problem with the given Hd, and weighting function truncation corresponds to applying a `rectangular window’ : +++: simple procedure (also for HP,BP,…) - - - : truncation in the time-domain results in `Gibbs effect’ in the frequency domain, i.e. large ripple in pass-band and stop-band (at band edge (=discontinuity)), which cannot be reduced by increasing the filter order N.

21. DSP-II p. 21 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design using `Windows’ Remedy : apply windows other than rectangular window: time-domain multiplication with a window function w[k] corresponds to frequency domain convolution with W(z) : candidate windows : Han, Hamming, Blackman, Kaiser,…. (see textbooks, see DSP-I) window choice/design = trade-off between side-lobe levels (define peak pass-/stop-band ripple) and width main-lobe (defines transition bandwidth)

22. DSP-II p. 22 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Equiripple Design Starting point is minimax criterion, e.g. Based on theory of Chebyshev approximation and the `alternation theorem’, which (roughly) states that the optimal d’s are such that the `max’ (maximum weighted approximation error) is obtained at L+2 extremal frequencies… …that hence will exhibit the same maximum ripple (`equiripple’) Iterative procedure for computing extremal frequencies, etc. (Remez exchange algorithm, Parks-McClellan algorithm) Very flexible, etc., available in many software packages Details omitted here (see textbooks)

23. DSP-II p. 23 Version 2005-2006 Chapter-2 FIR/IIR Filter Design FIR Filter Design Software FIR Filter design abundantly available in commercial software Matlab: b=fir1(n,Wn,type,window), windowed linear-phase FIR design, n is filter order, Wn defines band-edges, type is `high’,`stop’,… b=fir2(n,f,m,window), windowed FIR design based on inverse Fourier transform with frequency points f and corresponding magnitude response m b=remez(n,f,m), equiripple linear-phase FIR design with Parks-McClellan (Remez exchange) algorithm See exercise sessions

24. DSP-II p. 24 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR filters Rational transfer function : N poles (zeros of A(z)), N zeros (zeros of B(z)) infinitely long impulse response stable iff poles lie inside the unit circle corresponds to difference equation = also known as `ARMA’ (autoregressive-moving average)

25. DSP-II p. 25 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR filters Frequency response versus pole-zero location : (cfr. frequency response is z-transform evaluated on the unit circle) Example-1 : Low-pass filter poles at DC (z=1) Nyquist freq (z=-1) pole Re Im

26. DSP-II p. 26 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR filters Frequency response versus pole-zero location : Example-2 : Low-pass filter poles at zeros at DC Nyquist freq zero pole

27. DSP-II p. 27 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR filters Frequency response versus pole-zero location : Example-3 : Band-pass filter poles at zeros at DC zero pole

28. DSP-II p. 28 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR filters Frequency response versus pole-zero location : * pole near unit-circle introduces `peak’ in frequency response hence pass-band can be set by pole placement * zero near unit-circle introduces `dip’ in frequency response zero on unit-circle introduces `transmission zero’ in frequency response hence stop-band can be emphasized by zero placement

29. DSP-II p. 29 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design +++ low-order filters can produce sharp frequency response low computational cost - - - design more difficult stability should be checked/guaranteed phase response not easily controlled (e.g. no linear-phase IIR filters) coefficient sensitivity, quantization noise, etc. can be a problem (see Lecture-4)

30. DSP-II p. 30 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design by Optimization (I) Weighted Least Squares Design : IIR filter transfer function is specify desired frequency response (LP,HP,BP,…) optimization criterion is where is a weighting function stability constraint :

31. DSP-II p. 31 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design by Optimization (II) `Minimax’ Design : IIR filter transfer function is specify desired frequency response (LP,HP,BP,…) optimization criterion is where is a weighting function stability constraint :

32. DSP-II p. 32 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design by Optimization These optimization problems are significantly more complex than those for the FIR design case… : Problem-1: presence of denominator polynomial leads to rational approximation, and hence to highly non-linear optimization (instead of -for instance- quadratic optimization in the FIR design case) –A possible procedure (`Steiglitz-McBride’) consists in iteratively (k=1,2,…) minimizing …which leads to iterative Quadratic Programming, etc.. (similar for minimax)

33. DSP-II p. 33 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design by Optimization Problem-2: stability constraint (zeros of a high-order polynomial are related to the polynomial’s coefficients in a highly non-linear manner) –Solutions based on alternative stability constraints, that e.g. are affine functions of the filter coefficients, etc… –Topic of ongoing research, details omitted here

34. DSP-II p. 34 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design by Optimization Conclusion: (I) weighted least squares design (II) minimax design provide general `framework’, procedures to translate filter design problems into ``standard’’ optimization problems In practice (and in textbooks): emphasis on specific (ad-hoc) procedures : - IIR filter design based analog filter design (s-domain design) and analog->digital conversion - IIR filter design by modeling = direct z-domain design (Pade approximation, Prony, etc., not addressed here)

35. DSP-II p. 35 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog IIR Filter Design Commonly used analog filters : Lowpass Butterworth filters all-pole filters characterized by magnitude response…. (N=filter order) Poles of H(s)H(-s) are equally spaced points on a circle of radius in s-plane poles of H(s) N=4 poles of H(-s)

36. DSP-II p. 36 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog IIR Filter Design Commonly used analog filters : Lowpass Butterworth filters monotonic in pass-band & stop-band `maximum flat response’: (2N-1) derivatives are zero at and N

37. DSP-II p. 37 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog IIR Filter Design Commonly used analog filters : Lowpass Chebyshev filters (type-I) all-pole filters characterized by magnitude response (N=filter order) is related to passband ripple are Chebyshev polynomials:

38. DSP-II p. 38 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog IIR Filter Design Commonly used analog filters : Lowpass Chebyshev filters (type-I) All-pole filters, poles of H(s)H(-s) are on ellipse in s-plane Equiripple in the pass-band Monotone in the stop-band Lowpass Chebyshev filters (type-II) Pole-zero filters based on Chebyshev polynomials Monotone in the pass-band Equiripple in the stop-band Lowpass Elliptic (Cauer) filters Pole-zero filters based on Jacobian elliptic functions Equiripple in the pass-band and stop-band (hence) yield smallest-order for given set of specs

39. DSP-II p. 39 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog IIR Filter Design Frequency Transformations : Principle : prototype low-pass filter (e.g. cut-off frequency = 1 rad/sec) is transformed to properly scaled low-pass, high-pass, band- pass, band-stop,… filter example: replacing s by moves cut-off frequency to example: replacing s by turns LP into HP, with cut-off frequency example: replacing s by turns LP into BP etc...

40. DSP-II p. 40 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog -> Digital Principle : design analog filter (LP/HP/BP/…), and then convert it to a digital filter. Conversion methods: - convert differential equation into difference equation - convert continuous-time impulse response into discrete-time impulse response - convert transfer function H(s) into transfer function H(z) Requirement: the left-half plane of the s-plane should map into the inside of the unit circle in the z-plane, so that a stable analog filter is converted into a stable digital filter.

41. DSP-II p. 41 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog -> Digital Conversion methods: (I) convert differential equation into difference equation : -in a difference equation, a derivative dy/dt is replaced by a `backward difference’ (y(kT)-y(kT-T))/T=(y[k]-y[k-1])/T, where T=sampling interval. -similarly, a second derivative, etc… -eventually (details omitted), this corresponds to replacing s by (1-1/z)/T in Ha(s) (=analog transfer function) : -stable analog filters are mapped into stable digital filters, but pole location for digital filter confined to only a small region (o.k. only for LP or BP) jw s-plane z-plane j 1 Ignore this slide

42. DSP-II p. 42 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog -> Digital Conversion methods: (II) convert continuous-time impulse response into discrete-time impulse response : -given continuous-time impulse response ha(t), discrete-time impulse response is where T=sampling interval. -eventually (details omitted) this corresponds to a (many-to-one) mapping -aliasing (!) if continuous-time response has significant frequency content above the Nyquist frequency s-plane jw z-plane j 1 Ignore this slide

43. DSP-II p. 43 Version 2005-2006 Chapter-2 FIR/IIR Filter Design Analog -> Digital Conversion methods: (III) convert continuous-time system transfer function into discrete-time system transfer function : Bilinear Transform -mapping that transforms (whole!) jw-axis of the s-plane into unit circle in the z-plane only once, i.e. that avoids aliasing of the frequency components. -for low-frequencies, this is an approximation of -for high frequencies : significant frequency compression (`warping’) (sometimes pre-compensated by `pre-warping’) s-plane jw z-plane j 1

44. DSP-II p. 44 Version 2005-2006 Chapter-2 FIR/IIR Filter Design IIR Filter Design Software IIR filter design considerably more complicated than FIR design (stability, phase response, etc..) (Fortunately) IIR Filter design abundantly available in commercial software Matlab: [b,a]=butter/cheby1/cheby2/ellip(n,…,Wn), IIR LP/HP/BP/BS design based on analog prototypes, pre-warping, bilinear transform, … immediately gives H(z) (oef!) analog prototypes, transforms, … can also be called individually filter order estimation tool etc... See exercise sessions