1. 16 Oct'09Comp30291 Section 31 University of Manchester School of Computer Science Comp30291: Digital Media Processing 2009-10 Section 3 : Discrete-time LTI systems

2. 16 Oct'09Comp30291 Section 32 3.1Introduction Consider a DSP system as implemented by a digital processor. Takes discrete time input signal { x[ n ] }, & produces an output signal { y[ n ] }.

3. 16 Oct'09Comp30291 Section 33 {x[n]} is sequence whose value at t=nT is x[n]. Similarly for {y[n]}. T is sampling interval in seconds (T = 1/F S ). {x[n-N]} is sequence whose value at t=nT is x[n-N].  {x[n-N]} is {x[n]} with every sample delayed by N sampling intervals. Sequences representing digital signals

4. 16 Oct'09Comp30291 Section 34 (i) Discrete time ‘amplifier’: y[n] = A. x[n]. Described by ‘difference equation’: y[n] = A x[n]. Represented in diagram form by a ‘signal flow graph’: x[n] y[n] A Examples of discrete time systems

5. 16 Oct'09Comp30291 Section 35 (ii) Non-recursive digital filter Output at t=nT obtained by weighting & summing present & previous input samples: e.g. y[n] = A 0 x[n] + A 1 x[n-1] + A 2 x[n-2] + A 3 x[n-3] + A 4 x[n-4] This is a ‘non-recursive difference equation’ Represented by signal flow graph below. Boxes marked ‘ z -1 ‘ produce a delay of one sampling interval.

6. 16 Oct'09Comp30291 Section 36 (iii) Recursive digital filter Output at t= nT calculated from a recursive difference equation: e.g. y[n] = A 1 x[n] - B 2 y[n-1] Represented by signal flow graph below. z -1 x[n] A1A1 B2B2 y[n] Recursive means that previous values of y[n] as well as present & previous values of x[n] are used to calculate y[n].

7. 16 Oct'09Comp30291 Section 37 (iv) A non-linear system Output at t=nT calculated from some non-linear equation, e.g. y[n] = (x[n]) 2 Represented below: x[n] y[n]

8. 16 Oct'09Comp30291 Section 38 (I) A DSP system is linear if: Given any two discrete time signals {x 1 [n]} & {x 2 [n]}, if {x 1 [n]}  {y 1 [n]} & {x 2 [n]}  {y 2 [n]} then response to k 1 {x 1 [n]} + k 2 {x 2 [n]} must be  k 1 {y 1 [n]} + k 2 {y 2 [n]} for any values of k 1 and k 2, To multiply a sequence by k, multiply each element by k, k{x[n]} = {k x[n]}. To add two sequences together, add corresponding samples, {x[n]} + {y[n]} = {x[n] + y[n]}.) 3.2. Linearity & time-invariance

9. 16 Oct'09Comp30291 Section 39 (II) A DSP system is ‘time-invariant’ if: Given any discrete time signal {x[n]}, if response to {x[n]} is {y[n]}, response to {x[n-N]} must be {y[n-N]} for any N. Delaying input by N samples only delays output by N samples. An LTI system is both linear & time-invariant Examples (i), (ii) and (iii) are LTI whereas (iv) is not LTI.

10. 16 Oct'09Comp30291 Section 310 3.3. Discrete time unit impulse Useful to consider response of LTI systems to a discrete time unit impulse, or in short an impulse denoted by {d [n] } with: {d[n-N]} is delayed impulse where the only non-zero sample occurs at n=N rather than at n=0.

11. 16 Oct'09Comp30291 Section 311 When input is {d[n]}, output is impulse-response {h[n]}. If impulse-response of an LTI system is known, its response to any other input signal may be obtained. Impulse-response

12. 16 Oct'09Comp30291 Section 312 3.4. Implementing signal-flow-graphs Consider the non-recursive signal flow graph below with A1, A2, A3, A4, A5 set to specific constants. Notice the labels X1, X2, etc. z -1 z -1 z -1 z -1 x[n] A1 A2 A3 A4 A5 y[n] X1 Y X5 X4 X2 X3 Realised by MATLAB program & flow-diagram on next slides.

13. 16 Oct'09Comp30291 Section 313 Set X1, X2, X3, X4, X5 to zero Set values of A1,A2, A3, A4, A5 INPUT X1 Y = A1*X1 + A2*X2 + A3*X3 +A3*X4 + A5*X5 OUTPUT Y X5 = X4; X4 = X3; X3 = X2; X2 =X1 Flow-diagram for non-recursive signal-flow-graph

14. 16 Oct'09Comp30291 Section 314 clear all; A1=1; A2=2; A3=3; A4=-4; A5=5; X1=0; X2=0; X3=0; X4=0; X5=0; while 1 X1 = input( 'X1 = '); Y= A1*X1 + A2*X2 + A3*X3 + A4*X4 + A5*X5 ; disp([' Y = ' num2str(Y)]); X5 = X4 ; X4 = X3; X3 = X2 ; X2 =X1; end; MATLAB program for non-recursive signal flow graph

15. 16 Oct'09Comp30291 Section 315 More efficient version A = [1 2 3 -4 5 ]’ ; x = [0 0 0 0 0 ]’ ; while 1 x(1) = input( ‘x(1) = '); Y=0; for k = 1 : 5 Y = Y + A(k)*x(k); end; disp([' Y = ' num2str(Y)]); for k=5:-1:2 x(k) = x(k-1); end;

16. 16 Oct'09Comp30291 Section 316 Even more efficient version A = [1 2 3 -4 5 ]' ; x = [0 0 0 0 0 ]' ; while 1 x(1) = input( 'x(1) = '); Y = A(1)*x(1); for k = 5 : -1: 2 Y = Y + A(k)*x(k); x(k) = x(k-1); end; disp(['Y = ' num2str(Y)]); end;

17. 16 Oct'09Comp30291 Section 317 The ‘while 1’ statement initiates an infinite loop. Program runs for ever or until interrupted by ‘CONTROL+C’ Either of the following prints out value of Y: disp(['Y = ' num2str(Y)]); disp(sprintf(‘Y=%d’,Y)); A = [1 2 3 -4 5]’ makes A a column vector (not a row). Note ’. Comments Ready to be converted to DSP assembler. Y = Y + A(k)*x(k); x(k) = x(k-1); One DSP instruction: Mult-acc-shift - MACS

18. 16 Oct'09Comp30291 Section 318 clear all; x=[0 1 0 0 0 0 0 0 0 0]'; a = [1 2 3 -4 5]'; y=filter(a,1,x); y Use of ‘filter’ to implement non-rec signal-flow-graph - & thus to produce its impulse-response Just writing ‘y’ without ‘;’ prints out the array.

19. 16 Oct'09Comp30291 Section 319 Impulse-response for non-rec signal-flow-graph Use any of previous versions & enter values for X1 : 0, 0, 0, 1, 0, 0, 0, 0,.... Sequence of output samples printed out will be : 0, 0, 0, A1, A2, A3, A4, A5, 0, 0,.... Impulse-response can also be obtained by tabulation (later). Output must be zero until input becomes equal to 1 at n=0 Impulse response is: {..., 0,..., 0, A1, A2, A3, A4, A5, 0, 0,...,0,...} where the sample at n=0 is underlined. Only five non-zero output samples are observed. This is a ‘ finite impulse-response ‘ (FIR).

20. 16 Oct'09Comp30291 Section 320 (i) y[n] = x[n] + 2 x[n-1] + 3 x[n-2] - 4 x[n-3] + 5 x[n-4] (ii) y[n] = 4 x[n] - 0.5 y[n-1] Difference eqn (i) will produce a finite impulse-response. Difference eqn (ii) produces infinite response whose samples gradually reduce in amplitude but never quite become zero. Exercise 3.1 Calculate impulse-responses, by tabulation, for:

21. 16 Oct'09Comp30291 Section 321 nx[n] x[n-1] x[n-2] x[n-3] x[n-4] y[n] -1 0 0 0 0 0 0 010000 1 101000 2 200100 3 300010 -4 400001 5 500000 0 :::::: : Impulse-response for example (i) by tabulation Impulse response is: {.. 0,.., 0, 1, 2, 3, -4, 5, 0,.., 0,...} y[n] = x[n] + 2 x[n-1] + 3 x[n-2] - 4 x[n-3] + 5 x[n-4]

22. 16 Oct'09Comp30291 Section 322 nx[n]y[n-1] y[n] 010 4 104 -2 20-2 1 301 -0.5 40-0.5 0.25 500.25 -0.125 ::: : Impulse-response by tabulation for Example (ii) Impulse response is: {.., 0,.., 0, 4, -2, 1, -0.5, 0.25, -0.125,...) y[n] = 4 x[n] - 0.5 y[n-1]

23. 16 Oct'09Comp30291 Section 323 nx[n]y[n-1] y[n] 010 4 104 -8 20-8 16 3016 -32 40-32 64 50 64 128 ::: : Further example: Impulse-response by tabulation Impulse response is: {.., 0,.., 0, 8, 16, -32, 64, 128,...) This IIR filter is ‘unstable’ (see later) y[n] = 4 x[n] - 2 y[n-1]

24. 16 Oct'09Comp30291 Section 324 Difference equation (i) is "non-recursive" & produces a finite impulse response (FIR). Difference equation (ii) is " recursive ". Impulse-response of a recursive difference equation can have an infinite number of non-zero terms. In this case it is an infinite impulse-response (IIR). 3.5. FIR & IIR digital filters A ‘digital filter’ is a digitally implemented LTI discrete time system governed by a difference equation of finite order; e.g. : (i) y[n] = x[n] + 2 x[n-1] + 3 x[n-2] - 4 x[n-3] + 5 x[n-4] (ii) y[n] = 4 x[n] - 0.5 y[n-1]

25. 16 Oct'09Comp30291 Section 325 clear all; Y2=0; while 1 X1 = input( 'X1 = '); Y1= 4*X1 - 0.5*Y2 ; Y2 = Y1; % for next time round disp([' Y1 = ' num2str(Y)]); end; MATLAB program for recursive signal flow graph - example (ii)

26. 16 Oct'09Comp30291 Section 326 y = filter(A, B, x) filters signal in array x to create array y. For FIR example (i), A = [ 1 2 3 -4 5 ] & B = [1]. For IIR example (ii), A = [4], B = [1 0.5] Consider a third IIR example: y[n] = 2x[n] + 3x[n-1] + 4x[n-2] -0.5 y[n-1] - 0.25 y[n-2] In this case set A = [2 3 4] and B = [1 0.5 0.25]. Why are A & B defined in this way? Use of ‘filter’ for FIR & IIR digital filters

27. 16 Oct'09Comp30291 Section 327 A digital filter has a ‘system function’ which is with b 0 = 1 for the difference equation: A contains [a 0 a 1... a N ] & B contains [b 0, b 1,..., b M ]. Reasons for this & more details will be given later in course. ‘filter’ can be used without knowing why H(z) is defined in this way. Definition of A & B in ‘y = filter(A,B,x);’

28. 16 Oct'09Comp30291 Section 328 If impulse-response of an LTI system is {h[n]} its response to any input {x[n]} is an output {y[n] } whose samples are given by the following ‘convolution’ formulae: 3.6. Discrete time convolution Formulae are equivalent. Clearly, if we know impulse-response {h[n]} we can produce the response to any other input sequence, from either of these formulae. Proof of convolution in Appendix 3A.

29. 16 Oct'09Comp30291 Section 329 Calculate response of a system with impulse response: {h[n]} = {..., 0,..., 0, 1, 2, 3, -4, 5, 0,.....0,.... } to {x[n]} = {... 0,..., 0, 1, 2, 3, 0,..., 0,....} Solution: By first discrete time convolution formula, Example This is difference equation for an LTI system with impulse response {.., 0,.., 0, 1, 2, 3, -4, 5, 0,.., 0,..}

30. 16 Oct'09Comp30291 Section 330 Completing the Example Program discussed earlier implements this difference equation, We could run it, and enter {0 1 2 3 0 0 0 0...}, Output sequence produced is what we want. Alternatively, we could use tabulation as follows:

31. 16 Oct'09Comp30291 Section 331 Response of: y[n] = x[n] +2x[n-1]+3x[n-2]-4x[n-3] +5x[n-4] to input sequence {...,0,1,2,3,0,...} by tabulation n x[n] x[n-1] x[n-2] x[n-3] x[n-4] y[n] : : :: :: : -1 0 00 00 0 0 1 00 00 1 1 2 10 00 4 2 3 21 00 10 3 0 32 10 8 4 0 03 21 6 5 0 00 32 -2 6 0 00 03 15 7 0 00 00 0 : : :: :: :  {y[n]} = {.... 0,....., 0, 1, 4, 10, 8, 6, -2, 15, 0,...., 0,....}

32. 16 Oct'09Comp30291 Section 332 3.7. Stability An LTI system is stable if its impulse-response {h[n]} satisfies: This means that {h[n]} must be either an FIR or an IIR whose samples decay towards zero as n  .

33. 16 Oct'09Comp30291 Section 333 3.8. Causality An LTI system operating in real time must be ‘causal’ which means that its impulse-response {h[n]} must satisfy: h[n] = 0 for n < 0. Non-causal system would need “crystal ball ” to predict future.

34. 16 Oct'09Comp30291 Section 334 Illustration of stability & causality Looks stable but is not causal. n h[n] n Causal, but not stable h[n] Causal & looks stable. n

35. 16 Oct'09Comp30291 Section 335  =  T is ‘relative frequency’ of sampled sinusoid. Units of  are 'radians per sample'. 3.9. Relative Frequency Study effect of digital filters on sinusoids of different frequencies. Discrete time sinusoid obtained by sampling A cos(  t +  ). If sampling frequency is Fs Hz, and T=1/Fs, we obtain: x[n] = A cos(  nT +  ) = A cos(  n +  )

36. 16 Oct'09Comp30291 Section 336 Radians per sample To convert  back to true frequency (radians/s ) multiply by Fs. radians / sample  samples / second = radians / second Analogue signals in range 0 to F S /2 Hz =1/(2T) Restricts  to the range 0 to .

37. 16 Oct'09Comp30291 Section 337 T 3T -T -3T4T -4T cos(  t ) 1 3 -34 -4 cos(  T n ) t n  = 2  / 8  =2  / (8T)

38. 16 Oct'09Comp30291 Section 338 Values of  & corresponding true frequencies Relative frequency  True frequency (radians/sample) (radians/s) (Hz) 0 0 0  /6  f S /6 f S /12  /4  f S /4 f s /8  /3  f S /3 f s /6  /2  f S /2 f s /4 2  /3 2  f S /3 f s /3   f S f S /2 ‘radians / sample’  ‘samples / second’ = ‘radians / second’

39. 16 Oct'09Comp30291 Section 339 3.10. Relative frequency response It us useful to analyse the response to a sampled sinusoid: x[n] = Acos(  n +  ) To begin with, set A=1,  =0 and remember de Moivre’s Theorem: Easier to calculate response to the complex signal x[n] = e j  n than to cos(  n) directly.

40. 16 Oct'09Comp30291 Section 340 Relative frequency response (cont) If x[n] = e j  n is applied to a system with impulse-response {h[n]}, output would be, by convolution :

41. 16 Oct'09Comp30291 Section 341 H( e j  ) is the DTFT of { h[n] }. It is called the ‘relative frequency-response’ Complex number for any value of . Note similarity to the analogue Fourier transform. Discrete time Fourier Transform (DTFT)

42. 16 Oct'09Comp30291 Section 342 If input is x[n]=e j  n, output is same sequence with each element multiplied by H( e j  ). e j  n {h[n]} e j  n H(e j  ) LTI Recap

43. 16 Oct'09Comp30291 Section 343 3.11. Gain & phase responses G(  ) = |H( e j  )| is ‘gain’  (  ) = arg ( H( e j  ) ) is ‘phase lead’. Both vary with . Can express: H( e j  ) = G(  ) e j  (  ) If input is {A cos(  n)}, output is: { G(  )A cos(  n +  (  )) } When input is sampled sinusoid of relative frequency , output is sinusoid of same frequency , but with amplitude scaled by G(  ) & phase increased by  (  ).

44. 16 Oct'09Comp30291 Section 344 Gain & phase response graphs again   /4  /23  /4 0 20log 10 [G(  )] dB -()-()  -()-() G(  ) in dB

45. 16 Oct'09Comp30291 Section 345 Graphs of G(  ) &  (  ) against . G(  ) often converted to dBs by calculating 20 log 10 ( G(  ) ). Restrict  to lie in range 0 to  Adopt a linear horizontal frequency scale.

46. 16 Oct'09Comp30291 Section 346 Example Derive frequency-response of FIR digital filter below. Impulse-response is: {.., 0,.., 0, 1, 2, 3, -4, 5,0,.., 0,..}. By the formula established above, H( e j  ) = 1 + 2 e - j  + 3 e -2 j  - 4 e -3 j  + 5 e - 4 j 

47. 16 Oct'09Comp30291 Section 347 Example: Plot gain & phase responses of this FIR filter To do this ‘by hand’, we would need to take modulus & phase of expression for H( e j  ). Fortunately we almost never need to do this ‘by hand’ (except for very simple expressions). Best done by MATLAB. To calculate gain & phase responses for H( e j  ) = 1 + 2 e - j  + 3 e -2 j  - 4 e -3 j  + 5 e - 4 j  start MATLAB and type: freqz( [1 2 3 -4 5] );

48. 16 Oct'09Comp30291 Section 348

49. 16 Oct'09Comp30291 Section 349 ‘freqz’ graphs of gain & phase responses Frequency scale normalised to f S /2 & labelled 0 to 1 instead of 0 to . To plot freq-response for an IIR filter equally straightforward using ‘freqz’ if you know how to define the system function. See earlier. More about this in a later section.

50. 16 Oct'09Comp30291 Section 350 In next section we see how FIR filter coefficients can be chosen to achieve a particular type of gain & phase response. Tells us how the MATLAB function ‘fir1’ works in principle. In later sections, principles of IIR digital filter design will be considered also FIR & IIR digital filter design

51. 16 Oct'09Comp30291 Section 351 Phase delay Assume input is {A cos(  n)}, Output is: { G(  ) A cos (  n +  (  ) ) } = { G(  ) A cos (  [ n +  (  ) /  ] ) } = { G(  ) A cos (  [ n - (-  (  ) /  ) ] ) } Phase lead  (  ) delays sine-wave by -  (  )/  sampling intervals This is ‘phase-delay’.

52. 16 Oct'09Comp30291 Section 352 3.12. Linear phase response If -  (  )/  remains constant for all values of , i.e. if -  (  ) = k  for constant k, system has ‘linear phase’ response. Graph of -  (  ) against  on linear scale would then be straight line with slope k where k is "phase delay" i.e. the delay measured in sampling intervals. This need not be an integer.

53. 16 Oct'09Comp30291 Section 353 -  (  )/    /4  /23  /4 0  -()-() G(  ) in dB -  (  )/  Gain, phase & ‘phase-delay’ response graphs Not ‘linear-phase’.

54. 16 Oct'09Comp30291 Section 354 -  (  )/    /4  /23  /4 0  -()-() G(  ) in dB -  (  )/  Gain, phase & ‘phase-delay’ response graphs again This is ‘linear-phase’ as -  (  )/  is constant. k

55. 16 Oct'09Comp30291 Section 355 Why is linear phase a good property? All sinusoids delayed by same number of sampling intervals. Input signal expressed as Fourier series will have all its sinusoidal components delayed by same amount of time. No ‘phase distortion’ due to different delays at different frequencies. LTI systems are not necessarily linear phase. A certain class of digital filter can be made linear phase.

56. 16 Oct'09Comp30291 Section 356 3.13. Inverse DTFT Frequency-response H( e j  ) is DTFT of {h[n]}. Inverse DTFT formula allows {h[n]} to be deduced from H(e j  ) : This formula requires H(e j  ) for  in range -  to . Negative frequencies?? Not a problem because when { h[n] } is real, H( e -j  ) = H*( e j  ) where * denotes complex conj.

57. 16 Oct'09Comp30291 Section 357 Similarities with inverse analogue Fourier Transform Notice: (i) (1/2  ) factor, (ii) sign of j  n (n replaces t). (iii) variable of integration (d  ) Inverse DTFT Inverse FT

58. 16 Oct'09Comp30291 Section 358 Also note that, with inverse DTFT Range of integration is now -  to  rather than -  to . DTFT is a summation & inverse DTFT is an integral. This is because {h[n]} is a sequence whereas H(e j  ) is function of the continuous variable .

59. 16 Oct'09Comp30291 Section 359 3.14 Problems 1. Why is y[n]=(x[n]) 2 not LTI. 2. If {h[n]}= {.. 0,.., 0, 1, -1, 0,.. 0,.. }, calculate response to {x[n]} = {.... 0,.., 0, 1, 2, 3, 4, 0,.., 0,... }. 3.. Produce a signal flow graph for each of the following difference equations: (i) y[n] = x[n] + x[n-1] (ii) y[n] = x[n] - y[n-1] (iii) y[n] = 2x[n] +3x[n-1] +2x[n-2] - 0.5 y[n-1] -0.25 y[n-2] 4. For difference equations (i) & (ii) in question 3, determine the impulse- response, & deduce whether the system it represents is stable and/or causal. 5. Calculate, by tabulation, the output from difference equation (ii) in question 3 when the input is the impulse-response of difference equation (i). 6. If f S =8000 Hz, what true frequency corresponds to  /5 radians/sample? 7. Sketch the gain & phase responses of the system referred to in question 2. (You don't need a computer program.) Is it a linear phase system? 8. Calculate the impulse-response for y[n] = 4x[n] + 2y[n-1]. Is it stable? 9. Show that input {cos(  n)} produces an output {G(  )cos(  n+  (  ))}. 10. For y[n]=x[n]+2x[n-1]+3x[n-2]+2x[n-3]+x[n-4], sketch phase-response and comment on its significance. Show that  (  ) = -2 .

60. 16 Oct'09Comp30291 Section 360 Appendix 3A: Proof of discrete time convolution formulae Only use linearity, time-invariance & impulse-response. Since d[n-m] is non-zero only at n = m, given any sequence {x[n]}, {x[n]} is sum of infinite number of delayed impulses{d[n-m]} each multiplied by a single element, x[m]. Response to {d[n-m]} is {h[n-m]} for any value of m.  response to {x[n]} is :

61. 16 Oct'09Comp30291 Section 361 Replacing n-m by k gives the alternative formula. Study the graphical explanation of this proof in Section 3.17. n 2 3 x[n] 2 h[n] n {h[n]} = {..., 0,..., 0, 1, 2, 1, 2, 0,..., 0...} {x[n]} = {..., 0,..., 0, 1, 2, 3, 0, 0,..., 0,...}

62. 16 Oct'09Comp30291 Section 362 Discrete time convolution graphically n 2 3 x[n] d[n] n 2 2d[n-1] n 3 3d[n-2] n 2 h[n] n 2 4 2h[n-1] n 3 6 3h[n-2] n {h[n]} = {..., 0,..., 0, 1, 2, 1, 2, 0,..., 0,...} {x[n]} = {..., 0,..., 0, 1, 2, 3, 0, 0,..., 0,...}

63. 16 Oct'09Comp30291 Section 363 2 4 6 8 10 y[n] n Express: {x[n]} = {d[n]} + 2 {d[n-1]} + 3 {d[n-2]} Response is: {y[n]} = {h[n]} + 2 {h[n-1} + 3 {h[n-2]}

64. 16 Oct'09Comp30291 Section 364 2 4 6 8 10 y[n] n n 2 3 x[n] { d[n] }{h[n]} + 2{ d[n-1 ]} + 2 { h[n-1]} + 3 { d[n-2] } + 3 { h[n-2] } {h[n]} = {..., 0,..., 0, 1, 2, 1, 2, 0,..., 0,...} {x[n]} = {..., 0,..., 0, 1, 2, 3, 0, 0,..., 0,...}

65. 16 Oct'09Comp30291 Section 365 d[n] n 2 2d[n-1] n 3 3d[n-2] n 2 4 2h[n-1] n 3 6 3h[n-2] n 2 h[n] n