1. Lecture 11 Linear Systems

2. Past History Matters the purpose of this lecture is to give you tools that provide a way of relating what happened in the past to what’s happening today

3. Example: no past history needed Flame with time-varying heat h(t) Thermometer measuring temperature  (t) Flame instantaneously heats the thermometer Thermometer retains no heat  (t)  h(t)

4. Example: past history needed Flame with time-varying heat h(t) Thermometer measuring temperature  (t) Heats takes time to seep through plate Plate retains heat  (t=t’)  history of h(t) for time t<t’ Steel plate

5. Linear System  (t)  history of h(t) for all times, past and future linear in the sense that doubling h(t) doubles  (t) if h 1 (t) causes  1 (t) then 2h 1 (t) causes 2  1 (t) and also in the sense of additivity if h 1 (t) causes  1 (t) and if h 2 (t) causes  2 (t) then h 1 (t)+h 2 (t) causes  1 (t)+  2 (t)

6. Special Case: Causal Linear System  (t)  history of h(t=t’) for all past times, t’<t When the variable “t” actually represents “time”, then most linear systems are causal But when the independent variable represents something else (e.g. spatial position) then the linear system probably obeys a “non-causal” rule,  (x)  h(x=x’) for all positions x’ both to the left & right of x

7. Special Case: Time-Shift Invariant Linear System  (t)  history of h(t’), but only time difference (t-t’) matter The temperature depends only on the time elapsed since the flame was turned on, and not on the whether I performed it on Monday or Wednesday. Same shape (Might not be true if combustion depended on barometric pressure that changed from day-today)  (t)

8. How to write a Linear System  (t)  history of h(t’) for all times, past and future  (t 1 ) = … + g 1,-2 h(t -2 ) + g 1,-1 h(t -1 ) + g 1,0 h(t 0 ) + g 1,1 h(t 1 ) + g 1,2 h(t 2 ) + …  (t 2 ) = … + g 2,-2 h(t -2 ) + g 2,-1 h(t -1 ) + g 2,0 h(t 0 ) + g 2,1 h(t 1 ) + g 2,2 h(t 2 ) + … Doesn’t quite do it, since time is a continuous variable …

9. How to write a Linear System  (t)  history of h(t’) for all times, past and future  (t) =  -  +  g(t,t’) h(t’) dt’  (t 1 ) = … + g 1,-2 h(t -2 ) + g 1,-1 h(t -1 ) + g 1,0 h(t 0 ) + g 1,1 h(t 1 ) + g 1,2 h(t 2 ) + …

10. Special cases Causal  (t) =  -  t g(t,t’) h(t’) dt’ Time-shift invariant  (t) =  -  +  g(t-t’) h(t’) dt’ Causal and Time-shift invariant  (t) =  -  t g(t-t’) h(t’) dt’

11. Different ways to write the Time-shift invariant case  (t) =  -  t g(t-t’) h(t’) dt’ but rename t’ to   (t) =  -  t g(t-  ) h(  ) d  alternatively let  =t-t’ so d  =-dt’,  (t’=-  )= ,  (t’=t)=0  (t) =  0  g(  ) h(t-  ) d  Note symmetry of integrand

12. Interpretation of g(  ): Impulse Response Suppose h(t) is a impulse (spike) at time 0 t 0 h(t) Then the resulting  (t) is called the impulse response and denoted g(t): t 0  (t)=g(t)

13. Suppose h(t) is two impulses of different amplitude at two different times t t1t1 h(t) Then by additivity,  (t) is the sum of two time-shifted impulse responses of correspondingly scaled amplitudes t2t2 t  (t) t1t1 t2t2 t t1t1 t2t2 A  g(t-t 1 ) B  g(t-t 2 ) A  g(t-t 1 )+B  g(t-t 2 ) amplitude A amplitude B

14. An arbitrary function can be viewed as a sum of many impulses of different amplitudes occurring at a sequence of times H(t) Then by additivity,  (t) is the sum of all the corresponding responses:  (t)   t’<t g(t-t’) h(t’) tt’ amplitude h(t’) tt’  (t) =   t g(t-t’) h(t’) dt’ Same formula as before

15.  (t) =  -  t g(t-  ) h(  ) d  Or equivalently  (t) =  0  g(  ) h(t-  ) d  The h(  ) is “forward in time” The g(  ) is “forward in time” This form of integral is called a “convolution”

16. approximation of functions as time series h k =h[k  t] with k= … -2, -1, 0, 1, 2, … hkhk t tktk h(t) tt  (t) =  -  t g(t-  ) h(  ) d  k =  t  p=-  k g k-p h p or equivalently  (t) =  0  g(  ) h(t-  ) d  k =  t  p=o  g p h k-p

17. Matrix formulation  (t) =  -  t g(t-  ) h(  ) d  k =  t  p=-  k g k-p h p k=0:  0 =  t  p=-  0 g 0-p h p =  t{ … + g 0 h 0 } k=1:  1 =  t  p=-  1 g 1-p h p =  t{ … + g 1 h 0 + g 0 h 1 } k=2:  2 =  t  p=-  2 g 2-p h p =  t{… + g 2 h 0 + g 1 h 1 + g 0 h 2 } 01…N01…N h0h1…hNh0h1…hN g 0 0 0 0 0 0 g 1 g 0 0 0 0 0 … g N … g 3 g 2 g 1 g 0 =  t  Note problem with parts of the equation being “off the ends” of the matrix

18. Might be especially useful when we know g T k =  t  p=-  k g k-p H p 01…N01…N h0h1…hNh0h1…hN g 0 0 0 0 0 0 g 1 g 0 0 0 0 0 … g N … g 3 g 2 g 1 g 0 =  t   = G h

19. Alternate Matrix formulation  (t) =  0  g(  ) h(t-  ) d  k =  t  p=0  g p h k-p k=0:  0 =  t  p=0  g p h 0-p =  t{ g 0 h 0 + …} k=1:  1 =  t  p=0  g p h 1-p =  t{ g 0 h 1 + g 1 h 0 + …} k=2:  2 =  t  p=0  g p h 2-p =  t{ g 0 h 2 + g 1 h 1 + g 2 h 0 + … } 01…N01…N g0g1…gNg0g1…gN h 0 0 0 0 0 0 h 1 h 0 0 0 0 0 … h N … h 3 h 2 h 1 h 0 =  t  Note again the problem with parts of the equation being “off the ends” of the matrix

20.  k =  t  p=0  g p h k-p 01…N01…N g0g1…gNg0g1…gN h 0 0 0 0 0 0 h 1 h 0 0 0 0 0 … h N … h 3 h 2 h 1 h 0 =  t   = H g Might be especially useful when we know h

21. Returning to the flame problem Approximation: very conductive plate that heats very rapidly, but slowly cools down due to warming up the surrounding air d  /dt = -c  + h(t) Heat input from flame heat loss to air faster when plate hotter change in temperature of plate

22. Impulse response let h(t) =  (t-t’) d  /dt = -c  +  (t-t’) solution (take my word for it) unit impulse at time t’  (t) = g(t,t’) =  if t<t’ exp{-c(t-t’)} if t  t’

23. impulse response for c=1 and t’=2 t g(t,t’=2)

24. MatLab Code implements  =Gh N = 51; tmin = 0; tmax = 10; dt = (tmax-tmin)/(N-1); t = tmin+dt*[0:N-1]'; c=1; g = exp(-c*t); G=zeros(N,N); for p = [1:N] G(p:N,p) = dt*g(1:N-p+1); end f=0.2; h = 2+sin(2*pi*f*t); theta = G*h; define g(t) create matrix G setup time define h(t) compute  (t)

26. But suppose  and G are known  =Gh So h = G -1 

27. g(t) h true (t)  true (t)  obs (t)=  true (t)+noise h est (t) and h true (t)

28. another example in this case  t  and h(t) are known and g(t) is unknown

29. Thermometer measuring plate temperature  (t) Goal: infer “physics” of plate, as embodied in its impulse response function plateThermometer measuring flame heat h(t)

30.  =Hg So g = H -1 

31. g(t) h true (t)  true (t) Set up of problem

32.  obs (t)=  true (t)+noise h obs (t)=h true (t)+noise Simulate noisy data

33. Results g true (t) and g est (t) … yuck!

34. Fix-up try for shorter g(t) and use damped least-squares Damping:  2 =10

35. Another Fix-up try for shorter g(t) and use 2 nd derivative damping Damping:  2 =100