1. Lecture 22 Adjunct Methods

2. Part 1 Motivation

3. Motivating scenario We want to predict tomorrow’s weather, u(t) … We have a atmospheric model chugging away to predict temperature, pressure, etc. This model depends on a forcing f(t), for example, sea surface temperature, which we known only imperfectly

4. Yesterday’s model run t u(t) t yesterday f(t) tyty prediction t today But now we have new data for today t u(t) t yesterday f(t) tyty prediction t today t t new data

5. Today’s model run should include yesterday’s data to help constrain the poorly known forcing t u(t) t yesterday f(t) tyty old prediction t today t new data How do we adjust the forcing (which was imperfectly known, anyway) to better predict yesterday’s weather? new prediction old new

6. Part 2 The mathematics of continuous functions inner products linear operators and their adjoints

7. Discrete vectors u k and v k Continuous functions f(t) and g(t)

8. functions f(t) and g(t) Discrete approximation as a vectors u k =f(k  t) and v k =g(k  t)

9. Discrete dot product: c =  k u k v k = u  v Continuous inner product c =  f(t) g(t) dt = (f,g) scalar

10. discrete approximation as dot product (f,g) =  f(t) g(t) dt =  t  k u k v k =  t u  v with u k =f(k  t) and v k =g(k  t) inner product (f,g)

11. Discrete matrix: v j =  k M jk u k or v=Mu Continuous Linear operator f = Lg

12. What is a linear operator? Linear differential operator involving derivatives and known functions Lg = [ p(t) d/dt q(t) d/dt ] g(t) known

13. and/or Linear integral operator involving intergral and known functions Lg =  p(t,t’) g(t’) dt’ known

14. if L 1 g=f and L 2 f=g then L 1 =L 2 -1 and L 2 =L 1 -1 one linear operator is the inverse of the other

15. discrete approximations 1 0 0 0 … 0 -1 1 0 0 … 0 0 -1 1 0 … 0 …… … 0 0 -1 1 Lg=f with L = d/dt plus b.c. g(0)=known Lg=f with Lg =  0 t g(t’)dt’ plus b.b. g(0)=known Mu=v, M =  t -1 1 0 0 0 … 0 1 1 0 0 … 0 1 1 1 0 … 0 …… … 1 1 1 1 Mu=v, M =  t Sample differential operator plus b.c. Sample integral operator plus b.c.

16. Question concerning a dot product … given two matrices A and B when is (Au)  v = u  (Bv) ? Answer: when B=A T, since (Au)  v = (Au) T v = u T A T v = u T (A T v) = u  (A T v)

17. Question concerning an inner product … given two linear operators L 1 and L 2 when is ( L 1 f, g ) = ( f, L 2 g ) ? Answer: never mind, but let’s give it a name (L 1 f, g) = (f, L 2 g) when L 1 is the adjoint of L 2 let’s denote the adjoint relationship L 2 =L 1 * means “adjoint”

18. Transform if A=B T then B=A T A TT =A A T-1 =A -1T (A+B) T = A T +B T if A T =A then A is symmetric Adjoint if L 1 =L 2 * then L 2 =L 1 * L ** =L L *-1 =L -1* (L 1 +L 2 ) * = L 1 * +L 2 * if L * =L then L is self-adjoint

19. Calculating adjoints by integration by parts Let L = d/dt with b.c. zero at ±  (Lf, g) =  -  +  df/dt g dt = f g | -  +  -  -  +  f dg/dt dt = -  -  +  f dg/dt dt = (f, L*g) So L* = -d/dt with b.c. zero at ± 

20. Three simple adjoints L c(x) d/dt b.c.: function 0 at ±  d 2 /dt 2 b.c.: function and its first derivative 0 at ±  L* c(x) -d/dt b.c.: function 0 at ±  d 2 /dt 2 b.c.: function and its first derivative 0 at ± 

21. Part 3 Functional derivatives How to represent the idea that a perturbation in forcing, f(t) cause a perturbation in response, u(t)

22. Here’s the differential equation L u(t) = f(t) forcing Data d i linearly depends on u(t) through an inner product d i = (h i, u)

23. Differential Equation Lu=f A perturbation in f(t) causes a perturbation in u(t) f 0 (t)  f 0 (t)+  f(t) u 0 (t)  u 0 (t)+  u(t) Suppose  f(t) was localized at time t 0 :  f(t)=  (t-t 0 ) Then  u(t) is a function of  and t 0 :  u(t, ,t 0 ) Then the function (or Fréchet) derivative is:  u(t)/  f(t 0 ) = lim  0 [ u(t, ,t 0 ) – u(t, ,t 0 ) ] / 

24. An impulsive perturbation in forcing  (t-t 0 ) Causes a perturbation in response  u(t, ,t 0 ) Then the general perturbation  f in forcing causes the response  u =  (  u/  f)  f dt 0 = (  u/  f,  f )

25. t  (t-t 0 ) t0t0 uu t0t0 An impulsive perturbation in forcing Causes this response t ff t0t0 uu t0t0 A more complicated perturbation in forcing Causes this response defines  u/  a  u =  u/  a,  a)

26. Then the general perturbation  f in forcing causes the response  u =  (  u/  f)  f dt 0 = (  u/  f,  f ) In a discrete world:  u 1  u 2  u 3 =  t …  u N  u(t 1 )/  f(t 1 )  u(t 1 )/  f(t 2 )  u(t 1 )/  f(t 3 ) …  u(t 2 )/  f(t 1 )  u(t 2 )/  f(t 2 )  u(t 2 )/  f(t 3 ) …  u(t 3 )/  f(t 1 )  u(t 3 )/  f(t 2 )  u(t 3 )/  f(t 3 ) … …  u(t N )/  f(t 1 )  u(t N )/  f(t 2 )  u(t N )/  f(t 3 ) … f1f2f3…fNf1f2f3…fN Might solve with least-squares …

27. Part 4 Calculating the data kernel The functional derivative of data with respect to forcing

28. The Goal to find the data kernel, g i (t) which relates a perturbation in the data,  d i, to a perturbation in the forcing  f(t) through an inner product  d i = ( g i (t),  f(t) )

29. Note that since the data kernel satisfies it is a functional derivative g i (t) =  d i /  f(t)  d i = ( g i (t),  f(t) )

30. Step 1: assume that a function u(t) solves a linear differential equation with forcing f(t) L u(t) = f(t)

31. Step 2: assume the differential equation has green function F(t,t’) so the solution can be written: note that L -1 is the inverse of L, since f=Lu and u=L -1 f u(t) =  F(t,t’) f(t’) dt = (F(t,t’), f(t) )  L -1 f(t)

32. Step 3: assume that the data, d i, are related to the solution u(t) through an inner product d i = ( h i (t), u(t) )

33. Step 4: do some substitutions and manipulations d i = ( h i (t), u(t) ) = ( h i (t), L -1 f(t) ) = (L -1* h i (t), f(t) ) = (L *-1 h i (t), f(t) )

34. Step 4: since the problem is linear, this rule applies to perturbations of functions as well as to the functions themselves d i = (L *-1 h i (t), f(t) ) So  d i = (L *-1 h i (t),  f(t) )

35. Step 5: by comparing the definition of the data kernel  d i = ( g i (t),  f(t) ) to the result  d i = (L *-1 h i (t),  f(t) ) recognize that the data kernel is g i (t) = L *-1 h i (t)

36. Step 6: since the data kernel satisfies g i (t) = L *-1 h i (t) then it must satisfy the differential equation L * g i (t) = h i (t)

37. This is the desired results a way of calculating the data kernel, g i (t) by solving the differential equation L * g i (t) = h i (t)

38. Part 5 An example Note: In this example I use very simple differential equations that can be solved analytically. In a reality, you would be using much more complicated differential equations that but be solved numerically..

39. Example: Newtonian cooling equation du/dt + cu = f(t) L = d/dt + c u(t) is temperature f(t) is heating c is a constant

40. Green’s Function du/dt + cu =  (t-t’) F(t,t’) = H(t-t’) exp{ -c(t-t’) } unit step function

41. Adjoint differential equation L = d/dt + c The adjoint of d/dt is –d/dt and the adjoint of c is c So L* = -d/dt + c And so du/dt + cu = f(t) has corresponding adjoint equation -dg i /dt + cg i = h i

42. Greens Function of the Adjoint differential equation -dg i /dt + cg i =  (t-t’) has solution G(t,t’) = {1-H(t-t’)} exp{ c(t-t’) }

43. interpretation Suppose h i =  (t) so that the data d i is just u(t=0), temperature at time 0 Then G(t,t’=0) is the data kernel g i (t) Now suppose that we make an impulsive perturbation of heating at time t 0 :  f(t)=  (t-t 0 ) Then  d i = u(t=0) = ( g i (t),  f(t) ) = ( G(t,t’=0),  (t-t 0 ) ) = G(t 0,t’=0)

44. Interpretation, continued So for an impulsive perturbation of heating at time t 0  u(t=0) = G(t 0,t’=0) We would expect: no effect on temperature if heat applied after time t=0 large effect if applied just prior to t=0 minimal effect if it is applied way before t=1 No effect Large effectSmall effect

45. example H f0f0 u0u0 f u obs  u obs t

46. Forming data from u(t) Here I use an example of the data being averages of neighboring u’s d 1 = u(t 1 ) so h 1 = [1, 0, 0, 0, 0, 0 … 0] T d j = ½ { u(t j-1 ) + u(t j ) } for j>1 so h j = ½ [0, 0, 0, … 1, 1, … 0, 0, 0] T

47. d0d0  d obs t d obs

48. The problem Reconstruct  f from  d

49. Setup for Least Squares  d i = ( g i (t),  f(t) ) f1f2f3…fNf1f2f3…fN d1d2d3…dNd1d2d3…dN  g1g1 g2g2 g3g3 gNgN … time varies along columns …

50.  f true error t  f pre results

51. What about perturbations in the parameters of a differential equation Suppose L has a parameter a(t). Changing the parameter from a 0 (t) to a 0 (t)+  a(t) Changes the solution of Lu=f from u 0 (t) to u 0 (t)+  u(t)

52. approximation that makes perturbation in parameter act like a forcing L u = f with L = a(t) d/dt Suppose a(t) = a 0 (t) +  a(t) Then L = L 0 + L 1 = a 0 (t) d/dt +  a(t) d/dt write u(t) = u 0 (t) +  u(t) where u 0 (t) solves L 0 u 0 =f Lu=f  (L 0 + L 1 )(u 0 +  u) = f  (L 0 u 0 + L 0  u + L 1 u 0 + L 1  u ) = f  L 0  u = L 1 u 0 - L 1  u  L 0  u  - L 1 u 0

53. approximation that makes perturbation in parameter act like a forcing L u = f with L = a(t) d/dt Suppose a(t) = a 0 (t) +  a(t) Then L = L 0 + L 1 = a 0 (t) d/dt +  a(t) d/dt write u(t) = u 0 (t) +  u(t) where u 0 (t) solves L 0 u 0 =f Lu=f  (L 0 + L 1 )(u 0 +  u) = f  (L 0 u 0 + L 0  u + L 1 u 0 + L 1  u ) = f  L 0  u = L 1 u 0 - L 1  u  L 0  u  - L 1 u 0 acts as forcing