Lecture 4: Practical Examples

Remember this? m est = m A + M [ d obs – Gm A ] where M = [G T C d -1 G + C m -1 ] -1 G T C d -1

It’s exactly the same as solving this equation Cd-½GCm-½Cd-½GCm-½ C d -½ d  C m -½ m A m = which has the form Fm=h by simple least-squares! m = [F T F] -1 F T h This form of the equation is usually easier to set up m = [F T F] -1 F T h

in the uncorrelated case, the equation simplifies to d-1Gm-1d-1Gm-1 d-1dm-1mA d-1dm-1mA m = each data equation weighted by the variance of that datum each prior equation weighted by the variance of that prior

Example 1 1D Interpolation

Find a function f(x) that 1) goes through all your data points (observations) 2) does something smooth inbetween (prior information) This is interpolation … but why not just use least-squares ?

m – a vector of all the points at which you want to estimate the function, including the points for which you have observations d – a vector of just those points where you have observations So the equation Gm=d is very simple, a model parameter equals the data when the corresponding observation is available: … 0 … … 0 … …mi……mi… …dj……dj… = Just a single “1” per row

You then implement a smoothness constraint by first developing a matrix D that computes the non- smoothness of m … 0 … … 0 … D = One possibility is to use the finite- difference approximation of the second derivative And by realizing that: maximizing smoothness is the same as minimizing |Dm| 2 and minimizing |Dm| 2 is the same as choosing C m -1  D T D (along with m A =0).

First derivative [dm/dx] i  (1/  x) m i – m i-1  m i – m i-1 Second derivative [d 2 m/dx 2 ] i  [dm/dx] i+1 - [dm/dx] i = m i+1 – m i – m i + m i-1 = m i+1 – 2m i + m i-1

So the F m = h equation is: G  D d  m =  is a damping parameter that represent the relative weight of the smoothness constraint, that is, how certain we are that the solution is smooth.

m = 10…000 00…010 ……………… 00…001 d1d1 d7…d7… dNdN -- 0000  -2  000 ……………… 00…   00…0 --

example 101 equally spaced along the x-axis So 101 values of the function f(x) 40 of these values measured (the data, d) the rest are unknown Two prior information minimize 2 nd derivative for interior 99 x’s minimize 1 st derivative at left and right x’s (nice to have the same number of priors as unknowns, but not required)

 = data result f(x) x

 can be chosen by trial and error but usually the result fairly insensitive to , as long as its small

 varied over six orders of magnitude log 10 (Total Error) log 10 (  )

A purist might say that this is not really interpolation, because the curve goes through the data only in the limit  0 but for small  ’ s the error is extremely small

Example 2 Reconstructing 2D data known to obey a differential equation  2 f = 0 e.g. f(x,y) could be temperature

21 unknowns 21  21=441 unknowns 44 observed data

Prior information:  2 f = d 2 f/dx 2 + d 2 f/dy 2 = 0 in interior of the box n  f = 0 on edges of box (sides of box are insulating)

The biggest issue here is bookkeeping Conceptually, the model parameters are on a n  m grid m ij But they have to be reorganized into a vector m k to do the calculations m 11 m 12 m 13 …m 1n m 21 m 22 m 23 …m 2n m 31 m 32 m 33 …m 3n … m m1 m m2 m m3 …m mn m1m1 m2m2 m3m3 … mnmmnm e.g. m ij  m k with k=(i-1)*m+j Thus a large percentage of the code is concerned with converting back and forth between positions in the grid and positions in the corresponding vector. It can look pretty messy!

results

comparison

Example 3 Linear Systems

Scenario 1: no past history needed Flame with time-varying heat h(t) Thermometer measuring temperature  (t) Flame instantaneously heats the thermometer Thermometer retains no heat  (t)  h(t)

Scenario 2: past history needed Flame with time-varying heat h(t) Thermometer measuring temperature  (t) Heats takes time to seep through plate Plate retains heat  (t=t’)  history of h(t) for time t<t’ Steel plate

How to write a Linear System  (t)  history of h(t’) for all times in the past  (t 0 ) = … + g 0 h(t 0 ) + g 1 h(t -1 ) + g 2 h(t -2 ) + g 3 h(t -3 ) + g 4 h(t -4 ) + …  (t 1 ) = … + g 0 h(t 1 ) + g 1 h(t 0 ) + g 2 h(t -1 ) + g 3 h(t -2 ) + g 4 h(t -3 ) + … g is called the “impulse response” of the system

Matrix formulations 01…N01…N h0h1…hNh0h1…hN g g 1 g … g N … g 3 g 2 g 1 g 0 = Note problem with parts of the equation being “off the ends” of the matrix 01…N01…N g0g1…gNg0g1…gN h h 1 h … h N … h 3 h 2 h 1 h 0 =

This formulation might be especially useful when we know  and g and want to find h 01…N01…N h0h1…hNh0h1…hN g g 1 g … g N … g 3 g 2 g 1 g 0 =  = G h

01…N01…N g0g1…gNg0g1…gN h h 1 h … h N … h 3 h 2 h 1 h 0 =  = H g This formulation might be especially useful when we know  and  h and and want to find g

Thermometer measuring plate temperature  Goal: infer “physics” of plate, as embodied in its impulse response function, g plate Thermometer measuring flame heat h

g(t) h true (t)  true (t) Set up of problem

 obs (t)=  true (t)+noise h obs (t)=h true (t)+noise Simulate noisy data

Results g true (t) and g est (t) … yuck!

fix-up try for shorter g(t) and use 2 nd derivative damping Damping:  2 =100

Example 4 prediction error filter how well does the past predict the present?

 5 = g 1  4 + g 2  3 + g 3  2 + g 4  1 …  6 = g 1  5 + g 2  4 + g 3  3 + g 4  2 …  7 = g 1  6 + g 2  5 + g 3  4 + g 4  3 …  = g 0  5 + g 1  4 + g 2  3 + g 3  2 + g 4  1 …  = g 0  6 + g 1  5 + g 2  4 + g 3  3 + g 4  2 …  = g 0  7 + g 1  6 + g 2  5 + g 3  4 + g 4  3 … with g 0 = -1 Solve  g=0 by least squares with prior information g 0 =-1 matrix of  ’s use large damping

20 years of Laguardia Airport Temperatures, filter length M = 10 days gg 

filter length M = 10 days g

filter length M = 100 days gg 

g

 * g is the unpredictable part of 

Let’s try it with the Neuse River Hydrograph Dataset Filter length M=100 What’s that?  gg g

Close up of first year of data gg  Note that the prediction error,  * g, is spikier than the hydrograph data, . I think that this means that some of the dynamics of the river flow is being captured by the filter, g, and that the unpredictable part is mostly the forcing, that is, precipitation

Example 4 Tomography

Tomography: reconstructing an image from measurements made along rays CAT scan: density image, reconstructed from X-ray absorption Seismic Tomography: velocity image, reconstructed from seismic ray travel times MRI : proton density image, reconstructed from radio wave emission intensity along lines of constant precession frequency

source receiver ray data ray i =  ray i model(x,y) dL arc length

source receiver ray data ray i =  voxel j model j  L ij arc length of ray i in voxel j Discretize image into pixels or voxels

So the data kernel, G, is very simple …………… …………… ……  L ij …… …………… …………… Arc length of ray i in voxel j G =

Many elements will be zero …………… …………… ……  …… …………… …………… ray i does not go through voxel j G =

the hard parts are: 1. computing the ray paths, if they are more complicated than straight lines 2. book-keeping, e.g. figuring out which rays pass through which voxels

Sample seismic tomography problem here’s the true model, m true sources and receivers Note: for the equation Gm=d to be linear, m must be 1/velocity or “slownes”

Straight line ray paths

The true traveltime data, d true

In the previous plot, each ray is indexed by its closest distance to the origin, R, and it orientation,  ray R  R  Each ray makes plots as one point on the image, with its travel time indicated by its color

true model, d true estimated model, d est (solution via damped least squares)

true model, d true estimated model, d est After doubling the station/receiver density …