How do I know the answer if I’m not sure of the question? Putting robustness into estimation K. E. Schubert 11/7/00

Familiar Picture?

Basic Problem Picture of something that has been blurred x b A Picture of something that has been blurred If I know how it was blurred then I should be able to clean it up If system is invertible then I can get the original x b A†

Familiar Picture

Encountering Resistance Consider a simpler problem. Unknown resistor. Take current and voltage measurements. Plot them out. Want to fit a line to the points. No measurement is perfect. No exact fit to all the points. Want “best” fit.

Measured Values

Gauss’ Stellar Problem Orbit of Ceres. Errors were in people’s measurements Consider distance from the measurements to the equation to fit minimize the square of this distance min ||Ax-b||2 x=(ATA)-1ATb=A†b

Understanding Solution In our problem A, b are vectors Finding nearest scaled A to b Projection b A Ax Ax-b

Resistor Solved Want to find slope, 1/R i=(1/R)v Ax=b A vector of voltages b vector of currents x is slope 1/R=v†i

Best line

Reasonable Question What if I considered v=iR? Errors assumed in v now! R=i †v How do the measured resistances compare?

Comparison of Methods

Errors in Both A has errors (actual is A+dA) Want to minimize distance min ||(A+dA)x-b||2 Need to know something about dA Worst dA in bounded region Best dA in bounded region The dA that makes Ax=b consistent

Worst in a Bounded Region Keep worst case ok, rest will be fine ||dA||< (bounded region) Projection to farthest A+dA (A+dA)x-b b dA A (A+dA)x

Best in a Bounded Region Pick best dA but limit options ||dA||< (bounded region) Projection to nearest A+dA (A+dA)x-b b dA A (A+dA)x

Consistent Equation (TLS) Called Total Least Squares Projection nearest to A and b in new space No bound on dA, as big as need! b A (A+dA)x

General Regression Problems All of the techniques mentioned so far fall into the general category of regression (including least squares) Find a solution for most by taking the gradient and setting it equal to zero x=(ATA+I)-1ATb Equation for , which is solved by finding the roots of the equation (Newton’s or bisection)

Resistor by TLS

Simple Picture Consider a city skyline. Nice one dimensional picture. Only consider outline of buildings. Height is a function of horizontal distance. Nice one dimensional picture.

Hazy Day Smog and haze blur the image. Rounds the corners off. Want to get the corners back.

Least Squares Fails! Blurring works like a Gaussian distribution Don’t know the exact blur

TLS Too Optimistic! TLS assumes things are consistent Allows dA to be large

More Robust Solutions Picking a solution with some restrictions yields good results.

Conclusions Least Squares has nice properties and generally works well. Problems can arise in simple problems. Fundamental errors Must account for errors in basic system. Robust ~ works well for all nearby systems Can’t do as well or as bad (compromise)