2. How do I know the answer if I’m not sure of the question? Putting robustness into estimation K. E. Schubert 11/7/00

3. Familiar Picture?

4. Basic Problem u Picture of something that has been blurred u If I know how it was blurred then I should be able to clean it up u If system is invertible then I can get the original x b A x b A†A†

5. Familiar Picture

6. Encountering Resistance u Consider a simpler problem. – Unknown resistor. – Take current and voltage measurements. – Plot them out. – Want to fit a line to the points. u No measurement is perfect. – No exact fit to all the points. – Want “best” fit.

7. Measured Values

8. Gauss’ Stellar Problem u Orbit of Ceres. u Errors were in people’s measurements u Consider distance from the measurements to the equation to fit u minimize the square of this distance – min || Ax-b || 2 u x=(A T A) -1 A T b=A † b

9. Understanding Solution u In our problem A, b are vectors u Finding nearest scaled A to b u Projection b A Ax Ax-b

10. Resistor Solved uWuWant to find slope, 1/R uiui=(1/R)v uAuAx=b uAuA vector of voltages ubub vector of currents uxux is slope u1u1/R=v † i

11. Best line

12. Reasonable Question u What if I considered v=iR? u Errors assumed in v now! u R=i †v†v u How do the measured resistances compare?

13. Comparison of Methods

14. Errors in Both u A has errors (actual is A+dA) u Want to minimize distance – min || (A+dA)x-b || 2 u Need to know something about dA u Worst dA in bounded region u Best dA in bounded region u The dA that makes Ax=b consistent

15. Worst in a Bounded Region u Keep worst case ok, rest will be fine u || dA || <  (bounded region) u Projection to farthest A+dA b A (A+dA)x (A+dA)x-b dA

16. Best in a Bounded Region u Pick best dA but limit options u || dA || <  (bounded region) u Projection to nearest A+dA b A (A+dA)x (A+dA)x-b dA

17. Consistent Equation (TLS) u Called Total Least Squares u Projection nearest to A and b in new space u No bound on dA, as big as need! b A (A+dA)x

18. General Regression Problems u All of the techniques mentioned so far fall into the general category of regression (including least squares) u Find a solution for most by taking the gradient and setting it equal to zero u x=(A T A+  I) -1 A T b u Equation for , which is solved by finding the roots of the equation (Newton’s or bisection)

19. Resistor by TLS

20. Simple Picture u Consider a city skyline. – Only consider outline of buildings. – Height is a function of horizontal distance. u Nice one dimensional picture.

21. Hazy Day u Smog and haze blur the image. – Rounds the corners off. – Want to get the corners back.

22. Least Squares Fails! u Blurring works like a Gaussian distribution u Don’t know the exact blur

23. TLS Too Optimistic! u TLS assumes things are consistent u Allows dA to be large

24. More Robust Solutions u Picking a solution with some restrictions yields good results.

25. Conclusions u Least Squares has nice properties and generally works well. u Problems can arise in simple problems. – Fundamental errors u Must account for errors in basic system. u Robust ~ works well for all nearby systems – Can’t do as well or as bad (compromise)