1. How do I know the answer if I’m not sure of the question?
Putting robustness into estimation
K. E. Schubert
11/7/00

2. Familiar Picture?

3. Basic Problem Picture of something that has been blurredx b A
Picture of something that has been blurred
If I know how it was blurred then I should be able to clean it up
If system is invertible then I can get the original x b A†

4. Familiar Picture

5. Encountering Resistance
Consider a simpler problem.
Unknown resistor.
Take current and voltage measurements.
Plot them out.
Want to fit a line to the points.
No measurement is perfect.
No exact fit to all the points.
Want “best” fit.

6. Measured Values

7. Gauss’ Stellar Problem
Orbit of Ceres.
Errors were in people’s measurements
Consider distance from the measurements to the equation to fit
minimize the square of this distance
min ||Ax-b||2
x=(ATA)-1ATb=A†b

8. Understanding Solution
In our problem A, b are vectors
Finding nearest scaled A to b
Projection b A Ax
Ax-b

9. Resistor Solved Want to find slope, 1/R i=(1/R)v Ax=b
A vector of voltages
b vector of currents
x is slope
1/R=v†i

10. Best line

11. Reasonable Question What if I considered v=iR?
Errors assumed in v now!
R=i †v
How do the measured resistances compare?

12. Comparison of Methods

13. Errors in Both A has errors (actual is A+dA) Want to minimize distance
min ||(A+dA)x-b||2
Need to know something about dA
Worst dA in bounded region
Best dA in bounded region
The dA that makes Ax=b consistent

14. Worst in a Bounded Region
Keep worst case ok, rest will be fine
||dA||< (bounded region)
Projection to farthest A+dA
(A+dA)x-b b dA A
(A+dA)x

15. Best in a Bounded Region
Pick best dA but limit options
||dA||< (bounded region)
Projection to nearest A+dA
(A+dA)x-b b dA A
(A+dA)x

16. Consistent Equation (TLS)
Called Total Least Squares
Projection nearest to A and b in new space
No bound on dA, as big as need! b A
(A+dA)x

17. General Regression Problems
All of the techniques mentioned so far fall into the general category of regression (including least squares)
Find a solution for most by taking the gradient and setting it equal to zero
x=(ATA+I)-1ATb
Equation for , which is solved by finding the roots of the equation (Newton’s or bisection)

18. Resistor by TLS

19. Simple Picture Consider a city skyline. Nice one dimensional picture.
Only consider outline of buildings.
Height is a function of horizontal distance.
Nice one dimensional picture.

20. Hazy Day Smog and haze blur the image. Rounds the corners off.
Want to get the corners back.

21. Least Squares Fails! Blurring works like a Gaussian distribution
Don’t know the exact blur

22. TLS Too Optimistic! TLS assumes things are consistent
Allows dA to be large

23. More Robust Solutions
Picking a solution with some restrictions yields good results.

24. Conclusions
Least Squares has nice properties and generally works well.
Problems can arise in simple problems.
Fundamental errors
Must account for errors in basic system.
Robust ~ works well for all nearby systems
Can’t do as well or as bad (compromise)