1. Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert

2. Simple Problem l Consider the problem ax=b l l The resulting x value is l

3. Simple Problem 2 l Consider the problem ax=b l l The resulting x value is l

4. What’s Up? l The condition number (sensitivity to perturbations) is about 400. l A condition number of 1 is perfect. l Perturbation is 0.01, so 0.01*400=4. l Components of x can vary by this much!

5. What Can We Do? l Rather than solve it the standard way X=a\b X=(A T A) -1 a t b l Consider the following: X=(A T A+  i) -1 a t b  =.01 l Then:

6. Lucky Guess?

7. Does It Always Work? l No l Consider  X  0 Consider  i 2 (  i is singular value of A) X  ±  l Picking the wrong value can get junk

8. Skyline l Consider a 1 dimensional picture l Use height instead of color l Result looks like the silhouette of a city’s skyline l Have smog which blurs and softens l Don’t know exactly how much blur l Want to get sharp edges

9. Getting Garbage

10. Getting Improvement

11. Why Backward? l Forward errors Explicitly account for each error source (X+  1 )(y+  2 )=xy+(y  1 +x  2 +  1  2 ) l Backward errors Check that my algorithm acting on data will give me a solution that is “near” to the actual system acting on a nearby set of data I.E. My algorithm with good data should do about as well as a perfect calculation on ok data

12. Picture Please! Actual Data (x) Nearby Data (x*) Perfect Calculations My Algorithm Inherent errors in A b b* b est Errors due to algorithm

13. Least Squares l Usually we don’t have an invertible matrix l Need to find an estimated solution l Criterion: minimize ||ax-b|| l Normal equation A T A x = A T b l Solution X = (A T A) -1 a t b

14. Backward Error l Criterion: minimize ||Ax-b||/(||A|| ||x||+||b||) l Normal Equations l Solution:

15. Non Convex

16. Finding The Root

17. Look At Critical Region

18. Informal Algorithm l Get (A,b) l svd(A)  [u 1 u 2 ], ,v l U 1 b  b 1 Use rootfinder (bisection, Newton, etc.) to get  in [-  n 2,0] l v T (  2 -  I) -1  b 1  x

19. What You Get

20. Least Squares

21. Total Least Squares

22. Tikhonov

23. Backward Error

24. Original

25. Comparison

26. Final Thoughts l BE is always optimistic in that it presumes that the real system is “better” l Even with this it is “robust” l There is a perturbed version of this algorithm which can be either optimistic or pessimistic l That version is not fully proven