Confessions of a Numerical Analyst Keith Evan Schubert Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert

Simple Problem Consider the problem ax=b The resulting x value is

Simple Problem 2 Consider the problem ax=b The resulting x value is

What’s Up? The condition number (sensitivity to perturbations) is about 400. A condition number of 1 is perfect. Perturbation is 0.01, so 0.01*400=4. Components of x can vary by this much!

What Can We Do? Rather than solve it the standard way X=a\b X=(ATA)-1atb Consider the following: X=(ATA+i)-1atb  =.01 Then:

Lucky Guess?

Does It Always Work? No Consider  X0 Consider si2 (si is singular value of A) X± Picking the wrong value can get junk

Skyline Consider a 1 dimensional picture Use height instead of color Result looks like the silhouette of a city’s skyline Have smog which blurs and softens Don’t know exactly how much blur Want to get sharp edges

Getting Garbage

Getting Improvement

Why Backward? Forward errors Backward errors Explicitly account for each error source (X+d1)(y+d2)=xy+(yd1+xd2+d1d2) Backward errors Check that my algorithm acting on data will give me a solution that is “near” to the actual system acting on a nearby set of data I.E. My algorithm with good data should do about as well as a perfect calculation on ok data

Picture Please! Inherent errors in A b Perfect Calculations b* Errors due to algorithm best My Algorithm Actual Data (x) Nearby Data (x*)

Least Squares Usually we don’t have an invertible matrix Need to find an estimated solution Criterion: minimize ||ax-b|| Normal equation ATA x = ATb Solution X = (ATA)-1atb

Backward Error Criterion: minimize ||Ax-b||/(||A|| ||x||+||b||) Normal Equations Solution:

Non Convex

Finding The Root

Look At Critical Region

Informal Algorithm Get (A,b) svd(A)  [u1 u2],,v U1b  b1 Use rootfinder (bisection, Newton, etc.) to get  in [-sn2,0] vT(2- I)-1  b1  x

What You Get

Least Squares

Total Least Squares

Tikhonov

Backward Error

Original

Comparison

Final Thoughts BE is always optimistic in that it presumes that the real system is “better” Even with this it is “robust” There is a perturbed version of this algorithm which can be either optimistic or pessimistic That version is not fully proven