Computing ill-Conditioned Eigenvalues and Polynomial Roots Zhonggang Zeng Northeastern Illinois University International Conference on Matrix Theory and its Applications -- Shanghai

Can you solve (x-1.0 )100 = 0 Can you solve x100-100 x99 +4950 x98 - 161700 x97+3921225x96 - ... - 100 x +1 = 0

Eigenvalues of 1 0 1 1 ... ... 0 1 1 A = X X-1

The Wilkinson polynomial p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ... Wilkinson wrote in 1984: Speaking for myself I regard it as the most traumatic experience in my career as a numerical analyst.

Myths on multiple eigenvalues/roots: - multiple e’values/roots are ill-conditioned, or even intractable - extension of machine precision is necessary to calculate multiple roots - there is an “attainable precision” for multiple eigenvalues/roots: machine precision attainable precision = ----------------------------- multiplicity Example: for a 100-fold eigenvalue, to get 5 digits right 500 digits in machine precision 5 digits precision = ----------------------------------------- 100 in multiplicity

Conclusion: the problem is “bad” The forward error: 5 -- Ouch! Who’s responsible? The backward error: 5 x 10-10 -- method is good! Conclusion: the problem is “bad”

Who is asking a wrong question? What is the wrong question? If the answer is highly sensitive to perturbations, you have probably asked the wrong question. Maxims about numerical mathematics, computers, science and life, L. N. Trefethen. SIAM News A: “Customer” B: Numerical analyst Who is asking a wrong question? A: The polynomial or matrix B: The computing objective What is the wrong question?

Kahan’s pejorative manifolds All n-polynomials having certain multiplicity structure form a pejorative manifold xn + a1 xn-1+...+an-1 x + an <=> (a1 , ..., an-1 , an ) Example: ( x-t )2 = x2 + (-2t) x + t2 Pejorative manifold: a1= -2t a2= t2

Pejorative manifolds of 3-polynomials ( x - s )( x - t )2 = x3 + (-s-2t) x2 + (2st+t2) x + (-st2) a1= -s-2t a2= 2st+t2 a3= -st2 Pejorative manifold of multiplicity structure [1,2] ( x - s )3 = x3 + (-3s) x2 + (3s2) x + (-s3) a1 = -3s a2 = 3s2 a3 = -s3 Pejorative manifold of multiplicity structure [ 3 ]

Pejorative manifolds of 3-polynomials The wings: a1= -s-2t a2= 2st+t2 a3= -st2 The edge: a1 = -3s a2 = 3s2 a3 = -s3 General form of pejorative manifolds u = G(z)

Ill-condition is caused by solving polynomial W. Kahan, Conserving confluence curbs ill-condition, 1972 1. Ill-condition occurs when a polynomial/matrix is near a pejorative manifold. 2. A small “drift” of the problem on that pejorative manifold does not cause large forward error to the multiple roots, except 3. If a multiple root/eigenvalue is sensitive to small perturbation on the pejorative manifold, then the polynomial/matrix is near a pejorative submanifold of higher multiplicity. Ill-condition is caused by solving polynomial equations on a wrong manifold

Pejorative manifolds of 3-polynomials The wings: a1= -s-2t a2= 2st+t2 a3= -st2 The edge: a1 = -3s a2 = 3s2 a3 = -s3

Given a polynomial p(x) = xn + a1 xn-1+...+an-1 x + an / / / / / / / / / / / / / / / / / / Find ( z1, ..., zn ) such that p(x) = ( x - z1 )( x - z2 ) ... ( x - zn ) The wrong question: because you are asking for simple roots! Find distinct z1, ..., zm such that p(x) = ( x - z1 ) s1( x - z2 )s2 ... ( x - zm )sm s1+...+ sm = n, m < n The right question: do it on the pejorative manifold!

For ill-conditioned polynomial p(x)= xn + a1 xn-1+...+an-1 x + an ~ a = (a1 , ..., an-1 , an ) The objective: find u*=G(z*) that is nearest to p(x)~a

I.e. An over determined polynomial system Let ( x - z1 ) s1( x - z2 )s2 ... ( x - zm )sm = xn + g1 ( z1, ..., zm ) xn-1+...+gn-1 ( z1, ..., zm ) x + gn ( z1, ..., zm ) Then, p(x) = ( x - z1 ) s1( x - z2 )s2 ... ( x - zm )sm <==> g1 ( z1, ..., zm ) =a1 g2( z1, ..., zm ) =a2 ... ... ... gn ( z1, ..., zm ) =an n (m<n) m I.e. An over determined polynomial system G(z) = a

Project to tangent plane The polynomial a Project to tangent plane u1 = G(z0)+J(z0)(z1- z0) ~ tangent plane P0 : u = G(z0)+J(z0)(z- z0) pejorative root u*=G(z* ) initial iterate u0=G(z0) new iterate u1=G(z1) Pejorative manifold u = G( z ) Solve G(z0)+J(z0)( z - z0 ) = a for linear least squares solution z = z1 Solve G( z ) = a for nonlinear least squares solution z=z* G(z0)+J(z0)( z - z0 ) = a J(z0)( z - z0 ) = - [G(z0) - a ] z1 = z0 - [J(z0)+] [G(z0) - a]

zi+1=zi - J(zi )+[ G(zi )-a ], i=0,1,2 ... Theorem: If z=(z1, ..., zm) with z1, ..., zm distinct, then the Jacobian J(z) of G(z) is of full rank. Theorem: Let u*=G(z*) be nearest to p(x)~a, if 1. z*=(z*1, ..., z*m) with z*1, ..., z*m distinct; 2. z0 is sufficiently close to z*; 3. a is sufficiently close to u* then the iteration converges with a linear rate. Further assume that a = u* , then the convergence is quadratic.

The “pejorative” condition number v = G(z) u = G(y) || u - v ||2 = backward error || y - z ||2 = forward error u - v = G(y) - G(z) = J(z) (y - z) + h.o.t. || u - v ||2 = || J(z) (y - z) ||2 > s || y - z ||2 || y - z ||2 < (1/s) || u - v ||2 1/s is the pejorative condition number where s is the smallest singular value of J(z) .

Example (x-0.9)18(x-1.0)10(x-1.1)16 = 0 Step z1 z2 z3 -------------------------------------------------------------------- 0 .92 .95 1.12 1 .87 1.05 1.10 2 .92 .95 1.11 3 .88 1.01 1.10 4 .90 .97 1.12 5 .901 .992 1.101 6 .89993 .9998 1.1002 7 .9000003 .999998 1.1000007 8 .899999999997 .999999999991 1.100000000009 9 .900000000000006 .99999999999997 1.10000000000001 forward error: 6 x 10-15 backward error: 8 x 10-16 Pejorative condition: 58 Even clustered multiple roots are pejoratively well conditioned

Roots are correct up to 7 digits! Example (x-.3-.6i)100 (x-.1-.7i) 200 (x - .7-.5i) 300 (x-.3-.4i) 400 =0 Scary enough? Round coefficients to 6 digits. Z1 z2 z3 z4 .289 +.601i .100 +.702i .702 +.498i .301 +.399i .309 +.602i .097 +.698i .698 +.499i .299 +.401i .293 +.596i .101 +.7003i .7002 +.5005i .3007 +.4003i .300005 +.600006i .099998 +.6999992i .69999992+.4999993i .2999992 +.3999992i .3000002+.60000005i .09999995+.69999998i .69999997+.49999998i .29999997+.400000002i Roots are correct up to 7 digits! Pejorative condition: 0.58

What are the roots of the Wilkinson polynomial? Example: The Wilkinson polynomial p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ... There are 605 manifolds in total. It is near some manifolds, but which ones? Multiplicity backward error condition Estimated structure number error ------------------------------------------------------------------------ [1,1,1,1,1,1,1,1,1...,1] .000000000000003 550195997640164 1.6 [1,1,1,1,2,2,2,4,2,2,2] .000000003 29269411 .09 [1,1,1,2,3,4,5,3] .0000001 33563 .003 [1,1,2,3,4,6,3] .000001 6546 .007 [1,1,2,5,7,4] .000005 812 .004 [1,2,5,7,5] .00004 198 .008 [1,3,8,8] .0002 25 .005 [2,8,10] .003 6 .02 [5,15] .04 1 .04 [20] .9 .2 .2 What are the roots of the Wilkinson polynomial? Choose your poison!

The “right” question for ill-conditioned eigenproblem Given a matrix A Find a structured Schur form S and a matrix U such that AU - US = 0 U*U - I = 0 Over-determined!!! 3 1 3 l + l + + + A ~ m 2 S = m + m 2 1 2 Minimize || AU - US ||F2 + || U*U - I ||F2 ---- nonlinear least squares problem

+ O(10 -7) The pejorative condition number: 22.8 Example: A + E, where ||E|| A 1.0e-7 Step l m ------------------------------------------ 0 4.0 1.1 1 2.99 2.01 2 3.0006 1.9998 3 3.000001 1.9999997 4 3.00000001 1.99999992 3 + 3 + + + UT(A+E)U = + O(10 -7) 2 2 + 2 The pejorative condition number: 22.8

Conclusion 1. Ill-condition is cause by a wrong “identity” 2. Multiple eigenvalues/roots are pejoratively well conditioned, thereby tractable. 3. Extension of machine precision is NOT needed, a change in computing concept is. 4. To calculate ill-conditioned eigenvalues/roots, one has to figure out the pejorative structure (how?)