Perfidious Polynomials and Elusive Roots Zhonggang Zeng Northeastern Illinois University Nov. 2, 2001 at Northern Illinois University

What are the “best” functions? 1. Linear functions 2. Polynomials 3. Exponential functions 4. Trigonometric functions... Weierstrass Theorem: Any continuous function can be approximated to any accuracy by a polynomial of sufficiently high degree. Why by a polynomial ?

Polynomial root-finding x n + a 1 x n-1 + a 2 x n a n-1 x + a 0 = 0 has played a key role in the history of mathematics - Rational/irrational numbers, complex numbers - Fundamental Theorem of Algebra - Abel and Galois Theorem...

Can you solve ( x ) 100 = 0 Can you solve x x x x x x + 1 = 0

A = XX -1 Eigenvalues of

The Wilkinson polynomial p(x) = (x- 1 )(x- 2 )...(x- 20 ) = x x x Wilkinson wrote in 1984: Speaking for myself I regard it as the most traumatic experience in my career as a numerical analyst.

Classical textook methods for multiple roots Newton’s iteration x j+1 = x j - f(x j )/f’(x j ), j=0,1,2,... converges locally to a multiple root of f(x) with a linear rate. The modified Newton’s iteration x j+1 = x j - mf(x j )/f’(x j ), j=0,1,2,... converges locally to a m-fold root of f(x) with a quadratic rate. Newton’s iteration applied to g(x) = f(x)/f’(x) converges locally and quadratically to a root of f(x) regardliss of its multiplicity. None of them work!

Example: f(x) = (x-2) 7 (x-3)(x-4) in expanded form. Modified Newton’s iteration with m = 7 intended for root x = 2: x 1 = x 2 = x 3 = x 4 = x 5 = x 6 = x 7 = x 8 =

Myths on polynomial roots: - multiple roots are ill-conditioned, or even intractable - extension of machine precision is necessary to calculate multiple roots - there is an “attainable precision” for multiple roots: machine precision attainable precision = multiplicity Example: for 100-fold root, to get 5 digits right 500 digits in machine precision 5 digits precision = in multiplicity

How do we justify the answer ?

The condition number [Forward error] ~ [Condition number] [Backward error] A large condition number The problem is sensitive or, ill-conditioned From computational method From problem

The backward error: 5 x The forward error: 5 Conclusion: the problem is “bad” -- method is good! -- Ouch! Who’s responsible?

If the answer is highly sensitive to perturbations, you have probably asked the wrong question. Maxims about numerical mathematics, computers, science and life, L. N. Trefethen. SIAM News Who is asking a wrong question? What is the wrong question? A: “Customer” B: Numerical analyst A: The polynomial B: The computing objective

The question we used to ask The question we used to ask: (Fundamental Theorem of Algebra) Given a polynomial p(x) = x n + a 1 x n a n-1 x + a n find z = ( z 1,..., z n ) such that p(x) = ( x - z 1 )( x - z 2 )... ( x - z n ) Right - or - Wrong ?

Kahan’s pejorative manifolds x n + a 1 x n a n-1 x + a n (a 1,..., a n-1, a n ) All n-polynomials having certain multiplicity structure form a pejorative manifold Example: ( x-t ) 2 = x 2 + (-2t) x + t 2 Pejorative manifold: a 1 = -2t a 2 = t 2

Pejorative manifolds of 3-polynomials ( x - s )( x - t ) 2 = x 3 + (-s-2t) x 2 + (2st+t 2 ) x + (-st 2 ) ( x - s ) 3 = x 3 + (-3s) x 2 + (3s 2 ) x + (-s 3 ) Pejorative manifold of multiplicity structure [1,2] a 1 = -s-2t a 2 = 2st+t 2 a 3 = -st 2 Pejorative manifold of multiplicity structure [ 3 ] a 1 = -3s a 2 = 3s 2 a 3 = -s 3

Pejorative manifolds of 3-polynomials The wings: a 1 = -s-2t a 2 = 2st+t 2 a 3 = -st 2 The edge: a 1 = -3s a 2 = 3s 2 a 3 = -s 3 General form of pejorative manifolds u = G(z)

W. Kahan, Conserving confluence curbs ill-condition, Ill-condition occurs when a polynomial is near a pejorative manifold. 2. A small “drift” by a polynomial on that pejorative manifold does not cause large forward error to the multiple roots, except 3. If a multiple root is sensitive to small perturbation on the pejorative manifold, then the polynomial is near a pejorative submanifold of higher multiplicity. Ill-condition is caused by solving polynomial equations on a wrong manifold

Pejorative manifolds of 3-polynomials The wings: a 1 = -s-2t a 2 = 2st+t 2 a 3 = -st 2 The edge: a 1 = -3s a 2 = 3s 2 a 3 = -s 3

Given a polynomial p(x) = x n + a 1 x n a n-1 x + a n The wrong question: Find ( z 1,..., z n ) such that p(x) = ( x - z 1 )( x - z 2 )... ( x - z n ) because you are asking for simple roots! / / / / / / / / / The right question: Find distinct z 1,..., z m such that p(x) = ( x - z 1   1  x - z 2 )  2... ( x - z m )  m    m = n, m < n do it on the pejorative manifold!

For ill-conditioned polynomial p(x)= x n + a 1 x n a n-1 x + a n ~ a = (a 1,..., a n-1, a n ) The objective: find u * =G(z * ) that is nearest to p(x)~a

Let ( x - z 1   1  x - z 2 )  2... ( x - z m )  m = x n + g 1 ( z 1,..., z m ) x n g n-1 ( z 1,..., z m ) x + g n ( z 1,..., z m ) Then, p(x) = ( x - z 1   1  x - z 2 )  2... ( x - z m )  m g 1 ( z 1,..., z m ) =a 1 g 2 ( z 1,..., z m ) =a g n ( z 1,..., z m ) =a n I.e. An over determined polynomial system G(z) = a (m<n)n m

tangent plane P 0 : u = G(z 0 )+J(z 0 )(z- z 0 ) initial iterate u 0 =G(z 0 ) pejorative root u * =G(z * ) The polynomial a Project to tangent plane u 1 = G(z 0 )+J(z 0 )(z 1 - z 0 ) ~ new iterate u 1 =G(z 1 ) Pejorative manifold u = G( z ) Solve G( z ) = a for nonlinear least squares solution z=z * Solve G(z 0 )+J(z 0 )( z - z 0 ) = a for linear least squares solution z = z 1 G(z 0 )+J(z 0 )( z - z 0 ) = a J(z 0 )( z - z 0 ) = - [G(z 0 ) - a ] z 1 = z 0 - [J(z 0 ) + ] [G(z 0 ) - a]

Theorem: Let u * =G(z * ) be nearest to p(x)~a, if 1. z * =(z *1,..., z *m ) with z *1,..., z *m distinct; 2. z 0 is sufficiently close to z * ; 3. a is sufficiently close to u * then the iteration converges with a linear rate. Further assume that a = u *, then the convergence is quadratic. Theorem: If z=(z 1,..., z m ) with z 1,..., z m distinct, then the Jacobian J(z) of G(z) is of full rank. z i+1 =z i - J(z i ) + [ G(z i )-a ], i=0,1,2...

Calculation of G(z) and J(z) ( x - z 1   1  x - z 2 )  2... ( x - z m )  m = x n + g 1 ( z 1,..., z m ) x n g n-1 ( z 1,..., z m ) x + g n ( z 1,..., z m ) Explicit polynomail formula g j ( z 1,..., z m ) is neither necessary nor desirable. G(z) and every column of J(z) are calculated via simple numerical procedures, as key components of the algorithm.

The conventional condition number Let f(x * ) = 0 f(x  g(x  x(0) = x * f’(x  x’  g(x  g’(x  x’  / / / / / / x * x * x’(0) = - g(x * ) / f’(x * ) x  x * -  g(x * ) / f’(x * )  h.o.t. | x  x * | < | 1 / f’(x * )  |.|  g(x * ) | + h.o.t. Condition number | 1 / f’(x * )  | = infinity at multiple a root

The “pejorative” condition number u = G(y) v = G(z) || u - v || 2 = backward error || y - z || 2 = forward error u - v = G(y) - G(z) = J(z) (y - z) + h.o.t. || u - v || 2 = || J(z) (y - z) || 2 >  y - z || 2  y - z || 2 < (1/  )  u - v || 2 1/  is the pejorative condition number where  is the smallest singular value of J(z).

Example 1 ( x ) 100 = 0 To make the problem interesting: round the coefficients to 5 digits Step iterates conventional condition: infinity pejorative condition: Is it ill-conditioned ?

Example 2 (x- 0.9 ) 18 (x- 1.0 ) 10 (x- 1.1 ) 16 = 0 Step z 1 z 2 z forward error:6 x backward error:8 x Pejorative condition: 58 Even clustered multiple roots are pejoratively well conditioned

Multiplicity pejorative rootsbackward errorpejorative condition structure [1,1,...1] erratic [18,10,16] (.9000, , ) [17,11,16] (.8980,.9934, ) [14,16,14] (.8890,.9892, ) [10,24,10] (.8711,.9906, ) [ 2,40, 2] (.7393,.9917, ) [ 1,43] (.6559, ) [44] (.9925 ) If you are on a wrong manifold...

Example 3 (x-.3-.6i) 100 (x-.1-.7i) 200 (x i) 300 (x-.3-.4i) 400 =0 Scary enough? Round coefficients to 6 digits. Z 1 z 2 z 3 z i i i i i i i i i i i i i i i i i i i i Roots are correct up to 7 digits! Pejorative condition:0.58

Example 4: The Wilkinson polynomial p(x) = (x- 1 )(x- 2 )...(x- 20 ) = x x x There are 605 manifolds in total. It is near some manifolds, but which ones? Multiplicity backward error condition Estimated structure number error [1,1,1,1,1,1,1,1,1...,1] [1,1,1,1,2,2,2,4,2,2,2] [1,1,1,2,3,4,5,3] [1,1,2,3,4,6,3] [1,1,2,5,7,4] [1,2,5,7,5] [1,3,8,8] [2,8,10] [5,15] [20] What are the roots of the Wilkinson polynomial? Choose your poison!

If the estimated error is the sole criterion... Roots multiplicity * * * ** *** **** ***** *** The polynomial with these roots matches W. Polynomial for 6 digits The condition is manageable.

Summary of the simultaneous method: - Given a polynomial p(x) ~ a, decide the structure ( x - z 1   1  x - z 2 )  2... ( x - z m )  m implicitly define the coefficient operator G(z) and its Jacobian J(z) - Have an initial guess z 0 to the “pejorative root” - Start the nonlinear least squares iteration z i+1 =z i - J(z i ) + [ G(z i )-a ], i=0,1, If it converges verify (i) the backward error || G(z * )-a || (ii) the pejorative condition 1/  - Accept the pejorative root if both are tiny

Isolated multiple roots: The polynomial p(x) has an m-fold root p(x) = ( x-z * ) m. q(x-z * ) The clue: p(z * ) = p’(z * ) = p”(z * ) =... = p (m-1) (z * ) = 0 p (m) (z * ) =/= 0 F t1t1 t2t2... t m-1 x = p(x) - t 1 p (m) (x) p’(x) - t 2 p (m) (x) p (m-1) ( x) - t m-1 p (m) (x) p (m-1) ( x)

F t1t1 t2t2... t m-1 x = p(x) - t 1 p (m) (x) p’(x) - t 2 p (m) (x) p (m-1) ( x) - t m-1 p (m) (x) p (m-1) ( x) DF t1t1 t2t2... t m-1 x = diag( -p (m) (x), -p (m) (x),..., -p (m) (x), p (m) (x) ) z * is an m-fold root of p (0,0,..., z * ) is a simple zero of F

Theorem: Let p(x) = [( x-z * ) m. q(x-z * )] + [  m-1 (x-z * ) m  1 (x-z * ) +  0 ] Then F has a simple zero t2t2... t m-1 z t1t1 = z*z* 0 + [1/q(0)] 11...  m-2  m-1 /m 00 z = z * + [1/(mq(0))] [  m-1 ]

Conclusion 1. Ill-condition is cause by a wrong “identity” 4. To calculate multiple roots, one has to figure out the multiplicity structure (how?) 2. Multiple roots are pejoratively well conditioned, thereby tractable. 3. Extension of machine precision is not needed, a change in computing concept is. Question: Can Jordan Canonical form be computed?