Computation of the solutions of nonlinear polynomial systems By: E. C. Sherbrooke and N. M. Patrikalakis 1993 Presented by: Mahmod Mahameed

Agenda Introduction Notation and Definitions Formulation The Projected-Polyhedron Intersection Algorithm The Linear Programming Root Isolation Algorithm Main Algorithm Example and Conclusion

Introduction A fundamental problem in computer aided design and manufacturing is the efficient computation of all solutions to a system of nonlinear polynomial equations within some finite domain. This problem arises in many different applications, as varied as they are numerous. For example, in computer aided design it is often necessary to identify all characteristic points of an intersection curve between two surfaces, in order to trace out all of the branches of the curve, if the surfaces are piecewise rational polynomial, we must solve a system of nonlinear polynomial equations in order to identify these points In these thesis, we describe two techniques for eliciting all real solutions to a system of n nonlinear polynomial equations in n variables over an n- dimensional rectangular domain.

Notation and Definitions The Euclidean norm |x| of a vector x is defined to be the quantity: 𝑥 1 2 + 𝑥 2 2 +…+ 𝑥 𝑛 2 A multi-index I is an ordered n-tuple of nonnegative integers ( 𝑖 1 , 𝑖 2 , …, 𝑖 𝑛 ), We say that I is bounded by the multi-index M = (, 𝑚 2 , …, 𝑚 𝑛 ) if: 0 <= 𝑖 1 <= 𝑚 1 , 0 <= 𝑖 2 <= 𝑚 2 , …, 0 <= 𝑖 𝑛 <= 𝑚 𝑛 . We will use multi-indices as a shorthand for subscripting multi-dimensional arrays, for example, if I =(2,3),then 𝑤 𝐼 is equivalent to 𝑤 23 . We will use the notation 𝐼 𝑀 where M is a bounding multi-index, to indicate that a sum is to be taken over all possible multi-indices I which are bounded by M. For example if M = (1, 1), then 𝐼 𝑀 𝑤 𝐼 ≡ 𝑤 00 + 𝑤 01 + 𝑤 10 + 𝑤 11 .

Notation and Definitions The i-th Bernstein polynomial of degree m is defined by: 𝑏 𝑖,𝑚 (u) = 𝑚 𝑖 𝑢 𝑖 (1−𝑢) 𝑚−𝑖 The Bernstein multinomial determined by the multi-index I = ( 𝑖 1 , 𝑖 2 , .., 𝑖 𝑛 ) and the bounding multi-index M = ( 𝑚 1 , 𝑚 2 , .., 𝑚 𝑛 ) is defined by: 𝐵 𝐼,𝑀 (u) = 𝑏 𝑖 1 , 𝑚 1 ( 𝑢 1 ) 𝑏 𝑖 2 , 𝑚 2 ( 𝑢 2 ) … 𝑏 𝑖 𝑛 , 𝑚 𝑛 ( 𝑢 𝑛 ), where u = ( 𝑢 1 , 𝑢 2 , .., 𝑢 𝑛 ) ∈ 𝑅 𝑛 Let x = ( 𝑥 1 , 𝑥 2 , .., 𝑥 𝑛 ) be an element of Rn and let f : 𝑅 𝑛 -> R be a function of x. The graph of f is the function 𝐺 𝑓 : 𝑅 𝑛 -> 𝑅 𝑛+1 defined by: 𝐺 𝑓 (x) = (x, f(x))

Notation and Definitions Let x = ( 𝑥 1 , 𝑥 2 , .., 𝑥 𝑛 ) be an element of 𝑅 𝑛 . Then the i-th projection map 𝜋 𝑖 : 𝑅 𝑛 -> R is defined by 𝜋 𝑖 (x) = 𝑥 𝑖 Suppose P a set of m points of 𝑅 𝑛 . Let C(P) denote the set of all x such that x = 𝑖=1 𝑚 𝑐 𝑖 𝑝 𝑖 ,where for each i, 𝑝 𝑖 ∈𝑃 and 𝑐 𝑖 >=0, and 𝑖=1 𝑚 𝑐 𝑖 = 1 Then C(P) is called the convex hull of P. Suppose 𝑓 1 , 𝑓 2 ,.., 𝑓 𝑛 are functions of the parameters 𝑥 1 , 𝑥 2 , .., 𝑥 𝑛 . then the Jacobian matrix of the 𝑓 𝑖 , 𝜕{ 𝑓 1 , 𝑓 2 ,.., 𝑓 𝑛 } 𝜕{ 𝑥 1 , 𝑥 2 ,.., 𝑥 𝑛 } , is a square, n x n matrix whose element at row i, column j, is equal to 𝜕 𝑓 𝑖 𝜕 𝑥 𝑗

Supplemental Algorithms Converting from Power to the Bernstein basis: Input: n – the number of the dimensions. m – array indexed from 1 to n of the degrees. a – n-dimensional array of the Power basis coefficients. Output: b – array of the Bernstein coefficients.

Supplemental Algorithms Subdivision of a Multivariate Bernstein Polynomial (De Casteljau): Input: 𝑓 0 - The input Polynomial. n – The number of the dimensions. m – Array indexed from 1 to n of the degrees. a, b – Two arrays which define the domain S ∁ [0, 1] 𝑛 of 𝑓 0 (). Output: f – The output Polynomial. f( 𝑥 1 , 𝑥 2 , …, 𝑥 𝑛 ) = 𝑓 0 (a[1] + (b[1] – a[1]) 𝑥 1 , …,a[n] + (b[n] – a[n]) 𝑥 𝑛 ) 𝑓 0 , f are n-dimensional arrays indexed in i-th dimension from 0 to m[i].

Formulation Suppose we have a set of n functions of n variables 𝑓 1 , 𝑓 2 ,.., 𝑓 𝑛 , each is polynomial in the independent parameters 𝑢 1 , 𝑢 2 ,.., 𝑢 𝑛 . 𝑀 (𝑘) = ( 𝑚 1 (𝑘) , 𝑚 2 (𝑘) , …, 𝑚 𝑛 (𝑘) ) Multi-index of the degrees S = [ 𝑎 1 , 𝑏 1 ] x [ 𝑎 2 , 𝑏 2 ] x … x [ 𝑎 𝑛 , 𝑏 𝑛 ], Search domain We want to find all u = ( 𝑢 1 , 𝑢 2 , …, 𝑢 𝑛 ) ∈ S such that: 𝑓 1 (u) = 𝑓 2 (u) = … = 𝑓 𝑛 (u) = 0

Formulation Bernstein basis & C-H requires to transform from S to [0, 1] 𝑛 using: 𝑢 𝑖 = 𝑎 𝑖 + 𝑥 𝑖 ( 𝑏 𝑖 - 𝑎 𝑖 ) , and gives x ∈ [0, 1] 𝑛 such that: 𝑓 1 (x) = 𝑓 2 (x) = … = 𝑓 𝑛 (x) = 0 Bernstein basis gives an n-dimensional array of real coefficients 𝑤 𝑖 1 𝑖 2 … 𝑖 𝑛 (𝑘) such that for each k ∈ {1, 2, …, n}: 𝑓 𝑘 (x)= 𝐼 M (𝑘) 𝑤 𝐼 (𝑘) 𝐵 𝐼, M (𝑘) (x) By building the graph of 𝑓 𝑘 , 𝐹 𝑘 (x) = ( 𝑥 1 , 𝑥 2 …, 𝑥 𝑛 , 𝑓 𝑘 (x)) = (x, 𝑓 𝑘 (x)) The target is: 𝐹 1 (x) = 𝐹 2 (x) = … = 𝐹 𝑛 (x) = (x, 0)

Formulation Can be shown that: u = 𝑖=0 𝑚 𝑖 𝑚 𝑏 𝑖, 𝑚 (u) . u can be expressed as the weighted sum of Bernstein polynomials with coefficients spaced in the interval [0, 1] Using the summation unity property of the Bernstein basis ( 𝐼 𝑀 (𝐾) 𝐵 𝐼,𝑀 (u) = 1): 𝑥 𝑗 = 𝐼 𝑀 (𝐾) 𝑖 𝑗 𝑚 𝑗 (𝑘) 𝐵 𝐼, 𝑀 (𝐾) (x) Substituting gives the following representation: 𝐹 𝑘 (x) = 𝐼 𝑀 (𝐾) 𝑉 𝐼 (𝑘) 𝐵 𝐼, 𝑀 (𝐾) (x) , 𝑉 𝐼 (𝑘) = ( 𝑖 1 𝑚 1 (𝑘) , 𝑖 2 𝑚 2 (𝑘) , …, 𝑖 𝑛 𝑚 𝑛 (𝑘) , 𝑤 𝐼 (𝑘) )

Motivation for the Algorithm We will use properties of Bernstein basis in order to eliminate parts of the domain [0, 1] 𝑛 which do not contain any roots. Positivity: 𝐵 𝐼,𝑀 (u) ≥ 0 for u ∈ [0, 1] 𝑛 Partition of unity: 𝐼 𝑀 (𝐾) 𝐵 𝐼,𝑀 (u) = 1 Theorem: Let 𝑉 (𝑘) be the set of control points of 𝐹 𝑘 , and C( 𝑉 (𝑘) ) be the convex hull. Then for each k ∈ [1, 𝑛] : 𝐹 𝑘 (x) ∈ C( 𝑉 (𝑘) ) . Because every point on the hypersurface 𝐹 𝑘 lies within the convex hull of its control points, then we get the following theorem:

Motivation for the Algorithm Let x ∈ [0, 1] 𝑛 be a point satisfying: Let 𝑋 𝑛+1 be the set of points of 𝑅 𝑛+1 which lie on the hyperplane 𝑥 𝑛+1 , then: (x, 0) ∈ 𝑘=1 𝑛 (C( 𝑉 (𝑘) ) 𝑋 𝑛+1 ) Proof: Given that x satisfies so x satisfies 𝐹 𝑘 (x) = (x, 0) for each k, then the point (x, 0) ∈ C( 𝑉 (𝑘) ) , also any point of the form (x, 0) is automatically an element of 𝑋 𝑛+1 .

Motivation for the Algorithm First attempt to write the algorithm of solving the main problem is: Form the convex hulls of all the 𝐹 𝑘 and intersect them one another and with the hyperplane 𝑥 𝑛+1 = 0 . Call the intersection set A. Because A is the result of an intersection with the hyperplane 𝑥 𝑛+1 = 0, it can be expressed in the form A = 𝐴 ′ x {0} for some set 𝐴 ′ ∈ [0, 1] 𝑛 . so any root in [0, 1] 𝑛 must lie inside 𝐴 ′ . We will call an element of 𝐴 ′ a feasible point.

Motivation for the Algorithm But this attempt requires us to solve two problems Forming the convex hull of a set of m points in n dimensions is O( 𝑚 𝑛 2 +1 ) Even when this convex hulls are generated, the time-consuming process of intersecting them remains along with a large space requirement in which to store the result. These challenges are the motivation of our algorithms of finding the bounding boxes which will be presented.

Box generation Projected Polyhedron This method generates a bounding box by considering n two- dimensional problems on each iteration instead of one n+1 dimensional problem. Suppose we want to find [ 𝑎 𝑗 , 𝑏 𝑗 ], the j-th side of the bounding box B. 𝐷𝑒𝑓𝑖𝑛𝑒: 𝜋 𝑗 : 𝑅 𝑛+1 -> 𝑅 2 by: 𝜋 𝑗 (x) = ( 𝜋 𝑗 (x) , 𝜋 𝑛+1 (x)) We will generate the two-dimensional convex hulls of these projected points and intersect them with one another and with the x-axis. Let us call the set resulting from this intersection 𝐵 𝑗 𝐵 𝑗 is a subset of the x-axis, so we can say 𝐵 𝑗 = 𝐵 𝑗 ′ x {0}

Box generation Projected Polyhedron Let 𝐴 ′ be the set of feasible points as before. We want to show that 𝐴 ′ ∁ B = 𝐵 1 ′ x 𝐵 2 ′ x … x 𝐵 𝑛 ′ Theorem: Let 𝑉 𝑗,𝐼 (𝑘) = 𝜋 𝑗 ( 𝑉 𝐼 (𝑘) ) for specific j and k and for each I bounded by 𝑀 (𝑘) . Let 𝑉 𝑗 (𝑘) be the set of these two dimension points 𝑉 𝑗,𝐼 (𝑘) . Let C( 𝑉 𝑗 (𝑘) ) be the convex hull of 𝑉 𝑗 (𝑘) . Let 𝑋 2 be the x-axis (x, y=0) Let 𝐵 𝑗 = 𝑘=0 𝑛 (C( 𝑉 𝑗 (𝑘) ) 𝑋 2 ) , express 𝐵 𝑗 in the form 𝐵 𝑗 ′ x {0}. Then if B = 𝐵 1 ′ x 𝐵 2 ′ x … x 𝐵 𝑛 ′ , so 𝐴 ′ ∁ B.

Box generation Projected Polyhedron Proof: Let x ∈ 𝐴 ′ , then for some choice of coefficients { 𝐶 𝐼 (𝑘) } such that each 𝐶 𝐼 (𝑘) ≥ 0 and 𝐼 𝑀 (𝑘) 𝐶 𝐼 (𝑘) = 1 , we must have (x, 0) = 𝐼 𝑀 (𝑘) 𝐶 𝐼 (𝑘) 𝑉 𝐼 (𝑘) For all k, taking 𝜋 𝑗 of both sides, we obtain: ( 𝑥 𝑗 , 0) = 𝐼 𝑀 (𝑘) 𝐶 𝐼 (𝑘) 𝑉 𝑗, 𝐼 (𝑘) Therefore ( 𝑥 𝑗 , 0) ∈ C( 𝑉 𝑗 (𝑘) ) for all k, and automatically ( 𝑥 𝑗 , 0) ∈ 𝑋 2 , So ( 𝑥 𝑗 , 0) ∈ 𝐵 𝑗 which means that 𝑥 𝑗 ∈ 𝐵 𝑗 ′ , also this fact is true for all j So we can conclude: x∈ B which leads to the desired fact that 𝐴 ′ ∁ B.

Projected Polyhedron Analysis Let assume: n - number of variables, m – is the degree of each variable. The total execution time to perform the projection in one direction is: O( 𝑚 𝑛+1 ) There are a total of n convex hulls which need to be intersected in a given projection plane, So the total execution time is: O(n 𝑚 𝑛+1 ) There are n different projection planes for each side of the box, So the total execution time is: O( 𝑛 2 𝑚 𝑛+1 )

Box generation Linear Programming The approach now for finding every single side [ 𝑎 𝑖 , 𝑏 𝑖 ], is by finding the following statements: 𝑎 𝑖 = 𝑥∈𝐴 𝑚𝑎𝑥 { 𝑥 𝑖 } , 𝑏 𝑖 = 𝑥∈𝐴 𝑚𝑎𝑥 { 𝑥 𝑖 } The proof that 𝐴 ′ lies within the box determined by the 𝑎 𝑖 and the 𝑏 𝑖 is trivial. To use LP we need to rewrite the constraint equations by the fact x ∈ C( 𝑉 (𝑘) ) if : x= 𝐼 𝑀 (𝑘) 𝐶 𝐼 (𝑘) 𝑉 𝐼 (𝑘) Theorem: Point x ∈ 𝑅 𝑛+1 is in A if and only if there exist nonnegative coefficients 𝑐 𝐼 (𝑘) such that: (*) For each j between 1 and n and for k between 1 and n such that 𝑚 𝑗 (𝑘) ≠ 0: 𝐼 𝑀 (𝑘) 𝑐 𝐼 (𝑘) 𝑖 𝑗 𝑚 𝑗 (𝑘) = 𝑥 𝑗 (**) For each k between 1 and n: 𝐼 𝑀 (𝑘) 𝑐 𝐼 (𝑘) 𝑤 𝐼 (𝑘) = 0 (***) For each k between 1 and n: 𝐼 𝑀 (𝑘) 𝑐 𝐼 (𝑘) = 1

Box generation Linear Programming Suppose that we want to solve the min problem, finding the minimum value of 𝑥 𝑖 over all x ∈ A . Since (*) must hold for i = j and for all k, minimizing 𝑥 𝑖 is equivalent to minimizing. 𝑥 𝑖 = 𝐼 𝑀 (1) 𝑐 𝐼 (1) 𝑖 𝑖 𝑚 𝑖 (1) Linear programming are posed as follows: Given a p x q matrix C of constraint function coefficients, a q-vector u of nonnegative unknowns, a p-vector r of right-hand sides, and a q-vector c of objective function coefficients, minimize 𝑐 𝑇 𝑢 over all possible u which satisfy Cu = r.

Box generation Linear Programming With a small amount of algebra we can rewrite our minimization problem in the form of matrix problems. For finding the maximum of 𝑐 𝑇 𝑢 , we can simply negate all of the elements of c and obtain a minimization problem. The vector u of unknowns is all of the 𝑐 𝐼 (𝑘) ,as a result, q , the number of elements of u, is given by: q = 𝑘=1 𝑛 ( 𝑚 1 𝑘 + 1)( 𝑚 2 (𝑘) + 1)…( 𝑚 𝑛 (𝑘) + 1) = ( 𝑚+1) 𝑛 The 2n obtained by writing (**) and (***) are directly inserted into C

Box generation Linear Programming In order to put (*) into C we must eliminate the 𝑥 𝑗 from the equations by the following way: For all 1 ≤ j ≤ n and all 1 ≤ k ≤ n-1: if x is in A and if there exists a 𝑘 ′ which is the smallest integer such that k ≤ 𝑘 ′ ≤ n and 𝐼 𝑀 (𝑘) 𝑐 𝐼 (𝑘) 𝑖 𝑗 𝑚 𝑗 (𝑘) - 𝐼 𝑀 ( 𝑘 ′ ) 𝑐 𝐼 ( 𝑘 ′ ) 𝑖 𝑗 𝑚 𝑗 ( 𝑘 ′ ) = 0 Note that there is no more than 𝑛 2 - n equations of such 𝑘 ′ . So we get that: p ≤ 2n + 𝑛 2 - n = 𝑛 2 + n .

Linear Programming Analysis Let assume: n - number of variables, m – is the degree of each variable. The simplex method which solves the p x q C matrix to generate a solution is: O( 𝑝 2 q). As we saw the largest possible value of p is ( 𝑛 2 + n) , also saw that q = ( 𝑚+1) 𝑛 , then the complexity of one execution of the linear programming solver is O( 𝑛 4 𝑚 𝑛 )

Linear Programming Analysis Note that we need to execute the above method 2n times, n maximums and n minimums, so the final complexity is: O( 𝑛 5 𝑚 𝑛 ). As we saw from Projected Polyhedron, that the cost of subdivision on the current box is: O( 𝑛 2 𝑚 𝑛+1 ) So the total execution cost is: O( 𝑛 5 𝑚 𝑛 + 𝑛 2 𝑚 𝑛+1 )

Main Algorithm 1- Start with an initial box of search. 2- Use the multivariate De Casteljau subdivision to transform the n-functions from the initial 𝑅 𝑛 box to [0, 1] 𝑛 . 3- Find a sub-box of [0, 1] 𝑛 which contains all the roots. Projected Polyhedron method Linear Programming method

Main Algorithm 4- Using the reverse of the transformation we used in step 2, determine what this sub-box corresponds to in 𝑅 𝑛 : If the actual box in 𝑅 𝑛 is sufficiently small, conclude that there is a root inside and return the midpoint of the box as an approximation to it 5- If any dimensions of the sub-box in step 3 are not much smaller than 1 unit in length, split the box at the midpoint of each dimension which is causing trouble. (will get 2^k boxes, k is number of trouble dimensions) 6- Go back to step 2, once for each new box.

Notes Step 2 is needed in order to use the convex hull property. Step 5 is typically invoked when the box contains more than one root. On a typical iteration Step 5 will not be invoked and step 6 will only send one box back to step 2 for processing. The algorithm is finite – we have a finite tree calls from step 5-6 Which leads to the roots on the leafs of the tree. We don’t know how many children will result from a step of the algorithm, so the time will vary from step to step. But instead we can get a useful estimate by assessing the cost of each iteration of step 3.

Choosing an Algorithm Let assume that: n - number of variables, m – is the degree of each variable. The difference between both Algorithms is the third step which generates the bounding box. There are two cases to consider which way of generating the bounding box is preferred: If 𝑛 3 ≤ m, the second term of the LP complexity estimate is more important, and thus both the LP and PP algorithms have the same asymptotic time of O( 𝑛 2 𝑚 𝑛+1 ) . Otherwise, the PP algorithm is faster by a factor of 𝑛 3 𝑚 .

Choosing an Algorithm When we have a large number of roots in the specified domain, both algorithms are forced to perform a large number of subdivision steps. In such case The PP algorithm tends to perform better. The LP algorithm performs better when n is small and the number of iterations required is small. The LP algorithm performs better when n is small and there is multiple roots.

Examples Intersection two simple planar curves Suppose we want to intersect a circular arc with an elliptical arc: 𝑥 2 + 𝑦 2 - 1 = 0, 𝑥 2 4 + 4 𝑦 2 - 1 = 0, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 Simple algebra will show that the intersection point is: (0.894427191, 0.447213595). We want to solve the intersection problem with the two methods.

Projected Polyhedron Intersection two simple planar curves [ 𝑏 0,2 (𝑥) 𝑏 1,2 (𝑥) 𝑏 2,2 (𝑥) ] −1 −1 0 −1 −1 0 0 0 1 𝑏 0,2 (𝑦) 𝑏 1,2 (𝑦) 𝑏 2,2 (𝑦) = 0 [ 𝑏 0,2 (𝑥) 𝑏 1,2 (𝑥) 𝑏 2,2 (𝑥) ] −1 −1 3 −1 −1 3 3/4 3/4 13/4 𝑏 0,2 (𝑦) 𝑏 1,2 (𝑦) 𝑏 2,2 (𝑦) = 0

Projected Polyhedron Intersection two simple planar curves

Linear Programming Intersection two simple planar curves The constraint matrix C as follows: 0 0 −1 −1 1 0 0 0.5 −1 −1 1 0 0 1 0 0 1 0 0.5 0 −1 −1 1 0 0.5 0.5 −1 −1 1 0 0.5 1 0 0 1 0 1 0 0 0 1 0 1 0.5 0 0 1 0 1 1 1 1 1 0 0 0 1 0 0 1 0 −0.5 1 0 0 1 0 −1 −3 0 0 1 −0.5 0 1 0 0 1 −0.5 −0.5 1 0 0 1 −0.5 −1 −3 0 0 1 −1 0 3/4 0 0 1 −1 −0.5 3/4 0 0 1 −1 −1 −13/4 0 0 1 The vector of right-hand-sides r will be: 0 0 0 0 0 1 1

Linear Programming Intersection two simple planar curves

Conclusion Today we presented two methods of finding the bounding box, which helps us to solve the main problem - solutions of nonlinear polynomials – which is a fundamental problem in computer aided design and manufacturing . We also presented some comparison between the two methods and which one is preferred in terms of complexity, also presented two examples which clarifies the using of both methods. We should remember that when we are solving a problem with high degree and high dimensional, both algorithms tends to be expensive.

Thank you