Systems of Non-Linear Equations

Objective Finding the roots of a set of simultaneous nonlinear equations (n equations, n unknowns). where each of fi(x1, … xn) cannot be expressed in the form

Review of iterative methods for finding unknowns Finding x that satisfies f(x) = 0 (one equation, one unknown) - Fixed-Point Iteration - Newton-Raphson - Secant Solving Ax = b or Ax – b = 0, or finding multiple x's that simultaneously satisfy a system of linear equations - Gauss-Seidel - Jacobi Variation of Fixed-Point Iteration

Fixed Point Iteration To solve We can create updating formula as or Which formula will converge? What initial points should we pick?

Fixed Point Iteration Diverging Converging

Fixed Point Iteration – Converging Criteria For solving f(x) = 0 (one equation, one unknown), we have the updating formula Trough analysis, we derived the following relationship which tells us convergence is guaranteed if

Fixed Point Iteration – Converging Criteria For solving two equations with two unknowns, we have the updating formula Through similar reasoning, we can demonstrate that convergence can be guaranteed if

Fixed-Point Iteration – Summary Updating formula is easy to construct, but updating formula that satisfy (guarantees convergence) is not easy to construct. Slow convergent rate

Newton-Raphson (one equation, one unknown) Want to find the root of f(x) = 0. From 1st-Order Taylor Series Approximation, we have Idea: use the slope at xi to predict the location of the root. If xi+1 is the root, then f(xi+1) = 0. Thus we have Single-equation form

Newton-Raphson (two equations, two unknowns) Want to find x and y that satisfy From 1st-Order Taylor Series Approximation, we have Using similar reasoning, we have ui+1 = 0 and vi+1 = 0. continue …

Newton-Raphson (two equations, two unknowns) Replacing ui+1 = 0 and vi+1 = 0 in the equations yields continue …

Newton-Raphson (two equations, two unknowns) Solving the equations algebraically yields Alternatively, we may solve for xi+1 and yi+1 using well-known methods for solving systems of linear equations

Newton-Raphson Example To solve First evaluate With x0 = 1.5, y0 = 3.5, we have continue …

Newton-Raphson Example From these two formula, we can then calculate x1 and y1 as These process can be repeated until a "good enough" approximation is obtained.

Newton-Raphson (n equations, n unknowns) Want to find xi (i = 1, 2, …, n) that satisfy From 1st-Order Taylor Series Approximation, we have

Newton-Raphson (n equations, n unknowns) For each k = 0, 1, 2, …, n, setting fk,i+1 = 0 yields These equations can be expressed in matrix form as where

Newton-Raphson – Summary Updating formula is not convenient to construct. Excellent initial guesses are usually required to ensure convergence. If the iteration converges, it converges quickly.