1. Systems of Non-Linear Equations

2. Objective
Finding the roots of a set of simultaneous nonlinear equations (n equations, n unknowns).
where each of fi(x1, … xn) cannot be expressed in the form

3. Review of iterative methods for finding unknowns
Finding x that satisfies f(x) = 0 (one equation, one unknown)
- Fixed-Point Iteration
- Newton-Raphson
- Secant
Solving Ax = b or Ax – b = 0, or finding multiple x's that simultaneously satisfy a system of linear equations
- Gauss-Seidel
- Jacobi
Variation of Fixed-Point Iteration

4. Fixed Point Iteration To solve We can create updating formula as or
Which formula will converge?
What initial points should we pick?

5. Fixed Point Iteration
Diverging
Converging

6. Fixed Point Iteration – Converging Criteria
For solving f(x) = 0 (one equation, one unknown), we have the updating formula
Trough analysis, we derived the following relationship
which tells us convergence is guaranteed if

7. Fixed Point Iteration – Converging Criteria
For solving two equations with two unknowns, we have the updating formula
Through similar reasoning, we can demonstrate that convergence can be guaranteed if

8. Fixed-Point Iteration – Summary
Updating formula is easy to construct, but updating formula that satisfy (guarantees convergence)
is not easy to construct.
Slow convergent rate

9. Newton-Raphson (one equation, one unknown)
Want to find the root of f(x) = 0.
From 1st-Order Taylor Series Approximation, we have
Idea: use the slope at xi to predict the location of the root. If xi+1 is the root, then f(xi+1) = 0. Thus we have
Single-equation form

10. Newton-Raphson (two equations, two unknowns)
Want to find x and y that satisfy
From 1st-Order Taylor Series Approximation, we have
Using similar reasoning, we have ui+1 = 0 and vi+1 = 0.
continue …

11. Newton-Raphson (two equations, two unknowns)
Replacing ui+1 = 0 and vi+1 = 0 in the equations yields
continue …

12. Newton-Raphson (two equations, two unknowns)
Solving the equations algebraically yields
Alternatively, we may solve for xi+1 and yi+1 using well-known methods for solving systems of linear equations

13. Newton-Raphson Example
To solve
First evaluate
With x0 = 1.5, y0 = 3.5, we have
continue …

14. Newton-Raphson Example
From these two formula, we can then calculate x1 and y1 as
These process can be repeated until a "good enough" approximation is obtained.

15. Newton-Raphson (n equations, n unknowns)
Want to find xi (i = 1, 2, …, n) that satisfy
From 1st-Order Taylor Series Approximation, we have

16. Newton-Raphson (n equations, n unknowns)
For each k = 0, 1, 2, …, n, setting fk,i+1 = 0 yields
These equations can be expressed in matrix form as
where

17. Newton-Raphson – Summary
Updating formula is not convenient to construct.
Excellent initial guesses are usually required to ensure convergence.
If the iteration converges, it converges quickly.