Finding zeros (also called roots) of a function Overview: Define the problem Methods of solution Graphical Newton’s Bisection Secant

Define the problem For a function f(x) of a single independent variable, find all x0 such that f(x0)=0. (Some methods can be generalized to multiple independent variables) Other ways that the problem can be stated: find all x where graphs of f1(x) and f2(x) intersect find all extrema of f(x) on [a,b]

Graphical solutions Open MatLab Under “new” open the macro for function definition Type the following: function [ ]=graphical(fh,xa,xb) fplot(fh,[xa,xb]) grid on end fh is a “function handle” defined before graphical is called xa and xb are the range of the independent variable in the output plot

Calling graphical in the command window >>myf=@(x) x^3-x^2-2*x+1; >>graphical(myf,-1,1.5); Read approximate zero of myf(x) from graph

Alternative way to define myf >>myf=inline(‘x^3-x^2-2*x+1’); >>graphical(myf,-1,1.5); Every cubic has either 1 or 3 real zeros (why) How many real zeros does this cubic have? What are their approximate values? Discover by extending the range of graphical

Expand the range of the x axis >>myf=inline(‘x^3-x^2-2*x+1’); >>graphical(myf,-1,1.5); >>graphical(myf,-2,2.5); approximate zeros are -1.25, 0.5, 1.8

Use graphical to find the intersections of ex and 3x2 In MatLab the function name for ex is exp(x) In command window >>myf=@(x) ? You don’t know how many or where intersections occur Start by casting a wide net >>graphical(myf,-5,5) Refine, if needed, to get better estimates of zeros

Solution after refining range of x axis: approximate points of intersection of ex and 3x2 are -0.5, 1, and 3.75

Making a graph with MatLab’s plot function >>myf=inline(‘exp(x)-3*x.^2’); >>x=linspace(-1,4,50); >>plot(x,myf(x)); >>grid on Why x.^2 for x2?

Use graphical to estimate the zeros of f(x) = ln(x+1) + tan(2x) on [-1,9] Using MatLab documentation you could discover Function name for ln is log Function name for base 10 logarithm is log10

To make your graph more useful for estimating zeros, change range of y axis using >>axis([xmin, xmax, ymin, ymax])

f(x) = ln(x+1) + tan(2x) with better range on the y axis Where are the zeros? f(x) X

Each branch contains a zero. f(x) = ln(x+1) + tan(2x) Each branch contains a zero. Line connecting branches is plotting artifact f(x) X

Newton’s Method

Non-linear problem reduced to sequence of linear problems How do we get this result? Non-linear problem reduced to sequence of linear problems

Generalize to xk and xk+1

Develop a MatLab code for Newton’s method What is the basic idea Put basic idea into a pseudocode Translate pseudocode into MatLab syntax

Develop a pseudocode for Newton’s Method xk+1 = xk – f(xk)/f’(xk) Basic idea behind calculation; calculate sequence x1, x2, … xk, xk+1,… set x1 to the initial guess from graphical stop when change xk-> xk+1 is small assign root to xk+1 calculate f(xk+1)

pseudocode for Newton’s method xk+1 = xk – f(xk)/f’(xk) Example of iterative code function [r,fr]=newton(fh,dfh,x0) Initialization steps = 0 x <- x0 maxsteps convergence requirement on fractional change (minre) large initial value of fractional change (re) while (re>minre) && (steps<maxsteps) increment steps save current estimate of root calculate next estimate of root re <- absolute value of fractional change end while loop r = best estimate of root fr = fh(r) end function

MatLab code for Newton’s method created in editor using the function macro and saved as newton.m

Use Newton’s method to find all the values of x where the graphs of ex and 3x2 intersect. How many intersections and approximate values of x? Call graphical >>myf=inline(‘exp(x)-3*x.^2’); >>graphical(myf,-1,4)

Solution after refining range of x axis: approximate points of intersection of ex and 3x2 are -0.5, 1, and 3.75

Write a script to ind all intersections of ex and 3x2 by Newton’s method.

Write a script to ind all intersections of ex and 3x2 by Newton’s method. In the editor myf=@(x) exp(x)-3*x^2; mydf=@(x) exp(x)-6*x; [r1,fr1]=newton(myf,mydf,0.5); disp([r1,fr1]) [r2,fr2]=newton(myf,mydf,1); disp([r2,fr2]) [r3,fr3]=newton(myf,mydf,3.7); disp([r3,fr3]) Copy and paste into the command window

Script to refine multiple intersections of ex and 3x2 by Newton’s method Writing a script in the editor, whether you save it or not, has the advantage that: Typos don’t lead to a cluttered command window and Editor copy and paste allows less typing This screen shot shows copy and paste into the command window The script in the editor is called “Untitled” If you want to keep it, use “save as”

Assignment 1: due 1/17/19 Estimate all of the zero of x3-x2-2x+1 graphically. Write a MatLab code for Newton’s method. Use your code to refine the graphical estimates. Hand in copies of your graph and the command window where the functions were called and zeros returned.

Derive expected convergence rate of Newton’s method

The convergence of Newton’s method is usually quadratic. Error in the n+1 estimate is proportional to the square of error in nth estimate. Makes convergence fast for any “good” initial estimate. Proportionality constant is different for different initial guesses. Makes convergence slightly dependent on the initial guess. How do I change function newton to convergence data?

Extended pseudocode for Newton’s method function [r,fr,logre]=newton(fh,dfh,x0) Initialization steps = 0 x1 <- x0 maxsteps convergence requirement on fractional change (minre) assign a large initial value of fractional change (re) while (re>minre) && (steps<maxsteps) increment steps save current estimate of root calculate next estimate of root re <- absolute value of fractional change logre(steps)=log base 10(re) end while loop r = best estimate of root fr = fh(r) end function

Modify Newton’s method to get convergence data

Get convergence data to compare rates of convergence for different initial guesses Note alternative to “inline” Note suppressed output of logre

Write script to get convergence data for same root with different initial guesses How do we ensure convergence to the same root? Make a semi-log plot convergence data How do we make a semi-log plot? Put results for both guesses on the same set of axes How do we put 2 graphs on the same axes?

Find intersections of ex and 3x2 by Newton’s method. For initial guesses 0.5 and 1.5, Newton’s method should converge to the zero near 1, since max and min separate these initial guesses from other zeros.

Since the convergence data is saved as log10(myrel), we can use a linear plot. Log10(myrel) = -5 means convergence to 10-5 Note alternative to “inline” Note suppressed output of logre

Pseudocode for script to compare convergence Newton’s method for different intial guesses define function handles for f(x) and f’(x) Call newtonzeros with initial guess1 Save convergence data as array1 Call newtonzeros with initial guess2 Save convergence data as array2 Call plot(array1) Note: just one argument hold on Call plot(array2) hold off

Script to compare convergence of Newton’s method

Convergence of Newton’s method with different initial guesses Note: only axes and curves returned from MatLab X0=0.5 X0=1.5 Y axis is log10(re) -5 corresponds to re = 10-5 Convergence to root of ex – 3x2 in [0.5,1.5] with different x0 X axis is number of iterations

Assignment 2, Due 1/22/19 f(x) = ex - 3x2 has a zero in the interval [-1, 0]. Modify your Newton’s method code to return convergence data as log10(re). Use plot to compare the rates of convergence to the root with initial guesses 0 and -1. Verify that both initial guesses converge to the same zero. Hand in a copy of command window where Newton’s method was called Hand in your plot with labels (by hand is OK) on axes and curves to show which curve goes with which initial guess.

Usually the convergence of Newton’s method is quadratic. Error in the n+1 estimate of the root is proportional to the square of the error in the nth estimate of the root. Since xk+1 = xk – f(xk)/f ’(xk) involves f ’(x) in the denominator, convergence is not as fast when f ’(r)=0. Called a “multiple” root with multiplicity 2.

Ever other root has multiplicity at least 2 f(x) = cos(x) – cos(3x) Ever other root has multiplicity at least 2 X

Convergence problems with Newton’s method All can be avoided by a good initial guess

Bisection method: Does not require a derivative of f(x) Cannot have the convergence problems shown on previous slide Disadvantage: slow convergence

Bisection method for continuous functions Implies at least one zero between a and b c=(a+b)/2 Exactly one of the following will be true If f(c)=0 then c is a root. If f(c)f(a)<0 then a root is in [a,c]. If f(c)f(b)<0 then a root is in [c,b]. Which is the case illustrated here? Requires starting values a and b with f(a)f(b)<0 What is wrong with the choice of a and b illustrated here?

Starting values for bisection method Let c = (a+b)/2 f(a)f(b)<0, but a and b are not good starting values. Only one root should be between starting values. Use graphical solution to choose good starting values.

Basic idea of bisection method In each cycle of iteration with a and b as end points, c = (a+b)/2 is the next estimate of the zero If f(c) = 0 stop. c is the zero between a and b. Decrease the interval around the zero by half c becomes one end of the new interval If f(a)f(c) < 0, a is the other end: b <- c otherwise b is the other end: a <- c converged? no: recycle yes: stop

Basic pseudocode for bisection method function [r,fr]=bisection(fh,xa,xb) Initialize steps, maxsteps, re and minre (as in Newton) xc=(xa+xb)/2; while (re>minre) && (steps<maxsteps) increment steps save current estimate of root (oldxc) reassign either xa or xb to xc calculate a new xc=(xa+xb)/2 if re <- absolute value of fractional change in xc end while loop r = best estimate of root fr = fh(r) end function No check for a sign change of function between initial xa and xb Does not allow for f(xc)=0

Extended pseudocode for bisection method function [r,fr]=bisection(fh,xa,xb) If f(xa)f(xb)>0, write message and return Initialize steps and re (as in Newton) xc=(xa+xb)/2; if f(xc)=0 write message and return while (re>minre) && (steps<maxsteps) increment steps save current estimate of root (oldxc) reassign either xa or xb to xc, calculate a new xc (probably don’t need to check xc=0) if re <- absolute value of fractional change in xc end while loop r = best estimate of root fr = fh(r) end function Very unlikely that f(xc) is identically zero, unless starting values have are chosen to make that true

x=0 is a root. Don’t need a root finder f(x) = ln(x+1) + tan(2x) x=0 is a root. Don’t need a root finder If bisection call with xb = -xa, initial xc is a root Code should stop f(x) X

Implement logic of bisection method in MatLab Relational operators in MatLab < less than <= less than or equal == equal -= not equal > greater than >= greater than or equal

Matlab code for bisection method disp([‘landed on a zero’])

Test your bisection function f(x)=x5 + x3 +3 does not have a zero on [-1,1] Call your bisection function with xa=-1 and xb=1 Does it generates warning message? f(x) has a zero between -1.5 and -1 Did your bisection function find a zero at -1.1053?

Assignment 3, Due 1/24/19 Write bisection function that finds root and saves convergence data Use code to find the zero f(x) = ex - 3x2 in the interval [-1, 0]. On a semi-log plot, compare the rate of convergence of the bisection method with starting interval [-1,0] to that of Newton’s method with initial guesses -1 and 0. Verify that Bisection and Newton’s method converge to the same zero. Hand in a copy of the command window where functions were called. Hand in a plot of the convergence data for bisection and Newton’s method with axes and curves labeled.

-1 and 0 are good starting values for bisection method to converge to root near -0.5

Bisection method converges slower than Newton

Linear on semi_log_y plot

Exactly the result shown by this plot

Bisection method does not require the derivative of a function to find its zeros. However, convergence is slow. Secant method does not require a derivative and converges almost as fast as Newton’s method.

Newton’s method without derivatives ~ xn – f(xn)/f’(xn) Newton’s method without derivatives

Basic pseudocode for secant method is almost the same as for Newton’s method function [r,fr]=secant(fh,xn,xnm1) Initialize: steps, max steps, re, minre x = xn while (re>minre) && (steps<maxsteps) increment steps save current estimate of root calculate next estimate of root using numerical derivative re <- absolute value of fractional change end while loop r = best estimate of root fr = fh(r) end function xn should be the better of the 2 initial estimates of the root How do we confirm that this true? In some steps, estimate of the root may not improve How do we deal with this problem?

Convergence may not be uniform Always use best estimate of root as xn in each cycle If starting values are 1 and -1, what do we expect to happen?

Initial xn+1 is not as good as xn, (how do we know this?) Use best estimate of root as xn (what value is this?) Use bad first estimate as xn-1 Did the second attempt work?

Extended pseudocode for secant method function [r,fr]=secant(fh,xnm1,xn) Test if |fn|<|fnm1|; otherwise interchange xn and xnm1 Initialize steps maxsteps, minre and re (as in Newton) x=xn while (re>minre) && (steps<maxsteps) increment steps approximate derivative and calculate xnp1 if |fnp1|<|fn| xn=xnp1 and xnm1=xn if |fnp1|>|fn| keep xn and xnm1=xnp1 calculate re = |(xn-xnm1)/xn| end while loop r = best estimate of root fr = fh(r) end function

Assignment 4, Due 1/24/19 Write secant function that finds root and saves convergence data Use code to find zero f(x) = ex - 3x2 in the interval [-1,0]. On a semi-log plot, compare the rate of convergence of the secant method to that of the bisection method with the same starting values [-1,0] and to Newton’s method with initial guesses -1 and 0. Hand in a copy of the command window where you verified that all methods converged to same zero. Hand in a plot of the convergence data for secant, bisection, and Newton’s method with axes and curves labeled. Write a script for this problem when root is between 0.5 and 1.5.

Assignment 4, Due 1/24/19 Write secant function that finds root and saves convergence data Use code to find zero f(x) = ex - 3x2 in the interval [-1, 0]. On a semi-log plot, compare the rate of convergence of the secant method to that of the bisection method with the same starting values of -1 and 0. Compare these results to Newton’s method with initial guesses 0 and -1. Hand in a plot with labels (by hand is OK) on axes and the curves to show which curve goes with which initial guess and which method. Hand in a copy of the command window where you confirmed that all methods converge to the same root.

Move due date for HW4 to 1/29/19 Return graded HW1-HW4 by 1/31/19 Quiz on finding zeros of a function 2/5/19

Note similarity to Newton’s method (slide 20) en+1 = [ ] en2

Suggested problems from text (6th edition) Chapter 3.1 p85 Problems 1, 4, 9, 10 Computer problems 1, 2, 4, 7 Chapter 3.2 p101 Problems 4, 15, 17, 19 Computer problems 1, 2, 4, 8, 9, 14 Chapter 3.3 p119 Problems 2, 3, 5 Computer problems 1, 3, 5, 6, 7