Numerical Computation Lecture 15: Approximating Derivatives United International College

Last Time During the last class period we covered: Bezier Curves and Surfaces Readings: Class Slides Due Tomorrow: Bezier Surface Programming Assignment Questions?

Today We will cover: Numerical Approximations to Derivatives Readings: Pav, Chapter 7 Homework due on Friday !!

Derivatives The derivative represents the rate of change of a dependent variable y (= f(x) ) with respect to an independent variable x. (derivative = f '(x) ) Example: For f(x) = sin(x) , we have f '(1)= cos(1) For computational algorithms, we can seldom find the exact value of f '(x) as this involves symbolic calculations, so we have to estimate the value of f '(x).

Numerical Estimates of f’(x) There are three basic ways to estimate derivatives: Forward finite divided difference Backward finite divided difference Center finite divided difference All based on the Taylor Series Another form of this result is the following, where is some point between x and h.

Numerical Estimates of f’(x) Rearranging this last equation gives We will estimate the derivative by dropping the last term. If we can bound the second derivative by some constant, then this last term is O(h) and we get

Forward Divided Difference Formula for f’(x) Thus, This formula is called the first forward divided difference formula and the error in this formula is of order O(h). The error in this approximation is due to truncation of the last term (the second derivative term).

Forward Divided Difference Formula for f’(x) The truncation error can be made small by making h small. However, as h gets smaller, precision will be lost in this equation due to subtractive cancellation. The error in calculation for small h is called roundoff error. Generally the roundoff error will increase as h decreases.

Forward Divided Difference Formula for f’(x) Forward difference True derivative Approximation h x x -h x x +h

Backward Divided Difference Formula for f’(x) Backward difference True derivative Approximation h x x -h x x +h

Backward Divided Difference Formula for f’(x) Solving for f’(x) gives: Or, This is called the Backward Divided Difference Formula for f’(x).

Centered Divided Difference Formula for f’(x) Consider the Taylor series expansions for the forward and backward approximations, extended to the degree 3 terms: If we subtract these two and solve for f’(x) we get:

Centered Divided Difference Formula for f’(x) If f’’’ is continuous, we can find a bound on f’’’ and we get the centered divided difference formula: Note: Since the error is O(h2), this is a better approximation than the forward or backward approximations!

Centered Divided Difference Formula for f’(x) Centered difference True derivative Approximation 2h x x -h x x +h

f(x) f(x) x x f(x) f(x) x x forward finite divided difference approx. true derivative x x f(x) f(x) backward finite divided difference approx. centered finite divided difference approx. x x

Divided Difference Formula for f’’(x) Consider the Taylor series expansions for the forward and backward approximations, extended to the degree 4 terms: If we add these two and solve for f’’(x) we get:

Richardson Extrapolation Formula Recall: The forward difference formula for f’(x) had error O(h), while the centered difference formula had error O(h2). Can we do better? Consider again the Taylor expansions: Let Then,

Richardson Extrapolation Formula Also, We can use and to get: Note: This approximation for f’(x) has error of O(h4)!!

Ex19.2: Richardson Extrapolation Richardson Extrapolation Example We will use the extrapolation method above on the central difference approximation to estimate the first derivative of at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

Ex19.2: Richardson Extrapolation General Richardson Extrapolation We can use the Richardson method to increase the accuracy of numerical estimates to any series-based quantity. Suppose we want to calculate some quantity L and have found, through theory, an approximation to L: Let

Ex19.2: Richardson Extrapolation General Richardson Extrapolation Define: Theorem: (Richardson Extrapolation). There are constants akm such that Proof: By induction. (Skipped) Corollary:

Ex19.2: Richardson Extrapolation General Richardson Extrapolation Example: Consider f(x) = arctan(x). Suppose we want to find . Let and start with h = 0.01 Then, we compute D(n,m) in a pyramid fashion (as we did for Newton’s divided differences). The first column is just Class Exercise: Verify that these values are correct. Best Approximation is D(2,2) = 0.333333333333313 Actual Answer is ?

Ex19.2: Richardson Extrapolation General Richardson Extrapolation Example: Consider f(x) = arctan(x). What if we just used smaller and smaller values of h?

Ex19.2: Richardson Extrapolation General Richardson Extrapolation Example: The graph has values of h along the bottom and total error (truncation + roundoff) on the vertical axis. What can you conclude from this graph?