1. Numerical Computation
Lecture 15: Approximating Derivatives
United International College

2. Last Time During the last class period we covered:
Bezier Curves and Surfaces
Readings:
Class Slides
Due Tomorrow:
Bezier Surface Programming Assignment
Questions?

3. Today We will cover: Numerical Approximations to Derivatives Readings:
Pav, Chapter 7
Homework due on Friday !!

4. Derivatives
The derivative represents the rate of change of a dependent variable y (= f(x) ) with respect to an independent variable x. (derivative = f '(x) )
Example: For f(x) = sin(x) , we have
f '(1)= cos(1)
For computational algorithms, we can seldom find the exact value of f '(x) as this involves symbolic calculations, so we have to estimate the value of f '(x).

5. Numerical Estimates of f’(x)
There are three basic ways to estimate derivatives:
Forward finite divided difference
Backward finite divided difference
Center finite divided difference
All based on the Taylor Series
Another form of this result
is the following, where
is some point between x
and h.

6. Numerical Estimates of f’(x)
Rearranging this last equation gives
We will estimate the derivative by dropping the last term. If we can bound the second derivative by some constant, then this last term is O(h) and we get

7. Forward Divided Difference Formula for f’(x)
Thus,
This formula is called the first forward divided difference formula and the error in this formula is of order O(h).
The error in this approximation is due to truncation of the last term (the second derivative term).

8. Forward Divided Difference Formula for f’(x)
The truncation error can be made small by making h small. However, as h gets smaller, precision will be lost in this equation due to subtractive cancellation.
The error in calculation for small h is called roundoff error. Generally the roundoff error will increase as h decreases.

9. Forward Divided Difference Formula for f’(x)
Forward difference
True derivative
Approximation h x
x -h x x +h

10. Backward Divided Difference Formula for f’(x)
Backward difference
True derivative
Approximation h x
x -h x x +h

11. Backward Divided Difference Formula for f’(x)
Solving for f’(x) gives:
Or,
This is called the Backward Divided Difference Formula for f’(x).

12. Centered Divided Difference Formula for f’(x)
Consider the Taylor series expansions for the forward and backward approximations, extended to the degree 3 terms:
If we subtract these two and solve for f’(x) we get:

13. Centered Divided Difference Formula for f’(x)
If f’’’ is continuous, we can find a bound on f’’’ and we get the centered divided difference formula:
Note: Since the error is O(h2), this is a better approximation than the forward or backward approximations!

14. Centered Divided Difference Formula for f’(x)
Centered difference
True derivative
Approximation 2h x
x -h x x +h

15. f(x) f(x) x x f(x) f(x) x x forward finite divided difference approx.
true derivative x x
f(x)
f(x)
backward
finite divided
difference approx.
centered
finite divided
difference approx. x x

16. Divided Difference Formula for f’’(x)
Consider the Taylor series expansions for the forward and backward approximations, extended to the degree 4 terms:
If we add these two and solve for f’’(x) we get:

17. Richardson Extrapolation Formula
Recall: The forward difference formula for f’(x) had error O(h), while the centered difference formula had error O(h2). Can we do better?
Consider again the Taylor expansions:
Let Then,

18. Richardson Extrapolation Formula
Also,
We can use and to get:
Note: This approximation for f’(x) has error of O(h4)!!

19. Ex19.2: Richardson Extrapolation
Richardson Extrapolation Example
We will use the extrapolation method above on the central difference approximation to estimate the first derivative of
at x = 0.5 with h = 0.5 and (exact sol. = )

20. Ex19.2: Richardson Extrapolation
General Richardson Extrapolation
We can use the Richardson method to increase the accuracy of numerical estimates to any series-based quantity.
Suppose we want to calculate some quantity L and have found, through theory, an approximation to L:
Let

21. Ex19.2: Richardson Extrapolation
General Richardson Extrapolation
Define:
Theorem: (Richardson Extrapolation). There are constants akm such that
Proof: By induction. (Skipped)
Corollary:

22. Ex19.2: Richardson Extrapolation
General Richardson Extrapolation
Example: Consider f(x) = arctan(x). Suppose we want to find
. Let and start with
h = 0.01
Then, we compute D(n,m) in a pyramid fashion (as we did for Newton’s divided differences). The first column is just
Class Exercise: Verify that these values are correct.
Best Approximation is D(2,2) =
Actual Answer is ?

23. Ex19.2: Richardson Extrapolation
General Richardson Extrapolation
Example: Consider f(x) = arctan(x). What if we just used smaller and smaller values of h?

24. Ex19.2: Richardson Extrapolation
General Richardson Extrapolation
Example: The graph has values of h along the bottom and total error (truncation + roundoff) on the vertical axis. What can you conclude from this graph?