1. Sec:4.3
Total Numerical Error

2. Sec:4.3 Total Numerical Error
a centered difference approximation of the first derivative can be written as
forward finite difference
𝒇 𝒙 𝒊+𝟏 −𝒇 𝒙 𝒊 𝒉 − 𝒇 ′ 𝒙 𝒊 = 𝒇 𝟐 (𝝃) 𝟐 𝒉
Finite-difference
Approximation
True value
Truncation Error
We are using digital computers, the function values do include round-off error as in
𝒇 𝒙 𝒊 = 𝒇 𝒙 𝒊 + 𝒆 𝒊
𝒇 𝒙 𝒊+𝟏 = 𝒇 𝒙 𝒊+𝟏 + 𝒆 𝒊+𝟏
𝒇 𝒙 𝒊+𝟏 −𝒇 𝒙 𝒊 𝒉 − 𝒇 ′ 𝒙 𝒊 = 𝒆 𝒊+𝟏 − 𝒆 𝒊 𝒉 + 𝒇 𝟐 (𝝃) 𝟐 𝒉
Finite-difference
Approximation
True value
Round-off error
Truncation Error
Total Numerical Error

3. Sec:4.3 Total Numerical Error
𝒇 𝒙 𝒊+𝟏 −𝒇 𝒙 𝒊 𝒉 − 𝒇 ′ 𝒙 𝒊 ≤ 𝒆 𝒊+𝟏 − 𝒆 𝒊 𝒉 + 𝒇 𝟐 (𝝃) 𝟐 𝒉
We can see that the total error of the finite-difference approximation consists of a round-off error which increases with step size and a truncation error that decreases with step size.
Finite-difference
Approximation
True value
Round-off error
Truncation Error
Total Numerical Error
Assuming that the absolute value of each component of the round-off error has an
upper bound of ε
Round-off error
= 𝒆 𝒊+𝟏 − 𝒆 𝒊 𝒉 ≤ 𝟐𝜺 𝒉
𝒆 𝒊+𝟏 ≤𝜺 , 𝒆 𝒊 ≤𝜺
𝒆 𝒊+𝟏 − 𝒆 𝒊 ≤𝟐𝜺
assume that the second derivative has a maximum absolute value of M.
Truncation Error
= 𝒇 𝟐 (𝝃) 𝟐 𝒉≤ 𝑴 𝒉 𝟐
𝒇 𝟐 (𝝃) ≤𝑴
≤ 𝟐𝜺 𝒉 + 𝑴 𝒉 𝟐
ℎ 𝑜𝑝𝑡 = 4 𝜀 𝑀
Total Numerical Error
An optimal step size can be determined by differentiating setting the result equal to zero and solving for h

4. Sec:4.3 Total Numerical Error
Example:
use forward difference approximation of O(h) to estimate the first derivative of the following function at x = 0.5,
𝑓 (𝑥) = −0.1 𝑥 4 − 0.15 𝑥 3 − 0.5 𝑥 2 − 0.25𝑥 + 1.2
(a) Perform the same computation starting with h = 1. Then progressively divide the step size by a factor of 10 to demonstrate how round-off becomes dominant as the step size is reduced. Recall that the true value of the derivative is −
At first, round-off is minimal and the estimate is dominated by truncation error. A minimum error is reached at h = 10 −8 . Beyond this point, the error increases as round-off dominates.
step size finite difference true error
ℎ 𝑜𝑝𝑡 = 4 𝜀 𝑀
(b) Relate your results to
Because MATLAB has a precision of about 15 to 16 digits, a rough estimate of the upper bound on round-off would be about 𝜀=0.5× 10 −16 .
𝑓′′ 𝑥 = −(1.2 𝑥 𝑥 + 1)
𝑀≈ 𝑓′′ =1.75
ℎ 𝑜𝑝𝑡 = 4 𝜀 𝑀 = 4∗0.5× 10 − =1.07× 10 −8

5. Sec:4.3 Total Numerical Error
Example:
use forward difference approximation of O(h) to estimate the first derivative of the following function at x = 0.5,
𝑓 (𝑥) = −0.1 𝑥 4 − 0.15 𝑥 3 − 0.5 𝑥 2 − 0.25𝑥 + 1.2
Perform the same computation starting with h = 1. Then progressively divide the step size by a factor of 10 to demonstrate how round-off becomes dominant as the step size is reduced. Relate your results to Eq. (4.31). Recall that the true value of the derivative is −
format long
fun -0.1*x^4-0.15*x^3-0.5*x^2-0.25*x+1.2;
dfun -0.4*x^3-0.45*x^2-x-0.25;
x=0.5; exact = dfun(0.5); h =1;
for i=1:11
h_vec(i) = h;
[bd,fd,cd] = num_diff(fun,x,h);
fd_vec(i) = fd;
error(i) = exact - fd;
h = h/10;
end
res = [h_vec' fd_vec' error'];
fprintf(' step size finite difference true error\n');
fprintf('%14.10f %16.14f %16.13f\n', res');
plot(log10(h_vec),log10(error)),xlabel('Log(Step Size)'),ylabel('Log(Error)')
title('Plot of Error Versus Step Size')
grid on

6. Problems
1) (a) Use Taylor series to express 𝑓( 𝑥 𝑖−2 ) and 𝑓( 𝑥 𝑖−1 ) with center 𝑥 𝑖 .
(b) Use problem (1) to show that the following formula is of order 𝑶 𝒉 .
𝒇 𝒙 𝒊 −𝟐𝒇 𝒙 𝒊−𝟏 +𝒇 𝒙 𝒊−𝟐 𝒉 𝟐
2) Drive a formula for the condition number
𝒙 𝒇′( 𝒙 ) 𝒇 𝒙
3) Drive a formula for the optimal step size ℎ 𝑜𝑝𝑡 for a centered difference approximation of the first derivative
ℎ 𝑜𝑝𝑡 = 3 3 𝜀 𝑀