1. NUMERICAL DIFFERENTIATION AND INTEGRATION
ENGR 351
Numerical Methods for Engineers
Southern Illinois University Carbondale
College of Engineering
Dr. L.R. Chevalier
Dr. B.A. DeVantier

2. Numerical Differentiation and Integration
Calculus is the mathematics of change.
Engineers must continuously deal with systems and processes that change, making calculus an essential tool of our profession.
At the heart of calculus are the related mathematical concepts of differentiation and integration.

3. Differentiation
Dictionary definition of differentiate - “to mark off by differences, distinguish; ..to perceive the difference in or between”
Mathematical definition of derivative - rate of change of a dependent variable with respect to an independent variable

4. f(x) Dy Dx x

5. Integration The inverse process of differentiation
Dictionary definition of integrate - “to bring together, as parts, into a whole; to unite; to indicate the total amount”
Mathematically, it is the total value or summation of f(x)dx over a range of x. In fact the integration symbol is actually a stylized capital S intended to signify the connection between integration and summation.

6. f(x) x

7. Mathematical Background

8. Mathematical Background

9. Overview Newton-Cotes Integration Formulas Integration of Equations
Trapezoidal rule
Simpson’s Rules
Unequal Segments
Open Integration
Integration of Equations
Romberg Integration
Gauss Quadrature
Improper Integrals

10. Overview Numerical Differentiation Applied problems
High accuracy formulas
Richardson’s extrapolation
Unequal spaced data
Uncertain data
Applied problems

11. Specific Study Objectives
Understand the derivation of the Newton-Cotes formulas
Recognize that the trapezoidal and Simpson’s 1/3 and 3/8 rules represent the areas of 1st, 2nd, and 3rd order polynomials
Be able to choose the “best” among these formulas for any particular problem

12. Specific Study Objectives
Recognize the difference between open and closed integration formulas
Understand the theoretical basis of Richardson extrapolation and how it is applied in the Romberg integration algorithm and for numerical differentiation

13. Specific Study Objectives
Recognize why both Romberg integration and Gauss quadrature have utility when integrating equations (as opposed to tabular or discrete data).
Understand the application of high-accuracy numerical-differentiation.
Recognize data error on the processes of integration and differentiation.

14. Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated function or tabulated data with some approximating function that is easy to integrate

15. Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated function or tabulated data with some approximating function that is easy to integrate

16. Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated function or tabulated data with some approximating function that is easy to integrate
fn(x) is an nth order
polynomial

17. The approximation of an integral by the area under
- a first order polynomial
- a second order polynomial
We can also approximated the integral by using a
series of polynomials applied piece wise.

18. An approximation of an integral by the area under straight line segments.

19. Newton-Cotes Formulas
Closed form - data is at the beginning and end of the limits of integration
Open form - integration limits extend beyond the range of data.

20. Trapezoidal Rule First of the Newton-Cotes closed integration formulas
Corresponds to the case where the polynomial is a first order

21. A straight line can be represented as:

22. Integrate this equation. Results in the trapezoidal rule.

23. The concept is the same but the trapezoid is on its side.
base
height
height
height
width
base

24. Error of the Trapezoidal Rule
This indicates that is the function being integrated is
linear, the trapezoidal rule will be exact.
Otherwise, for section with second and higher order
derivatives (that is with curvature) error can occur.
A reasonable estimate of x is the average value of
b and a

25. Multiple Application of the Trapezoidal Rule
Improve the accuracy by dividing the integration interval into a number of smaller segments
Apply the method to each segment
Resulting equations are called multiple-application or composite integration formulas

26. Multiple Application of the Trapezoidal Rule
where there are n+1 equally spaced base points.

27. We can group terms to express a general form} }
width
average height

28. }
width
average height
The average height represents a weighted average
of the function values
Note that the interior points are given twice the weight
of the two end points

29. EXAMPLE Evaluate the following integral using the
trapezoidal rule and h = 0.1

30. Simpson’s 1/3 Rule
Corresponds to the case where the function is a second order polynomial

31. Simpson’s 1/3 Rule
Designate a and b as x0 and x2, and estimate f2(x) as a second order Lagrange polynomial

32. Simpson’s 1/3 Rule
After integration and algebraic manipulation, we get the following equations }
width
average height

33. Error Single application of Trapezoidal Rule.
Single application of Simpson’s 1/3 Rule

34. Multiple Application of Simpson’s 1/3 Rule

35. The odd points represent the middle term for each application
The odd points represent the middle term for each application. Hence carry the weight 4
The even points are common to adjacent applications and are counted twice.

36. Simpson’s 3/8 Rule
Corresponds to the case where the function is a third order polynomial

37. Integration of Unequal Segments
Experimental and field study data is often unevenly spaced
In previous equations we grouped the term (i.e. hi) which represented segment width.

38. Integration of Unequal Segments
We should also consider alternately using higher order equations if we can find data in consecutively even segments
trapezoidal
rule
1/3
rule
trapezoidal
rule
3/8
rule

39. EXAMPLE Integrate the following using the trapezoidal rule,
Simpson’s 1/3 Rule and a multiple application of
the trapezoidal rule with n=2. Compare results with
the analytical solution.

40. Simpson’s 1/3 Rule
f(2) =

41. Multiple Application of the Trapezoidal Rule
We are obviously not doing very well on our estimates.
Lets consider a scheme where we “weight” the estimates
....end of example

42. Integration of Equations
Integration of analytical as opposed to tabular functions
Romberg Integration
Richardson’s Extrapolation
Romberg Integration Algorithm
Gauss Quadrature
Improper Integrals

43. Richardson’s Extrapolation
Use two estimates of an integral to compute a third more accurate approximation
The estimate and error associated with a multiple application trapezoidal rule can be represented generally as:
I = I(h) + E(h)
where I is the exact value of the integral
I(h) is the approximation from an n-segment application
E(h) is the truncation error
h is the step size (b-a)/n

44. Make two separate estimates using step sizes of h1 and h2 .
I(h1) + E(h1) = I(h2) + E(h2)
Recall the error of the multiple-application of the trapezoidal
rule
Assume that is constant regardless of the step size

45. Substitute into previous equation:
I(h1) + E(h1) = I(h2) + E(h2)

46. Thus we have developed an estimate of the truncation error in terms of the integral estimates and their step sizes. This estimate can then be substituted into:
I = I(h2) + E(h2)
to yield an improved estimate of the integral:

47. What is the equation for the special case where the interval is halved?
i.e. h2 = h1 / 2

49. EXAMPLE
Use Richardson’s extrapolation to evaluate:

50. ROMBERG INTEGRATION
We can continue to improve the estimate by successive
halving of the step size to yield a general formula:
k = 2; j = 1
Note:
the subscripts
m and l refer to
more and less
accurate estimates

51. Following a similar pattern to Newton divided differences, Romberg’s Table can be produced
Error orders for j values
i = i = i = i = 4
j O(h2) O(h4) O(h6) O(h8)
1 h I1, I1, I1, I1,4
2 h/ I2, I2, I2,3
3 h/ I3, I3,2
4 h/ I4,1
Trapezoidal Simpson’s 1/3 Simpson’s 3/ Boole’s
Rule Rule Rule Rule

52. Gauss Quadrature
Extend the area
under the straight
line
f(x) x
f(x) x

53. Method of Undetermined Coefficients
Recall the trapezoidal rule
Before analyzing
this method,
answer this
question.
What are two
functions that
should be evaluated
exactly
by the trapezoidal
rule?
This can also be expressed as
where the c’s are constant

54. The two cases that should be evaluated exactly
by the trapezoidal rule: 1) y = constant
2) a straight line
f(x)
y = 1
f(x)
y = x
-(b-a)/2 x
(b-a)/2
-(b-a)/2 x
(b-a)/2

55. Thus, the following equalities should hold.
FOR y=1
since f(a) = f(b) =1
FOR y =x
since f(a) = x =-(b-a)/2
and
f(b) = x =(b-a)/2

56. Evaluating both integrals
For y = 1
For y = x
Now we have two equations and two unknowns, c0 and c1.
Solving simultaneously, we get :
c0 = c1 = (b-a)/2
Substitute this back into:

57. We get the equivalent of the trapezoidal rule.
DERIVATION OF THE TWO-POINT
GAUSS-LEGENDRE FORMULA
Lets raise the level of sophistication by:
- considering two points between -1 and 1
- i.e. “open integration”

58. f(x)
-1 x x1 1 x
Previously ,we assumed that the equation fit the integrals of a constant and linear function.
Extend the reasoning by assuming that it also fits the integral of a parabolic and a cubic function.

59. f(xi) is either 1, xi, xi2 or xi3
Solve these equations simultaneously

60. This results in the following
The interesting result is that the simple addition
of the function values at

61. However, we have set the limit of integration at -1 and 1.
This was done to simplify the mathematics. A simple
change in variables can be use to translate other limits.
Assume that the new variable xd is related to the
original variable x in a linear fashion.
x = a0 + a1xd
Let the lower limit x = a correspond to xd = -1 and the upper
limit x=b correspond to xd=1
a = a0 + a1(-1) b = a0 + a1(1)

62. a = a0 + a1(-1) b = a0 + a1(1)
SOLVE THESE EQUATIONS
SIMULTANEOUSLY
substitute

63. These equations are substituted for x and dx respectively.
Let’s do an example to appreciate the theory
behind this numerical method.

64. EXAMPLE
Estimate the following using two-point Gauss Legendre:

65. Higher-Point Formulas
For two point, we determined that c0 =c1 = 1
For three point:
c0 = x0=-0.775
c1= x1=0.0
c2= x2=0.775

66. Higher-Point Formulas
Your text goes on to provide additional weighting
factors (ci’s) and function arguments (xi’s)
in Table p. 623.

67. Improper Integrals How do we deal with integrals that do not
have finite limits
and bounded
integrands?
Our answer will focus
on improper integrals
where one limit (upper or
lower) is infinity.

68. Choose -A so that is is sufficiently large to
approach zero asymptotically.
Once chosen, evaluate the second part using a
Newton-Cotes formula.
Evaluate the first part with the following identity.

69. Numerical Differentiation
Forward finite divided difference
Backward finite divided difference
Center finite divided difference
All based on the Taylor Series

70. Forward Finite Difference

71. Forward Divided Difference
f(x)
(x i+1,y i+1)
actual
estimate
(xi, yi) x

72. Error is proportional to
the step size
first forward divided difference
O(h2) error is proportional to the square of the step size
O(h3) error is proportional to the cube of the step size

73. f(x)
actual
(xi,yi)
estimate
(xi-1,yi-1) x

74. Backward Difference Approximation of the
First Derivative
Expand the Taylor series backwards
The error is still O(h)

75. Centered Difference Approximation of the
First Derivative
Subtract backward difference approximation
from forward Taylor series expansion

76. f(x)
actual
(xi+1,yi+1)
(xi,yi)
estimate
(xi-1,yi-1) x

77. f(x) f(x) x x f(x) f(x) x x forward true derivative finite divided
difference approx.
true derivative x x
f(x)
f(x)
backward
finite divided
difference approx.
centered
finite divided
difference approx. x x

78. Numerical Differentiation
Forward finite divided differences Fig. 23.1
Backward finite divided differences Fig. 23.2
Centered finite divided differences Fig. 23.3
First - Fourth derivative

79. Richardson Extrapolation
Two ways to improve derivative estimates
decrease step size
use a higher order formula that employs more points
Third approach, based on Richardson extrapolation, uses two derivatives estimates to compute a third, more accurate approximation

80. Richardson Extrapolation
For a centered difference
approximation with
O(h2) the application of
this formula will yield
a new derivative estimate
of O(h4)

81. EXAMPLE
Given the following function, use Richardson’s extrapolation to determine the derivative at 0.5.
f(x) = -0.1x x x x +1.2
Note:
f(0) = 1.2
f(0.25) =1.1035
f(0.75) = 0.636
f(1) = 0.2

82. Derivatives of Unequally Spaced Data
Common in data from experiments or field studies
Fit a second order Lagrange interpolating polynomial to each set of three adjacent points, since this polynomial does not require that the points be equispaced
Differentiate analytically

83. Derivative and Integral Estimates for Data with Errors
In addition to unequal spacing, the other problem related to differentiating empirical data is measurement error
Differentiation amplifies error
Integration tends to be more forgiving
Primary approach for determining derivatives of imprecise data is to use least squares regression to fit a smooth, differentiable function to the data
In absence of other information, a lower order polynomial regression is a good first choice