1. E. T. S. I. Caminos, Canales y Puertos1 Lecture 2 Engineering Computation

2. E. T. S. I. Caminos, Canales y Puertos2 Smearing Occurs when individual terms are larger than summation itself. Consider the exponential series with x = -10 Consider formulas such as: With 7-decimal-digit accuracy: exact answer = 4.54 10 -05 computed answer = – 6.26 10 -05 (45 terms) wrong sign ! Largest intermediate terms are: 9th = –2,755.732 & 10th = 2,755.732 Round-off Error due to Arithmetic Operations

3. E. T. S. I. Caminos, Canales y Puertos3 Error caused by the nature of the numerical technique employed to approximate the solution. Example: Maclaurin series expansion of e x If we use a truncated version of the series: Then the Truncation Error is: Truncation Error

4. E. T. S. I. Caminos, Canales y Puertos4 Approximations and Rounding Errors Precautions: –Sums of large and small numbers: due to equaling the exponent. They are common in sums of infinite series where the individual terms are very small when compared with the accumulated sum. This error can be reduced by summing first the small terms and using double precision. –Cancellation of the subtraction: The subtraction of very similar numbers. –Smearing: The individual terms are larger than the total sum. –Inner products: They are prone to rounding errors. Thus, it is convenient to use double precision in this type of calculations.

5. E. T. S. I. Caminos, Canales y Puertos5 Error Propagation Errors which appear because we are basing current calculations on previous calculations which also incurred some form of error Stability and Condition Number Numerically Unstable: Computations which are so sensitive to round-off errors that errors grow uncontrollably during calculations. Condition: sensitivity to such uncertainty; "well conditioned" vs. "ill conditioned" Condition Number: measure of the condition; i.e., extent to which uncertainty in x is amplified by ƒ(x) C.N.  1 ===> "well-conditioned" C.N. >> 1 ===> "ill-conditioned"

6. E. T. S. I. Caminos, Canales y Puertos6 Basic Idea: Predict the value of a function, ƒ, at a point x i+1 based on the value of the function and all of its derivatives, ƒ, ƒ', ƒ",… at a neighboring point x i Given x i, ƒ(x i ), ƒ'(x i ), ƒ"(x i ),... ƒ n+1 (x i ), we can predict or approximate ƒ(x i+1 ) Taylor Series Expansion

7. E. T. S. I. Caminos, Canales y Puertos7 General Form: h = "step size" = x i+1 – x i R n = remainder to account for all other terms = O (h n+1 ) with x not exactly known "on the order of h n+1 " Note: f(x) must be a function with n+1 continuous derivatives with x i    x i+1 Taylor Series Expansion

8. E. T. S. I. Caminos, Canales y Puertos8  0 th order T.S. approx. (n = 0): f(x i+1 ) = f(x i ) + O (h 1 )  1 st order T.S. approx. (n = 1): f(x i+1 ) = f(x i ) + hf '(x i ) + O (h 2 )  2 nd order T.S. approx. (n = 2):  n th order T.S. approximation will be exact for an n th order polynomial Taylor Series Expansion

9. E. T. S. I. Caminos, Canales y Puertos9 Zero order First order Second order f(x i ) f(x i+1 )  f(x i ) f(x i+1 )  f(x i )+f '(x i )h f(x i+1 )  f(x i )+f '(x i )h+ )+f "(x i )h 2 /2! f(x i+1 ) True f(x ) x i+1 xixi h x Taylor Series Expansion

10. E. T. S. I. Caminos, Canales y Puertos10 Objective: Evaluate the derivatives of function, ƒ(x i ), without doing it analytically. When would we want to do this? 1. function is too complicated to differentiate analytically: 2. function is not defined by an equation, i.e., given a set of data points (x i, ƒ(x i )), i=1,…,n i01234 x i 1.03.05.07.09.0 ƒ(x i )2.34.15.55.75.9 Numerical Differentiation from Taylor Series Expansion

11. E. T. S. I. Caminos, Canales y Puertos11 Numerical Differentiation from Taylor Series Expansion –First derivative with backward difference.

12. E. T. S. I. Caminos, Canales y Puertos12 Backward Difference Approx.: First Derivative: first backward difference Letting h = x i - x i-1 Numerical Differentiation from Taylor Series Expansion

13. E. T. S. I. Caminos, Canales y Puertos13 Using data below calculate ƒ'(x 1 ) : i01234 x i 1.03.05.07.09.0 ƒ(x i )2.34.15.55.75.9 First Backward Finite-Divided-Difference at x 1 : f ' (x 1 )  0.9 { + O (h) } Example of 1st Backward FDD

14. E. T. S. I. Caminos, Canales y Puertos14 Second Derivative: + O([x i-2 – x i ] 3 ) withh = x i – x i-1 and 2h = x i – x i-2 The 2 nd order approximation to ƒ(x i-2 ) becomes: ƒ(x i-2 ) = ƒ(x i ) – 2hƒ'(x i ) + 2h 2 ƒ"(x i ) +O (h 3 ) [1] 2nd order approximation to ƒ(x i-1 ): [2] Backward Difference Approximation

15. E. T. S. I. Caminos, Canales y Puertos15 Subtracting 2*[2] from [1] yields: f(x i-2 ) – 2f(x i-1 ) = –f(x i ) + h 2 f"(x i ) + O (h 3 ) Rearranging: h 2 ƒ"(x i ) = f(x i ) – 2f(x i-1 ) + f(x i-2 ) + O (h 3 ) Second backward difference Backward Difference Approximation

16. E. T. S. I. Caminos, Canales y Puertos16 Using data below calculate ƒ"(x 2 ) : i01234 x i 1.03.05.07.09.0 ƒ(x i )2.34.15.55.75.9 Second Backward Finite-Divided-Difference at x 2 : f " (5.0)  - 0.1 { + O (h) } Example of 2nd Backward FDD

17. E. T. S. I. Caminos, Canales y Puertos17 What points are used for each form? Backward: …, ƒ(x i-2 ), ƒ(x i-1 ), ƒ(x i ), ƒ(x i+1 ), ƒ(x i+2 ), … Forward: …, ƒ(x i-2 ), ƒ(x i-1 ), ƒ(x i ), ƒ(x i+1 ), ƒ(x i+2 ), … Centered: …, ƒ(x i-2 ), ƒ(x i-1 ), ƒ(x i ), ƒ(x i+1 ), ƒ(x i+2 ), … Other Forms of Numerical Differentiation

18. E. T. S. I. Caminos, Canales y Puertos18 Taylor Series and Truncation errors - Higher order divided differences. - Second finite central divided difference

19. E. T. S. I. Caminos, Canales y Puertos19 Forward: Centered: Other Forms of Numerical Differentiation

20. E. T. S. I. Caminos, Canales y Puertos20 Taylor Series and Truncation errors Use of the Taylor series to calculate derivatives. –First derivative with forward difference.

21. E. T. S. I. Caminos, Canales y Puertos21 Taylor Series and Truncation errors –First derivative with central differences.

22. E. T. S. I. Caminos, Canales y Puertos22 Questions: Which is a better approximation? Forward, Centered, or Backward? Why? When would you use which? Note: We also can get higher order forward, centered, and backward difference derivative approximations [C&C Chapter 23, tabulated in Figs. 23.1-3] Taylor Series and Truncation errors

23. E. T. S. I. Caminos, Canales y Puertos23 Determine h to minimize the total error of a forward finite- divided difference approximation for: x i    x i+1 Round-off Error: Truncation Error: with  = machine epsilon. As a result: Example Combining Roundoff and Truncation Error

24. E. T. S. I. Caminos, Canales y Puertos24  Total error  =  truncation error  +  roundoff error  E = | Total Error |  NOTE: Truncation error decreases as h decreases Round-off error increases as h decreases + Example Combining Roundoff and Truncation Error

25. E. T. S. I. Caminos, Canales y Puertos25 Example Combining Roundoff and Truncation Error

26. E. T. S. I. Caminos, Canales y Puertos26 To minimize total error E with respect to h, set the first derivative to zero: Solve for h and approximate f "(  ) as f "(x i ): Example Combining Roundoff and Truncation Error

27. E. T. S. I. Caminos, Canales y Puertos27 Using the first forward-divided-difference approximation with error O(h) and a 5-decimal-digit machine: e = b 1-t = 10 1-5 = 10 -4 = 0.0001 f '(x) =  ;f "(x) = 0 Linear Application: Determine h that will minimize total error for calculating f’(x) for f(x) =  x at x = 1 Example Combining Roundoff and Truncation Error

28. E. T. S. I. Caminos, Canales y Puertos28 f(x+h)={exact: 3.1415} h  (x+h) f(x+h)-f(x)[f(x+h)-f(x)]/h 03.1415 0.0000013.141500 0.000013.14160.000110 0.00013.14190.00044.0 0.0013.14470.00323.2 0.013.17300.03153.15 0.13. 45570.31423.142 16.28313.14163.146 Underlined digits are subject to round-off error. They are likely to be in error by ± one or two units. This does not cause much problem when h = 1, but causes large errors in the final result when h < 10 -4. Example Combining Roundoff and Truncation Error

29. E. T. S. I. Caminos, Canales y Puertos29 Nonlinear Application: Determine h for minimizing the total error for computing f’(x) for ƒ(x) = e x at x = 3 Using the first forward-divided-difference approximation with error O(h) and a 5-decimal-digit machine: e = b 1-t = 10 1-5 = 10 -4 = 0.0001 f(x) = f '(x) = f "(x) = e x = 20.0855; or about 0.01 Example Combining Roundoff and Truncation Error

30. E. T. S. I. Caminos, Canales y Puertos30 Underlined digits subject to roundoff error. Bold digits in error due to truncation. full precision hf(x+h)=e x+h f(x+h)-f(x)[f(x+h)-f(x)][f(x+h)-f(x)]h 020.085{exact = 20.085} 0.0000120.0850.0020.086 0.000120.0870.0022020.086 0.00120.1050.0202020.096 0.0120.2870.20220.220.18 0.122.1982.11321.1321.12 154.59834.51334.52334.512 RoundoffTruncation Example Combining Roundoff and Truncation Error