ECIV 201 Computational Methods for Civil Engineers Richard P. Ray, Ph.D., P.E. Error Analysis

Approximations and Round-Off Errors For many engineering problems, we cannot obtain analytical solutions. Numerical methods yield approximate results, results that are close to the exact analytical solution. We cannot exactly compute the errors associated with numerical methods. – Only rarely given data are exact, since they originate from measurements. Therefore there is probably error in the input information. – Algorithm itself usually introduces errors as well, e.g., unavoidable round-offs, etc … – The output information will then contain error from both of these sources. 2

Approximation and Round-off Errors  How confident we are in our approximate result?  The question is “how much error is present in our calculation and is it tolerable?”

 Accuracy. How close is a computed or measured value to the true value  Precision (or reproducibility). How close is a computed or measured value to previously computed or measured values.  Inaccuracy (or bias). A systematic deviation from the actual value.  Imprecision (or uncertainty). Magnitude of scatter. 4

5

Significant Figures  Number of significant figures indicates precision. Significant digits of a number are those that can be used with confidence, e.g., the number of certain digits plus one estimated digit. 53,800How many significant figures? 5.38 x x x Zeros are sometimes used to locate the decimal point not significant figures

Error Definitions True Value = Approximation + Error E t = True value – Approximation (+/-) 7 True error

 For numerical methods, the true value will be known only when we deal with functions that can be solved analytically (simple systems). In real world applications, we usually will not know the answer a priori. Then  Iterative approach, example Newton’s method 8 (+ / -)

 Use absolute value.  Computations are repeated until stopping criterion is satisfied.  If the following criterion is met you can be sure that the result is correct to at least n significant figures. 9 Pre-specified % tolerance based on the knowledge of your solution

Round-off Errors  Numbers such as , e, or cannot be expressed by a fixed number of significant figures.  Computers use a base-2 representation, they cannot precisely represent certain exact base-10 numbers.  Fractional quantities are typically represented in computer using “floating point” form, e.g., 10 exponent Base of the number system used mantissa Integer part

11

12

 x10 3 in a floating point base-10 system Suppose only 4 decimal places to be stored  Normalized to remove the leading zeroes. Multiply the mantissa by 10 and lower the exponent by x Additional significant figure is retained

Therefore for a base-10 system 0.1 ≤m<1 for a base-2 system0.5 ≤m<1  Floating point representation allows both fractions and very large numbers to be expressed on the computer. However,  Floating point numbers take up more room.  Take longer to process than integer numbers.  Round-off errors are introduced because mantissa holds only a finite number of significant figures. 14

Chopping Example:  = to be stored on a base-10 system carrying 7 significant digits.  = chopping error  t = If rounded  =  t =  Some machines use chopping, because rounding adds to the computational overhead. Since number of significant figures is large enough, resulting chopping error is negligible. 15

Truncation Errors and the Taylor Series  Non-elementary functions such as trigonometric, exponential, and others are expressed in an approximate fashion using Taylor series when their values, derivatives, and integrals are computed.  Any smooth function can be approximated as a polynomial. Taylor series provides a means to predict the value of a function at one point in terms of the function value and its derivatives at another point. 16

17

Example: To get the cos(x) for small x: If x=0.5 cos(0.5)= … = From the supporting theory, for this series, the error is no greater than the first omitted term. 18

Any smooth function can be approximated as a polynomial. f(x i+1 ) ≈ f(x i ) zero order approximation, only true if x i+1 and x i are very close to each other. f(x i+1 ) ≈ f(x i ) + f′(x i ) (x i+1 -x i )first order approximation, in form of a straight line 19

20 (x i+1 -x i )= hstep size (define first) Reminder term, R n, accounts for all terms from (n+1) to infinity. n th order approximation

 is not known exactly, lies somewhere between x i+1 >  >x i. Need to determine f n+1 (x), to do this you need f'(x). If we knew f(x), there wouldn’t be any need to perform the Taylor series expansion. However, R=O(h n+1 ), (n+1) th order, the order of truncation error is h n+1. O(h), halving the step size will halve the error. O(h 2 ), halving the step size will quarter the error. 21

 Truncation error is decreased by addition of terms to the Taylor series.  If h is sufficiently small, only a few terms may be required to obtain an approximation close enough to the actual value for practical purposes. Example: Calculate series, correct to the 3 digits. 22

Error Propagation  fl(x) refers to the floating point (or computer) representation of the real number x. Because a computer can hold a finite number of significant figures for a given number, there may be an error (round-off error) associated with the floating point representation. The error is determined by the precision of the computer (  ). 23

 Suppose that we have a function f(x) that is dependent on a single independent variable x. fl(x) is an approximation of x and we would like to estimate the effect of discrepancy between x and fl(x) on the value of the function: 24

Figure

Also, let  t, the fractional relative error, be the error associated with fl(x). Then Rearranging, we get 26 Machine epsilon, upper boundary

 Case 1: Addition of x 1 and x 2 with associated errors  t1 and  t2 yields the following result: fl(x 1 )=x 1 (1+  t1 ) fl(x 2 )=x 2 (1+  t2 ) fl(x 1 )+fl(x 2 )=  t1 x 1 +  t2 x 2 +x 1 +x 2 27 A large error could result from addition if x 1 and x 2 are almost equal magnitude but opposite sign, therefore one should avoid subtracting nearly equal numbers.

 Generalization: Suppose the numbers fl(x 1 ), fl(x 2 ), fl(x 3 ), …, fl(x n ) are approximations to x 1, x 2, x 3, …,x n and that in each case the maximum possible error is E. fl(x i )-E ≤ x i ≤ fl(x i )+EE ti ≤E It follows by addition that 28 nE So that Therefore, the maximum possible error in the sum of fl(x i ) is.

 Case 2: Multiplication of x 1 and x 2 with associated errors e t1 and e t2 results in: 29

 Since  t1,  t2 are both small, the term  t1  t2 should be small relative to  t1 +  t2. Thus the magnitude of the error associated with one multiplication or division step should be  t1 +  t2.  t1 ≤  (upper bound)  Although error of one calculation may not be significant, if 100 calculations were done, the error is then approximately 100 . The magnitude of error associated with a calculation is directly proportional to the number of multiplication steps. 30

 Overflow: Any number larger than the largest number that can be expressed on a computer will result in an overflow.  Underflow (Hole) : Any positive number smaller than the smallest number that can be represented on a computer will result an underflow.  Stable Algorithm: In extended calculations, it is likely that many round-offs will be made. Each of these plays the role of an input error for the remainder of the computation, impacting the eventual output. Algorithms for which the cumulative effect of all such errors are limited, so that a useful result is generated, are called “stable” algorithms. When accumulation is devastating and the solution is overwhelmed by the error, such algorithms are called unstable. 31

Figure