1 Error Analysis Part 1 The Basics

2 Key Concepts Analytical vs. numerical Methods Representation of floating-point numbers Concept of significant digits Distinguishing different kinds of errors –Round-off / chopping / truncation errors –True/approximate absolute and relative errors –Acceptable errors

3 Analytical vs. Numerical Methods Find the intersection of y 1 = 2x + 3 y 2 = x + 2 Find the intersection of y 1 = x y 2 = cos(x)

4 Analytical vs. Numerical Methods Analytical Methods –Accurate solution –Difficult and not always possible Numerical Methods –Approximation of true solution –What method to use? How good is our approximation? (Error Analysis) How efficient is our method? (Algorithm design, Convergence rate) Does our methods always work? (Convergence)

5 Number Representation Do machines represent integers and floating-point numbers using the same representation? How does computer represent integers? How does computer represent floating- point numbers?

6 Representation of Integers 13 as 8-bit unsigned integers (no negative #) = = 0 x x x x x x x x 2 0 =

7 Exercise What is the equivalent decimal number represented by the following binary number? = ?

8 Representation of Floating-point Numbers = x 10 3 in an "imaginary" base-10 floating-point system

9 Normalized Representation (and notations used in this course) σ is the sign β is the base, e is the exponent –binary : β =2 –decimal : β =10 1/β ≤ m < 1(i.e., a 1 ≠ 0) –binary: 0.5 ≤ m < 1 –decimal: 0.1 ≤ m < 1

10 Representation of Floating-point Numbers SignSigned exponent ( e )Mantissa ( a )

11 Exercise What is the normalized floating-point representation of (for β = 2 )? What is the normalized floating-point representation of (for β = 2 )? What is the equivalent decimal value of ( ) 2 x 2 3 ?

12 There are discrete points on the number lines that can be represented by our computer. How about the space between ?

13 Implication of floating-point representations Only limited range of quantities may be represented –Number too larger  overflow –Number too small (too close to 0)  underflow Only a finite number of quantities may be represented –round-off or chopping errors

14 Exercise Consider the following floating-point representation –The mantissa has only 3 bits –Exponent, e, ranges from -4 to 4 Can you give an integer that cannot be represented by this representation? Can you give an integer between 0 and 14 that cannot be represented by this representation?

15 IEEE 754 Floating-point Representation Size in bits Sign (0=+ve, 1 = -ve) ExponentBias of the exponent Mantissa Single precision (float) 32 bits1 bit8 bits (-126 to +127) bits double precision (double) 64 bits1 bit11 bits (-1022 to +1023) bits Larger exponent  Wider range of numbers Longer mantissa  Higher precision

16 Note on IEEE 754 Representation Exponents of all 0's and 1's are reserved for special numbers. Zero is a special value denoted with an exponent field of zero and a mantissa field of zero, and we could have +0 and -0. +∞ an -∞ are denoted with an exponent of all 1's and a mantissa field of all 0's. NaN (Not-a-number) is denoted with an exponent of all 1's and a non-zero mantissa field.

17 Errors and Significant Digits I paid $10 for 7 oranges. What is unit price of each orange? $ (that is the exact output from my computer !!) Is there any difference between $ and $1.4? Is there any difference between $1.4 and $1.40?

18 Significant figures, or digits The significant digits of a number are those that can be used with confidence. They correspond to the number of certain digits plus one estimated digits. x = 3.5 (2 significant digits)  3.45 ≤ x < 3.55 x = (5 significant digits)  ≤ x <

19 Excercise Suppose x = –Show the value of x up to 4 significant digits. –Show the value of x up to 10 significant digits. Calculate 22/7 up to 5 significant digits.

20 Concepts of Significant Digits Suppose x = is the true solution Which of the following calculated values is/are accurate to 3 significant digits with respect to x ? a = b = c = d =

21 Concepts of Significant Digits x A (approximate value) has m significant digits (with respect to x T, the true value) if the absolute error | x T - x A | has magnitude less than or equal to 5 in the ( m + 1) st digit of x T counting to the right from the first non-zero digit in x T.  3 significant digits e.g. 1:

22  4 significant digits e.g. 2:  2 significant digits e.g. 3:

23 Excercise Suppose x = is the true solution Which of the following calculated values is/are accurate to 3 significant digits with respect to x ? a = (| x - a | = ) b = (| x - b | = ) c = (| x - c | = ) d = (| x - d | = )

24 Scientific Notation How do we express the number 45,300 meaningfully? 4.53 x 10 4 to denote the number is known to 3 significant figures x 10 4 to denote the number is known to 4 significant figures x 10 4 to denote the number is known to 5 significant figures.

25 Implications As numerical methods yield approximate results, we must therefore develop criteria to specify how confidence we are in our approximate result. Usually, in terms of 1) Significant digits, or 2) Absolute/relative error bounds

26 Error Definition (True Error) x T – true value x A – approximate value True Error in x A (exact value of the error) = True Relative Error in x A = True Percentage Relative Error in x A =

27 Error Definition e.g., Error = Relative error =

28 If we do not know the true value x T, we can replace it by an estimation of the true value. The result is, we have the approximate error, and the approximate relative error instead. Error Definition (Approximate Error)

29 x A (i) – approximate value in the i th iteration of an iterative approach Approximate Error in x A = Approximate Relative Error in x A = Approximate Percentage Relative Error in x A Error Definition (Approximate Error)

30 Example: Maclaurin Series When x = 0.5 TermsResultε t (True percentage relative error) ε a (Approx. percentage relative error) 11( )/ = 39.3% 21.5( )/ = 9.02%(1.5-1)/1.5 = 33.3% %( )/1.625 = 7.69% %1.27% %0.158% %0.0158%

31 How many terms should we use? TermsResultεtεt εaεa % %33.3% %7.69% %1.27% %0.158% %0.0158% Computation stops when |ε a | < ε s –ε s = pre-determined acceptable percentage relative error If we want the result to be correct to at least n significant digits, it is suggested that we set ε s = (0.5 x 10 2-n )%

32 Summary Floating-point number representation and its implication –Round-off and chopping errors Significant digits The definitions of –True errors, true relative errors, true percentage errors, –Approximate errors, approximate relative errors, approximate percentage relative errors

33 Next Errors do not occur only in the space between the discrete values (rounding or chopping error) Errors also appear in many stages. Propagation of round-off errors Truncation errors