1. 1 Chapter 1 Introductory Concepts and Calculus Review

2. 2 Introduction The subjects The subjects The derivation of the algorithms The derivation of the algorithms The implementation of the algorithms The implementation of the algorithms Analyze the algorithms mathematically Analyze the algorithms mathematically Accuracy, efficiency, and stability Accuracy, efficiency, and stability

3. 3 1.1 Basic Tools of Calculus 1.1.1 Taylor ’ s Theorem Integral mean value theorem

4. 4 Three particular expansions of Taylor ’ s Theorem where x 0 = ?

5. 5 Three particular expansions of Taylor ’ s Theorem where x 0 = 0

6. 6 Example : e x Finally, n can be found! (here n = 9 ) If we want then Let x 0 = 0

7. 7 Example : e x p 2 (x) p 9 (x) exp(x)

8. 8 Example : e x

9. 9 The result tells us The result tells us We can approximate the exponential function to within 10 -6 accuracy using a specific polynomial, and this accuracy holds for all x in a specified interval. We can approximate the exponential function to within 10 -6 accuracy using a specific polynomial, and this accuracy holds for all x in a specified interval.

10. 10 Example 1.1 Let f (x) = (x+1) 1/2, then the second-order Taylor polynomial (computed about x 0 = 0 ) is computed as follows: Let f (x) = (x+1) 1/2, then the second-order Taylor polynomial (computed about x 0 = 0 ) is computed as follows: 2

11. 11 Example 1.2: sin Function: Function: Accuracy: Accuracy:

12. 12 Example 1.3: arctan Function: Function: http://zh.wikipedia.org/wiki/File:Atan_acot_plot.svg

13. 13 Example 1.3: arctan Function: Function: Error term Let and

14. 14 Example 1.3 : arctan Please determine the error in a ninth-degree Taylor approximation to the arctangent function. Please determine the error in a ninth-degree Taylor approximation to the arctangent function. Since 2n +1 = 9 implies that n = 4, we have Since 2n +1 = 9 implies that n = 4, we have

15. 15 Taylor ’ s Theorem Expansion Let x  x + h and x 0  x

16. 16 1.1.2 Mean Value and Extreme Value Theorems http://en.wikipedia.org/wiki/Mean_value_theorem

17. 17 1.1.2 Mean Value and Extreme Value Theorems W

18. 18 1.1.2 Mean Value and Extreme Value Theorems Mm Critical point

19. 19 1.1.2 Mean Value and Extreme Value Theorems The integral mean value theorem (a corollary of the intermediate value theorem) states that a function continuous on an interval takes on its average value somewhere in the interval. More exactly, if is continuous on, then there exists in such that.

20. 20 1.1.2 Mean Value and Extreme Value Theorems

21. 21 1.2 Error, Approximate Equality, and Asymptotic Order Notation 1.2.1 Error A : a quantity( 數量 ) we want to compute A : a quantity( 數量 ) we want to compute A h : an approximation( 近似值 ) to that quantity A h : an approximation( 近似值 ) to that quantity Relative error ( 相對誤差 ) is better. Relative error ( 相對誤差 ) is better. These errors are both computational errors. These errors are both computational errors.

22. 22 1.2.2 Notation: Approximate Equality Approximate equality Approximate equality It is an equivalence relation, and satisfy the following properties: It is an equivalence relation, and satisfy the following properties: Transitive( 遞移性 ): Transitive( 遞移性 ): Symmetric( 對稱性 ): Symmetric( 對稱性 ): Reflexive( 反身性 ): Reflexive( 反身性 ):

23. 23 1.2.3 Notation: Asymptotic Order (Big O)

24. 24 Example 1.4 Let Let Simple calculus shows that Simple calculus shows that so that we have so that we have Here Here

25. 25 1.2.3 Notation: Asymptotic Order (Big O)

26. 26 Example 1.6

27. 27

28. 28 1.3 A Primer on Computer Arithmetic Computer arithmetic is generally inexact. Computer arithmetic is generally inexact. While the errors are very small, they can accumulate and dominate the calculation. While the errors are very small, they can accumulate and dominate the calculation. Example: floating-point arithmetic Example: floating-point arithmetic Reference: An Introduction to Computer Science, Chapter 3, Excess System (Excess_127 or Excess_1023) Reference: An Introduction to Computer Science, Chapter 3, Excess System (Excess_127 or Excess_1023) is the sign of the number, f is the fraction ( 0 <= f <= 1 ), is the base of the internal number system

29. IEEE standards for floating-point representation （底數 尾數）

30. Example Show the representation of the normalized number + 2 6 x 1.01000111001 Solution The sign is positive. The Excess_127 representation of the exponent is 133. You add extra 0s on the right to make it 23 bits. The number in memory is stored as: 0 10000101 01000111001000000000000

31. 31 Errors Rounding error v.s. chopping error Rounding error v.s. chopping error Rounding: 四捨五入 Rounding: 四捨五入 Chopping: 無條件捨去 Chopping: 無條件捨去 Discussion: Discussion: Rounding is more accurate but chopping is faster. Rounding is more accurate but chopping is faster. The chopping error is indeed lager than the rounding error. The chopping error is indeed lager than the rounding error.

32. 32 Example Rounding error Rounding error Chopping error Chopping error

33. 33 Subtractive Cancellation If a and b are accurate to 16 decimal digits. What about their difference c = a - b ? If a and b are accurate to 16 decimal digits. What about their difference c = a - b ? Example: Example: The result c is accurate to 12 digits. The result c is accurate to 12 digits. This is because we were subtracting two nearly equal numbers. This is because we were subtracting two nearly equal numbers.

34. 34 Example Function : Function : We know that : We know that : Taylor ’ s Theorem : Taylor ’ s Theorem :

35. 35 ……

36. 36 1.5 Simple Approximations Error function: Error function: (probability theory) (probability theory) It is not possible to evaluate this integral by means of the fundamental theorem of calculus. It is not possible to evaluate this integral by means of the fundamental theorem of calculus. Use Taylor ’ s Theorem to approximate. Use Taylor ’ s Theorem to approximate. where where

37. 37 Substitution:Define So that we have Set where c depends on t and

38. 38 Apply the Integral Mean Value Theorem: The structured form: where

39. 39 Use the big O notation: Use the big O notation: Use the approximate equality notation: Use the approximate equality notation: Simplify: if the values of x between 0 and 2 Simplify: if the values of x between 0 and 2 if k >=1 if k >=1 thus thus

40. 40 Fundamental Idea When confronted with a computation that cannot be done exactly, we often replace that relevant function with something simpler that approximates it, and carry out the computation exactly on the simple approximation. When confronted with a computation that cannot be done exactly, we often replace that relevant function with something simpler that approximates it, and carry out the computation exactly on the simple approximation.