1. 1 An Introduction to the Quality of Computed Solutions Sven Hammarling NAG Ltd, Oxford sven@nag.co.uk

2. 2 Plan of Talk Introduction Floating point numbers and IEEE arithmetic Condition, stability and error analysis with examples Implications for software. –LAPACK and NAG Other approaches Summary

3. 3 Introduction to NAG

4. 4 NAG History 1970 -Nottingham Algorithms Group 1971 - Mark 1 NAG Library 1973 - NAG moved to Oxford, renamed Numerical Algorithms Group 1976 -NAG Ltd, a non-profit company 1978 -NAG Inc established in USA 1980 -NAG Ltd financially self-sufficient 1990 -NAG GmbH established 1998 - Nihon NAG KK established

5. 5 What does non-profit mean? No shareholders, no owner surplus is re-invested in the Company

6. 6 How many people work at NAG? Distributors worldwide

7. 7 NAG Products and Services Product Lines –Numerical Libraries –Statistical Systems –NAGWare: Compiler and tools –Visualization and Graphics –PDE Solutions Consultancy Customer Support www.nag.co.uk or www.nag.comwww.nag.co.ukwww.nag.com

8. 8 NAG Numerical Libraries F77 F90 C Parallel SMPMPI FL90Plus

9. 9 Quality of Computed Solutions

10. 10 Quality of Computed Solutions The quality of computed solutions is concerned with assessing how good a computed solution is in some appropriate measure

11. 11 Software Quality Quality software should implement reliable algorithms and should provide measures of solution quality

12. 12 Floating Point Numbers Floating point numbers are a subset of the real numbers that can be conveniently represented in the finite word length of a computer, without unduly restricting the range of numbers represented For example, the IEEE standard uses 64 bits to represent double precision numbers in the approximate range

13. 13 Floating Point Numbers - Representation

14. 14 Floating Point Numbers - Example

15. 15 Floating Point Numbers - Example (cont’d) 0 1 2 3

16. 16 IEEE Arithmetic Standard ANSI/IEEE Standard 754-1985 is a standard for binary arithmetic (b = 2). The standard specifies: Floating point number formats Results of the basic floating point operations Rounding modes Signed zero, infinity ( ) and not-a-number (NaN) Floating point exceptions and their handling Conversion between formats Most modern machines use IEEE arithmetic.

17. 17 IEEE Arithmetic Formats

18. 18 Why Worry about Computed Solutions? Tacoma bridge collapse North Sea oil rig collapse Vancouver stock exchange index, Jan 1982 to Nov 1983 Ariane 5 rocket, flight 501, failure, 4 June 1996 Patriot missile, 25 Feb 1991 Auckland Bridge London millennium bridge

19. 19 Tacoma Bridge

20. 20 Auckland Bridge, 1975

21. 21 Millennium Bridge, 2000

22. 22 Web Sites Disasters attributable to bad numerical computing: http://www.math.psu.edu/dna/disasters/ http://www.math.psu.edu/dna/disasters/ Numerical problems: RISKS-LIST: http://catless.ncl.ac.uk/Risks/ http://catless.ncl.ac.uk/Risks/ London millennium bridge: http://www.arup.com/MillenniumBridge/ http://www.arup.com/MillenniumBridge/

23. 23 Example - Means

24. 24 Example - Sample Variance

25. 25 Excel Example: Standard Deviation

26. 26 Overflow/Underflow Example: Modulus of a Complex Number

27. 27 Excel Example: Standard Deviation - Overflow

28. 28 Stability The stability of a method for solving a problem is concerned with the sensitivity of the method to (rounding) errors in the solution process A method that guarantees as accurate a solution as the data warrants is said to be stable, otherwise the method is unstable

29. 29 Condition The condition of a problem is concerned with the sensitivity of the problem to perturbations in the data A problem is ill-conditioned if small changes in the data cause relatively large changes in the solution. Otherwise a problem is well- conditioned

30. 30 Condition Examples

33. 33 Condition Examples (cont’d)

35. 35 Condition Examples (cont’d)

37. 37 Condition Number for Linear Equations

38. 38 Condition Number Example

39. 39 Stability Examples

40. 40 Stability Examples (Contd) recursive.m

41. 41 Stability Examples (Contd)

42. 42 Stability Examples (cont’d) brecursive.m

43. 43 Error Analysis Error analysis is concerned with establishing whether or not an algorithm is stable for the problem in hand A forward error analysis is concerned with how close the computed solution is to the exact solution A backward error analysis is concerned with how well the computed solution satisfies the problem to be solved

44. Example

45. 45 The Purpose of Error Analysis “The clear identification of the factors determining the stability of an algorithm soon led to the development of better algorithms. The proper understanding of inverse iteration for eigenvectors and the development of the QR algorithm by Francis are the crowning achievements of this line of research. “For me, then, the primary purpose of the rounding error analysis was insight.” Wilkinson, 1986 Bulletin of the IMA, Vol 22, p197

46. 46

47. 47 Backward Error and Perturbation Analysis

48. Condition and Error Analysis

49. 49 LAPACK Systems of linear equations Linear least squares problems Eigenvalue and singular value problems, including generalized problems Matrix factorizations Condition and error estimates The BLAS as a portability layer Linear Algebra PACKage for high-performance computers Dense and banded linear algebra for Shared Memory

51. LAPACK and Error Bounds “In Addition to providing faster routines than previously available, LAPACK provides more comprehensive and better error bounds. Our goal is to provide error bounds for most quantities computed by LAPACK.” LAPACK Users’ Guide, Chapter 4 – Accuracy and Stability

52. Error Bounds in LAPACK The Users’ Guide gives details of error bounds and code fragments to compute those bounds. In many cases the routines return the bounds directly

53. Error Bounds in LAPACK: Example

54. 54 ODE Software Example NAG routine D02PZF returns global error estimates for the Runge-Kutta solvers D02PCF and D02PDF: D02PCZ( RMSERR, ERRMAX, TERRMX,…) RMSERR: Approximate Root mean square error ERRMAX: Max approximate true error TERRMX: Point at which max approximate true error occurred (These routines are based upon RKSUITE)

55. What to do if Software does not Provide Error Estimates? Run the problem with perturbed data Better still, use a software tool such as PRECISE which allows you to perform a stochastic analysis Put pressure on developers to provide estimates

56. PRECISE Provides a module for statistical backward error analysis Provides a module for sensitivity analysis http://www.cerfacs.fr/algor/Softs/PRECISE/ index.htmlhttp://www.cerfacs.fr/algor/Softs/PRECISE/ index.html

57. Quality of Reliable Software

58. 58 CADNA CADNA is another package with the same aims as PRECISE http://www-anp.lip6.fr/cadna/Accueil.php It is now free to academics

59. 59 Cancellation Using four figure decimal arithmetic

60. 60 Interval Analysis Interval arithmetic works on intervals that contain the exact solution Interval analysis can provide automatic error bounds for a number of problems See www.cs.utep.edu/interval-comp/main.html

61. 61 Summary Error analysis gives insight In linear algebra we usually aim for a backward error analysis Be concerned about the quality of computed solutions Use quality software, complain about low quality software Write quality software and be proud of your results

62. 62 Quote “You have been solving these damn problems better than I can pose them.” Sir Edward Bullard, Director NPL, in a remark to Wilkinson. (mid 1950s.) See NAG Newsletter 2/83, page 11

63. References [1] E. Anderson, Z. Bai, C. H. Bischof, J. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Blackford & D. C. Sorensen. LAPACK Users’ Guide, 3 rd Edition, SIAM, 1999. [2] F. Chaitin-Chatelin & V. Frayssé. Lectures on Finite Precision Computations, SIAM, 1996. [3] G. H. Golub & C. F. Van Loan. Matrix Computations, 3 rd Edition, The Johns Hopkins University Press, 1996. [4] N. J. Higham. Accuracy and Stability of Numerical Algorithms, 2 nd edition, SIAM, 2002. [5] www.cs.utep.edu/interval-comp/main.htmlwww.cs.utep.edu/interval-comp/main.html [6] www.nag.co.uk/numeric/numerical_libraries.asp

64. References (cont’d) [7] F. S. Acton. Numerical Methods that Usually Work. Harper and Row, 1970. [8] G. E. Forsythe. What is a Satisfactory Quadratic Equation Solver. In Constructive Aspects of the Fundamental Theorem of Algebra, B. Dejon and P. Henrici, P., editors, pages 53-61, Wiley, 1969. [9] G. E. Forsythe. Pitfalls in Computation, or Why a Math Book isn't enough. Amer. Math. Monthly, 9: 931-995, 1970. [10] J. H. Wilkinson. The Perfidious Polynomial. In Studies in Numerical Analysis, G. H. Golub, editor, volume 24 of Studies in Mathematics, Mathematical Association of America, Washington D.C., pages 1-24, 1984.