1. Test Construction for Mathematical Functions Victor Kuliamin Institute for System Programming, Russian Academy of Sciences, Moscow

2. 2 / 25 02.06.2015 Mathematical Modeling Astrophysics Geosciences Biosciences Social Sciences Confidence?

3. 3 / 25 02.06.2015 Mathematical Libraries Floating-point numbers + arithmetics Basic math functions EElementary SSpecial Specialized libraries LLinear algebra NNumber theory NNumeric calculus DDynamic systems OOptimization …… : sqrt, pow, exp, log, sin, atan, cosh, … : erf, tgamma, j0, y1, …

4. 4 / 25 02.06.2015 How they should work? Intuition – everybody knows sin (?) Standards  IEEE 754 (Floating-point arithmetics) FP numbers, operations: +,-,*,/, sqrt  ISO 9899 (C language and libraries) 28 real functions  IEEE 1003.1 (POSIX) 34 real + 16 complex functions  ISO 10697.1-3 (Language independent arithmetics) Elementary real and complex functions

5. 5 / 25 02.06.2015 IEEE 754 Floating-point Numbers Normal : E > 0 & E < 2 k –1 X = (–1) S ·2 (E–B) ·(1+M/2 (n–k–1) ) Denormal : E = 0 X = (–1) S ·2 (–B+1) ·(M/2 (n–k–1) ) Special: E = 2 k –1  M = 0 : + , –   M ≠ 0 : NaN sign k+1 n-1n-1 0 exponentmantissa 0111111010010000000000000000000 01k n, k S E M B = 2 (k–1) –1 2 (–1) ·1.101 2 = 13/16 0, -0 1/0 = + , (–1)/0 = –  0/0 = NaN n = 32, k = 8 – float n = 64, k = 11 – double n = 79, k = 15– ext. double n = 128, k = 15– quadruple

6. 6 / 25 02.06.2015 Standard Requirements : IEEE 754 +, –, *, /, sqrt, remainder Correct rounding – 4 rounding modes  to +   to –   to 0  to the nearest Exception flags  INVALID:Incorrect arguments  DIVISION-BY-ZERO:Infinite result  OVERFLOW:Too big result  UNDERFLOW:Too small (or denormal) result  INEXACT:Inexact result 0

7. 7 / 25 02.06.2015 Standard Requirements : ISO C & POSIX ISO/IEC 9899 (C language) : 28 real functions  Exact values : sin(0) = 0, exp(0) = 1, …  DIVISION-BY-ZERO flag : log(0), atanh(1), pow(0,x), Г(-n)  NaN results and INVALID flag outside of domains IEEE 1003.1 (POSIX) : 34 real + 16 complex  All IEEE 754 flags (except for INEXACT) for real functions  Error settings Domain error ~ INVALID, Range error ~ OVERFLOW or UNDERFLOW  If x is denormal f(x) = x for each f(x)~x in 0 (sin, asin, sinh, expm1…) Contradiction with 3 rounding modes!

8. 8 / 25 02.06.2015 Standard Requirements : ISO 10697  Real and complex elementary functions (no erf, gamma, bessel)  Only symmetric rounding modes (no rounding to +  or to –  ) Preservation of sign Preservation of monotonicity Inaccuracy 0.5-2.0 ulp Evenness and oddity Exact values: cosh(0) = 1, log(1) = 0, … Asymptotics near 0 : cos(x) ~ 1, sin(x) ~ x, … Relations: expm1 = sinh, atan <= ↓( π /2 ), … for sin, cos, tan – small arguments only

9. 9 / 25 02.06.2015 Summary of Requirements Domain boundaries and poles (+ flags) Exact values, limits and asymptotics Preservation of sign and monotonicity Symmetries Evenness, periodicity, others : Г(1+x) = x·Г(x) Relations and range boundaries Correct rounding (according to mode) CComputational accuracy IInteroperability and portability of libraries and applications

10. 10 / 25 02.06.2015 Contradictions with Correct Rounding Correct rounding Oddity (sym. with –x, 1/x) Range boundaries POSIX : f(x) = x for denormal x and f(x)~x in 0

11. 11 / 25 02.06.2015 Test Data Construction Bit structure of FP numbers  Boundaries 0, - 0, + , - , NaN Least and greatest positive and negative, normal and denormal  Mantissa patterns FFFFFFFFFFFFF 16 FFFFF00000000 16 555550000FFFF 16 Both arguments and values of function Intervals of uniform function behavior Points hard to compute correctly rounded function value

12. 12 / 25 02.06.2015 Intervals and their boundaries max 0 Poles and overflow points Zeroes and extremes Tangents and asymtotics – horizontal and diagonal

13. 13 / 25 02.06.2015 Table Maker Dilemma (TMD) tan(1.1101111111111111111111111111111111111111111100011111 2 ·2 -22 ) = 1.1110000000000000000000000000000000000000000101010001 0 1 78 010… 2 ·2 -22 sin(1.1110000000000000000000000000000000000000011100001000 2 ·2 -19 ) = 1.1101111111111111111111111111111111111100000010111000 0 67 11101… 2 ·2 -19 Rounding to the nearest f = x.xxxxxxxxxx|0111111111...1xxx... f = x.xxxxxxxxxx|1000000000...0xxx... Rounding to 0, + , -  f = x.xxxxxxxxxx|0000000000...0xxx... f = x.xxxxxxxxxx|1111111111...1xxx... ? !

14. 14 / 25 02.06.2015 Number of Hard Points Probabilistic evaluation Uniform independent bits distribution Total N = 2 (n-k-1) values ~N·2 -m have m consecutive equal bits Real data for sin on exponent -16 Eval.  0, + , -   N 540.51 53112 52244 51466 5081012 49161921 4832 37 47647067 46128142106 45256280239 44512547518 4310241073996 42204821031985 41409641874040 40819283258142

15. 15 / 25 02.06.2015 Hard Points Calculation Techniques Exhaustive search Continued fractions (Kahan, 1983) Dyadic method (Tang, 1989; Kahan, 1994) Reduced search (Lefevre, 1997) Lattice reduction (Gonnet, 2002; Stehle, Lefevre, Zimmermann, 2003) Integer secants method (2007) Feasible only for single precision numbers X ≈ N·π; X = M·2 m ; 2 (n – k – 1) <= M < 2 (n – k)  π ≈ (2 m ·M)/N 3386417804515981120643892082331156599120239393299838035242121518428537 5540647742216209302675834747096020680456860263629892718144118637084998 6972132271594662263430201169763297290792255889271083061603403854134215 4669787134871905353772776431251615694251273653 · π/2 = 1.0110101011000101101100100110001011001010000111111110 1 857 011… 2 ·2 849 tan(1.0110101011000101101100100110001011001010000111111111 2 ·2 849 ) = -1.1101100110111010100110100111100101110101011000110101 1010… 2 ·2 60 sqrt(N·2 m ) ≈ M + ½; 2 (n-k-1) <= M, N < 2 (n-k)  2 (m+2) ·N = (2·M + 1) 2 – j  (2·M + 1) 2 = j (mod 2 (m+2) ) j = 15 sqrt(1.0010010101100101011001011100101011011100101111110100 2 ) = 1.0001001000001111100110011001111010011001001101110100 0 1 50 000… 2

16. 16 / 25 02.06.2015 Integer Secants Method Feasible near a tangent with integer linear coefficient The most hard points are near intersections of parallel secants with function graph F(x) = f(x) – a·x – b = c 1 x 2 + c 2 x 3 + c 3 x 4 + … F(x) = c 1 (G(x) ) 2, G(x) = x + d 1 x 2 + d 2 x 3 +… G(x) = y  x = H(y), H is reversed series x m = H(sqrt(m/c 1 2 z ))  F(x m ) = m/2 z 2–z2–z

17. 17 / 25 02.06.2015 Achievements Hard points  float (single precision) sqrt, cbrt, exp, sin, cos  double Some hard points with ≥ 48 additional bits can be found in crlibm tests http://lipforge.ens-lyon.fr/projects/crlibm All with ≥ 46 additional bits for sqrt Neighborhood of 0, with ≥ 40 additional bits for sin, asin, cos, acos, tan, atan, sinh, asinh, cosh, tanh, atanh, exp, exp2, expm1, log1p, j0 Neighborhood of +/- , with ≥ 40 additional bits exp, atan, tanh  extended double All with ≥ 53 additional bits for sqrt Neighborhood of 0, with ≥ 51 additional bits for sin, exp Test suites developed  double : sqrt, atan, exp, sin

18. 18 / 25 02.06.2015 Tested Libraries IDProcessor archLibraryOS x86_64 + glibcx86_64glibc 2.3.4Linux i686 + glibci686glibc 2.5,2.7Linux ia64 glibc 2.3.6,2.4Linux ppc32 glibc 2.3.5Linux ppc64 glibc 2.7Linux s390 glibc 2.4Linux i686 + VC8i686Visual C 8Windows XP x86_64 + VC6x86_64Visual C 6Windows XP sparcUltraSparc IIISun libcSolaris 10

19. 19 / 25 02.06.2015 Test Results : sqrt IDErrors – everywhere INVALID flag x86_64 + glibcNo comput errors i686 + glibculps: 1, 0, 0, 0(12,7%) ia64No comput errors ppc32No comput errors ppc64No comput errors s390No comput errors i686 + VC8ulps: 0, 1, 1, 1(40%,42%,42%), errno x86_64 + VC6ulps: 0, 1, 1, 1(40%,42%,42%), errno sparcNot-NaN for negative, no comput errors

20. 20 / 25 02.06.2015 Test Results : atan IDErrors – everywhere INVALID flag x86_64 + glibculps: 1, 22, 18, 18(71%,38%,38%, 6%) i686 + glibculps: 1, 1, 1, 1(27%,20%,20%,20%), up >π /2 ia64ulps: 1, 1, 1, 1(26% for all), up >π /2, non-POSIX ppc32ulps: 1, 22, 18, 18(5%,38%,38%, 6%) ppc64ulps: 1, 22, 18, 18(5%,38%,38%, 6%) s390ulps: 1, 22, 18, 18(5%,38%,38%, 6%) i686 + VC8ulps: 1, 2, 2, 1(30%,51%,51%,25%) x86_64 + VC6ulps: 1, 2, 2, 2(26%,51%,51%,50%) sparculps: 1, 1, 1, 1(22%,22%,22%,25%), up >π /2

21. 21 / 25 02.06.2015 Test Results : exp IDErrors – everywhere INV, (ex ia64) - UND, OVER, -inf x86_64 + glibcup, down, to 0 – huge, up ->1, down,0 – all, +inf1 i686 + glibculps: 1, 1, 1, 1(9%,12%,15%,16%), +inf1 ia64ulps: 1, 1, 1, 1(3%,4%,4%,4%), +inf1 ppc32up, down, to 0 – huge, up ->1, down,0 – all, +inf1 ppc64up, down, to 0 – huge, up – all, down – -0, +inf1 s390up, down, to 0 – huge, up – all, down – -0, U+O, +inf1 i686 + VC8ulps: 16, 17, 16, 16(71%,99%,40%,40%), +inf2, 0 x86_64 + VC6ulps: 1, 2, 1, 1(41%,90%,13%,13%), +inf2, 0x sparculps: 1, 1, 1, 1(6%,10%,10%,10%), +inf -> float exp( -16.9666900121946469 ) = 1.34027189065658032e+300 exp( 706.945585650006592 ) = -1.26136053650993277e+308 exp( 0.524284463190085037 ) = 1.11022302462515654e-16

22. 22 / 25 02.06.2015 Test Results : sin IDErrors – everywhere INV, DOMAIN x86_64 + glibculps: to nearest 1 (0,3%), other - huge i686 + glibcerrors increase to +/-inf ia64ulps: 1, 1, 1, 1(5%,5%,5%,6%) ppc32ulps: to nearest 1 (0,3%), other – huge ppc64ulps: to nearest 1 (0,3%), other – huge s390ulps: to nearest 1 (0,3%), other – huge i686 + VC8errors increase to +/-inf, bug for negative args x86_64 + VC6errors increase to +/-inf sparculps: 1, 2, 2, 2(8%,45%,45%,9%) sin( 8.50270011698847838 ) = 7.99976837364664238e+22 sin( -1.76788240868979430e-31 ) = 0.0980171403295605760

23. 23 / 25 02.06.2015 Future Development Complete set of hard points for some function (with ≥ 40 additional bits in double and with ≥ 51 bits in extended double precision) Complete list of POSIX functions Multiple variable functions

24. 24 / 25 02.06.2015 Conclusion No adequate standards for math libraries Several standards, sometimes contradictive, highly incomplete Correct rounding is suitable concept for standardization  It gives interoperability and almost all nice properties of implementations  It is feasible – crlibm, INRIA Need in methods for hard points calculation Current methods give only partial solution

25. 25 / 25 02.06.2015 Contact E-mail:kuliamin@ispras.ru Web:www.ispras.ru/~kuliaminwww.ispras.ru/~kuliamin Thank you! Questions?