University of Maryland Dynamic Floating-Point Error Detection Mike Lam, Jeff Hollingsworth and Pete Stewart

University of Maryland 2 Motivation Finite precision -> roundoff error Compromises ill-conditioned calculations Hard to detect and diagnose Increasingly important as HPC grows Single-precision is faster on GPUs Double-precision fails on long-running computations Previous solutions are problematic Numerical analysis requires training Manual re-writing and testing in higher precision is tedious and time-consuming

University of Maryland 3 Our Solution Instrument floating-point instructions Automatic Minimize developer effort Ensure analysis consistency and correctness Binary-level Include shared libraries w/o source code Include compiler optimizations Runtime Data-sensitive

University of Maryland 4 Our Solution Three parts Utility that inserts binary instrumentation Runtime shared library with analysis routines GUI log viewer General overview Find floating-point instructions and insert calls to shared library Run instrumented program View output with GUI

University of Maryland 5 Our Solution Dyninst-based instrumentation Cross-platform No special hardware required Stack walking and binary rewriting Java GUI Cross-platform Minimal development effort

University of Maryland 6 Our Solution Cancellation detection Instrument addition & subtraction Compare runtime operand values Report cancelled digits Side-by-side (“shadow”) calculations Instrument all floating-point instructions Higher/lower precision Different representation (i.e. rationals) Report final errors

University of Maryland 7 Cancellation Detection Overview Loss of significant digits during operations For each addition/subtraction: Extract value of each operand Calculate result and compare magnitudes (binary exponents) If e ans < max(e x,e y ) there is a cancellation For each cancellation event: Record a “priority:” max(e x,e y ) - e ans Save event information to log

University of Maryland 8

9

10 Gaussian Elimination A -> [L,U] Comparison of eight methods Classical Classical w/ partial pivoting Classical w/ full pivoting Bordering (“Sherman’s march”) “Pickett’s charge” “Pickett’s charge” w/ partial pivoting Crout’s method Crout’s method w/ partial pivoting

University of Maryland 11 Gaussian Elimination

University of Maryland 12 Gaussian Elimination Classical vs. Bordering

University of Maryland 13 Gaussian Elimination ClassicalBordering Operations Cancellations399 Cancels/ops14%3% Average bits

University of Maryland 14 SPEC Benchmarks Results are hard to interpret without domain knowledge Overheads:

University of Maryland 15 Roundoff Error Sparse “shadow value” table Maps memory addresses to alternate values Shadow values can be single-, double-, quad- or arbitrary-precision Other ideas: rationals, # of significant digits, etc. Instrument every FP instruction Extract operation type and operand addresses Perform the same operation on corresponding shadow values Output shadow values and errors upon termination

University of Maryland 16

University of Maryland 17 More Gaussian Elimination Maximum relative error 25x2550x50100x100 Partial pivoting9.3e-102.3e-21.0 Full pivoting1.3e-152.4e-154.8e-15

University of Maryland 18 Issues & Possible Solutions Expensive overheads ( X) Optimize with inline snippets Reduce workload with data flow analysis Following values through compiler optimizations Selectively instrument MOV instructions Filtering false positives Deduce “root cause” of error using data flow

University of Maryland 19 Conclusion Analysis of floating-point error is hard Our tool provides automatic analysis of such error Work in progress

University of Maryland 20 Thank you!