1. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 1 Robert Könighofer and Roderick Bloem IAIK – Graz University of Technology robert.koenighofer@iaik.tugraz.at www.iaik.tugraz.at This work was supported in part by the European Commission through project DIAMOND (FP7-2009-IST-4-248613). FMCAD 2011Robert KönighoferAustin, 2011-10-31 Automated Error Localization and Correction for Imperative Programs

2. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 2 FMCAD 2011Robert KönighoferAustin, 2011-10-31  Addressed problem  Debugging approach  Program analysis  Error localization  Error correction  Experimental results  Summary and conclusions Outline

3. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 3 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Addressed Problem  Automate debugging of simple software  Input:  Incorrect program  Specification:  Assertions  Reference Implementation  Output:  Potential error locations  Potential repairs int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; }

4. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 4 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

5. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 5 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

6. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 6 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; }

7. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 7 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

8. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 8 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis  Can handle all incorrect expressions int max(int x, int y) { int res = x; int tmp = y > x; if(tmp) res = x; assert(res>=x && res>=y); return res; } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1 c0c0 c2c2 c1c1

9. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 9 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Pre-Processing  Components may be faulty  Transformation: bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

10. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 10 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Program Analysis  When is my buggy program correct?  We use Symbolic Execution  Allows incomplete analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } Symbolic Execution

11. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 11 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Program Analysis  When is my program correct? bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; }

12. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 12 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Program Analysis  When is my buggy program correct? bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Diagnostic Data

13. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 13 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

14. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 14 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

15. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 15 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness

16. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 16 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

17. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 17 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

18. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 18 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = repair(…); assert(res>=x && res>=y); return res; } c0c0 c1c1

19. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 19 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = repair(…); if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

20. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 20 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = repair(…); if(y > x) res = repair(…); assert(res>=x && res>=y); return res; } c0c0 c1c1

21. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 21 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Program Analysis Diagnostic Data:  Checking for Repairability:

22. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 22 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Program Analysis Diagnostic Data:  Checking for Repairability:

23. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 23 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Use SMT-Solver to check repairability  Problem: Quantifier alternation  Check for a finite set of input values only  False positives

24. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 24 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Localization  Use SMT-Solver to check repairability  Problem: Quantifier alternation  Check for a finite set of input values only  False positives  A set D is a diagnosis iff Repairable(AC_c 0, AC_c 1, …,AC_c n ) = true with AC_c k = false iff c k  D  Computing diagnoses naively: check every set  Our algorithm: Unsatisfiable cores + hitting sets

25. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 25 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

26. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 26 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  We know which components are faulty  E.g.: D 1 ={c 8 }, D 2 ={c 2, c 4 }  Error Correction  Compute replacements for the faulty components  E.g.:c 8  x+9 c 2  a <= b and c 4  1

27. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 27 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Use information about faulty components: Diagnostic Data c 1 =incorrect AC_c 0 =true AC_c 1 =false Simplified

28. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 28 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Goal: make Correct’(…) equal to true

29. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 29 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Goal: make Correct’(…) equal to true  Simpler problem:  Checking a given expression is easy  Example: repair_c 1 (x,y,res) = x + 4  Correct’’(0,5) = false  ‘x + 4’ is not a valid repair  Solver can also return a counterexample

30. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 30 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Finding a substitution for repair() is more difficult  Idea: Replace repair(…) with a template for expressions  E.g.: repair(x,y,res) = k 0 + k 1 *x + k 2 *y + k 3 *res

31. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 31 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Finding a substitution for repair() is more difficult  Idea: Replace repair(…) with a template for expressions  E.g.: repair(x,y,res) = k 0 + k 1 *x + k 2 *y + k 3 *res  Find k 0, k 1, k 2, k 3 such that Correct’’(X, Y, k 0, k 1, k 2, k 3 ) holds for all X,Y  Now: find parameter values instead of expressions

32. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 32 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Solvers cannot handle conditions like efficiently  Can find k 0, k 1, k 2, k 3 such that Correct(X, Y, k 0, k 1, k 2, k 3 ) is true for some inputs X, Y.  Can compute repairs that are correct for some inputs

33. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 33 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach: Error Correction  Iterative repair refinement: Compute Repair Input Database Check if Correct Counterexample Repair Correct Repair No Repair Exists Done Fail See also: K.-H. Chang, I. L. Markov, V. Bertacco: Fixing design error with counterexamples and resynthesis. ASP-DAC 2007. A. Solar-Lezama, L. Tancau, R. Bodik, V. Saraswat, S. A. Seshia: Combinatorial sketching for finite programs. ASPLOS 2006.

34. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 34 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs

35. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 35 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Implementation  For simple C programs  Tool: FoREnSiC  Various debugging methods  Program analysis:  Concolic execution  Extension of CREST  SMT-Solver:  Yices and Z3  Linear Integer Arithmetic  Bitvector Arithmetic under Development

36. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 36 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components): DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas227022711175291297 tcas7212354210702041459 tcas8212253710796.3142 tcas1621235431072222764 tcas1721255411070112666 tcas18212453810759.32744 tcas19212354010709.6142 tcas3603.0---447.04796 [1] A. Solar-Lezama, L. Tancau, R. Bodik, V. Saraswat, S. A: Seshia. Combinatorial sketching for finite programs. ASPLOS 2006. [2] http://pleuma.cc.gatech.edu/aristotle/Tools/subjects/http://pleuma.cc.gatech.edu/aristotle/Tools/subjects/

37. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 37 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components):  Do not take too seriously; different solvers, objectives, … DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas227022711175291297 tcas7212354210702041459 tcas8212253710796.3142 tcas1621235431072222764 tcas1721255411070112666 tcas18212453810759.32744 tcas19212354010709.6142 tcas3603.0---447.04796

38. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 38 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components):  Accurate diagnoses and correct repairs in reasonable time! DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas227022711175291297 tcas7212354210702041459 tcas8212253710796.3142 tcas1621235431072222764 tcas1721255411070112666 tcas18212453810759.32744 tcas19212354010709.6142 tcas3603.0---447.04796

39. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 39 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Experimental Results: Example: tcas35  Correct: Buggy:  Diagnosis: {c 5,c 6 } ClimbIn ? UpSep+100 : UpSepClimbIn ? UpSep : UpSep+100 c4c4 c5c5 c6c6

40. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 40 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Experimental Results: Example: tcas35  Correct: Buggy:  Diagnosis: {c 5,c 6 }  Repair: ClimbIn ? UpSep+100 : UpSepClimbIn ? UpSep : UpSep+100 c4c4 c5c5 c6c6 It.c5c5 c6c6 1 00  2 7280  3 -ClimbIn+10000  4 -ClimbIn+580  5 UpSep+3520  6 UpSep-10000  7 UpSep+1000  8 UpSep-OtherTrAlt+11  9 UpSep+1UpSep-1000  It.c5c5 c6c6 10 UpSep+2CurVerSep-795  11 UpSep+33114  12 UpSep+100UpSep-1000  13 UpSep+100-OtherTrAlt+11  14 UpSep+4CurVerSep  15 UpSep+1003114  16 UpSep+5UpSep+1000  17 UpSep+100OwnTrAlt  18 UpSep+100UpSep

41. Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 41 FMCAD 2011Robert KönighoferAustin, 2011-10-31 Summary and Conclusion  Problem: debug incorrect programs  Debugging method:  Works nicely, but a lot of room for improvement! Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving