Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 1 Robert Könighofer and Roderick Bloem IAIK – Graz University of Technology This work was supported in part by the European Commission through project DIAMOND (FP IST ). FMCAD 2011Robert KönighoferAustin, Automated Error Localization and Correction for Imperative Programs

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 2 FMCAD 2011Robert KönighoferAustin,  Addressed problem  Debugging approach  Program analysis  Error localization  Error correction  Experimental results  Summary and conclusions Outline

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 3 FMCAD 2011Robert KönighoferAustin, Addressed Problem  Automate debugging of simple software  Input:  Incorrect program  Specification:  Assertions  Reference Implementation  Output:  Potential error locations  Potential repairs int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; }

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 4 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 5 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 6 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; }

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 7 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 8 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Pre-Processing  Identification of Components  Right-hand side (RHS) of assignments  Efficient program analysis  Can handle all incorrect expressions int max(int x, int y) { int res = x; int tmp = y > x; if(tmp) res = x; assert(res>=x && res>=y); return res; } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1 c0c0 c2c2 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 9 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Pre-Processing  Components may be faulty  Transformation: bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 10 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Program Analysis  When is my buggy program correct?  We use Symbolic Execution  Allows incomplete analysis int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } Symbolic Execution

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 11 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Program Analysis  When is my program correct? bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; }

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 12 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Program Analysis  When is my buggy program correct? bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Diagnostic Data

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 13 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 14 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 15 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 16 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 17 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 18 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = x; if(y > x) res = repair(…); assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 19 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = repair(…); if(y > x) res = x; assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 20 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Goal: compute diagnoses Sets of components that  … if assumed to be incorrect, make the program repairable  … can be modified to make the program correct  … may be responsible for the incorrectness Assumption c 0 c 1 RepairableDiagnosis correct no correctincorrectyes{c 1 } incorrectcorrectno incorrect yes{c 0, c 1 } int max(int x, int y) { int res = repair(…); if(y > x) res = repair(…); assert(res>=x && res>=y); return res; } c0c0 c1c1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 21 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Program Analysis Diagnostic Data:  Checking for Repairability:

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 22 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization bool AC_c0; // AC = ‘Assume Correct’ bool AC_c1; int max(int x, int y) { int res = AC_c0 ? x : repair_c0(x,y); if(y > x) res = AC_c1 ? x : repair_c1(x,y,res); assert(res>=x && res>=y); return res; } Program Analysis Diagnostic Data:  Checking for Repairability:

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 23 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Use SMT-Solver to check repairability  Problem: Quantifier alternation  Check for a finite set of input values only  False positives

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 24 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Localization  Use SMT-Solver to check repairability  Problem: Quantifier alternation  Check for a finite set of input values only  False positives  A set D is a diagnosis iff Repairable(AC_c 0, AC_c 1, …,AC_c n ) = true with AC_c k = false iff c k  D  Computing diagnoses naively: check every set  Our algorithm: Unsatisfiable cores + hitting sets

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 25 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 26 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  We know which components are faulty  E.g.: D 1 ={c 8 }, D 2 ={c 2, c 4 }  Error Correction  Compute replacements for the faulty components  E.g.:c 8  x+9 c 2  a <= b and c 4  1

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 27 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Use information about faulty components: Diagnostic Data c 1 =incorrect AC_c 0 =true AC_c 1 =false Simplified

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 28 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Goal: make Correct’(…) equal to true

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 29 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Goal: make Correct’(…) equal to true  Simpler problem:  Checking a given expression is easy  Example: repair_c 1 (x,y,res) = x + 4  Correct’’(0,5) = false  ‘x + 4’ is not a valid repair  Solver can also return a counterexample

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 30 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Finding a substitution for repair() is more difficult  Idea: Replace repair(…) with a template for expressions  E.g.: repair(x,y,res) = k 0 + k 1 *x + k 2 *y + k 3 *res

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 31 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Finding a substitution for repair() is more difficult  Idea: Replace repair(…) with a template for expressions  E.g.: repair(x,y,res) = k 0 + k 1 *x + k 2 *y + k 3 *res  Find k 0, k 1, k 2, k 3 such that Correct’’(X, Y, k 0, k 1, k 2, k 3 ) holds for all X,Y  Now: find parameter values instead of expressions

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 32 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Solvers cannot handle conditions like efficiently  Can find k 0, k 1, k 2, k 3 such that Correct(X, Y, k 0, k 1, k 2, k 3 ) is true for some inputs X, Y.  Can compute repairs that are correct for some inputs

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 33 FMCAD 2011Robert KönighoferAustin, Debugging Approach: Error Correction  Iterative repair refinement: Compute Repair Input Database Check if Correct Counterexample Repair Correct Repair No Repair Exists Done Fail See also: K.-H. Chang, I. L. Markov, V. Bertacco: Fixing design error with counterexamples and resynthesis. ASP-DAC A. Solar-Lezama, L. Tancau, R. Bodik, V. Saraswat, S. A. Seshia: Combinatorial sketching for finite programs. ASPLOS 2006.

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 34 FMCAD 2011Robert KönighoferAustin, Debugging Approach Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 35 FMCAD 2011Robert KönighoferAustin, Implementation  For simple C programs  Tool: FoREnSiC  Various debugging methods  Program analysis:  Concolic execution  Extension of CREST  SMT-Solver:  Yices and Z3  Linear Integer Arithmetic  Bitvector Arithmetic under Development

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 36 FMCAD 2011Robert KönighoferAustin, Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components): DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas tcas tcas tcas tcas tcas tcas tcas [1] A. Solar-Lezama, L. Tancau, R. Bodik, V. Saraswat, S. A: Seshia. Combinatorial sketching for finite programs. ASPLOS [2]

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 37 FMCAD 2011Robert KönighoferAustin, Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components):  Do not take too seriously; different solvers, objectives, … DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas tcas tcas tcas tcas tcas tcas tcas

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 38 FMCAD 2011Robert KönighoferAustin, Experimental Results: Comparison with Sketch [1]  TCAS example [2] (180 LOC, 12 inputs, 44 components):  Accurate diagnoses and correct repairs in reasonable time! DiagnosisRepairSketch (8 bit) FoundTime [sec] FoundTime [sec] Template Variables Memory [MB] Time [sec] Template Variables Memory [MB] tcas tcas tcas tcas tcas tcas tcas tcas

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 39 FMCAD 2011Robert KönighoferAustin, Experimental Results: Example: tcas35  Correct: Buggy:  Diagnosis: {c 5,c 6 } ClimbIn ? UpSep+100 : UpSepClimbIn ? UpSep : UpSep+100 c4c4 c5c5 c6c6

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 40 FMCAD 2011Robert KönighoferAustin, Experimental Results: Example: tcas35  Correct: Buggy:  Diagnosis: {c 5,c 6 }  Repair: ClimbIn ? UpSep+100 : UpSepClimbIn ? UpSep : UpSep+100 c4c4 c5c5 c6c6 It.c5c5 c6c   3 -ClimbIn  4 -ClimbIn+580  5 UpSep+3520  6 UpSep  7 UpSep+1000  8 UpSep-OtherTrAlt+11  9 UpSep+1UpSep-1000  It.c5c5 c6c6 10 UpSep+2CurVerSep-795  11 UpSep  12 UpSep+100UpSep-1000  13 UpSep+100-OtherTrAlt+11  14 UpSep+4CurVerSep  15 UpSep  16 UpSep+5UpSep+1000  17 UpSep+100OwnTrAlt  18 UpSep+100UpSep

Institute for Applied Information Processing and Communications (IAIK) – Secure & Correct Systems 41 FMCAD 2011Robert KönighoferAustin, Summary and Conclusion  Problem: debug incorrect programs  Debugging method:  Works nicely, but a lot of room for improvement! Incorrect Program Pre-Processing Program Analysis Diagnostic Data Error Localization Error Correction Diagnoses Repairs Symbolic Execution SMT Solving