1. Inferring Specifications to Detect Errors in Code Mana Taghdiri Presented by: Robert Seater MIT Computer Science & AI Lab

2. ASE 20042 Outline Problem – function specs Key Insights Algorithmic Details Experimental results Related work Conclusions

3. ASE 20043 Problem Statement Software model checkers ignore code modularity Procedures treated at control flow and in-lined Procedure boundaries not exploited This is odd ! Can cause trouble scaling

4. ASE 20044 Problem Statement In contrast, Traditional methods are based on code structure Examples: ESC/Java, Jalloy User has to provide procedure specs C2C1 C P S1S2 C2C1 C P’ S1’S2’ C1 C P S1 S1’

5. ASE 20045 Example procedure f(bool b, List x, List y) { g(b, x); h(y); } procedure g(bool b, List x) { if (b) then mutate x; else mutate’ x; } if (b) then x is acyclic

6. ASE 20046 Generic Analyzer User’s View programproperty counter example satisfies specification Our Approach

7. ASE 20047 Our Approach A procedure-based automatic analysis Procedures => modularity => help scale Procedure specs are inferred automatically Property-dependent Call site-dependent A counterexample-guided refinement technique To detect bugs Sound error reports Incomplete

8. ASE 20048 Example procedure f(bool b, List x, List y) { g(b, x); h(y); } procedure g(bool b, List x) { if (b) then mutate x; else mutate’ x; } if (b) then x is acyclic b=true x=empty g(b,x)=mangled mess y=… h(y)=… mangled(empty) is cyclic b=true x=empty g(b,x)=mutated x y=… h(y)=… mutate(empty) is cyclic

9. ASE 20049 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid?

10. ASE 200410 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Written in subset of Java Finitize: unwind loops and recursion based on a user-provided number

11. ASE 200411 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Partial spec of selected procedure About structure of the heap Written in logical constraints (Alloy)

12. ASE 200412 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Procedures under-constrained Initially just frame conditions (generated bottom up)

13. ASE 200413 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Semantics preserving translation to Alloy Same technique as Jalloy

14. ASE 200414 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Alloy Analyzer SAT solver back end (ZChaff) Solutions satisfy current spec but violate property

15. ASE 200415 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Solutions = counter-examples No solutions => no counterexample to property within the finite bounds No guarantees about the general case

16. ASE 200416 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? May be a counterexample Trace in abstract program Assigns behavior to each procedure: before/after pairs of program states for the procedure call site

17. ASE 200417 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? behaviors assigned to procedures may be too relaxed Examine each procedure individually Check behavior against full procedure, not just current (partial) spec

18. ASE 200418 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? If all procedures have valid behaviors, then counterexample is real Bug in original code!

19. ASE 200419 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Some procedure p is invalid Refine the spec of p just enough to exclude the bad trace Use unsat core to choose which constraints to add

20. ASE 200420 Algorithmic Overview program Check validitysolve refine spec (unsat core) abstract trace negated property logical constraints abstract program satisfies property counter example: bug! unsat? sat? valid? invalid? Repeat until conclusive result Guaranteed to terminate At worst, each spec is just the entire procedure

21. ASE 200421 Experiments Preliminary results Graph manipulation code Structural properties All properties hold in the code shows benefits of procedure abstraction Comparison with Jalloy Same translation technique Same SAT solver Inlines all procedure calls

22. ASE 200422 Experimental Results Loop Iter Heap Size JalloyOur MethodImprove ment #Var#ClauseTime#Var#ClauseTime 448216181261549281026091.67 551455534704162861119002981.65 64135543055540670214013123.33 651813743760234985721776832.82 List.removeAll : The result list is a subset of the original list No refinement iterations needed!

23. ASE 200423 Experimental Results Loop Iter Heap Size JalloyOur MethodImprove ment #Var#ClauseTime#Var#ClauseTime 3327112562416132846589512.2 4466566151323164618713507820.5 4587710214959206952423383277.63 54-- >9006807147948>112 55-- >900103462526336>25 64-- >9007499162079>100 Graph.remove : If given node set is empty, graph edges aren’t changed No refinement iterations needed!

24. ASE 200424 Experimental Results Loop Iter Heap Size JalloyOur MethodImprove ment #Var#ClauseTime#Var#ClauseTime 3327147562984459271165276.29 44666611514891231105723450139.46 458780321512922415682368901072.09 5410801624691435913075274461720.9 5514108734746658618549429481913.07 Graph.remove : If given node set is empty, graph nodes aren’t changed Needs 3 refinement iterations!

25. ASE 200425 Related Work Jalloy Structural properties SAT-based analysis Inlines procedure calls – not scalable SLAM Predicate abstraction Not structural properties

26. ASE 200426 Related Work ESC/Java Based on theorem prover Needs user-provided procedure specification Flanagan’s Method Extension to ESC Translates code to CLP Check satisfiability VeriFun – iteratively refined predicate abstraction

27. ASE 200427 Related Work Bandera Static slicing based on property User-provided data abstraction Intermediate model may be analyzed by a set of model checkers Daikon Specification extraction tool Not dependent on context or property Dynamic Unsound (sound over test cases)

28. ASE 200428 Conclusions Checks structural properties of the heap Exploits modularity from function calls Infers context-dependent specs automatically Uses counterexamples to refine specs Uses unsat core to produce small specs