1. 1 Employing decision procedures for automatic analysis and verification of heap-manipulating programs Greta Yorsh under the supervision of Mooly Sagiv and Alexander Rabinovich

2. 2 class List { List n = null; List append(List a, List b)... } List test(List z) { List x, y, t = z; x = new List(); y = new List(); while (t != null) { x.n = y; y = new List(); t = t.n; } t = append(x,y); x = append(t,z); return x; } programmer x is not nullx and y are disjoint lists x is acyclic while loop terminates It’s not gonna crash because.. Introduction

3. 3 class List { List n = null; List append(List a, List b)... } List test(List z) { List x, y, t = z; x = new List(); y = new List(); while (t != null) { x.n = y; y = new List(); t = t.n; } t = append(x,y); x = append(t,z); return x; } source code programmer

4. 4 source code compiler programmer tester  executable test inputs

5. 5  source code compiler programmer executable tester test inputs product client

6. 6  source code compiler programmer executable tester test inputs product 2.0 client

7. 7 compiler programmer source code verifier  Let X be the set of all inputs…

8. 8 programmer source code compiler verifier client executable product Let X be the set of all inputs…  What is software verification? Why do we need software verification? Why is software verification difficult? How can we verify software?

9. 9 Deductive verification [Hoare’76] –ask the programmer –use specifications –preconditions, postconditions, loop invariants –use (automatic/interactive) theorem prover Abstract interpretation [Coutsot&Cousot’77] –automatically infers loop invariants –use abstraction to over-approximate program’s behavior 9 Software Verification Techniques

10. 10 Abstraction Choose abstraction: an abstract value represent sets of program states Over-approximate all reachable program states Check property over abstract values –Proof –Potential error Most-precise abstract value Trade-off precision and cost

11. 11 @requires precondition @ensures postcondition Leverage Specifications Specifications provide insights from the programmer programmer source code  verifier  specifications

12. 12 List test(List z) { List x, y, t = z; x = new List(); y = new List(); while (t != null) { x.n = y; y = new List(); t = t.n; } t = append(x,y); x = append(t,z); return x; } class List { List n = null;... List append(List a, List b) { if (a == null) return b; List d = a; while (d.n != null) d = d.n; d.n = b; return a; } @requires acyclic(a) & disjoint(a,b) @ensures acyclic(ret) @requires acyclic(z) @ensures acyclic(ret) Example

13. 13 Modular Analysis Each procedure is analyzed only once, in isolation Allows more precise analysis of each procedure List test(List z) {... t = append(x,y);... return x; } class List { List n = null;... List append(List a, List b) {... } assume acyclic(t) @requires acyclic(a) & disjoint(a,b) @ensures acyclic(ret) @requires acyclic(z) @ensures acyclic(ret) assert acyclic(x) & disjoint(x,y)

14. 14 Challenge Abstract interpretation is not designed to take specifications into account –operates on abstract values –does not support symbolic operations assert, assume… How to implement these operations ? Which specification language to use?

15. 15Results Assist program analysis tools to employ formal specifications –algorithms for computing symbolic operations [TACAS’04, VMCAI’04] –leverage testing to reduce cost of symbolic operations [ISSTA’06] –case-study on modular analysis [AIOOL’05] Develop specification language for recursive data-structures that can be incorporated in program analysis –Decidability and undecidability for transitive closure logics [CSL’04] –Verification via structure simulation [CAV’04] –LRP: decidable logic for reasoning about linked data-structures [FoSSaCS’06, JLAP] 15

16. 16 Outline Two algorithms for computing assume Logic of Reachable Patterns

17. 17 Computing assume What is the result of assume  ? The most-precise abstract value that represent all states that satisfy  Why is assume difficult? Formal specification often more expressive than abstract values (describes more sets of concrete states) 

18. 18 Naïve Algorithm for assume  Requires –finite abstract domain A –  is a formula in L –for every abstract value a, the set of states represented by a is expressed by the formula  (a) in L –theorem prover for L Choose the least abstract value a s.t.    (a) is valid Number of calls  |A| 

19. 19 Better Algorithms Goal: number of calls << |A| Two algorithms and their tradeoffs –number of calls  height(A) << |A| –usually, |A| is at least 2 height(A)

20. 20 Computing from below Computing assume from below 

21. 21  Computing from below Computing assume from below [ VMCAI’04, ISSTA’06 ] Finite-height domain A Theorem prover for L Model generator for L Extension to infinite-height domains –guarantee termination using widening –no guarantee of precision Leveraging concrete execution to reduce number of calls [ISSTA’06] is    (a) valid ?

22. 22  Computing from below Computing assume from below [ VMCAI’04, ISSTA’06 ] Finite-height domain A Theorem prover for L Model generator for L Extension to infinite-height domains –guarantee termination using widening –no guarantee of precision Leveraging concrete execution to reduce number of calls [ISSTA’06] is    (a) valid ? Intermediate result is not sound

23. 23 Computing from above Computing assume from above  [TACAS’04] Specialized for Canonical Abstraction Theorem prover for FOTC Intermediate result is always sound

24. 24 Computing Computing assume From below [ VMCAI’04, ISSTA’06 ] Finite-height domain A Theorem prover for L Model generator for L Intermediate result is not sound From above [TACAS’04] Specialized for Canonical Abstraction Theorem prover for FOTC Intermediate result is always sound  

25. 25 Employing Theorem Provers Theorem prover or model generation might fail –no guarantee of precision –we can guarantee soundness and termination Theorem prover should match the expressive power of the abstraction and the properties of interest

26. 26 Choosing Specification Language Expressive –specifications: assertions, postconditions... –characterization the abstraction  (a) –queries from the algorithms Support for automatic reasoning –theorem prover –model generator (for assume from below) Predictable –guarantee termination of the analysis –guarantee the most-precise result w.r.t. to the abstraction

27. 27 Goal Specification language for reasoning about programs that manipulate dynamically allocated memory (heap) and recursive data-structures –Expressive –Natural for writing specifications –Decidable

28. 28 Heap-Manipulating Programs Imperative programs (in C, C++, Java) –dynamic memory allocation –recursive data-structures –destructive updates of fields x.next = y Unbounded size of the heap Arbitrary structure of the heap –objects can be heap shared Heap structure can be mutated Invariants involve unbounded objects and reachability between objects Temporary violations of data-structure invariants

29. 29 MSO FO+TC FO Decidability with Reachability Allow regular properties of arbitrary heaps –disjointness –cyclicity –reversal –sharing

30. 30 Logic of Reachable Patterns (LRP) Expressive –regular properties of arbitrary heaps –invariants involve reachability –temporary violations of data-structure invariants Decidable –validity, satisfiability, model generation –non-trivial proof of tree-like model property Useful for verification

31. 31 Interesting Heap Properties x  *  y –y is reachable from x x  +  x –x is cyclic f f

32. 32 List pointed-to by x is not shared x[ * ]uns f where uns f is a pattern uns f (v 0 )  (v 1 v 0 )  (v 2 v 0 )  (v 1 = v 2 ) Unshared Lists v2v2 v1v1 v0v0 f f f ff

33. 33 Doubly-linked list pointed-to by x x[ * ]inv f,b where inv f,b is a pattern inv f,b (v 0 )  (v 0 v 1 )  (v 1 v 0 ) Doubly-linked Lists f f b v0v0 v1v1 f b

34. 34 List with head pointers x[ * ]p where p is a pattern p (v 0 )  (v 1 v 0 )  (v 1 x) x ffffff g g g g g f g f

35. 35 x y’ precondition x  *  null postcondition y’  *  null  x [ * ] inv n,n’ Reversal of Singly-linked List n n n’

36. 36 LRP: Principles –Arbitrary structure of the heap –Allow arbitrary boolean combinations of reachability constraints –Restrict quantification: no alternations –Use regular expressions to define reachability –Syntactically limit the patterns

37. 37 Summary Approach to software verification –Prove properties of programs using abstraction –Trade-off between precision and cost of abstraction –Modular program analysis Thesis results –Novel algorithms enable program analysis with user-provided specifications –LRP: decidable logic for reasoning about recursive data- structures Foundations of modular analysis of heap-manipulating programs

38. 38 Future Directions Reasoning about data-structures –develop specification languages –develop decision procedures Modular analysis of real programs –using user-provided specifications –specification inference 38

39. 39 Research Topics and Collaborators Advisors Mooly Sagiv Alexander Rabinovich Thesis Results Symbolic implementation of abstract operations for shape analysis [TACAS’04] –with Tom Reps and Mooly Sagiv Combining testing and abstraction [ISSTA’06] –with Tom Ball and Mooly Sagiv Logic of Reachable Patterns [FoSSaCS’06,JLAP] –with Alexander Rabinovich, Mooly Sagiv, Antoine Meyer, Ahmed Bouajjani Additional Results Symbolic implementation of best transformer [VMCAI’04] –with Tom Reps and Mooly Sagiv Case-study on assume-guarantee [AIOOL’05] –with Alexey Skidanov, Tom Reps, Mooly Sagiv Verificiation via Structure Simulation [CAV’04] –with Neil Immerman, Tom Reps, Mooly Sagiv Decidability and undecidability for transitive closure logics [CSL’04] –with Neil Immerman, Tom Reps, Mooly Sagiv Abstraction for falsification [CAV’05] –with Tom Ball, Orna Kupferman Combination method for generating interpolants [CADE’05] –with Madan Musuvathi Modular typestate verification [POPL’08] –with Eran Yahav, Satish Chandra 39 Ongoing work Decision procedure for LRP, extensions –with Mooly Sagiv, Alexander Rabinovich Decidability and model generation for fragments of separation logic –with Josh Berdine, Byron Cook, Mooly Sagiv