1. Shape Analysis via 3-Valued Logic Mooly Sagiv Thomas Reps Reinhard Wilhelm http://www.cs.tau.ac.il/~msagiv/toplas02.ps http://www.cs.tau.ac.il/~rumster/TVLA

2. ... and also Tel-Aviv University –N. Dor –T. Lev-Ami –R. Manevich –E. Yahav Universität des Saarlandes –Jörg Bauer –Ronald Biber University of Wisconsin –F. DiMaio –D. Gopan –A. Loginov –A. Mulhern IBM Research –G. Ramalingam –J. Field –D. Goyal

3. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

4. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

5. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

6. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

7. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

8. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

9. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

10. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

11. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

12. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

13. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

14. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

15. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

16. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

17. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

18. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

19. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

20. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

21. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

22. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL Materialization

23. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

24. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

25. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

26. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

27. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

28. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

29. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

30. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

31. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

32. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

33. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

34. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

35. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

36. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

37. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

38. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

39. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

40. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

41. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

42. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

43. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

44. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

45. Original Problem: Shape Analysis (Jones and Muchnick 1981) Characterize dynamically allocated data –x points to an acyclic list, cyclic list, tree, dag, etc. –show that data-structure invariants hold Identify may-alias relationships Establish “disjointedness” properties –x and y point to structures that do not share cells

46. Properties of reverse(x) On entry, x points to an acyclic list On each iteration, x & y point to disjoint acyclic lists All the pointer dereferences are safe No memory leaks On exit, y points to an acyclic list On exit, x = = NULL On exit, all cells reachable from y were reachable from x on entry, and vice versa On exit, the order between neighbors in the y-list is opposite to the order they had in the x-list on entry

47. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures

48. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re-established

49. A ‘Yacc’ for Shape Analysis: TVLA Parametric framework –Some instantiations  known analyses –Other instantiations  new analyses

50. A ‘Yacc’ for Shape Analysis: TVLA Parametric framework –Some instantiations  known analyses –Other instantiations  new analyses Applications beyond shape analysis –Partial correctness of sorting algorithms –Safety of mobile code –Deadlock detection in multi-threaded programs –Partial correctness of mark-and-sweep gc alg. –Correct usage of Java iterators

51. A ‘Yacc’ for Static Analysis: TVLA Parametric framework –Some instantiations  known analyses –Other instantiations  new analyses Applications beyond shape analysis –Partial correctness of sorting algorithms –Safety of mobile code –Deadlock detection in multi-threaded programs –Partial correctness of mark-and-sweep gc alg. –Correct usage of Java iterators

52. x yt NULL x y t x y t return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt x y t x y t x y t x y t x y t x y t x y t x y t x y t x y t

53. Formalizing “... ” Informal: x Formal: x Summary node

54. A ‘Yacc’ for Static Analysis (Using Logic) Correctness proofs via inductive-assertion method Proof derivation via weakest-precondition calculus “Annotate your loops with invariants!” WP

55. Plan Operational semantics using first order logic Canonic Abstraction Abstract interpretation using canonic abstraction Discussion

56. Recorded Information (for reverse)

57. u1u1 u2u2 u3u3 u4u4 x y

58. Formulas: For Observing Properties Are x and y pointer aliases?  v: x(v)  y(v)

59. Formulas for Observing Properties Does x point to a non NULL element  v: x(v) Are x and y pointer aliases?  v: x(v)  y(v) Does x point to a cell with a self cycle?  v : x(v)  n(v,v)

60. x y u1u1 u2u2 u3u3 u4u4 Are x and y Pointer Aliases?  v: x(v)  y(v) x y u1u1  Yes

61. x ’ (v) = x(v) y ’ (v) = 0 t ’ (v) = t(v) n ’ (v1,v2) = n(v1,v2) Predicate-Update Formulas for ‘y = NULL’

62. x y u1u1 u2u2 u3u3 u4u4 y ’ (v) = 0 

63. Predicate-Update Formulas for ‘y = x’ x ’ (v) = x(v) y ’ (v) = x(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) = n(v 1,v 2 )

64. x u1u1 u2u2 u3u3 u4u4 Predicate-Update Formulas for ‘y = x’ y ’ (v) = x(v) 10001000 y

65. Predicate-Update Formulas for ‘x = x  n’ x ’ (v) =  v 1 : x(v 1 )  n(v 1,v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v 1, v 2 ) = n(v 1, v 2 )

66. x u1u1 u2u2 u3u3 u4u4 Predicate-Update Formulas for ‘x = x  n’ y x ’ (v) =  v 1 : x(v 1 )  n(v 1,v)   x

67. Predicate-Update Formulas for ‘y  n = t’ x ’ (v) = x(v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) =  y(v 1 )  n(v 1,v 2 )  y(v 1 )  t(v 2 )

68. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re-established

69. Two- vs. Three-Valued Logic 01 Two-valued logic {0,1} {0}{1} Three-valued logic {0}  {0,1} {1}  {0,1}

70. Two- vs. Three-Valued Logic Two-valued logicThree-valued logic

71. Two- vs. Three-Valued Logic Three-valued logic 0 1 Two-valued logic {1} {0,1} {0} 1 ½ 0

72. Two- vs. Three-Valued Logic 01 Two-valued logic {0}{1} Three-valued logic {0,1}

73. Two- vs. Three-Valued Logic 01 Two-valued logic ½ 01 Three-valued logic 0  ½ 1  ½

74. 1: True 0: False 1/2: Unknown A join semi-lattice: 0  1 = 1/2 Kleene Three-Valued Logic   1/2 Information order

75. Boolean Connectives [Kleene]

76. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234        

77. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234

78. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234   

79. Canonical Abstraction Partition the individuals into equivalence classes based on the values of their unary predicates Collapse other predicates via  –p S (u’ 1,..., u’ k ) =  {p B (u 1,..., u k ) | f(u 1 )=u’ 1,..., f(u’ k )=u’ k ) } At most 2 n abstract individuals

80. Safety Properties of Concurrent Java Programs (Eran Yahav, POPL 01) Potentially unbounded heap Potentially unbounded number of threads Defects checked –RW interference –WW interference –Total deadlock –Nested monitors –Illegal thread interactions

81. l_0: while (true) { l_1:synchronized(sharedLock) { l_C:// critical actions l_2:} l_3: } Example - Mutual Exclusion

82. Concrete Configuration is_thread at[l_C] rval[this] held_by blocked is_thread at[l_1] rval[this] is_thread at[l_0] is_thread at[l_0] is_thread at[l_1] rval[this] blocked is_thread at[l_1]

83. Abstract Configuration is_thread at[l_C] rval[this] held_by blocked is_thread at[l_1] rval[this] is_thread at[l_0]

84. Embedding u1u1 u2u2 u3u3 u4u4 x u5u5 u6u6 u 12 u 34 u 56 x u 123 u 456 x

85. Canonical Abstraction: An Embedding Whose Result is of Bounded Size u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234

86. Property-Extraction Principle Questions about a family of two-valued stores can be answered conservatively by evaluating a formula in a three-valued store Formula evaluates to 1  formula holds in every store in the family Formula evaluates to 0  formula does not hold in any store in the family Formula evaluates to 1/2  formula may hold in some; not hold in others  

87. The Embedding Theorem If a structure B can be embedded in a structure S by an onto function f, such that basic predicates are preserved, i.e., p B (u 1,.., u k )  p S (f(u 1 ),..., f(u k )) Then every formula  is preserved: –If    = 1 in S, then    = 1 in B –If    = 0 in S, then    = 0 in B –If    = 1/2 in S, then    could be 0 or 1 in B

88. The Embedding Theorem y x u1u1 u 34 u2u2 y x u1u1 u 234 y x u1u1 u3u3 u2u2 u4u4 x y u 1234  v: x(v)  y(v) Maybe No

89. Cyclic versus Acyclic Lists  x 31 7191 u1u1 u x u1u1 u x

90. The Instrumentation Principle Increase precision by storing the truth-value of some chosen formulas Nullary predicates –Observations on all the concrete stores Unary Predicates –Sets of individuals with designated properties Binary Predicates –Relationships between individuals

91. Cyclicity Predicate  x 31 7191 u1u1 u x u1u1 u x c[x]() =  v 1,v 2 : x(v 1 )  n * (v 1,v 2 )  n + (v 2, v 2 ) c[x]()=0

92. Cyclicity Predicate  x 31 7191 u1u1 u x u1u1 u x c[x]() =  v 1,v 2 : x(v 1 )  n * (v 1,v 2 )  n + (v 2, v 2 ) c[x]()=1

93. is = 0 Heap Sharing Predicate  x 31 7191 is(v) =  v 1,v 2 : n(v 1,v)  n(v 2,v)  v 1  v 2 u1u1 u x u1u1 u x is = 0

94. Heap Sharing Predicate  x 31 7191 is(v) =  v 1,v 2 : n(v 1,v)  n(v 2,v)  v 1  v 2 u1u1 u x u1u1 u x is = 0 is = 1

95. Sortedness Predicate inOrder(v) =  v1: n(v,v1)  dle(v, v1) u1u1 u x u1u1 u x inOrder = 1 n n  x 51 7191 inOrder = 1 n n n        

96. Sortedness Predicate  x 51 4591 inOrder(v) =  v1: n(v,v1)  dle(v, v1) u xx inOrder = 0inOrder = 1 n n n n n inOrder = 0          n  

97. Reachability Predicate r(v1, v2) = n * (v1,v2) u1u1 u x u1u1 u x n n x n n n r r r r r r r r

98. reachable-from-variable-x(v) acyclic-along-dimension-d(v) –à la ADDS c fb (v) =  v1: f(v, v1)  b(v1, v) tree(v) dag(v) AVL trees: –balanced(v), left-heavy(v), right-heavy(v) –... but not via height arithmetic Additional Instrumentation Predicates Need FO + TC

99. Shape Analysis via Abstract Interpretation Start with a set of 3-valured structures at the program entry Iteratively compute a set of 3-valued structures for every program point Apply canonic abstraction after every statement Check safety properties against 3-valued structures

100. Predicate-Update Formulas for “y = x” y ’ (v) = x(v) Old: u1u1 u x y New: u1u1 u x 

101. Predicate-Update Formulas for “x = x  n” x ’ (v) =  v 1 : x(v 1 )  n(v 1,v) y Old: u1u1 u x y New: u1u1 u x  

102. x yt NULL return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt u1u1 u x y u1u1 u x y ’ (v) = x(v) 1010

103. x yt NULL x y t x y t return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt x y t x y t x y t x y t x y t x y t x y t x y t x y t x y t

104. Naïve Transformer (x = x  n) x y Evaluate update formulas y x

105. How Are We Doing? Conservative Convenient But not very precise  –Advancing a pointer down a list loses precision –Cannot distinguish an acyclic list from a cyclic list

106. x ’ (v) = x(v) y ’ (v) = 0 t ’ (v) = t(v) n ’ (v 1,v 2 ) = n(v 1,v 2 ) is ’ (v) = is(v) Predicate-Update Formulae for ‘y = NULL’

107. Predicate-Update Formulae for ‘y = x’ x ’ (v) = x(v) y ’ (v) = x(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) = n(v 1,v 2 ) is ’ (v) = is(v)

108. Predicate-Update Formulae for ‘x = x  n’ x ’ (v) =  v 1 : x(v 1 )  n(v 1,v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) = n(v 1, v 2 ) is ’ (v) = is(v)

109. x ’ (v) = x(v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) =  y(v 1 )  n(v 1,v 2 )  y(v 1 )  t(v 2 ) is ’ (v) = Predicate-Update Formulae for ‘y  n = t’  v 1,v 2 : (is(v)  n ’ (v 1,v)  n ’ (v 2,v)  v 1  v 2 )  (t(v)  n(v 1,v)   y(v 1 ))

110. x ’ (v) = x(v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v 1,v 2 ) =  y(v 1 )  n(v 1,v 2 )  y(v 1 )  t(v 2 ) is ’ (v) = Predicate-Update Formulae for ‘y  n = t’ (  ((  v 1 : y(v 1 )  n(v 1,v 2 ))  t(v))  is(v))  ( ((  v 1 : y(v 1 )  n(v 1,v 2 ))  t(v))  (is(v)  t(v))   v 1,v 2 : n ’ (v 1,v)  n ’ (v 1,v)  v 1  v 2 )

111. Formalizing “... ” Informal: x y Formal: x y

112. Formalizing “... ” Informal: x y t2t2 t1t1 Formal: x y t2t2 t1t1

113. Formalizing “... ” Informal: x y Formal: x y reachable from variable x reachable from variable y r[x]r[x] r[y]r[y] r[x]r[x] r[y]r[y]

114. Formalizing “... ” Informal: x y t2t2 t1t1 Formal: t2t2 t1t1 r[x],r[t 1 ] r[y],r[t 2 ] r[x],r[t 1 ] r[y],r[t 2 ] x y r[y]r[y] r[x]r[x]r[x]r[x] r[y]r[y]

115. Updating Auxiliary Information t1t1 {r[t 1 ],r[x],r[y]}{p[t 1 ],r[x],r[y]} x {p[x],p[y]} {r[x],r[y]} y t1t1 {r[t 1 ],r[x]}{p[t 1 ],r[x]} x {p[x]} {r[x]} y = NULL

116. Materialization x = x  n Informal: x y y x x = x  n Formal: x y x y x y x = x  n y x

117. Materialization x = x  n [Chase, Wegman, & Zadeck 90] x y u1u1 u2u2 y x u1u1 u2u2 x = x  n [Sagiv, Reps, & Wilhelm 96, 98] x y u1u1 u2u2 y x u1u1 u3u3 u2u2

118. Naïve Transformer (x = x  n) x y Evaluate update formulas y x

119. Best Transformer (x = x  n) x y y x y x blur y x y x ...... Evaluate update formulas y x y x...... inverse embedding canonic abstraction

120. “Focus”-Based Transformer (x = x  n) x y y x y x Focus(x  n) “Partial  ” y x y x Evaluate update Formulas (Kleene) y x y x blur

121. The Focusing Principle “Bring the structure into better focus” –Selectively force 1/2 to 0 or 1 –Avoid indefiniteness Then apply the predicate-update formulae

122. (1) Focus on  v 1 : x(v 1 )  n(v 1,v) u1u1 x y u x y u1u1 u x y y u1u1 u.1 x  u1u1 u.0 u

123. x ’ (v) =  v 1 : x(v 1 )  n(v 1,v) (2) Evaluate Predicate-Update Formulae u1u1 u x y x y u1u1 u1u1 u y x y u1u1 x u.1 u.0 x y u1u1 u.1 u.0 y u1u1 u u

124. The Coercion Principle Increase precision by exploiting some structural properties possessed by all stores Structural properties captured by constraints Apply a constraint solver

125. (3) Apply Constraint Solver y u1u1 u1u1 u y x y u1u1 u1u1 u y x x u.1 u.0 y u1u1 x u.1 y u1u1 uu

126. n(v 1, v )  n(v 2, v)  v 1  v 2  is(v)  is(v)  n(v 1, v)  v 1  v 2   n(v 2, v) (3) Apply Constraint Solver x y u1u1 u.1 u.0 1 1  1  0  

127. (3) Apply Constraint Solver x y u1u1 u.1 u.0

128. (3) Apply Constraint Solver x y u1u1 u.1 u.0  is(v)  n(v 1, v)  v 1  v 2   n(v 2, v) 1 1  1  n(v 1, v )  n(v 2, v)  v 1  v 2  is(v)   0

129. (3) Apply Constraint Solver x y u1u1 u.0 x(v 1 )  x(v 2 )  v 1 = v 2 u.1 1  11 

130. (3) Apply Constraint Solver x y u1u1 u.0

131. Example: InsertSort Run Demo List InsertSort(List x) { List r, pr, rn, l, pl; r = x; pr = NULL; while (r != NULL) { l = x; rn = r  n; pl = NULL; while (l != r) { if (l  data > r  data) { pr  n = rn; r  n = l; if (pl = = NULL) x = r; else pl  n = r; r = pr; break; } pl = l; l = l  n; } pr = r; r = rn; } return x; } typedef struct list_cell { int data; struct list_cell *n; } *List;

132. Ongoing Work Deriving update-formulas (ESOP 03) Automatic Refinement (A. Loginov) Exploring encapsulation (J. Bauer & N. Rinezkey) Integrating arithmetic (F. Dimaio, N. Dor, D. Gopan) Improving scalability (R. Manevich) –Using BDDs (SAS 02) –Widening A generic Java front-end (R. Manevich) Compile-time GC (R. Shaham) Temporal properties (E. Yahav) Assume-Guarantee Reasoning (G. Yorsh) Providing concrete counter examples (G. Erez)

133. TVLA Experience Quite fast on small programs –But runs on medium programs too Not a panacea More instrumentation may lead to faster (and more precise) analysis Manually updating instrumentation predicates is difficult

134. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships –Track aliasing on 3-valued structures Dynamic storage allocation –No bound on the size of run-time data structures –Canonical abstraction  finite-sized 3-valued structures Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re- established –Query the 3-valued structures that arise at the exit

135. Abstraction via logic – model-based –Logical + numeric abstraction –Finite differencing of logical formulasFinite differencing of logical formulas Abstraction via logic – proof-based –Shape analysis using theorem proversShape analysis using theorem provers –Symbolic implementation of the best transformer Applications –Verifying temporal properties of concurrent softwareVerifying temporal properties of concurrent software Projects in Progress

136. Logical + Numeric Abstractions x u1u1 u2u2 x u u’u’’u’’’ (1,2,3,4) u u’ u’’ u’’’ Position in list (1,2) {1} x [2,4] u1u1 u2u2 (1,4)

137. Logical + Numeric Abstractions y = x  next (1,2) {1} x [2,4] x u1u1 u2u2 u1u1 u2u2 (1,4) u1u1 u 2.1 u 2.0 x y (1,2,3) {1} x {2} x [3,4] u1u1 u 2.1 u 2.0 (1,2,4)

138. u1u1 u 2.1 u 2.0 Logical + Numeric Abstractions u1u1 u 2.1 u 2.0 x y

139. Example: Sortedness inOrder(v) =  v 1 : n(v, v 1 )  (  v    v 1  )  x 51 7191 inOrder = 1       xx   Yes sorted =  v: inOrder(v)

140. Example: Sortedness  x 51 4591 inOrder(v) =  v 1 : n(v, v 1 )  (  v    v 1  )       inOrder = 1inOrder = 0inOrder = 1 sorted =  v: inOrder(v) No      inOrder = 0inOrder = 1 xx

141. Example: InsertSort Run Demo List InsertSort(List x) { List r, pr, rn, l, pl; r = x; pr = NULL; while (r != NULL) { l = x; rn = r  n; pl = NULL; while (l != r) { if (l  data > r  data) { pr  n = rn; r  n = l; if (pl = = NULL) x = r; else pl  n = r; r = pr; break; } pl = l; l = l  n; } pr = r; r = rn; } return x; } typedef struct list_cell { int data; struct list_cell *n; } *List;

142. Abstraction via logic – model-based –Logical + numeric abstraction –Finite differencing of logical formulas [A. Loginov] Abstraction via logic – proof-based –Shape analysis using theorem provers [G. Yorsh]Shape analysis using theorem provers [G. Yorsh] –Symbolic implementation of the best transformer [T. Reps] Applications –Verifying temporal properties of concurrent software [E. Yahav]Verifying temporal properties of concurrent software [E. Yahav] Projects in Progress

143. Work in Progress: Automatic Generation of Update Formulas for Instrumentation Predicates Obligation: Must define predicate-update formulas to update the extra predicates

144. Work in Progress: Automatic Generation of Update Formulas for Instrumentation Predicates Currently, user provides –Update formulas for core predicates:  c,st (v) –Definitions of instrumentation predicates:  p (v) –Update formulas for instrumentation predicates: p,st (v) Consistently defined? Goal: Create p,st from  p and the  c,st dddd

145. Abstraction via logic – model-based –Logical + numeric abstraction –Finite differencing of logical formulas [A. Loginov] Abstraction via logic – proof-based –Shape analysis using theorem provers [G. Yorsh] –Symbolic implementation of the best transformer [T. Reps] Applications –Verifying temporal properties of concurrent software [E. Yahav]Verifying temporal properties of concurrent software [E. Yahav] Projects in Progress

146. Abstract Abstract Interpretation Concrete Memory configurations Descriptors of memory configurations   T#T# T  

147. Abstract Best Abstract Transformer Concrete  T  Memory configurations Descriptors of memory configurations 

148. Symbolic Operations: Three Value-Spaces Formulas Concrete Values Abstract Values   T T 

149. Symbolic Operations: Three Value-Spaces FormulasConcrete Values Abstract Values  T  T#T#

150. Symbolic Operations: Three Value-Spaces FormulasConcrete Values Abstract Values u1u1 x u x x...    

151. Abstraction via logic – model-based –Logical + numeric abstraction –Finite differencing of logical formulas [A. Loginov] Abstraction via logic – proof-based –Shape analysis using theorem provers [G. Yorsh] –Symbolic implementation of the best transformer [T. Reps] Applications –Verifying temporal properties of concurrent software [E. Yahav] Projects in Progress

152. Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis A memory configuration: thread3 inCritical lock1 isAcquired thread1 atStart thread2 atStart thread4 atStart csLock heldBy

153. An abstract memory configuration: thread inCritical lock1 isAcquired thread ’ atStart csLock heldBy Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis

154. Model checking means: Explore the space of possible transitions among abstract memory configurations Java Threads Are Heap-Allocated Objects  Thread Analysis  Shape Analysis

155. Spatial and Temporal Properties P   Q PQ  v.  u. x(u)  n + (u,v)    (  u. x(u)  n + (u,v)) n x n y n x n y

156. Spatial and Temporal Properties Relate memory locations across program configurations Specify properties relating to allocation and deallocation Example: concurrent garbage collection –Safety: “Only nodes reachable from the roots are collected” –Liveness: “All garbage nodes are eventually collected” Propositional temporal logic is not enough ETL: a more expressive specification language

157. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships –Track aliasing on 3-valued structures Dynamic storage allocation –No bound on the size of run-time data structures –Canonical abstraction  finite-sized 3-valued structures Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re- established –Query the 3-valued structures that arise at the exit

158. A ‘Yacc’ for Static Analysis (Using Logic) First-order structures (= predicate tables) –Hold recorded information –Model-theoretic approach, not proof-theoretic Formulas –Means for observing and updating information Abstraction controlled by choice of instrumentation predicates –What distinctions do you need to observe? –“Predicate abstraction” is the special case of nullary predicates only

159. A ‘Yacc’ for Static Analysis Based on Logic: Who Cares? Broad scope of potential applicability –Not just shape properties: predicates are not restricted to be links! –Discrete systems in which a relational (+ numeric) structure evolves –Transition: evolution of relational + numeric state

161. Example: Mark and Sweep void Sweep() { unexplored = Universe collected =  while (unexplored   ) { x = SelectAndRemove(unexplored) if (x  marked) collected = collected  {x} } assert(collected = = Universe – Reachset(root) ) } void Mark(Node root) { if (root != NULL) { pending =  pending = pending  {root} marked =  while (pending   ) { x = SelectAndRemove(pending) marked = marked  {x} t = x  left if (t  NULL) if (t  marked) pending = pending  {t} t = x  right if (t  NULL) if (t  marked) pending = pending  {t} } assert(marked = = Reachset(root)) } Run Demo

162. What Stores Does a 3-Valued Structure Represent? Example 3-valued structure –individuals: {u 1 } –predicates: graphical presentation concrete stores represented x u1u1 33 x 88 x 37 x

163. Example 3-valued structure graphical presentation concrete stores What Stores Does a 3-Valued Structure Represent? u1u1 u x u1u1 u x  x 31 7191

164. Example 3-valued structure graphical presentation concrete stores u1u1 u x u1u1 u x  x 31 7191 What Stores Does a 3-Valued Structure Represent?

165. ETL Examples Every object allocated is eventually freed –  (  v.  v    v) Every request is eventually assigned to a handler thread –   r:request.  r    t:thread. handles(t,r)

166. Evolution Temporal Logic (ETL) Defined over infinite traces Based on linear temporal logic, first-order logic, plus transitive closure –Formulas may include free variables Express properties of how structures evolve Special operators  v object v allocated  v object v deallocated Predicates represent properties of interest –E.g., blocked(t,l), held_by(l,t)

167. ETL Traces Sequence of configurations, augmented by –Evolution edges that relate individuals of successive configurations –Sets of allocations and deallocations … deallocationallocationthreadobject evolution edge

168. Fine, but… Traces are infinite –How do you represent an infinite program trace? The number of traces is possibly infinite –How do you represent the set of program traces?

169. ETL Traces via Logical Structures … deallocationallocationthreadlockworld evolution edge

170. ETL Traces via Logical Structures … succ deallocationallocationthreadlockworld existence edge evolution edge

171. ETL Traces via Logical Structures … deallocationallocationthreadlockworld evolution edge

172. Fine, but… Traces are infinite –How do you represent an infinite program trace? The number of traces is possibly infinite –How do you represent the set of program traces? Use canonical abstraction to abstract traces –A summary world can represent the infinite tails of an infinite number of unexplored trace –Canonical abstractions are finite

173. Equality Under Abstraction twe   evolution twe(u,u) = 1 twe(u,u) = ½

174. Abstract Interpretation (Over-)approximate possibly infinite set of infinite traces by a finite set of finite abstract traces Successive approximation –Start with an abstract trace that represents all traces –Repeatedly refine the state by exploring longer abstract traces Evaluate the property of interest over abstract traces

175. Current Implementation Manually –Convert ETL to FO+TC –Define instrumentation predicates for temporal subformulas Let TVLA do the rest Properties proven: –Termination of linked-list manipulation –Response (fair/unfair)