1. Program Analysis and Verification 0368-4479 Noam Rinetzky Lecture 10: Shape Analysis 1 Slides credit: Roman Manevich, Mooly Sagiv, Eran Yahav

2. Abstract Interpretation [Cousot’77] Mathematical foundation of static analysis – Abstract domains Abstract states Join (  ) – Transformer functions Abstract steps – Chaotic iteration Abstract computation Structured Programs Lattices (D, , , , ,  ) Lattices (D, , , , ,  ) Monotonic functions Fixpoints

3. The collecting lattice Lattice for a given control-flow node v: L v =(2 State, , , , , State) Lattice for entire control-flow graph with nodes V: L CFG = Map(V, L v ) We will use this lattice as a baseline for static analysis and define abstractions of its elements

4. Galois Connection 4 1   c 2 (c)(c) 3  (  (c))  The most precise (least) element in A representing c CA c   (  (c))

5. Galois Connection Given two complete lattices C = (D C,  C,  C,  C,  C,  C )– concrete domain A = (D A,  A,  A,  A,  A,  A )– abstract domain A Galois Connection (GC) is quadruple (C, , , A) that relates C and A via the monotone functions – The abstraction function  : D C  D A – The concretization function  : D A  D C for every concrete element c  D C and abstract element a  D A  (  (a))  a and c   (  (c)) Alternatively  (c)  a iff c   (a) 5

6. Abstract (conservative) interpretation set of states operational semantics (concrete semantics) statement S set of states  abstract representation abstract semantics statement S abstract representation concretization

7. Shape Analysis 7 Automatically verify properties of programs manipulating dynamically allocated storage Identify all possible shapes (layout) of the heap

8. Analyzing Singly Linked Lists

9. Limitations of pointer analysis 9 // Singly-linked list // data type. class SLL { int data; public SLL n; // next cell SLL(Object data) { this.data = data; this.n = null; } } // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; }

10. Flow&Field-sensitive Analysis 10 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; }

11. Flow&Field-sensitive Analysis 11 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t

12. Flow&Field-sensitive Analysis 12 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t L1 n

13. Flow&Field-sensitive Analysis 13 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n

14. Flow&Field-sensitive Analysis 14 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

15. Flow&Field-sensitive Analysis 15 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

16. Flow&Field-sensitive Analysis 16 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

17. Flow&Field-sensitive Analysis 17 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n tmp null h t L1 n  tmp != null tmp == null

18. Flow&Field-sensitive Analysis 18 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n tmp null h t L1 n 

19. Flow&Field-sensitive Analysis 19 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n tmp == null ?

20. Flow&Field-sensitive Analysis 20 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

21. Flow&Field-sensitive Analysis 21 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n Possible null dereference!

22. Flow&Field-sensitive Analysis 22 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n Fixed-point for first loop

23. Flow&Field-sensitive Analysis 23 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

24. Flow&Field-sensitive Analysis 24 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n

25. Flow&Field-sensitive Analysis 25 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } tmp null h t L1 n L2 n Possible null dereference!

26. What was the problem? Pointer analysis abstract all objects allocated at same program location into one summary object. However, objects allocated at same memory location may behave very differently – E.g., object is first/last one in the list – Assign extra predicates to distinguish between objects with different roles Number of objects represented by summary object  1 – does not allow strong updates – Distinguish between concrete objects (#=1) and abstract objects (#  1) Join operator very coarse – abstracts away important distinctions (tmp=null/tmp!=null) – Apply disjunctive completion 26

27. Improved solution Pointer analysis abstract all objects allocated at same program location into one summary object. However, objects allocated at same memory location may behave very differently – E.g., object is first/last one in the list – Add extra instrumentation predicates to distinguish between objects with different roles Number of objects represented by summary object  1 – does not allow strong updates – Distinguish between concrete objects (#=1) and abstract objects (#  1) Join operator very coarse – abstracts away important distinctions (tmp=null/tmp!=null) – Apply disjunctive completion 27

28. Adding properties to objects Let’s first drop allocation site information and instead… Define a unary predicate x(v) for each pointer variable x meaning x points to x Predicate holds for at most one node Merge together nodes with same sets of predicates 28

29. Flow&Field-sensitive Analysis 29 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t

30. Flow&Field-sensitive Analysis 30 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {h, t} n

31. Flow&Field-sensitive Analysis 31 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {h, t} tmp n

32. Flow&Field-sensitive Analysis 32 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {h, t} tmp n {tmp} n No null dereference

33. Flow&Field-sensitive Analysis 33 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {h, t} tmp n {tmp} n

34. Flow&Field-sensitive Analysis 34 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n {h, tmp} n

35. Flow&Field-sensitive Analysis 35 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n {h, tmp} n  null h t {h, t} tmp n

36. Flow&Field-sensitive Analysis 36 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n {h, tmp} n  null h t {h, t} tmp n Do we need to analyze this shape graph again?

37. Flow&Field-sensitive Analysis 37 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n {h, tmp} n  null h t {h, t} tmp n

38. Flow&Field-sensitive Analysis 38 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { h} n { tmp} n No null dereference

39. Flow&Field-sensitive Analysis 39 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { h} n { tmp} n

40. Flow&Field-sensitive Analysis 40 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n { h, tmp} n

41. Flow&Field-sensitive Analysis 41 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n { h, tmp} n

42. Flow&Field-sensitive Analysis 42 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n { h} n { tmp} n No null dereference

43. Flow&Field-sensitive Analysis 43 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n { h} n { tmp} n

44. Flow&Field-sensitive Analysis 44 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n n {h, tmp} n

45. Flow&Field-sensitive Analysis 45 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h, tmp} n n Summarization How many objects does it represent?

46. Flow&Field-sensitive Analysis 46 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h} n n { tmp} No null dereference n

47. Flow&Field-sensitive Analysis 47 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h} n n { tmp} n

48. Flow&Field-sensitive Analysis 48 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h, tmp} n n Fixed-point for first loop

49. Flow&Field-sensitive Analysis 49 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h, tmp} n n

50. Flow&Field-sensitive Analysis 50 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { } n {h, tmp} n n

51. Flow&Field-sensitive Analysis 51 // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; } null h t {t} tmp n { tmp?} n { tmp} n n

52. Best (induced) transformer 52 CA 2 3 f f # (a)=  (f(  (a))) 1 f#f# 4 Problem:  incomputable directly

53. Best transformer for tmp=tmp.n 53 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n { } n {h, tmp} n null h t {t} tmp n { } n {h, tmp} n { } 

54. Best transformer for tmp=tmp.n:  54 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n { } n {h, tmp} n null h t {t} tmp n { } n {h, tmp} n { }  null h t {t} tmp n { } n {h, tmp} n { } …

55. Best transformer for  tmp=tmp.n  55 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n {tmp } n {h} n null h t {t} tmp n { } n {h} n { } {tmp } null h t {t} tmp n { } n {h} n { } {tmp } …

56. Best transformer for tmp=tmp.n:  56 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n {tmp } n {h} n null h t {t} tmp n { } n {h} n {tmp } null h t {t} tmp n { } n {h} n {tmp } …

57. Handling updates on summary nodes Transformers accessing only concrete nodes are easy Transformers accessing summary nodes are complicated Can’t concretize summary nodes – represents potentially unbounded number of concrete nodes We need to split into cases by “materializing” concrete nodes from summary node – Introduce a new temporary predicate tmp.n – Partial concretization 57

58. Transformer for tmp=tmp.n:  ’ 58 null h t {t} tmp n { } n {h, tmp} n n ’’ Case 1: Exactly 1 object. Case 2: >1 objects.

59. Transformer for tmp=tmp.n:  ’ 59 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n {tmp.n } n {h, tmp} n null h t {t} tmp n { } n {h, tmp} n {tmp.n } ’’ tmp.n is a concrete node Summary node spaghetti

60. Transformer  tmp=tmp.n  60 null h t {t} tmp n { } n {h, tmp} n n null h t {t} tmp n {tmp, tmp.n } n {h} n null h t {t} tmp n { } n {h} n {tmp, tmp.n }

61. Transformer for tmp=tmp.n:  61 null h t {t} tmp n {tmp} n {h} n null h t {t} tmp n { } n {h} n {tmp} // Build a list SLL h=null, t = null; L1: h=t= new SLL(-1); SLL tmp = null; while (…) { int data = getData(…); L2: tmp = new SLL(data); tmp.n = h; h = tmp; } // Process elements tmp = h; while (tmp != t) { assert tmp != null; tmp.data += 1; tmp = tmp.n; }

62. Transformer via partial-concretization 62 CA 2 3 f f # (a)=  ’(f # ’(  ’(a))) 1 f#f# 4 A’ 5 f #’ 6 ’’ ’’

63. Recap Adding more properties to nodes refines abstraction Can add temporary properties for partial concretization – Materialize concrete nodes from summary nodes – Allows turning weak updates into strong ones – Focus operation in shape-analysis lingo – Not trivial in general and requires more semantic reduction to clean up impossible edges – General algorithms available via 3-valued logic and implemented in TVLA system 63

64. 3-Value logic based shape analysis

65. Sequential Stack void push (int v) { Node  x = malloc(sizeof(Node)); x->d = v; x->n = Top; Top = x; } 65 int pop() { if (Top == NULL) return EMPTY; Node *s = Top->n; int r = Top->d; Top = s; return r; } Want to Verify No Null Dereference Underlying list remains acyclic after each operation Want to Verify No Null Dereference Underlying list remains acyclic after each operation

66. Shape Analysis via 3-valued Logic 1) Abstraction – 3-valued logical structure – canonical abstraction 2) Transformers – via logical formulae – soundness by construction embedding theorem, [SRW02] 66

67. Concrete State represent a concrete state as a two-valued logical structure – Individuals = heap allocated objects – Unary predicates = object properties – Binary predicates = relations parametric vocabulary 67 Top nn u1u2u3 (storeless, no heap addresses)

68. Concrete State S = over a vocabulary P U – universe  - interpretation, mapping each predicate from p to its truth value in S 68 Top nn u1u2u3  U = { u1, u2, u3}  P = { Top, n }   (n)(u1,u2) = 1,  (n)(u1,u3)=0,  (n)(u2,u1)=0,…   (Top)(u1)=1,  (Top)(u2)=0,  (Top)(u3)=0

69.  v1,v2: n(v1, v2)  n*(v2, v1) Formulae for Observing Properties void push (int v) { Node  x = malloc(sizeof(Node)); x  d = v; x  n = Top; Top = x; } 69 Top nn u1u2u3 1 No Cycles  v1,v2: n(v1, v2)  n*(v2, v1) 1  w:Top(w) Top != null  w: x(w) No node precedes Top  v1,v2: n(v1, v2)  Top(v2) No node precedes Top  v1,v2: n(v1, v2)  Top(v2) 1

70. Concrete Interpretation Rules 70

71. Example: s = Top  n 71 Top s ’ (v) =  v1: Top(v1)  n(v1,v) u1 u2u3 Top s u1 u2u3 Top u11 u20 u30 nu1u2U3 u1010 u2001 u3000 s u10 u20 u30 Top u11 u20 u30 nu1u2U3 u1010 u2001 u3000 s u10 u21 u30

72. Collecting Semantics 72  {  st(w)  (S) | S  CSS[w] }  (w,v)  E(G), w  Assignments(G) { } CSS [v] = if v = entry  { S | S  CSS[w] }  (w,v)  E(G), w  Skip(G)  { S | S  CSS[w] and S  cond(w)}  (w,v)  True-Branches(G)  { S | S  CSS[w] and S   cond(w)} (w,v)  False-Branches(G) othrewise

73. Collecting Semantics At every program point – a potentially infinite set of two-valued logical structures Representing (at least) all possible heaps that can arise at the program point Next step: find a bounded abstract representation 73

74. 3-Valued Logic 1 = true 0 = false 1/2 = unknown  A join semi-lattice, 0  1 = 1/2 74 1/2 0 1 information order logical order

75. 3-Valued Logical Structures A set of individuals (nodes) U Relation meaning – Interpretation of relation symbols in P p 0 ()  {0,1, 1/2} p 1 (v)  {0,1, 1/2} p 2 (u,v)  {0,1, 1/2} A join semi-lattice: 0  1 = 1/2 75

76. Boolean Connectives [Kleene]

77. Property Space 3-struct[P] = the set of 3-valued logical structures over a vocabulary (set of predicates) P Abstract domain –  (3-Struct[P]) –  is  77

78. Embedding Order Given two structures S =, S’ = and an onto function f : U  U’ mapping individuals in U to individuals in U’ We say that f embeds S in S’ (denoted by S  S’) if – for every predicate symbol p  P of arity k: u1, …, uk  U,  (p)(u1,..., uk)   ’(p)(f(u1),..., f (uk)) – and for all u’  U’ ( | { u | f (u) = u’ } | > 1)   ’(sm)(u’) We say that S can be embedded in S’ (denoted by S  S’) if there exists a function f such that S  f S’ 78

80. Tight Embedding S’ = is a tight embedding of S= with respect to a function f if: – S’ does not lose unnecessary information  ’(u’ 1,…, u’ k ) =  {  (u 1..., u k ) | f(u 1 )=u’ 1,..., f(u k )=u’ k } One way to get tight embedding is canonical abstraction 80

81. Canonical Abstraction 81 Top nn u1u2u3 u1u2u3 [Sagiv, Reps, Wilhelm, TOPLAS02]

82. Canonical Abstraction 82 Top nn u1u2u3 u1u2u3 Top [Sagiv, Reps, Wilhelm, TOPLAS02]

83. Canonical Abstraction 83 Top nn u1u2u3 u1u2u3 Top

84. Canonical Abstraction 84 Top nn u1u2u3 u1u2u3 Top

85. Canonical Abstraction 85 Top nn u1u2u3 u1u2u3 Top

86. Canonical Abstraction 86 Top nn u1u2u3 u1u2u3 Top

87. Canonical Abstraction (  ) Merge all nodes with the same unary predicate values into a single summary node Join predicate values  ’(u’ 1,..., u’ k ) =  {  (u 1,..., u k ) | f(u 1 )=u’ 1,..., f(u k )=u’ k } Converts a state of arbitrary size into a 3-valued abstract state of bounded size  (C) =  {  (c) | c  C } 87

88. Information Loss 88 Top nn nn n Canonical abstraction Top n n

89. Instrumentation Predicates Record additional derived information via predicates 89 r x (v) =  v1: x(v1)  n*(v1,v) c(v) =  v1: n(v1, v)  n*(v, v1) Top nn c c c c nn n c c Canonical abstraction

90. Embedding Theorem: Conservatively Observing Properties 90 No Cycles  v1,v2: n(v1, v2)  n*(v2, v1) 1/2 Top r Top No cycles (derived)  v:  c(v) 1

91. Operational Semantics void push (int v) { Node  x = malloc(sizeof(Node)); x->d = v; x->n = Top; Top = x; } 91 int pop() { if (Top == NULL) return EMPTY; Node *s = Top->n; int r = Top->d; Top = s; return r; } s ’ (v) =  v1: Top(v1)  n(v1,v)  s = Top->n  Top nn u1u2u3 Top nn u1u2u3 s

92. Abstract Semantics 92 Top s = Top  n ? r Top Top r Top s s ’ (v) =  v1: Top(v1)  n(v1,v)  s = Top->n 

93. Top 93 Best Transformer (s = Top  n) Concrete Semantics  Canonical Abstraction Abstract Semantics ? r Top............ Top r Top Top r Top............ Top s r Top Top s r Top Top s r Top Top s r Top s ’ (v) =  v1: Top(v1)  n(v1,v)

94. Semantic Reduction Improve the precision of the analysis by recovering properties of the program semantics A Galois connection (C, , , A) An operation op:A  A is a semantic reduction when –  l  L 2 op(l)  l and –  (op(l)) =  (l) l CA   op

95. The Focus Operation Focus: Formula  (  (3-Struct)   (3-Struct)) Generalizes materialization For every formula  – Focus(  )(X) yields structure in which  evaluates to a definite values in all assignments – Only maximal in terms of embedding – Focus(  ) is a semantic reduction – But Focus(  )(X) may be undefined for some X

96. Top r Top Top r Top 96 Partial Concretization Based on Transformer (s=Top  n) Abstract Semantics Partial Concretization Canonical Abstraction Abstract Semantics Top s r Top Top s r Top Top r Top s Top s r Top Top r Top s ’ (v) =  v1: Top(v1)  n(v1,v)  u: top(u)  n(u, v) Focus (Top  n)

97. 97 Partial Concretization Locally refine the abstract domain per statement Soundness is immediate Employed in other shape analysis algorithms [Distefano et.al., TACAS’06, Evan et.al., SAS’07, POPL’08]

98. The Coercion Principle Another Semantic Reduction Can be applied after Focus or after Update or both Increase precision by exploiting some structural properties possessed by all stores (Global invariants) Structural properties captured by constraints Apply a constraint solver

99. Apply Constraint Solver Top r Top x x Top r Top x x x x rxrx r x, r y n n n n n n n n y y n n Top r Top  x x rxrx r x, r y n n n n y y n n

100. Sources of Constraints Properties of the operational semantics Domain specific knowledge – Instrumentation predicates User supplied

101. Example Constraints x(v1)  x(v2)  eq(v1, v2) n(v, v1)  n(v,v2)  eq(v1, v2) n(v1, v)  n(v2,v)   eq(v1, v2)  is(v) n*(v3, v4)  t[n](v1, v2)

102. Abstract Transformers: Summary Kleene evaluation yields sound solution Focus is a statement-specific partial concretization Coerce applies global constraints

103. Abstract Semantics 103  { t_embed(coerce(  st(w)  3 (focus F(w) (SS[w] ))))  (w,v)  E(G), w  Assignments(G) { } SS [v] = if v = entry  { S | S  SS[w] }  (w,v)  E(G), w  Skip(G)  { t_embed(S) | S  coerce(  st(w)  3 (focus F(w) (SS[w] ))) and S  3 cond(w)}  (w,v)  True-Branches(G)  { t_embed(S) | S  coerce(  st(w)  3 (focus F(w) (SS[w] ))) and S  3  cond(w)}  (w,v)  False-Branches(G) othrewise

104. Recap Abstraction – canonical abstraction – recording derived information Transformers – partial concretization (focus) – constraint solver (coerce) – sound information extraction 104

105. Stack Push void push (int v) { Node  x = alloc(sizeof(Node)); x  d = v; x  n = Top; Top = x; } 105 Top x x x x x x emp x x x x x x Top  v:  c(v)  v: x(v)  v1,v2: n(v1, v2)  Top(v2) …