1 Shape Analysis via 3-Valued Logic Mooly Sagiv Tel Aviv University Shape analysis with applications Chapter 4.6

2 Main Results A new abstract domain for static analysis Abstract dynamically allocated memory Abstract interpretation by Klenee’s 3-valued logic evaluation A system TVLA –Input: Structural operational semantics Input Program –Output: the result of the analysis

3 Motivation Dynamically allocated storage and pointers are an essential programming tools –Object oriented –Modularity –Data structure But –Error prone –Inefficient Static analysis can be very useful here

4 A Pathological C Program a = malloc(…) ; b = a; free (a); c = malloc (…); if (b == c) printf(“unexpected equality”);

5 Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for (elem = c; c != NULL; elem = elem->next;) if (elem->val == value) return TRUE; return FALSE

6 Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for (elem = c; c != NULL; elem = elem->next;) if (elem->val == value) return TRUE; return FALSE potential null de-reference

7 Memory leakage Elements* reverse(Elements *c) { Elements *h,*g; h = NULL; while (c!= NULL) { g = c->next; h = c; c->next = h; c = g; } return h; typedef struct element { int value; struct element *next; } Elements

8 Memory leakage Elements* reverse(Elements *c) { Elements *h,*g; h = NULL; while (c!= NULL) { g = c->next; h = c; c->next = h; c = g; } return h; leakage of address pointed-by h typedef struct element { int value; struct element *next; } Elements

9 class Make { private Worklist worklist; public static void main (String[] args) { Make m = new Make(); m.initializeWorklist(args); m.processWorklist(); } void initializeWorklist(String[] args) {...; worklist = new Worklist();... // add some items to worklist} void processWorklist() { Set s = worklist.unprocessedItems(); for (Iterator i = s.iterator(); i.hasNext()){ Object item = i.next(); if (...) processItem(item); } } void processItem(Object i){...; doSubproblem(...);} void doSubproblem(...) {... worklist.addItem(newitem);... } } public class Worklist { Set s; public Worklist() {...; s = new HashSet();... } public void addItem(Object item) { s.add(item); } public Set unprocessedItems() { return s; } } return rev; }

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL Materialization

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

52 Original Problem: Shape Analysis Characterize dynamically allocated data –x points to an acyclic list, cyclic list, tree, dag, etc. –data-structure invariants Identify may-alias relationships Establish “disjointedness” properties –x and y point to structures that do not share cells

53 Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures –No syntactic names for locations Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re- established

54 Formalizing “... ” Informal: x Formal: x Summary node

55 Applications: Software Tools Static detection of memory errors (cleanness) –dereferencing NULL pointers –dereferencing dangling pointers –memory leaks What is in the heap? –list? doubly-linked list? tree? DAG? –disjoint? intertwined? Static detection of logical errors –Is a shape invariant restored?

56 Properties of reverse(x) On entry: x points to an acyclic list On exit: y points to an acyclic list On exit: x = = NULL On each iteration, x and y point to disjoint acyclic lists All the pointer dereferences are safe No memory leaks

57 Plan Motivation SWhile An SOS for SWhile An SOS for SWhile using predicate calculus Simple Abstract interpretation using 3-valued logics More precise abstract interpretation+ TVLA (next meeting)

58 The SWhile Programming Language Abstract Syntax a := x | x.sel | null | n | a 1 op a a 2 b := true | false | not b | b 1 op b b 2 | a 1 op r a 2 S := [x := a] l | [x.sel := a] l | [x := malloc()] l | [skip] l | S 1 ; S 2 | if [b] l then S 1 else S 2 | while [b] l do S sel:= car | cdr

59 Dereference of NULL pointers [elem := c;] 1 [found := false;] 2 while ([c != null] 3 && [!found] 4 ) ( if ([elem->car= value] 5 ) then [found := true] 6 else [elem = elem->cdr] 7 )

60 Structural Operational Semantics The program state consists of: –current allocated objects –a mapping from variables into atoms, objects, and null –a car mapping from objects into atoms, objects, and null –a cdr mapping from objects into atoms, objects, and null malloc() allocates more objects assignments update the state

61 Structural Operational Semantics The program state S= : –current allocated objects O –atoms (integers, Booleans) A –env: Var *  A  O  {null} –car: A  A  O  {null} –cdr: A  A  O  {null} {l 1, l 2, l 3 }, [x  l 1,y  null], [l 1  1, l 2  2, l 3  3], [l 1  l2, l 2  l3, l 3  null] null 123 x y l1l1 l2l2 l3l3

62 The meaning of expressions A  a  : S  A  O  {null} A  at  (s) = at A  x  ( ) = env(x) A  x.cdr  ( ) = cdr(env(x)) env(x)  O undefinedotherwise A  x.car  ( ) = car(env(x)) env(x)  O undefinedotherwise

63 Structural Semantics for SWhile axioms [ass v sos ]  (O, e[x  A  a  s], car, cdr) [ass car sos ]  (O, e, car[e(x)  A  a  s], cdr) where env(x)  O [ass cdr sos ]  (O, e, car, cdr[e(x)  A  a  s]) where env(x)  O [skip sos ]  s [ass m sos ]  (O  {n}, e[x  n], car, cdr) where n  O

64 Structural Semantics for SWhile (rules) [comp 1 sos ]   [comp 2 sos ]  s’  [if tt sos ]  if B  b  s=tt [if ff sos ]  if B  b  s=ff

65 Summary The SOS is natural Can handle: –errors, e.g., null dereferences –free –garbage collection But does not lead to an analysis –The set of potential objects is unbounded Solution: Three-Valued Kleene Predicate Logic

66 Predicate Logic Vocabulary –A finite set of predicate symbols P each with a fixed arity –A finite set of function symbols Logical Structures S provide meaning for predicates –A set of individuals (nodes) U –P S : U S  {0, 1} First-Order Formulas over  express logical structure properties

67 P = {x 1, y 1, car 2, cdr 2 } U S ={l 1, l 2, l 3 } x S =[l 1  1, l 2  0, l 3  0]y S =[l 1  0, l 2  0, l 3  0}, car S =[  0,  0,  0,  0,  0,  0,  0,  0,  0 ] null 123 x y l1l1 l2l2 l3l3 {l 1, l 2, l 3 }, [x  l 1,y  null], [l 1  1, l 2  2, l 3  3], [l 1  l2, l 2  l3, l 3  null] cdr S =[  0,  1,  0,  0,  0,  1,  0,  0,  0 ]

68 Formal Semantics of First Order Formulae For a structure S= Formulae  with LVar free variables Assignment z: LVar  U S    S (z): {0, 1}  1  S ( z)=1  0  S ( z)=1  v 1 =v 2  S (z) = 1 z(v 1 ) = z(v 2 ) 0 z(v 1 )  z(v 2 )  p (v 1, v 2, …, v k )  S (z)=p S (z(v 1 ), z(v 2 ), …, z(v k ))

69 Formal Semantics of First Order Formulae For a structure S= Formulae  with LVar free variables Assignment z: LVar  U S    S (z): {0, 1}   1   2  S (z)=max (   1  S (z),   2  S (z))   1   2  S (z)=min (   1  S (z),   2  S (z))   1  S (z)=1-   1  S (z)   v:  1  S (z)=max {   1  S (z[v  u]) : u  U S }

70 Using Predicate Logic to describe states in SOS U=O For a pointer variable x define a unary predicate –x(u )=1 when env(x)=u and u is an object Two binary predicates: –car(u 1, u 2 ) = 1 when car(u 1 )=u 2 and u 2 is object –cdr(u 1, u 2 ) = 1 when cdr(u 1 )=u 2 and u 2 is object

71 SOS (Using Predicate Logic) First-order structures (= predicate tables) –hold recorded information Formulae –means for observing information Predicate-update formulae –operational semantics –update recorded information

72 Recorded Information (for reverse)

73 Recorded Information (for reverse) u1u1 u2u2 u3u3 u4u4 x y

74 Formulae for Observing Properties Are x and y pointer aliases?  v: x(v)  y(v) Does x point to a cell with a self cycle?  v : x(v)  n(v,v)

75 x y u1u1 u2u2 u3u3 u4u4 Are x and y Pointer Aliases?  v: x(v)  y(v) x y u1u1  Yes

76 x ’ (v) = x(v) y ’ (v) = 0 t ’ (v) = t(v) n ’ (v1,v2) = n(v1,v2) Predicate-Update Formulae for ‘y = NULL’

77 x y u1u1 u2u2 u3u3 u4u4 Predicate-Update Formulae for ‘y = NULL’ y ’ (v) = 0 

78 Predicate-Update Formulae for ‘y = x’ x ’ (v) = x(v) y ’ (v) = x(v) t ’ (v) = t(v) n ’ (v1,v2) = n(v1,v2)

79 x u1u1 u2u2 u3u3 u4u4 Predicate-Update Formulae for ‘y = x’ y ’ (v) = x(v) y 

80 Predicate-Update Formulae for ‘x = x  n’ x ’ (v) =  v1: x(v1)  n(v1,v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v1, v2) = n(v1, v2)

81 x u1u1 u2u2 u3u3 u4u4 Predicate-Update Formulae for ‘x = x  n’ y x ’ (v) =  v1: x(v1)  n(v1,v)   x

82 Predicate-Update Formulae for ‘y  n = t’ x ’ (v) = x(v) y ’ (v) = y(v) t ’ (v) = t(v) n ’ (v1,v2) =  y(v1)  n(v1,v2)  y(v1)  t(v2)

83 Two- vs. Three-Valued Logic 01 Two-valued logic {0,1} {0}{1} Three-valued logic {0}  {0,1} {1}  {0,1}

84 Two- vs. Three-Valued Logic Two-valued logicThree-valued logic

85 Two- vs. Three-Valued Logic Three-valued logic 0 1 Two-valued logic {1} {0,1} {0} 1 ½ 0

86 1: True 0: False 1/2: Unknown A join semi-lattice: 0  1 = 1/2 Three-Valued Logic   1/2 Information order

87 Boolean Connectives [Kleene]

88 The Abstraction Principle Partition the individuals into equivalence classes based on the values of their unary predicates Collapse other predicates via 

89 The Abstraction Principle u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234        

90 What Stores Does a 3-Valued Structure Represent? Example 3-valued structure –individuals: {u 1 } –predicates: graphical presentation concrete stores represented x u1u1 33 x 88 x 37 x

91 Example 3-valued structure graphical presentation concrete stores What Stores Does a 3-Valued Structure Represent? u1u1 u x u1u1 u x  x

92 Example 3-valued structure graphical presentation concrete stores u1u1 u x u1u1 u x  x What Stores Does a 3-Valued Structure Represent?

93 Property-Extraction Principle Questions about store properties can be answered conservatively by evaluating formulae in three-valued logic Formula evaluates to 1  formula always holds in every store Formula evaluates to 0  formula never holds in any store Formula evaluates to 1/2  don’t know  

94 The Embedding Theorem If a big structure B can be embedded in a structure S via a surjective (onto) function f such that basic predicates are preserved, i.e., p B (u 1,.., u k )  p S (f(u 1 ),..., f(u k )) Then, every formula  is preserved –  = 1 in S   = 1 in B –  =0 in S   =0 in B –  = 1/2 in S  don’t know

95 Are x and y Pointer Aliases? u1u1 u x y  v: x(v)  y(v)    Yes 1

96 Is Cell u Heap-Shared?  v1,v2: n(v1,u)  n(v2,u)  v1  v2 u Yes 1  

97 Maybe Is Cell u Heap-Shared?  v1,v2: n(v1,u)  n(v2,u)  v1  v2 u1u1 u x y 1/2   1

98 The Instrumentation Principle Increase precision by storing the truth- value of some designated formulae Introduce predicate-update formulae to update the extra predicates

99 is = 0 Example: Heap Sharing  x is(v) =  v1,v2: n(v1,v)  n(v2,v)  v1  v2 u1u1 u x u1u1 u x is = 0 is = 1 is = 1/2

100 is = 0 Example: Heap Sharing  x is(v) =  v1,v2: n(v1,v)  n(v2,v)  v1  v2 u1u1 u x u1u1 u x is = 0

101 is = 0 Example: Heap Sharing  x is(v) =  v1,v2: n(v1,v)  n(v2,v)  v1  v2 u1u1 u x u1u1 u x is = 0 is = 1

102 Is Cell u Heap-Shared?  v1,v2: n(v1,u)  n(v2,u)  v1  v2 u1u1 u x y  1/2   1 is = 0 No!

103 Example2: Sortedness inOrder(v) =  v1: n(v,v1)  dle(v, v1) u1u1 u x u1u1 u x inOrder = 1 n n  x inOrder = 1 n n n        

104 inOrder = 1 Example2: Sortedness  x inOrder(v) =  v1: n(v,v1)  dle(v, v1) u xx inOrder = 0inOrder = 1 n n n n n inOrder = 0          n  

105 Shape Analysis via Abstract Interpretation Iteratively compute a set of 3-valued structures for every program point Every statement transforms structures according to the predicate-update formulae –use 3-valued logic instead of 2-valued logic –use exactly the predicate-update formulae of the concrete semantics!!

106 Predicate-Update Formulae for “y = x” y ’ (v) = x(v) Old: u1u1 u x y New: u1u1 u x 

107 Predicate-Update Formulae for “x = x  n” x ’ (v) =  v1: x(v1)  n(v1,v) y Old: u1u1 u x y New: u1u1 u x  

108 Summary Predicate logics allows naturally expressing SOS for languages with pointers and dynamically allocated structures 3-valued logic provides a sound solution More precise solution+TVLA (next meeting)