3-Valued Logic Analyzer (TVP) Part II Tal Lev-Ami and Mooly Sagiv

Outline u The Shape Analysis Problem u Solving Shape Analysis with TVLA –Structural Operational Semantics –Predicate logic –Embedding –(Imprecise) Abstract Interpretation –Instrumentation Predicates –Focus –Coerce u Bibliography

Shape Analysis u Determine the possible shapes of a dynamically allocated data structure at given program point u Relevant questions: –Does a variable point to an acyclic list? –Does a variable point to a doubly-linked list? –Does a variable point p to an allocated element every time p is dereferenced? –Can a procedure create a memory-leak

Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for ( elem = c; c != NULL;elem = elem->next;) if (elem->val == value) return TRUE; return FALSE NULL dereference

Memory leakage Elements* reverse(Elements *c) { Elements *h,*g; h = NULL; while (c!= NULL) { g = c->next; h = c; c->next = h; c = g; } return h; leakage of address pointed-by h

The SWhile Programming Language Abstract Syntax a := x | x.sel | null | n | a 1 op a a 2 b := true | false | not b | b 1 op b b 2 | a 1 op r a 2 S := [x := a] l | [x.sel := a] l | [x := malloc()] l | [skip] l | S 1 ; S 2 | if [b] l then S 1 else S 2 | while [b] l do S sel:= car | cdr

Dereference of NULL pointers [elem := c;] 1 [found := false;] 2 while ([c != null] 3 && [!found] 4 ) ( if ([elem->car= value] 5 ) then [found := true] 6 else [elem = elem->cdr] 7 ) NULL dereference

Structural Operational Semantics for languages with dynamically allocated objects u The program state consists of: –current allocated objects –a mapping from variables into atoms, objects, and null –a car mapping from objects into atoms, objects, and null –a cdr mapping from objects into atoms, objects, and null –…–… u malloc() allocates more objects u assignments update the state

Structural Operational Semantics u The program state S=(O, env, car, cdr): –current allocated objects O –atoms (integers, Booleans) A –env: Var *  A  O  {null} –car: A  A  O  {null} –cdr: A  A  O  {null} u The meaning of expressions A  a  : S  A  O  {null} –A  at  (s) = at –A  x  ((O, env, car, cdr)) = env(x) –A  x.car  ((O, env, car, cdr)) = car(env(x)) –A  x.cdr  ((O, env, car, cdr)) = cdr(env(x))

Structural Semantics for SWhile axioms [ass v sos ]  (O, e[x  A  a  s], car, cdr) [ass car sos ]  (O, e, car[e(x)  A  a  s], cdr) [ass cdr sos ]  (O, e, car, cdr[e(x)  A  a  s]) [skip sos ]  s [ass m sos ]  (O  {n}, e[x  n], car, cdr) where n  O

Structural Semantics for SWhile rules [comp 1 sos ]   [comp 2 sos ]  s’  [if tt sos ]  if B  b  s=tt [if ff sos ]  if B  b  s=ff

Summary u The SOS is natural u Can handle: –errors, e.g., null dereferences –free –garbage collection u But does not lead to an analysis –The set of potential objects is unbound u Solution: Three-Valued Kleene Predicate Logic

Predicate Logic u Vocabulary –A finite set of predicate symbols P each with a fixed arity –A finite set of function symbols u Logical Structures S provide meaning for predicates –A set of individuals (nodes) U –P S : U S  {0, 1} u First-Order Formulas over  express logical structure properties

Using Predicate Logic to describe states in SOS u U=O u For a Boolean variable x define a nullary predicate (proposition) b[x] –b[x] = 1 when env(x)=1 u For a pointer variable x define a unary predicate –p[x](u)=1 when env(x)=u and u is an object u Two binary predicates: –s[car](u1, u2) = 1 when car(u1)=u2 and u2 is object –s[cdr](u1, u2) = 1 when cdr(u1)=u2 and u2 is object

Running Example [elem := c;] 1 [found := false;] 2 while ([c != null] 3 && [!found] 4 ) ( if ([elem->car= value] 5 ) then [found := true] 6 else [elem = elem->cdr] 7 )

%s Pvar {elem, c} %s Bvar {found} %s Sel {car, cdr} #include "pred.tvp" % #include "cond.tvp" #include "stat.tvp" % /* [elem := c;] 1 */ l_1 Copy_Var(elem, c) l_2 /* [found := false;] 2 */ l_2 Set_False(found) l_3 /* while ([c != null] 3 && [!found] 4 ) ( */ l_3 Is_Not_Null_Var (c) l_4 l_3 Is_Null_Var (c) l_end l_4 Is_False(found) l_5 l_4 Is_True(found) l_end /*if ([elem->car= value] 5 ) */ l_5 Uninterpreted_Cond() l_6 l_5 Uninterpreted_Cond() l_7 /*then [found := true] 6 */l_6 Set_True(found) l_3 /*else [elem = elem->cdr] 7 */ l_7 Get_Sel(cdr, elem, elem) l_3 /* ) */ % l_1, l_end

foreach (z in Bvar) { %p b[z]() } pred.tvp foreach (z in Pvar) { %p p[z](v) unique box } foreach (sel in Sel) { %p s[sel](v1, v2) function }

Actions u Use first order formulae over  to express the SOS u Every action can have: –title %t –focus formula %f –precondition formula %p –error messages %message –new formula %new –predicate-update formulas {} –retain formula

cond.tvp (part 1) %action Uninterpreted_Cond() { %t "uninterpreted-Condition" } %action Is_True(x1) { %t x1 %p b[x1]() { b[x1]() = 1 } %action Is_False(x1) { %t "!" + x1 %p !b[x1]() { b[x1]() = 0 }

cond.tvp (part 2) %action Is_Not_Null_Var(x1) { %t x1 + " != null" %p E(v) p[x1](v) } %action Is_Null_Var(x1) { %t x1 + " = null" %p !(E(v) p[x1](v)) }

stat.tvp (part 1) %action Skip() { %t "Skip" } %action Set_True(x1) { %t x1 + " := true" { b[x1]() = 1 } %action Set_False(x1) { %t x1 + " := false" { b[x1]() = 0 }

stat.tvp (part 2) %action Copy_Var(x1, x2) { %t x1 + " := " + x2 { p[x1](v) = p[x2](v) }

stat.tvp (part 3) %action Get_Sel(sel, x1, x2) { %t x1 + " := " + x2 + “.” + sel %message (!E(v) p[x2](v)) -> "an illegal dereference to" + sel + " component of " + x2 { p[x1](v) = E(v_1) p[x2](v_1) & s[sel](v_1, v) }

stat.tvp (part 4) %action Set_Sel_Null(x1, sel) { %t x1 + "." + sel + " := null" %message (!E(v) p[x1](v)) -> "an illegal dereference to" + sel + " component of " + x1 { s[sel](v_1, v_2) = s[sel](v_1, v_2) & !p[x1](v_1) }

stat.tvp (part 5) %action Set_Sel(x1, sel, x2) { %t x1 + “.” + sel + " := " + x2 %message (E(v, v1) p[x1](v) & s[sel](v, v1)) -> "Internal Error! assume that " + x1 + "." + sel + ==NULL" %message (!E(v) p[x1](v)) -> "an illegal dereference to" + sel + " component of " + x1 { s[sel](v_1, v_2) = s[sel](v_1, v_2) | p[x1](v_1) & p[x2](v_2) }

stat.tvp (part 6) %action Malloc(x1) { %t x1 + " := malloc()" %new { p[x1](v) = isNew(v) }

3-Valued Kleene Logic u A logic with 3-values –0 -false –1 - true –1/2 - don’t know u Operators are conservatively interpreted –1/2 means either true or false 01 1/2 Logical order information order 0  1=1/2

Kleene Interpretation of Operators (logical-and)

Kleene Interpretation of Operators (logical-or)

Kleene Interpretation of Operators (logical-negation)

Kleene Interpretation of Operators (logical-implication)

3-Valued Predicate Logic u Vocabulary –A finite set of predicate symbols P –A special unary predicate sm »sm(u)=0 when u represents a unique concrete node »sm(u)=1/2 when u may represent more than one concrete node u 3-valued Logical Structures S provide meaning for predicates –A (bounded) set of individuals (nodes) U –P S : U S  {0, 1/2, 1} u First-Order Formulas over  express logical structure properties u Interpret  as maximum on logical order

The Blur Operation u Abstract an arbitrary structure into a structure of bounded size u Select a set of unary predicates as abstraction-predicates u Map all the nodes with the same value of abstraction predicates into a single summary node u Join the values of other predicates

The Embedding Theorem u If a big structure B can be embedded in a structure S via a surjective (onto) function f such that all predicate values are preserved, i.e., p B (u 1,.., u k )  p S (f(u 1 ),..., f(u k )) u Then, every formula  is preserved  is preserved –  = 1 in S   = 1 in B –  =0 in S   =0 in B –  = 1/2 in S  don’t know

Naive Program Analysis via 3-valued predicate logic u Chaotic iterations u Start with the initial 3-valued structure u Execute every action in three phases: –check if precondition is satisfied –execute update formulas –execute blur –Command line tvla prgm prgm -action pub

prgm.tvs %n = {u, u0} %p = { sm = {u:1/2} s[cdr] = {u->u:1/2, u0->u:1/2} p[c] = {u0} }

More Precise Shape Analysis u Distinguish between cyclic and acyclic lists u Use Focus to guarantee that important formulas do not evaluate to 1/2 u Use Coerce to maintain global invariants u It all works –Singly linked lists (reverse, insert, delete, del_all) –Sortedness (bubble-sort, insetion-sort, reverse) –Doubly linked lists (insert, delete –Mobile code (router) –Java multithreading (interference, concurrent-queue)

The Instrumentation Principle u Increase precision by storing the truth- value of some designated formulae u Introduce predicate-update formulae to update the extra predicates

is = 0 Example: Heap Sharing  x is[cdr](v) =  v1,v2: cdr(v1,v)  cdr(v2,v)  v1  v2 u1u1 u x u1u1 u x is = 0

Example: Heap Sharing  x is[cdr](v) =  v1,v2: cdr(v1,v)  cdr(v2,v)  v1  v2 u1u1 u x u1u1 u x is = 0 is = 1

foreach (z in Bvar) { %p b[z]() } pred.tvp foreach (z in Pvar) { %p p[z](v) unique box } foreach (sel in Sel) { %p s[sel](v1, v2) function } foreach (sel in Sel) { %i is[sel](v) = E(v1, v2) sel(v_1) & sel(v2, v) & v_1 != v_2 }

stat.tvp (part 4) %action Set_Sel_Null(x1, sel) { %t x1 + "." + sel + " := null" %message (!E(v) p[x1](v)) -> "an illegal dereference to" + sel + " component of " + x1 { s[sel](v_1, v_2) = s[sel](v_1, v_2) & !p[x1](v_1) is[sel](v) = is(v) & (!(E(v_1) x1(v_1) & sel(v_1, v)) | E(v_1, v_2) v_1 != v_2 & (sel(v_1, v) & !x1(v_1)) & (sel(v_2, v) & !x1(v_2))) }

stat.tvp (part 5) %action Set_Sel(x1, sel, x2) { %t x1 + “.” + sel + " := " + x2 %message (E(v, v1) p[x1](v) & s[sel](v, v1)) -> "Internal Error! assume that " + x1 + "." + sel + ==NULL" %message (!E(v) p[x1](v)) -> "an illegal dereference to" + sel + " component of " + x1 { s[sel](v_1, v_2) = s[sel](v_1, v_2) | p[x1](v_1) & p[x2](v_2) is[sel](v) = is[sel](v) | E(v_1) x2(v) & sel(v_1, v) }

u reachable-from-variable-x(v)  v1:x(v1)  cdr*(v1,v) u cyclic-along-dimension-d(v) cdr+(v, v) u ordered element inOrder(v)  v1:cdr(v, v_1)  v->d d u doubly linked lists Additional Instrumentation Predicates

The Focusing Principle u To increase precision –“Bring the predicate-update formula into focus” (Force 1/2 to 0 or 1) –Then apply the predicate-update formulas

(1) Focus on  v 1 : x(v 1 )  cdr(v 1,v) u1u1 x y u xy u1u1 u xy y u1u1 u.1 x  u1u1 u.0 u

x(v) =  v 1 : x(v 1 )  cdr(v 1,v) (2) Evaluate Predicate-Update Formulae xy u1u1 u xy y u1u1 u.1 x  u1u1 u.0 u u1u1 u x u1u1 u.1 x u.0 y x y u1u1 u

The Coercion Principle u Increase precision by exploiting some structural properties possessed by all stores (Global invariants) u Structural properties captured by constraints u Apply a constraint solver

(3) Apply Constraint Solver u1u1 u x u1u1 u.1 x u.0 y x y u1u1 u u1u1 u x x y u1u1 u u1u1 u.1 x u.0 y

Conclusion u TVLA allows construction of non trivial analyses u But it is no panacea –Expressing operational semantics using logical formulas is not always easy –Need instrumentation to be reasonably precise (sometimes help efficiency as well) u Open problems: –A debugger for TVLA –Frontends –Algorithmic problems: »Space optimizations

Bibliography u Chapter 2.6 u (Invited talk CC’2000) u Parametric Shape Analysis based on 3-valued logics (the general theory) u The system and its applications