1 Verifying Temporal Heap Properties Specified via Evolution Logic Eran Yahav, Tom Reps, Mooly Sagiv and Reinhard Wilhelm ESOP 2003

2 Introduction Goals: specify and verify temporal properties of sequential and concurrent heap manipulating programs specify the way objects evolve across program execution focus on Java-like programs support the following  Java-like Concurrency  Dynamic allocation/deallocation of objects  Dynamic allocation/deallocation of threads

3 Relate memory locations across program configurations Allow specification relating to allocation and deallocation of objects Example: concurrent GC  Safety – only objects not reachable from the roots are collected  Liveness – all garbage objects are eventually collected Propositional temporal logic is not enough Motivate use of more expressive specification language Spatial and Temporal Properties spatialtemporal

4 Spatial and Temporal Properties L1: while (x != null) { L2:e = x L3:x = x.n L4:e.n = null L5:free(e) } n x n at[L1] Every object is eventually pointed-to by x … n x n x at[L1] …

5 Spatial and Temporal Properties Every allocated object is eventually deallocated Every allocated request is eventually assigned handler thread An object is eventually removed from pointer- based data structure Each opened file remains open until used …

6 Challenges Varying domains  Set of objects in the heap likely to change during program execution Dynamic allocation and deallocation No a priori bound on number of objects/threads Progress  Abstraction of transitions/traces  Progress may be lost under abstraction

7 Plan Program Configurations and Traces Specification  Evolution Temporal Logic (ETL)  Meaning of ETL formulae Verification  Reducing ETL to FO TC Representing ETL Traces via FO Structures Compiling ETL formulae to FO TC formulae  Abstract Interpretation  Prototype implementation Summary

8 Program Configurations A concrete program configuration encodes  global store  program-location of every thread  status of locks and threads First-order logical structures used to represent program configurations

9 Concrete Configuration n x n at[L3] e

10 Concrete Configuration at[l_C] rval[f] held_by blocked at[l_1] rval[f] at[l_0] at[l_1] rval[f] blocked

11 Program Traces Infinite sequence of program configurations Each step is a single program action Individuals may vary between configurations  Dynamic allocation / deallocation … xxxeex at[L1]at[L2]at[L3]at[L4]at[L5]at[L1] exx

12 Evolution Temporal Logic (ETL) Based on first-order linear temporal logic   v.  (v),  v.  (v), TC  X ,  U , ,  State formulae may include free variables  Relate memory locations across configurations (worlds)   v. x(v)  e(v) Special operators   v object v allocated   v object v deallocated Predicates represent properties of interest  For heap references – x(v),n(v 1,v 2 ),…  for threads and locks – blocked(t,l), held_by(l,t),…

13 ETL Examples Every object is eventually pointed-to by x   v.  x(v) Every allocated object is eventually deallocated   (  v.  v    v) Every allocated request is eventually assigned handler thread    r:request.  r    t:thread. handles(t,r) An object v is eventually removed from a pointer-based data structure s  …    u:s(u)  n*(u,v)…

14 ETL Semantics Infinite sequence of configurations World locality  An individual may exist in at most one world  Equality is world-local Evolution  Explicit representation of evolution relation of individuals across worlds  Explicitly represent allocated and deallocated individuals

15 ETL Traces deallocatedobjectevolution edge … x at[L1] x at[L2] x at[L1] x at[L3] e e at[L4] xe at[L5] x

16 Meaning of ETL formulae … x at[L1] x at[L2] x at[L1] x at[L3] e e at[L4] xe at[L5] x x(v)  x(v)  v. 

17 Temporally Separable Properties Properties which do not relate individuals of different configurations Temporal operators only over closed FO formulae Corresponds to propositional temporal logic   v.x(v)  v’.  n(v,v’)   P, P=  v.x(v)  v’.  n(v,v’) P … xxxex at[L1]at[L2]at[L3]at[L5]at[L1] ex …

18 Spatially Separable Properties Universally quantified propositional specification Each object should obey the specification separately  Typestate verification Examples:   v.  x(v)   f:file.   (read(f)  closed(f)) … x at[L1] x at[L2] x at[L1] x at[L3] e e at[L4] xe at[L5] x

19 ETL Traces as FO Structures … x at[L1] x at[L2] x at[L1] x at[L3] e e at[L4] xe at[L5] x

20 ETL Traces as FO Structures … xxx xe exex at[L1]at[L2]at[L1] at[L3] at[L4]at[L5] deallocationobjectworld existence edge evolution edge succ

21 ETL Traces as FO Structures … x at[L1] x at[L2] x at[L1] x at[L3] e e at[L4] xe at[L5] x

22 Representing ETL Traces via First-order Structures Explicitly encode possible worlds and accessibility relation  World individuals  Successor edges relate worlds  Each non-world individual exists in at most one world  Existence predicate relates non-world individuals to the world in which they exist Designated predicates  succ(w 1,w 2 )  exists(o,w)  evolves(o 1,o 2 ) Adapted from Lewis’s “counterpart semantics”

23 Extracting ETL properties ETL properties compiled into plain FO TC formulae ETL trace encoded as FO structure Evaluate ETL over ETL-trace by evaluating corresponding FO TC formula over FO structure

24 All is well, but… ETL traces are infinite Number of traces for a program is possibly infinite

25 Abstract Interpretation (Over-) Approximate possibly infinite set of infinite traces by finite set of finite abstract traces Successive Approximations  Compute the greatest fixed point  Start with an abstract trace representing initial configuration with all possible suffixes  Repeatedly refine the results by exploring longer finite prefixes  Longer abstract trace  represents fewer concrete traces Evaluate property over abstract traces in the fixed point Use 3-valued logical structures for abstract traces

26 Canonic Abstraction x at[L1] currWorld x at[L1] x at[L2] succ x at[L3] e succ … currWorld succ x at[L2] x at[L3] e succ

27 Abstraction Example rval[v] heldBy blocked rval[v] succ rval[v] heldBy rval[v] heldBy blocked t0 at[l_1] t0 at[l_1] at[l_c] t0 at[l_1] t0 at[l_1] at[l_c] currWorldinitialWorld rval[v] succ rval[v] heldBy rval[v] blocked at[l_1] at[l_2] at[l_c] t0 at[l_1] rval[v] t0 at[l_1] rval[v] blocked initialWorld rval[v] at[l_1] t0 at[l_1] rval[v] succ currWorld … succ

28 Growing Abstract Traces Partial Concretization (Focus) Apply update  Append new configuration to abstract trace  New configuration reflects update effect  Add Successor edge into new configuration Evolution edges into evolved individuals  Update currWorld predicate Abstraction

29 currWorld x at[L1] succ concretization abstraction update … x at[L1] x at[L2]at[L3] x x … x at[L1] at[L2]at[L3] x x … x e x at[L1] at[L2]at[L3] x at[L4] x … x e currWorld x at[L1] succ x at[L2] succ

30 Greatest Fixed Point x at[L1] succ … x at[L1] x at[L2]at[L3] x x … x at[L1] at[L2]at[L3] x x … x e x at[L1] at[L2]at[L3] x at[L4] x … x e

31 Greatest Fixed Point … x at[L1] x at[L2]at[L3] x x … x at[L1] at[L2]at[L3] x x … x e x at[L1] at[L2]at[L3] x at[L4] x … x e x at[L1] succ x at[L2] succ

32 Recording History Improve precision add predicates for subformulae of the ETL formula Record state of subformulae satisfaction over the trace Tailor abstraction according to property of interest

33 Progress Progress may be lost under abstraction Common for liveness to require augmentation with progress information  Can express progress measure for linked data structures in ETL  e.g., progress of a linked data structure traversal Number of items reachable from a program variable decreases

34 Implementation Manually  Convert ETL to FO TC  Define instrumentation predicates for temporal subformulae Let TVLA do the rest Properties proved  Termination of linked list manipulation  Response (fair/unfair) Takes a lot of time

35 In the paper… Soundness Technicalities  Transworld Equality  Instrumentation predicates

36 Related Work Model Checking Birth and Death / Distefano,Rensink,Katoen [TCS ‘02]  Decidable temporal logic  Allows referring to moment of allocation and deallocation  Does not allow relationships between objects  Simple abstraction – collapse all non-reachable objects

37 Summary ETL allows specification of heap evolution properties Automatically verify ETL properties  Represent ETL traces via FO structures  Represent ETL properties as FO TC formulae  Evaluate FO TC formula over 3-valued FO structures representing sets of traces Common for liveness properties to require reduction or progress monitors  Progress expressed as ETL formulae

38 Future Work More precise and efficient algorithms for verifying ETL  Tableau-like verification method ETL subclasses  Already used spatially separable properties for memory management properties [SAS’03]

39

40

41 ETL Examples    v.x(v)   v. e(v)  v  v.  x(v)  v.x(v)

42 ETL to FO TC

43 Subtle Issues Fairness  We can express explicit scheduling queue  Other notions of fairness under dynamic allocation?  Reduction Constant domain semantics  Requires user to specify existence or use syntactically different quantifiers for global/local quantification Monotone domain semantics  Easy to understand, a viable alternative

44 Progress x at[L1] currWorld succ x at[L2] succ x at[L3] e succ x at[L4] e succ

45 Example While (x != null) { e = x x = x.n e.n = null free(e) } n e n x e n x n xe n x n ex ex Empty list … e = x x = x.n e.n = null free(e)

46 Why not Constant Domains? Requires user to explicitly specify existence  or use syntactically different quantifiers for global/local quantification Explicit evolution edges allow to abstract away from implementation details  Can handle various allocation semantics  Can handle copy-garbage-collector

47 ETL to FO TC  v  w 0 initialWorld(w 0 )  exists(w 0, v)   w,v’ succ*(w 0,w)  evolution*(v,v’)  exists(w,v’)  P(v)

48 Growing Abstract Traces currWorld x at[L1] succ x at[L1] succ x at[L2] succ