Program Verification with Graph Types and Monadic Second-order Logic Anders Møller Michael I. Schwartzbach BRICS, Aarhus University

Introduction Motivation: find bugs optimize code in programs that heavily rely on pointers, such as data type implementations General approach: model the heap, including records and pointer fields describe properties in some assertion language verify correctness of assertions Variations in expressive power and use(/need) of annotations. Program Verification with Graph Types and Monadic Second-order Logic

Three approaches Shape Types [Fradet & Le Métayer, POPL’97] based on context-free graph grammars Shape Graphs [Sagiv, Reps & Wilhelm, POPL’96+’99] based on classical abstract interpretation and program analysis Graph Types [Klarlund & Schwartzbach, POPL’93] based on regular expressions and monadic second-order logic Program Verification with Graph Types and Monadic Second-order Logic

Overview Shape Types Shape Graphs Graph Types Verification with graph types Monadic 2nd-order Logic (M2L) Hoare logic Implementation: PALE – the Pointer Assertion Logic Engine Example: red_black_insert.pale Program Verification with Graph Types and Monadic Second-order Logic

Shape Types [Fradet & Le Métayer, POPL’97] Define classes of heap structures with context-free graph grammars Model program execution with single-step transformers Program Verification with Graph Types and Monadic Second-order Logic

Shape Types - example An example: the doubly-linked list with a pointer to the first element can be described by a multiset of tuples: {p a1, pred a1 a1, next a1 a2, pred a2 a1, next a2 a3, pred a3 a2, next a3 a3} Such lists can be described by a context-free graph grammar: Doubly = p x, pred x x, L x L x = next x y, pred y x, L y L x = next x x where Doubly and L are non-terminals. pred next next next a1 a2 a3 p pred pred Program Verification with Graph Types and Monadic Second-order Logic

Shape Types - transformers A program step is modeled by a single-step transformer, e.g: next a b, pred b a, next b c, pred c b  next a c, pred c a “Shape invariance”: rewriting a graph using a transformer must preserve the shape type – the paper describes a conservative algorithm Program Verification with Graph Types and Monadic Second-order Logic

Shape Types – practical problems Requires extensive modifications of the programming language Shape invariance, grammar equivalence, and grammar inclusion are undecidable decidable for a rather unelegant subclass no experimental results Apparently, only considers single steps of execution does not permit temporary but intentional invalidation of properties, which always occurs in programs Program Verification with Graph Types and Monadic Second-order Logic

Shape Graphs [Sagiv, Reps & Wilhelm, POPL’96] Idea: approximate a concrete heap with a shape graph: nodes that are not directly reachable from program variables are collapsed into a summary node nodes are labeled with sharing information (a single bit) (many variants) Use abstract interpretation (standard forward least-fixpoint iterative data-flow analysis) to find a (set of) shape graphs for each program point. Note: uses no annotations, but scales poorly compared to its precision Program Verification with Graph Types and Monadic Second-order Logic

Parametric Shape Analysis [Sagiv, Reps & Wilhelm, POPL’99] Use a 3-valued first-order logic with transitive closure Represent shape graphs as logical structures Parametric in the choice of instrumentation predicates Assertions can be specified in the logic Core predicates: ptr_x(v) (“does the program variable x point to the record v?”) succ_f(v,w) (“does the f field of v point to w?”) sm(v) (“does v represent more than one concrete record?”) Example instrumentation predicates: reachable(v) (“is v reachable from some program variable?”) shared(v) (“does more than one field point to v?”) More general approach, but same problems with scalability, and requires manual construction of interpretation of predicates Similarities with the approach based on monadic second-order logic (next...) Program Verification with Graph Types and Monadic Second-order Logic

Graph Types [Klarlund & Schwartzbach, POPL’93] A Graph type is a recursive type with auxiliary pointers: the recursive type defines a spanning tree (the “backbone”) the auxiliary pointers provide short-cuts across the backbone or into other trees they must be functionally determined by the backbone (“wellformedness”) they are defined by “routing expressions” Graph types are interesting because many common shapes can be expressed monadic second-order logic on finite trees can be used for verification – and this is one of the most expressive logics that is practically decidable Program Verification with Graph Types and Monadic Second-order Logic

Graph types – routing expressions A routing expression is a regular expression over navigation directives: a: “move down” along a given pointer field : “move up” one step backwards ^: “check at root” of the backbone $: “check at leaf” T:v “check type and variant” combined with choice (r1+r2), sequence (r1 r2), and repetition (r*) Program Verification with Graph Types and Monadic Second-order Logic

Graph Types – examples A few examples: list with pointer to the last element H  (first: L, last: L[first tail* $ ]) L  (tail: L) L  () doubly-linked cyclic list D  (next: D, prev: D[ + ^next* $]) D  (next: D[* ^], prev: D[ +^]) binary tree in which all the leaves are joined in a cyclic list J  (left, right: J) J  (next: J[right* (left right + ^) left*]) H L L L L first tail tail last Program Verification with Graph Types and Monadic Second-order Logic

Monadic Second-order Logic on finite Trees (M2L-Tree) Allows routing expressions and wellformedness to be expressed Provides an expressive assertion language for imperative programs manipulating graph type structures ::= ¬ |  |  |  |  | 1x. | 1x. | 2X. | 2X. (formulas) | t=t | tT | T=T | TT | ... T ::= X | TT | TT | T\T |  (set terms) t ::= x | t.left | t.right | t.up (position terms) Decidable using tree automata. Implementation: the MONA tool (www.brics.dk/mona) Program Verification with Graph Types and Monadic Second-order Logic

Graph Types and M2L-Tree for Program Verification Pointer Assertion Logic (PAL): M2L–Tree + generalized routing expressions (as syntactic sugar) Use PAL formulas in graph type declarations (to specify the auxiliary pointers and express wellformedness) as program assertions (e.g. invariants, pre- and post-conditions) Goal: automatically verify correctness of these program annotations Program Verification with Graph Types and Monadic Second-order Logic

Hoare Logic Require invariants at all while-loops and procedure calls (extra assertions are also allowed) Split the program into Hoare triples: {pre} stm {post} (graph types need only be valid at cut-points) Verify each triple separately (only loop-free code left) including check for null-pointer dereferences and other memory errors Note: highly modular, but requires invariants Program Verification with Graph Types and Monadic Second-order Logic

Verifying the Hoare triples Reduce everything to M2L-Tree and use the MONA tool. Use a technique of transductions to encode statements: A collection of M2L-Tree store predicates describes a set of stores at a given program point, e.g: succ_T_d(v,w) is true if v denotes a record of type T with a pointer field d pointing to the record w ptr_p(v) is true if v denotes the record pointed to by the program variable p Each statement is simulated by predicate transformation, e.g: p = q.next; is simulated by updating the ptr_p(v) predicate to ptr_p’(v) = w. ptr_q(w)  succ_T_next(w,v) This technique is sound and complete for loop-free code! Program Verification with Graph Types and Monadic Second-order Logic

Challenges and Extensions How to split into Hoare triples without too many cut-points? (each cut-point requires an invariant) use a non-standard form of Hoare triples How to handle procedure calls? (procedure calls are known to cause problems for Hoare logics) use call invariants, pre- and post-conditions, and logical variables How to obtain a working decision procedure? (M2L-Tree has a non-elementary complexity!) use advanced features of the MONA tool How precisely can record data (i.e. the non-pointer fields) be expressed? some data abstraction is possible, e.g. sorted lists can be expressed ... Program Verification with Graph Types and Monadic Second-order Logic

Example: red_black_insert.pale (1) A red-black tree is a binary tree whose records are red or black and have parent pointers a red record cannot have a red successor the root is black the number of black records is the same for all direct paths from the root to a leaf 1) is a graph type 2) and 3) can be captured as PAL formula invariants 4) cannot be expressed Program Verification with Graph Types and Monadic Second-order Logic

Example: red_black_insert.pale (2) type Node = { bool color; data left,right:Node; pointer p:Node[this^Node.left union this^Node.right={p}]; } pred red(pointer n:Node) = n.color=false; pred black(pointer n:Node) = n.color=true | n=null; pred inv(pointer n:Node) = black(n) & allpos m of Node: n<(left+right)*>m => (red(m) => black(m.left) & black(m.right)); Program Verification with Graph Types and Monadic Second-order Logic

Example: red_black_insert.pale (3) proc redblackinsert(data t,root:Node):Node [t.left=null & t.right=null & inv(root)] { pointer y,x:Node; x = t; root = treeinsert(x,root) [treeinsert.Z=x & treeinsert.Q=root]; x.color = false; while [x!=null & root<(left+right)*>x & almostinv1(root,x) & (black(root) | x=root) & (x!=root & red(x.p) => red(x))] (x!=root & x.p.color=false) { if (x.p=x.p.p.left) { y = x.p.p.right; if (y!=null & y.color=false) { x.p.color = true; y.color = true; x.p.p.color = false; x = x.p.p; } else { if (x=x.p.right) { x = x.p; root = leftrotate(x,root) [leftrotate.X=x & root<(left+right)*>x & red(leftrotate.Y)]; split [x.p.left=x & x.p.p.left=x.p & black(x.p.p.right) & root<(left+right)*>x & almostinv2(root,x) & black(x.left) & black(x.right) & black(x.p.right)]; root = rightrotate(x.p.p,root) [rightrotate.Y.left=x & root<(left+right)*>x & red(rightrotate.X) & rightrotate.Q=root & x!=null]; root.color = true; } } else { ... }} return root; } [inv(return)] + auxiliary procedures leftrotate, rightrotate, and treeinsert (total ~135 lines of program code) Program Verification with Graph Types and Monadic Second-order Logic

Example: red_black_insert.pale (4) Insert invariants and pre- and post-conditions, expressing partial correctness of red_black_insert and the auxiliary procedures Run the PALE tool Result: after ~10000 tree automaton operations and 55 seconds, PALE replies that all assertions are valid there can be no null-pointer dereferences or memory leaks the graph type is wellformed and valid at all cut-points If verification fails, a counterexample is returned. Note: very detailed information, also useful for optimizing code Program Verification with Graph Types and Monadic Second-order Logic

http://www.brics.dk/PALE “The Pointer Assertion Logic Engine” (submitted) The PALE tool (8000 lines of C, GPL licence) Detailed description of the translation into MONA code More examples (singly-linked lists, doubly-linked lists with tail pointers, threaded trees, sorted lists) Program Verification with Graph Types and Monadic Second-order Logic