Predicate Abstraction for Software Verification Shaz Qadeer Compaq Systems Research Center (joint work with Cormac Flanagan)

Systems software OS components –file system, buffer cache manager Abstract from low-level to high-level operations –tedious low-level programming detail Expected to work –with multiple concurrent clients –in the presence of crashes

Outline Background –verification condition –strongest postcondition –loop invariants Inferring loop invariants –predicate abstraction –universally-quantified invariants Frangipani Related work

Verification condition x := a; if (x < b) { x := b; } assert x = max(a,b); sp(P,true)  x  a  x  b Program P Check for validity:

Statement S x := e A ; B assume e A B while {I} e do B end Guarded command language (Dijkstra 76) Variables have arbitrary values in program’s initial state if e then A else B end  (assume e ; A) (assume  e ; B)

Strongest postcondition semantics sp(S,Q)  y.(Q[x  y]  x=e[x  y]) sp(B,sp(A,Q)) e  Q sp(A,Q)  sp(B,Q) Statement S x := e A ; B assume e A B Denote sp(S, true) by sp(S)

Checking loop invariants Given loop Need to check –Invariant holds on entry to loop sp( C )  I –Invariant holds at end of every iteration sp( C; H; assume I  e; B)  I where H = havoc variables modified in B C; {I} while e do B end

Desugaring loops S = “while {I} e do B end” H = havoc variables modified in B desugar(S) = assert I ; H ; assume I ; ( (assume e ; B; assert I; assume false) assume  e)

Example assume 0 < a.length; i := 0; while (i < a.length) { a[i] := 0; i := i + 1; } assert a[0] = 0; {  int j; 0 <= j  j < i  a[j] = 0, 0<=i}

Translation assume 0 < a.length; i := 0; assert  int j; 0 <= j  j < i  a[j] = 0; assert 0 <= i; havoc a[*], i; assume  int j; 0 <= j  j < i  a[j] = 0; assume 0 <= i; ( assume (i < a.length); a[i] := 0; i := i + 1; assert  int j; 0 <= j  j < i  a[j] = 0; assert 0 <= i; assume false;) assume  (i < a.length) assert a[0] = 0;

ESC/Java with loop invariants loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==> bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED); loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED; loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED || (0 <= t.inum && t.inum < FS.IMAX)); */ for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }

Outline Background –verification condition –strongest postcondition –loop invariants Inferring loop invariants –predicate abstraction –universally-quantified invariants Frangipani Related work

Inferring loop invariants Could try –I 0 = sp( C ) –I n+1 = I n  sp( C; H; assume I n  e; B) I n is an invariant for first n iterations Problem: May not converge C; {I} while e do B end

Divergence State Space I0I0 I1I1 I2I2 InIn...

Predicate abstraction Invariant is boolean combination of predicates Split problem into two parts 1Find a suitable set of predicates 2Find the boolean combination of these predicates

Predicate abstraction (contd.) Predicates over program variables –a = expr1, b = expr2, c = expr3, d = expr4  : boolean expression over {a,b,c,d}  formula over program variables  : formula over program variables  boolean expression over {a, b, c, d}  (F) = least boolean function G over {a,b,c,d} such that F   (G)

Abstract state space Predicates { a, b, c, d } They generate an abstract space of size 2 4 = 16 a  b a  b  a  b abab cdcd c  d  c  d cdcd F  (F) State Space

Inferring loop invariants Compute –I 0 =  (sp( C )) –I n+1 = I n   (sp( C; H; assume E   (I n) ; B)) Converges yielding valid loop invariant –Best loop invariant for the given predicates

Method 1 (slow!) Is F  a  b  c  d satisfiable? No! Can compute  (F) by asking 2 n such queries a  b a  b  a  b abab cdcd c  d  c  d cdcd F  (F) XXXX X X XX X XX

Method 2 (Das-Dill-Park 99) Order the variables: a < b < c < d a b c FaFa FabFab F  a  b  c F  a  b  (F) a  b a  b  a  b abab cdcd c  d  c  d cdcd F XXXX X X XX X XX O(2 n+1 ) queries Sensitive to ordering

Method 3 (Shankar-Saidi 99) O(3 n ) queries No ordering required Queries of size 1: F  a, F  a, F  b, etc. Number = n C 1  2 1 Queries of size 2: F  a  b, F  a  b, etc. Number = n C 2  2 2 Queries of size n:... Number = n C n  2 n....

New method F  a  b  c  d ? No! a  b a  b  a  b abab cdcd c  d  c  d cdcd F  (F) XXXX X X XX X XX F  a  c  d ? No! F  c  d ? No! Removed 1/4 of state space in 3 queries! =  (  c   d)  (  a   c)  (  a   b)  ( c   d)

Universally-quantified loop invariants loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==> bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED); loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED; loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED || (0 <= t.inum && t.inum < FS.IMAX)); */ for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }

Inferring  -quantified loop invariants assume 0 < a.length; i := 0; loop_invariant 0 <= i,  int j; 0 <= j  j < i  a[j] = 0 */ while (i < a.length) { a[i] := 0; i := i + 1; } assert a[0] = 0;

Inferring  -quantified loop invariants assume 0 < a.length; i := 0; loop_predicate 0 <= i,  int j; 0 <= j  j < i  a[j] = 0 */ while (i < a.length) { a[i] := 0; i := i + 1; } assert a[0] = 0;

Inferring  -quantified loop invariants assume 0 < a.length; i := 0; skolem_constant int j */ loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */ while (i < a.length) { a[i] := 0; i := i + 1; } assert a[0] = 0;

Inferring  -quantified loop invariants I 0 =  (sp(i := 0)) I n+1 = I n   (sp(i := 0; havoc i, a[*]; assume I n  i < a.length; a[i] := 0; i := i+1)) loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */ TFTF TFTTTTFFTTFT I0I0 TTTT I1I1

Inferring  -quantified loop invariants loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */ TFTF TFTTTTFFTTFT TTTT I0I0 I1I1  0  i   (0  j)   (j < i)  a[j] = 0  0  j  j < i

Inferring  -quantified loop invariants loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */ TFTF TFTTTTFFTTFT TTTT I0I0 I1I1  0  i   int j; 0  j  j < i  a[j] = 0   int j; 0  j  j < i

Outline Background –verification condition –strongest postcondition –loop invariants Inferring loop invariants –predicate abstraction –universally-quantified invariants Frangipani Related work

Frangipani Distributed file system (Thekkath, Mann, Lee 1997) Built on top of Petal virtual disk (Lee, Thekkath 1996) Implements the file abstraction from the virtual disk block abstraction Exports file and directory operations - create, delete, rename etc. - to clients

Verification Loop invariant inference built on top of ESC/Java (Detlefs, Leino, Nelson, Saxe 1998) ESC/Java uses automatic theorem prover Simplify (Nelson 81) Frangipani data structures and “create” modeled abstractly in Java Assume –single thread of execution –no crashes

... ibusy idisk bdisk bbusy F TT TT FFF FF F F Inode { int inum; int addr; int mode; int length; } Block { int addr; Entry[] entries; } Entry { String name; int inum; } addr Data structures on disk: Data structures in memory:... vArray vbusy TTFF T F vArray contains inodes distinct entries must contain distinct inodes itov: int  int

invariant  int i, j: vbusy[i]  ((vArray[i].inum = j)  (itov[j] = i)); invariant  int i: ibusy[i]  bbusy[idisk[i].addr]; invariant idisk  bdisk; invariant  int i: idisk[i]  null  idisk[i].inum = i;. */ requires vbusy[num]  vArray[num].mode = DIR */ ensures \result = ERROR  there is an entry with name s in directory vArray[num] */ int create(int num, String s) {. }

Main loop in “create” loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==> bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED); loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED; loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED || (0 <= t.inum && t.inum < FS.IMAX)); */ for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }

Performance

Related work Inferring/computing loop invariants Predicate abstraction –Graf-Saidi 97 –Bensalem-Lakhnech-Owre 98, Colon-Uribe 98 –Saidi-Shankar 99, Das-Dill-Park 99 –Ball-Majumdar-Millstein-Rajamani 2001

Future Work Heuristics for generating predicates Multiple threads –locks, semaphores,... –thread-modular reasoning Linked lists, reachability Target C