Houdini: An Annotation Assistant for ESC/Java Cormac Flanagan and K. Rustan M. Leino Compaq Systems Research Center

Software QA via Testing u Useful (the dominant methodology), but.. u Costly l half of development cost is testing l finds errors late in development cycle u Incomplete l often fails to ensure needed reliability l hard to test all configurations

Software QA via Static Checking u Statically verify many correctness properties u Type systems catch many errors l e.g. “Cannot multiply a number and a string” u Would like to catch additional errors l e.g. “Array index out of bounds at line 10” u And verify other correctness properties l assertions l object invariants l lightweight method specifications

Extended Static Checker Architecture The translator “understands” the semantics of Java. A verification condition is a logical formula that, ideally, is valid if and only if the program is free of the kinds of error under consideration. The automatic theorem prover is invisible to users. Counterexamples are turned into precise warning messages. ESC/Java Java method + annotations Translator Verification conditions Automatic theorem prover Counterexamples Post-processor Warning messages Index out of bounds on line 218 Method does not preserve object invariant on line 223

ESC/Java Example class Rational { int num, denom; Rational(int n, int d) { num = n; denom = d; } double getDouble() { return ((double)num)/denom; } public static void main(String[] a) { int n = readInt(), d = readInt(); if( d == 0 ) return; Rational r = new Rational(d,n); print( r.getDouble() ); }... } Warning: possible division by zero Warning: invariant possibly not established invariant denom != 0; requires d != 0; Warning: precondition possibly not established

ESC/Java Experience u Tested on 40 KLOC, caught a variety of defects u Ready for educational/research use? Yes! l u Ready for software engineering use? Not really. l annotation overhead significant l annotations increase program size by 10% l requires 1 programmer-hour to annotate 300 lines of code u Need annotation inference for ESC/Java!

Houdini Architecture Class A { String s; … } Generate set of candidate annotations Class A { String s; … … … } Annotation Refutation Loop

Generating Candidate Annotations u Invariants generated heuristically from program text For fields int i,j guess invariant i cmp j; invariant i cmp 0; where cmp  {, >= } For field Object[] a guess invariant a != null; invariant a.length cmp i; invariant (forall int k; 0 <= k && k a[k] != null); u Similar heuristics for preconditions and postconditions

Removing Invalid Annotations State Space Powerset Lattice Initial states... G  Refute some annotations Fixpoint Reachable states Candidate set

Houdini Architecture Class A { String s; … } Generate set of candidate annotations ESC/Java Warning: Invariant not established Warning:... Annotation remover Class A { String s; … … … } Annotation Refutation Loop

Houdini Example class Rational { int num, denom; Rational(int n, int d) { num = n; denom = d; } double getDouble() { return ((double)num)/denom; } public static void main(String[] a) { int n = readInt(), d = readInt(); if( d == 0 ) return; Rational r = new Rational(d,n); print( r.getDouble() ); }... } Warning: invariant possibly not established Warning: precondition possibly not established invariant num != 0; requires d != 0; invariant denom != 0; requires n != 0; u No warnings refuting annotations l Remaining annotations are valid l Houdini algorithm terminates Warning: possible division by zero

Houdini Architecture Class A { String s; … } Generate set of candidate annotations ESC/Java Warning: Invariant not established Warning:... Annotation remover web page generator /#* */ Class A... } NETSCAPE Class A { String s; … … … }

Finding the cause of a warning class Rational { int num, denom; Rational(int n, int d) { num = n; denom = d; } double getDouble() { return ((double)num)/denom; } public static void main(String[] a) { int n = readInt(), d = readInt(); if( d == 0 ) return; Rational r = new Rational(d,n); print( r.getDouble() ); }... } invariant num != 0; requires d != 0; invariant denom != 0; requires n != 0; Warning: possible division by zero Hyperlink

Houdini Example (corrected) class Rational { int num, denom; Rational(int n, int d) { num = n; denom = d; } double getDouble() { return ((double)num)/denom; } public static void main(String[] a) { int n = readInt(), d = readInt(); if( d == 0 ) return; Rational r = new Rational(n,d); print( r.getDouble() ); }... } Warning: invariant possibly not established Warning: precondition possibly not established invariant num != 0; requires d != 0; invariant denom != 0; requires n != 0; u No warnings refuting annotations l Remaining annotations are valid l Houdini algorithm terminates u No warnings about primitive operations l Division by zero error is impossible

Houdini Architecture Class A { String s; … } Generate set of candidate annotations ESC/Java Warning: Invariant not established Warning:... Annotation remover web page generator /#* */ Class A... } NETSCAPE Library Spec Class L { … … … } Class A { String s; … … … }

Houdini is a Two-Level Analysis u Interprocedural analysis l Uses ESC/Java (weakest preconditions, theorem proving) l Precise, not scalable u Intraprocedural analysis l Abstract interpretation based on powerset lattice l Less precise, but more scalable l Can add annotations manually l Houdini’s heuristics are extensible Eg. to reason about whether int[][] a is rectangular, guess (forall int i,j; 0 <= i && i < a.length && 0 <= j && j < a.length ==> a[i].length == a[j].length);

Evaluation

Houdini for Other Modular Checkers u Houdini originally designed for ESC/Java u But could be ported to other modular checkers Ported to rccjava (Race Condition Checker for Java) Requires new heuristics for guessing annotations Straightforward port Infers useful locking annotations u Houdini for your favorite modular checker?

Conclusions u Houdini is an effective annotation assistant l Infers many useful annotations l Significantly reduces number of ESC/Java warnings u Future work l Refine guessing heuristics Guess fewer “useless” annotations Guess additional properties (aliasing, container classes) l Refine user interface l Check 500,000 LOC