1. User-defined type checkers for error detection and prevention in Java Michael D. Ernst MIT Computer Science & AI Lab http://pag.csail.mit.edu/jsr308/

2. Code without type qualifiers class DAG { Set edges; //... List getNeighbors( Vertex v) { List neighbors = new LinkedList (); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; }

3. @NonNull type qualifier @NonNullDefault class DAG { Set edges; //... List getNeighbors( Vertex v) { List neighbors = new LinkedList (); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; }

4. @NonNull type qualifier @NonNullDefault class DAG { @NonNull Set edges; //... @NonNull List getNeighbors( @NonNull Vertex v) { List neighbors = new LinkedList (); for (Edge e : edges) if (e.from() == v) // OK neighbors.add(e.to()); // OK return neighbors; }

5. @Interned type qualifier @NonNullDefault class DAG { Set edges; //... List getNeighbors(@Interned Vertex v) { List neighbors = new LinkedList (); for (Edge e : edges) if (e.from() == v) // OK neighbors.add(e.to()); return neighbors; }

6. @ReadOnly type qualifier @NonNullDefault class DAG { Set edges; //... List getNeighbors(@ReadOnly Vertex v) @ReadOnly { List neighbors = new LinkedList (); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; }

7. Using a checker Write annotations on the code –Or use an inference tool Compiler plugin for javac javac -processor NonnullChecker MyFile.java Produces additional errors and warnings –All existing Java checks are performed Type-system-specific checks –For NonNull: dereferences –For Interned: equality Error to assign supertype to subtype: myObject = myNonNullObject; // OK myInternedObject = myObject; // error myObject = myReadOnlyObject; // error –Similarly for arguments, returns, overriding, …

8. Outline NonNull checker Interned checker Javari checker IGJ checker Creating your own checker

9. NonNull checker Problem: NullPointerExceptions Example: Object obj; // might be null @NonNull Object nnobj; // never null nnobj.toString(); // OK obj.toString(); // possible NPE obj =...; // OK nnobj = obj; // nnobj may become null Type system:

10. NonNull case study Programs: –Annotation file utilities (5 KLOC) –Lookup (4 KLOC, twice) –NonNull checker (1 KLOC) –Checkers framework (5 KLOC) Two different defaults: NonNull and Nullable Annotations with & without checker available

11. NonNull results 1 annotation per 50 lines of code –Inference tools exist (for NonNull and other qualifiers) 44 errors –if (x.y().z()) { somethingOptional(); } where no work need be done if x.y() is null 83 application invariants –@NonNull indicates whether a variable may be null –@NonNull cannot indicate conditions that hold when a variable is null entry_re == null iff exit_re == null 43 tool weaknesses (many already fixed)

12. Comparison with other tools Program to be checked: Lookup (4KLOC) Other tools also find non-NPE bugs To reduce false positives, they discard many warnings Our checker7 errors (every error!)7 false pos. FindBugs1 warning: dead code1 false pos. JLint08 PMD00

13. Good defaults reduce user effort Nullable default: –Dictated by backward compatibility –Many instances of @NonNull – too verbose –Flow-sensitive analysis inferred many NonNull types for local variables, reducing annotation burden NonNull default –Less verbose in signatures Draws attention to exceptions rather than the rule –More annotations in method bodies Our system: Nullable only for local variables –Not for generics on local variables –An alternative to type inference; effect may be similar

14. Interned checker Interning (aka hash-consing, canonicalization) chooses a canonical representative –Saves memory –Permits use of == for comparisons Problem: Equality errors Example: String s; @Interned String is, is2;... is = myString.intern()... // OK if (is == is2) {... } // OK if (s == is) {... } // unsafe equality is = s; // unsafe assignment

15. Interned case study Program: Daikon (250 KLOC) –Daikon’s key scalability problem is memory –1170 lines of code/comment refer to interning 72% of files have none, 87% have 0, 1, or 2 –200 run-time assertions –Emacs plug-in for String equality checking 127 annotations in 11 files (12KLOC)

16. Interned results 9 errors –Missing calls to intern 2 performance bugs –Unnecessary interning in inner loop of file reading 1 design flaw –VarInfoName is not interned within its implementation, but all escaping objects are –Too hard to understand complex interning 14 false positives (used @SuppressWarnings )

17. Javari checker Problem: Unintended side effects (mutations) Example: List sort(@ReadOnly List arg) {... // swap two elements E tmp = arg.get(i); arg.put(i, arg.get(j)); // illegal mutation arg.put(j, tmp); // illegal mutation } Type system:

18. Javari case study Programs (8KLOC) –JOlden benchmarks –Javari checker –Some JDK classes

19. Javari results Mutability bug in Javari checker –Global variable indicating checker state was set on entering an inner class, not reset on exiting –The bug-fix allocates a new object instead 3 false positives due to varargs –void format(Object...) { … }

20. IGJ checker Problem: Unintended side effects (mutations) Type system: –ReadOnly : Reference immutability (aliases may modify) –Immutable : Object immutability (object cannot change) Syntax uses annotations, not generics

21. IGJ case study Programs (128KLOC) –JOlden benchmarks –htmlparser library –tinySQL library –SVNKit subversion client –IGJ checker 4760 annotations (62133 possible locations)

22. IGJ results Representation exposure errors Constructors that left object in inconsistent state 26 false positives (arrays, varargs, casting) In SVNKit: –some getters have side effects –some setters have no side effects Used both reference and object immutability Preferred Annotation IGJ to original dialect: arrays, and –List > –@ReadOnly List More opportunities for immutability in new code than old

23. Writing your own checker Base checker performs all standard checks (assignment, method arguments) Override only a couple methods, for special rules –For NonNull: dereferences –For Interned: equality Check what you want: –General properties (tainting) –Project-specific (string formatting)

24. JSR 308: Annotation on Java types Permits annotations in new locations List myList; class Properties extends @Immutable Map {…} Planned for inclusion in Java 7 Implementation is backwards compatible: –Write annotations in comments: List myList; /*@NonNull*/ String s; –Compiles with any Java compiler

25. Pluggable type system benefits Easy to use Scalable Reveals errors Prevents even more errors Download the checkers from http://pag.csail.mit.edu/jsr308/ –Prerelease of annotation extensions in Java 7 Backward-compatible implementation –Includes inference tools for some type checkers –Includes a framework for writing new checkers