1. Java Bytecode Verification for @NonNull Types Chris Male, David J. Pearce, Alex Potanin and Constantine Dymnikov Victoria University of Wellington, New Zealand

2. void f(@NonNull Integer x) { x.toString(); // safe! } void f(@NonNull Integer x) { x.toString(); // safe! } Introduction Suppose: –Bytecode verifier to enforce @NonNull –NullPointerExceptions eliminated for good! –Useful for optimising away null-checks

3. Bytecode Verification Standard Bytecode Verifier –Abstract Store: Location  Type Integer StringInteger 012 String Local Variable ArrayStack

4. Bytecode Verification Standard Bytecode Verifier –Abstract Store: Location  Type Can we extend to @NonNull types ? –Abstract Store: Location  @NonNull Type Integer StringInteger 012 String Local Variable ArrayStack Integer@NN StringInteger 01 String 2

5. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 1

6. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 IntegerString 1

7. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerString 1

8. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerStringIntegerString 1

9. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerStringIntegerStringIntegerString 1

10. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerStringIntegerStringIntegerStringIntegerString 1

11. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerStringIntegerStringIntegerStringIntegerString 1

12. @NonNull Verification Problem static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return 0 Integer StringIntegerStringIntegerStringIntegerStringIntegerString 1

13. Type Aliasing IDEA: add another level of indirection! –Abstract store: Location  TypeObject  Type Local Var ArrayStack Integer @NN String 01 2 String Abstract “Meta Heap” 0 1 2

14. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return

15. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return Integer String 01

16. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return Integer String 01 Integer String 01

17. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return Integer String 01 Integer String 01 @NN Integer String 01

18. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return Integer String 01 Integer String 01 @NN Integer String 01 01 @NN Integer

19. static int f(Integer,String): 0: aload_0 1: ifnull 8 4: aload_0 5: invokevirtual … 8: return Integer String 01 Integer String 01 @NN Integer String 01 01 @NN Integer String 01 @NN Integer

20. Abstract Semantics expressed as transition system: aload 2 getfield … ifacmp_eq (true)

21. Abstract Store Subtyping Store subtyping needed for termination –For when stores are “joined” astore 1astore 2 aload 1

22. To show termination requires: –The Transfer Function to be Monotonic –That the Abstract Stores form a Lattice  

23. Equivalence of Stores Type Objects have identity (like Java Objects) –i.e. type objects with the same type can be distinct –Thus, different abstract stores can be equivalent Integer 01 2 String 0 1 2 Integer 01 2 String 0 1 2

24. Equivalence of Stores Type Objects have identity (like Java Objects) –i.e. type objects with the same type can be distinct –Thus, different abstract stores can be equivalent Integer 01 2 String 0 1 2 Integer 01 2 String 0 1 2

26. Consider the following Java code: –Can conclude no NullPointerException ? class Test { String x; … void f() { if(x != null) { x.toString(); }}} class Test { String x; … void f() { if(x != null) { x.toString(); }}} Field-Load Fix

27. Consider the following Java code: –Now it’s OK! class Test { String x; … void f() { if(x != null) { x.toString(); }}} class Test { String x; … void f() { if(x != null) { x.toString(); }}} Field-Load Fix class Test { String x; … void f() { String _x = x; if(_x != null) { _x.toString(); }}} class Test { String x; … void f() { String _x = x; if(_x != null) { _x.toString(); }}}

28. When programmer knows reference can’t be null, but verifier doesn’t! –To deal with these, we must add spurious null check Thread.currentThread().getThreadGroup().toString(); Context Fix void f(HashMap x) { if(x.contains(“Dave”)) { x.get(“Dave”).toString(); }} void f(HashMap x) { if(x.contains(“Dave”)) { x.get(“Dave”).toString(); }}

29. Experimental Results Annotated some benchmarks by hand:

30. Field-Load Fixes Context Fixes Other Fixes java/lang (14K)656136 java/io (11K)598221 jakarta_oro (8K)5332729 javacc (28K)10913774

31. Experiences This is good JavaDoc!! /** * … * @throws NullPointerException if sb is null */ public boolean contentEquals(@NonNull StringBuffer sb) { synchronized(sb) { … } } /** * … * @throws NullPointerException if sb is null */ public boolean contentEquals(@NonNull StringBuffer sb) { synchronized(sb) { … } }

32. /** * Tests if this string starts with the specified prefix * beginning at specified index. * * @param prefix the prefix. * @param toffset where to begin looking in the string. * @return true if the character sequence … */ public boolean startsWith(@NonNull String prefix, … ) { … char pa[] = prefix.value; … } /** * Tests if this string starts with the specified prefix * beginning at specified index. * * @param prefix the prefix. * @param toffset where to begin looking in the string. * @return true if the character sequence … */ public boolean startsWith(@NonNull String prefix, … ) { … char pa[] = prefix.value; … } Experiences (cont’d)

33. /** * Tests if this string starts with the specified prefix * beginning at specified index. * * @param prefix the prefix. * @param toffset where to begin looking in the string. * @return true if the character sequence … */ public boolean startsWith(@NonNull String prefix, … ) { … char pa[] = prefix.value; … } /** * Tests if this string starts with the specified prefix * beginning at specified index. * * @param prefix the prefix. * @param toffset where to begin looking in the string. * @return true if the character sequence … */ public boolean startsWith(@NonNull String prefix, … ) { … char pa[] = prefix.value; … } Experiences (cont’d) Overall, found 83/1101 methods were misdocumented!

34. Generics We allow @NonNull types in generics –Some classes cause a problem: Vector v HashMap m Vector v HashMap m class HashMap { … V get(K key) { … ; return null; }} class HashMap { … V get(K key) { … ; return null; }}

35. Conclusion Bytecode Verification of @NonNull Types –Must deal with aliasing between stack and locals –Subtyping of Abstract Stores non-trivial –Situations where programmer knows better –Problem with generic collections like HashMap Contact: david.pearce@mcs.vuw.ac.nz

36. Equivalence of Stores Type Objects have identity (like Java Objects) –i.e. type objects with the same type can be distinct –Thus, different abstract stores can be equivalent Integer 01 2 String 0 1 2 Integer 01 2 String 0 1

37. Implementation Considerations Constructors Generics – hash map problems Field Retyping Casting/Arrays New Operator Instance Of Static blocks Effect of interface limitation

38. Graph Isomorphism 12 3 12 3 12 3 12 3 12 3 12 3 12 3 12 3