1. A Universe-Type-Based Verification Technique for Mutable Static Fields and Methods Alexander J Summers Sophia Drossopoulou Imperial College London Peter Müller Microsoft Research Redmond

2. What is it all about? ►A new verification technique ►Extend Visibility Technique – handle static fields, methods and invariants ►Visible state semantics –safely handle/restrict call-backs ►Multiple-object invariants ►Global data structures –mutable static fields and static methods ►Expressive invariants: quantification over instances ►Minimal code annotations using Universe Types

3. Heap topology

11. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

12. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

13. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation

14. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me

15. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me

16. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper

17. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper

18. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper ►All object references must respect ownership

19. Universe types ►Describe relative location of objects ►Universe modifiers rep : an object I own, part of my representation peer : an object with the same owner as me any, readonly, lost, etc. : not in this paper ►All object references must respect ownership

20. Framework Paper (ECOOP 2008) ►Identifies 7 parameters to describe a technique X invariants expected at visible states of a method V invariants vulnerable to execution of a method D invariants which may depend on a particular field B invariants which must be proven before a method call E invariants which must be proven at end of a method call U permitted receivers for field updates (who updates fields) C permitted receivers for method calls (who calls who) ►Identifies 5 sufficient conditions, in terms of these –e.g., before a permitted method call, all invariants expected by the new method which are not currently known to hold, must be proven

21. Visible State Semantics ►Invariants need only hold at ‘visible states’ –beginning of a method call –end of a method call –may be temporarily broken in between ►Flexible, but must handle call-backs with care –avoid control returning to an object temporarily broken ►This problem can be avoided by: –avoiding ‘loops’ in sequences of calls, or, –requiring invariants to be proven before such calls ►Ensure expected invariants hold for a new receiver

22. Visibility Technique (Müller et al.)

23. ►Calls are restricted

24. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps

25. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps

26. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers

27. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

28. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

29. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

30. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

31. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps

32. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

33. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

34. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken

35. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily

36. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily

37. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal

38. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

39. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

40. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist

41. Visibility Technique (Müller et al.) ►Calls are restricted –calls ‘down’ to reps –calls ‘across’ to peers ►Expect invariants of peers & transitive reps ►Calls down may leave invariants broken –temporarily –no calls ‘up’ are legal ►peer call-backs exist –extra proof obligations

42. Static fields and methods ►Static fields can control/refine instantiation of class –Restricting/counting number of instances (Singleton) –Maintaining invariants across all instances Instances of Thread are assigned unique identifiers –Sharing data across all instances String can maintain a ‘pool’ of shared instances for use ►Static fields are internal ‘representation’ of the class –Motivates static rep fields –Objects owned by classes (or objects) –Classes do not have owners themselves ►Static methods are methods of the class –Treat classes as potential receivers, like objects

43. Heap topology

44. ►Classes in the topology

45. Heap topology ►Classes in the topology

46. Heap topology ►Classes in the topology –Classes may be owners

47. Heap topology ►Classes in the topology –Classes may be owners Multiple trees

48. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

49. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

50. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields

51. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods

52. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

53. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

54. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

55. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls

56. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants

57. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before

58. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees

59. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods?

60. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods? ?

61. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods?

62. Heap topology ►Classes in the topology –Classes may be owners Multiple trees Static rep fields –Classes may be receivers static methods –Same rules apply for instance method calls –Expected invariants Current tree: as before Other unvisited trees –Who calls static methods? ?

63. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

64. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

65. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

66. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

67. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

68. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs

69. Heap topology ►Who calls static methods? –class has no owner/peers –VT implies only self-calls –Unrestricted static calls? flexible, useful source of call-backs ►Idea to avoid call-backs: –A static method can only be called if the class is not a prior receiver –depends on call-stack –approximate with effects

70. Effect annotations ►Ensure that a class cannot receive a call-back ►Annotate methods with a set effects(c,m) –Our effects sets are sets of class-names –Which classes might have static methods called on them, as a result of executing method m of class c? –Predict this set of classes (conservatively) ►Static method of c is legal only if c is not in effects ►Effects sets can be computed iteratively ►Further restriction necessary: –An overridden method may not have any extra effects –Ensures effects conservatively predict runtime calls

71. Soundness (outline) ►If the invariants of an object (or class) do not hold, then either it or one of its peers must be a receiver somewhere on the call-stack. ►If an object o is a receiver on the call-stack, the most recently-preceding class receiver to o is the ‘root’ of the tree in which o resides. ►Effects are conservative: if a static method of c is called, c was in effects of all methods on the stack ►Call-backs to classes are restricted to self-calls. ►Call-backs to objects are restricted to peer calls. ►Proof obligations imposed are sufficient

72. Problematic example Class MyClass extends Object { boolean equals(Object o) { System.out.println(new String(“equals”)); return (o == this); }

73. Problematic example Class MyClass extends Object { boolean equals(Object o) { System.out.println(new String(“equals”)); return (o == this); } } ►String must be in the effects of MyClass::equals() ►String must be in the effects of Object::equals() ►Annotation overhead problem ►Information-hiding problem ►Practicality problem

74. Levels

75. ►Divide forest into (totally ordered) levels

76. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes.

77. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

78. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

79. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’

80. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs

81. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs –Effects only computed for one level at a time

82. Levels ►Divide forest into (totally ordered) levels ►Restriction: lower-level classes do not mention higher-level classes. –Static method calls can be made ‘down’ but not ‘up’ –Calls to lower levels can never result in call-backs –Effects only computed for one level at a time ►MyClass is now legal

83. Soundness (outline) ►If c ≥ c’ then level (c) ≥ level (c’) ►If an object o is transitively owned by c, and c’ is the dynamic class of o, then level (c)≥ level (c’) ►If a sequence of legal calls can be made starting from receiver r and ending with receiver r’, then level (r) ≥ level (r’) ►If a sequence of legal calls starts and ends with receiver r, then for all intermediate receivers r’, level (r’) = level (r) ►Effects sets for one level are enough to guarantee no call-backs to that level

84. Finally.. ►We refine static invariants to allow quantification over instances –e.g., all Thread instances have distinct identifiers ►Use ECOOP paper to calculate necessary changes –Satisfy the 5 soundness conditions presented there –These imply sufficient changes to the 7 parameters ►Note: we allow static invariants to quantify over instances, whereas JML allows instance invariants to mention static fields –Similar expressiveness in logical terms –Different visible state semantics (future work)

85. Related work ►JML supports some statics and Universe Types –Limited support for both in combination –No static rep or peer fields: only readonly ►Leino and Müller extend Boogie technique to statics –supports static rep fields –refine work with a class ordering –restrictions on static initialisation ►Jacobs et. al. present modular verification for multithreaded programs in the context of Spec# –partially order locks, to avoid deadlock –similar to our levels, but without flexibility of effects

86. Possible future work ►Practicality: examples, case studies ►Static initialisation –Trees ‘appear’ in the topology –Rules for verifying static initialisers ►Formal definitions and proofs ►Extend framework (ECOOP) to cover our technique –formal proofs ‘for free’ ►Cover static factory methods –Ownership Transfer ►Considering ‘levels’ for other problems ►Path types to allow further flexibility

87. Any questions? Thank you for listening!