1. Ownership and Immutability in Generic Java (OIGJ) Yoav Zibin +, Alex Potanin * Paley Li *, Mahmood Ali ^, and Michael Ernst $ Presenter: Yossi Gil + + IBM * Victoria,NZ ^ MIT $ Washington

2. 2/22 Ownership + Immutability Our previous work OGJ: added Ownership to Java IGJ: added Immutability to Java This work OIGJ: combine Ownership + Immutability The sum is greater than its parts IGJ could not type-check existing code for creating immutable cyclic data-structures (e.g., lists, trees) We found a non-trivial connection between ownership and immutability

3. 3/22 Contributions No refactoring of existing code Prototype implementation No syntax changes (uses type-annotations in Java 7) No runtime overhead Backward compatible Verified that Java’s collection classes are properly encapsulated (using few annotations) Flexibility OIGJ can type-check more code than previous work: cyclic structures, the factory and visitor design patterns Formalization Formalized the concepts of raw/cooked immutable objects and wildcards as owner parameters Proved soundness

4. 4/22 Problem 1: Representation exposure Internal representation leaks to the outside private doesn’t offer real protection! Real life example! class Class { private List signers; public List getSigners() { return this.signers; } http://java.sun.com/security/getSigners.html Bug: the system thinks that code signed by one identity is signed by a different identity Forgot to copy signers!

5. 5/22 Solution for Representation Exposure Ownership! Class should own the list signers No outside alias can exist Ownership can be nested: note the tree structure Class signers entry1entry2entryN … elem1elem2elemN … X X X

6. 6/22 Ownership: Owner-as-dominator Dominators in graph theory Given: a directed rooted graph X dominates Y if any path from the root to Y passes X Owner-as-dominator Object graph; roots are the static variables An object cannot leak outside its owner, i.e., Any path from a root to an object passes its owner Conclusion: No aliases to internal state

7. 7/22 Problem 2: Unintended Modification Modification is not explicit in the language can Map.get() modify the map? for (Object key : map.keySet()) { map.get(key); } throws ConcurrentModificationException for the following map new LinkedHashMap(100, 1, true) Reorders elements according to last-accessed (like a cache)

8. 8/22 Solution: Immutability Varieties of Immutability Class immutability (like String or Integer in Java) Object immutability The same class may have both mutable and immutable instances Reference immutability A particular reference cannot be used to mutate its referent (but other aliases might cause mutations) class Student { @Immutable Date dateOfBirth; … void setTutor(@ReadOnly Student tutor) @Mutable { … } } Method may modify the this object Example in IGJ syntax

9. 9/23 Objects vs. References Objects mutable or immutable Creation of an immutable object Raw state: Fields can be assigned Cooked state: Fields cannot be assigned References mutable, immutable, or readonly

10. Challenge: Cyclic Immutability Cooking a cyclic data-structure is complicated Many objects must be raw simultaneously to manipulate backward pointers Then everything must become immutable simultaneously OIGJ’s novel idea: Prolong the cooking phase by using ownership information Enables creation of immutable cyclic structures

11. 11/22 Cooking immutable objects Previous work An object becomes cooked when its constructor finishes OIGJ’s observation An object becomes cooked when its owner’s constructor finishes The outside world will not see this cooking phase The complex object with its representation becomes immutable simultenously

12. 12/22 Cooking LinkedList (1 of 2) No refactoring – the original code must compile in OIGJ 1 : LinkedList(Collection c) { 2 : this(); 3 : Entry succ = this.header, pred = succ.prev; 4 : for (E e : c) { 5 : Entry entry = 6 : new Entry (e,succ,pred); 7 : // An entry is modified after it’s constructor finished 8 : pred.next = entry; pred = entry; 9 : } 10: succ.prev = pred; 11: } 1 : LinkedList(Collection c) { 2 : this(); 3 : Entry succ = this.header, pred = succ.prev; 4 : for (E e : c) { 5 : Entry entry = 6 : new Entry (e,succ,pred); 7 : // An entry is modified after it’s constructor finished 8 : pred.next = entry; pred = entry; 9 : } 10: succ.prev = pred; 11: } Sun’s code is similar

13. 13/22 Cooking LinkedList (2 of 2) The list owns its entries Therefore, it can mutate them, even after their constructor finished 1 : LinkedList(@ReadOnly Collection c) @Raw { 2 : this(); 3 : @This @I Entry succ = this.header, pred = succ.prev; 4 : for (E e : c) { 5 : @This @I Entry entry = 6 : new @This @I Entry (e,succ,pred); 7 : // An entry is modified after it’s constructor finished 8 : pred.next = entry; pred = entry; 9 : } 10: succ.prev = pred; 11: } 1 : LinkedList(@ReadOnly Collection c) @Raw { 2 : this(); 3 : @This @I Entry succ = this.header, pred = succ.prev; 4 : for (E e : c) { 5 : @This @I Entry entry = 6 : new @This @I Entry (e,succ,pred); 7 : // An entry is modified after it’s constructor finished 8 : pred.next = entry; pred = entry; 9 : } 10: succ.prev = pred; 11: } Sun’s code is similar Code in OIGJ; Annotations next slide.

14. 14/22 Hierarchies in OIGJ Immutability hierarchy ReadOnly – no modification Raw – object under construction Ownership hierarchy World – anyone can access This – this owns the object World This ReadOnly Raw Mutable Immut

15. 15/22 OIGJ syntax: fields (1 of 2) Two annotations per type 1:class Foo { 2: // An immutable reference to an immutable date. @O @Immut Date imD = new @O @Immut Date (); 3: // A mutable reference to a mutable date. @O @Mutable Date mutD = new @O @Mutable Date(); 4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable. @O @ReadOnly Date roD =... ? imD : mutD; 5: // A date with the same owner and immutability as this @O @I Date sameD; 6: // A date owned by this ; it cannot leak. @This @I Date ownedD; 7: // Anyone can access this date. @World @I Date publicD; 1:class Foo { 2: // An immutable reference to an immutable date. @O @Immut Date imD = new @O @Immut Date (); 3: // A mutable reference to a mutable date. @O @Mutable Date mutD = new @O @Mutable Date(); 4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable. @O @ReadOnly Date roD =... ? imD : mutD; 5: // A date with the same owner and immutability as this @O @I Date sameD; 6: // A date owned by this ; it cannot leak. @This @I Date ownedD; 7: // Anyone can access this date. @World @I Date publicD;

16. 16/22 OIGJ syntax: methods (2 of 2) Method receiver’s annotation has a dual purpose: Determines if the method is applicable. Inside the method, the bound of @I is the annotation. 8 : // Can be called on any receiver; cannot mutate this. int readonlyMethod() @ReadOnly {...} 9 : // Can be called only on mutable receivers; can mutate this. void mutatingMethod() @Mutable {...} 10: // Constructor that can create (im)mutable objects. Foo(@O @I Date d) @Raw { 11: this.sameD = d; 12: this.ownedD = new @This @I Date (); 13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...); 14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...); 15: } 8 : // Can be called on any receiver; cannot mutate this. int readonlyMethod() @ReadOnly {...} 9 : // Can be called only on mutable receivers; can mutate this. void mutatingMethod() @Mutable {...} 10: // Constructor that can create (im)mutable objects. Foo(@O @I Date d) @Raw { 11: this.sameD = d; 12: this.ownedD = new @This @I Date (); 13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...); 14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...); 15: }

17. 17/22 Formalization: Featherweight OIGJ Novel idea: Cookers Every object o in the heap is of the form: o’ is the owner of o o” is the cooker of o, i.e., when the constructor of o” finishes then o becomes cooked We keep track of the set of ongoing constructors Subtyping rules connect cookers and owners o  Foo or o” Proved soundness and type preservation

18. 18/22 Case studies Implementation uses the checkers framework Only 1600 lines of code (but still a prototype) Requires type annotations available in Java 7 Java’s Collections case study 77 classes, 33K lines of code 85 ownership-related annotations 46 immutability-related annotations

19. 19/22 Case studies conclusions Verified that collections own their representation Method clone is problematic clone makes a shallow copy that breaks ownership Our suggestion: compiler-generated clone that nullifies fields, and then calls a copy-constructor

20. 20/22 Previous Work Universes Relaxed owner-as-dominator to owner-as-modifier ReadOnly references can be freely shared Constrains modification instead of aliasing, i.e., only the owner can modify an object Reference immutability: C++’s const Javari

21. 21/22 Future work Inferring ownership and immutability annotations Bigger case study Extending OIGJ owner-as-modifier uniqueness or external uniqueness

22. 22/22 Conclusions Ownership Immutability Generic Java (OIGJ) Simple, intuitive, small Static – no runtime penalties (like generics) Backward compatible, no JVM changes Case study proving usefulness Formal proof of soundness Paper submitted to OOPSLA. Links: http://ecs.victoria.ac.nz/twiki/pub/Main/Technical ReportSeries/ http://code.google.com/p/checker-framework/ http://code.google.com/p/ownership-immutability/

24. 24/22 OIGJ typing rules Ownership nesting Field access Field assignment Method invocation Method guards Raw parameter can be used only in method guards (see the paper for all rules, such as inner classes, covariant subtyping)

25. 25/22 Ownership example in OGJ This -owned fields/methods can be accessed only via this class Class { @This List signers; // This-owned field public @This List getSigners1() { return this.signers; } public @World List getSigners2() { return new @World LinkedList(this.signers); } public void example(Class other) { this.signers = …; // Ok other.signers = …; // Illegal this.getSigners1(); // Ok other.getSigners1(); // Illegal other.getSigners2(); // Ok }

26. 26/22 OIGJ syntax: fields (1 of 2) Two new generic parameters were added 1:class Foo { 2: // An immutable reference to an immutable date. Date imD = new Date (); 3: // A mutable reference to a mutable date. Date mutD = new Date (); 4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable. Date roD =... ? imD : mutD; 5: // A date with the same owner and immutability as this Date sameD; 6: // A date owned by this ; it cannot leak. Date ownedD; 7: // Anyone can access this date. Date publicD; 1:class Foo { 2: // An immutable reference to an immutable date. Date imD = new Date (); 3: // A mutable reference to a mutable date. Date mutD = new Date (); 4: // A readonly reference to any date. Both roD and imD cannot mutate // their referent, however the referent of roD might be mutated by an // alias, whereas the referent of imD is immutable. Date roD =... ? imD : mutD; 5: // A date with the same owner and immutability as this Date sameD; 6: // A date owned by this ; it cannot leak. Date ownedD; 7: // Anyone can access this date. Date publicD;

27. 27/22 OIGJ syntax: methods (2 of 2) Method guard ? has a dual purpose: The method is included only if T extends U Inside the method, the bound of T is U. 8 : // Can be called on any receiver; cannot mutate this. ? int readonlyMethod(){...} 9 : // Can be called only on mutable receivers; can mutate this. ? void mutatingMethod(){...} 10: // Constructor that can create (im)mutable objects. ? Foo(Date d) { 11: this.sameD = d; 12: this.ownedD = new Date (); 13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...); 14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...); 15: } 8 : // Can be called on any receiver; cannot mutate this. ? int readonlyMethod(){...} 9 : // Can be called only on mutable receivers; can mutate this. ? void mutatingMethod(){...} 10: // Constructor that can create (im)mutable objects. ? Foo(Date d) { 11: this.sameD = d; 12: this.ownedD = new Date (); 13: // Illegal, because sameD came from the outside. // this.sameD.setTime(...); 14: // OK, because Raw is transitive for owned fields. this.ownedD.setTime(...); 15: }

28. 28/22 LinkedList Example 1 : class Entry { 2 : E element; 3 : Entry next, prev; … 4 : } 5 : class LinkedList { 6 : Entry header; … 7 : ? LinkedList( 8 : Collection c) { 9 : this(); this.addAll(c); 10: } 11: ? void addAll( 12: Collection c) { 13: Entry succ = this.header, 14: pred = succ.prev; 15: for (E e : c) { 16: Entry en = 17: new Entry (e,succ,pred); 18: pred.next = en; pred = en; } 19: succ.prev = pred; 20: } 1 : class Entry { 2 : E element; 3 : Entry next, prev; … 4 : } 5 : class LinkedList { 6 : Entry header; … 7 : ? LinkedList( 8 : Collection c) { 9 : this(); this.addAll(c); 10: } 11: ? void addAll( 12: Collection c) { 13: Entry succ = this.header, 14: pred = succ.prev; 15: for (E e : c) { 16: Entry en = 17: new Entry (e,succ,pred); 18: pred.next = en; pred = en; } 19: succ.prev = pred; 20: }

29. 29/22 Ownership nesting List > l1; // Legal nesting List > l2; // Illegal! List > l1; // Legal nesting List > l2; // Illegal! The main owner parameter must be inside any other owner parameter. public static Object alias_l2; It’s illegal because we can store l2 in this variable:

30. 30/22 Field access/assignment 1: class Foo { 2: Date ownedD; // this -owned field 3: Date sameD; 4: ? void bar(Foo other) { 5: this.ownedD = …; // Legal: assign via this 6: other.ownedD = …; // Illegal: not via this 7: other.sameD = …; // Legal: not this -owned 8: } 1: class Foo { 2: Date ownedD; // this -owned field 3: Date sameD; 4: ? void bar(Foo other) { 5: this.ownedD = …; // Legal: assign via this 6: other.ownedD = …; // Illegal: not via this 7: other.sameD = …; // Legal: not this -owned 8: } this -owned fields can be accessed/assigned only via this

31. 31/22 Field assignment 1 : class Foo { 2 : Date sameD; 3 : ? void bar( 4 : Foo mutableFoo, 5 : Foo readonlyFoo, 6 : Foo rawFoo1, 7 : Foo rawFoo2) { 8 : mutableFoo.sameD = …; // Legal: object is Mutable 9 : readonlyFoo.sameD = …; // Illegal: object is not Raw nor Mutable 10: rawFoo1.sameD = …; // Illegal: object is not this nor this -owned 11: rawFoo2.sameD = …; // Legal: object is Raw and this -owned 12: this.sameD = …; // Legal: object is Raw and this 13: } 1 : class Foo { 2 : Date sameD; 3 : ? void bar( 4 : Foo mutableFoo, 5 : Foo readonlyFoo, 6 : Foo rawFoo1, 7 : Foo rawFoo2) { 8 : mutableFoo.sameD = …; // Legal: object is Mutable 9 : readonlyFoo.sameD = …; // Illegal: object is not Raw nor Mutable 10: rawFoo1.sameD = …; // Illegal: object is not this nor this -owned 11: rawFoo2.sameD = …; // Legal: object is Raw and this -owned 12: this.sameD = …; // Legal: object is Raw and this 13: } 1)A field can be assigned only if the object is Raw or Mutable. 2)If it is Raw, then the object must be this or this -owned.

32. 32/22 Method invocation 1 : class Foo { 2 : Date m1() { … } // Parameter is this -owned 3 : ? void m2() { … } 4 : ? void bar( 5 : Foo rawFoo1, 6 : Foo rawFoo2) { 7 : this.m1(); // Legal: object is this 8 : rawFoo2.m1(); // Illegal: object is not this 9 : rawFoo1.m2(); // Illegal: object is not this nor this -owned 10: rawFoo2.m2(); // Legal: both Raw and object is this -owned 11: this.m2(); // Legal: both Raw and object is this 12: } 1 : class Foo { 2 : Date m1() { … } // Parameter is this -owned 3 : ? void m2() { … } 4 : ? void bar( 5 : Foo rawFoo1, 6 : Foo rawFoo2) { 7 : this.m1(); // Legal: object is this 8 : rawFoo2.m1(); // Illegal: object is not this 9 : rawFoo1.m2(); // Illegal: object is not this nor this -owned 10: rawFoo2.m2(); // Legal: both Raw and object is this -owned 11: this.m2(); // Legal: both Raw and object is this 12: } Method invocation is the same as field access/assignment: 1)If any parameter is this -owned, then the receiver must be this. 2)If the guard is Raw and the object is Raw, then the receiver must be this or this -owned.

33. 33/22 Method guards 1: class Foo { 2: ? void rawM() { … } 3: ? void bar( 4: Foo readonlyFoo, 5: Foo mutableFoo) { 6: readonlyFoo.rawM(); // Illegal: ReadOnly is not a subtype of Raw // The bound of I in this method is Mutable 7: mutableFoo.rawM(); // Legal: Mutable is a subtype of Raw 8: this.rawM(); // Legal: Mutable is a subtype of Raw 9: } 1: class Foo { 2: ? void rawM() { … } 3: ? void bar( 4: Foo readonlyFoo, 5: Foo mutableFoo) { 6: readonlyFoo.rawM(); // Illegal: ReadOnly is not a subtype of Raw // The bound of I in this method is Mutable 7: mutableFoo.rawM(); // Legal: Mutable is a subtype of Raw 8: this.rawM(); // Legal: Mutable is a subtype of Raw 9: } Guard “ ? ” has a dual purpose: 1)The receiver’s T must be a subtype of U. 2)Inside the method, the bound of T is U. Conditional Java (cJ) proposed method guards for Java