1. K. Rustan M. Leino RiSE, Joint work with: Peter Müller (ETH Zurich) Jan Smans (KU Leuven) Special thanks to Mike Barnett VMCAI, Madrid, Spain, 18 January 2010

2. Interleaving of thread executions Unbounded number of: threads, locks, … We need some basis for doing the reasoning A way of thinking!

3. Experimental language with focus on: Shared-memory concurrency Static verification Key features Memory access governed by a model of permissions Sharing via locks with monitor invariants Copy-free non-blocking channels Deadlock checking, dynamic lock re-ordering Other features Classes; Mutual exclusion and readers/writers locks; Fractional permissions; Two-state monitor invariants; Asynchronous method calls; Memory leak checking; Logic predicates and functions; Ghost and prophecy variables

4. Access to a memory location requires permission Permissions are held by activation records Syntax for talking about permission to y: acc(y)

6. method Main() { var c := new Counter; call c.Inc(); } method Main() { var c := new Counter; call c.Inc(); } method Inc() requires acc(y); ensures acc(y); { y := y + 1; } method Inc() requires acc(y); ensures acc(y); { y := y + 1; } acc(c.y)

7. call == fork + join is semantically like … but is compiled to more efficient code call x,y := o.M(E, F); fork tk := o.M(E, F); join x,y := tk; fork tk := o.M(E, F); join x,y := tk;

9. class XYZ { var x: int; var y: int; var z: int; method Main() { var c := new XYZ; fork c.A(); fork c.B(); } … } class XYZ { var x: int; var y: int; var z: int; method Main() { var c := new XYZ; fork c.A(); fork c.B(); } … } method A() requires acc(x); { x := x + 1; } method A() requires acc(x); { x := x + 1; } method B() requires acc(y) && acc(z); { y := y + z; } method B() requires acc(y) && acc(z); { y := y + z; }

10. acc(y)write permission to y rd(y)read permission to y At any one time, at most one thread can have write permission to a location

11. class Fib { var x: int; var y: int; var z: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } class Fib { var x: int; var y: int; var z: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } method A() requires rd(x) && acc(y) { y := x + 21; } method A() requires rd(x) && acc(y) { y := x + 21; } method B() requires rd(x) && acc(z) { z := x + 34; } method B() requires rd(x) && acc(z) { z := x + 34; }

12. acc(y)100% permission to y acc(y, p)p% permission to y rd(y)read permission to y Write access requires 100% Read access requires >0% = +  acc(y)acc(y,69) acc(y,31) rd(y) acc(y,  )

13. What if two threads want write access to the same location? method A() … { y := y + 21; } method A() … { y := y + 21; } method B() … { y := y + 34; } method B() … { y := y + 34; } class Fib { var y: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } class Fib { var y: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } acc(c.y)

14. method A() … { acquire this; y := y + 21; release this; } method A() … { acquire this; y := y + 21; release this; } method B() … { acquire this; y := y + 34; release this; } method B() … { acquire this; y := y + 34; release this; } class Fib { var y: int; invariant acc(y); method Main() { var c := new Fib; share c; fork c.A(); fork c.B(); } … } class Fib { var y: int; invariant acc(y); method Main() { var c := new Fib; share c; fork c.A(); fork c.B(); } … } acc(c.y) acc(y)

15. The concepts holding a lock, and having permissions are orthogonal to one another In particular: Holding a lock does not imply any right to read or modify shared variables Their connection is: Acquiring a lock obtains some permissions Releasing a lock gives up some permissions

16. Like other specifications, monitors can hold both permissions and conditions Example: invariant acc(y) && 0 ≤ y acc(y)

17. [Chalice encoding by Bart Jacobs]

18. class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } … } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } … }

19. class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { … } method Mutate() requires c.Valid; ensures c.Valid; { … } } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { … } method Mutate() requires c.Valid; ensures c.Valid; { … } }

20. class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } acc (c.x) acc (c.y) x ≤ y c.Validc.Valid

21. class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } acc (c.x) acc (c.y) x ≤ y c.Validc.Valid c.Validc.Valid

22. channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z;

23. channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } acc (c.y) acc (c.x)

24. channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } acc (c.y) c.y ≤ z

25. Deadlocks are prevented by making sure no such cycle can ever occur The program partially order locks The program is checked to acquire locks in strict ascending order A deadlock is the situation where a nonempty set (cycle) of threads each waits for a resource (e.g., lock) that is held by another thread in the set

26. Wait order is a dense partial order (Mu, <<) with a bottom element  << is the strict version of << The wait level of an object o is stored in a mutable ghost field o.mu Accessing o.mu requires appropriate permissions, as for other fields

27. With these preconditions, both methods verify The conjunction of the preconditions is false, so the methods can never be invoked at the same time method M() { acquire a; acquire b; … } method M() { acquire a; acquire b; … } method N() { acquire b; acquire a; … } method N() { acquire b; acquire a; … } requires rd(a.mu); requires rd(b.mu); requires rd(a.mu) requires rd(b.mu) requires waitlevel << b.mu; requires b.mu << a.mu; requires waitlevel << a.mu; requires a.mu << b.mu;

28. Recall, the wait level of an object o is stored in the ghost field o.mu Initially, the.mu field is  The.mu field is set by the share statement: picks some wait level strictly between L and H, and sets o.mu to that level Provided L << H and neither denotes an extreme element, such a wait level exists, since the order is dense Changing o.mu requires acc(o.mu), as usual share o between L and H;

29. Given as translation to Boogie ChaliceChalice Z3Z3 BoogieBoogie Boogie is an intermediate verification language

30. Chalice: o.f Boogie:Heap[ o, f ] where Heap is declared to be a map from objects and field names to values To encode permissions, use another map: Mask

31. call M() = Exhale[[ P ]]; Inhale[[ Q ]] method M() requires P; ensures Q; method M() requires P; ensures Q;

32. Defined by structural induction For expression P without permission predicates Exhale P≡assert P Inhale P≡assume P Exhale acc(o.f, p) ≡ assert p ≤ Mask[o,f]; Mask[o,f] := Mask[o,f] – p; Inhale acc(o.f, p) ≡ if (Mask[o,f] == 0) { havoc Heap[o,f]; } Mask[o,f] := Mask[o,f] + p;

33. call M() = assert 100 ≤ Mask[this,y]; Mask[this,y] := Mask[this,y] – 100; oldH := Heap; if (Mask[this,y] = 0) { havoc Heap[this,y]; } Mask[this,y] := Mask[this,y] + 100; assume Heap[this,y] = oldH[this,y] * oldH [this,y]; class Cell { var y: int; method Square() requires acc(y); ensures acc(y) && y = old(y*y); class Cell { var y: int; method Square() requires acc(y); ensures acc(y) && y = old(y*y);

35. Logical constant: Let the verifier figure out K! method Square(c: Cell) requires acc(c.y); ensures acc(c.y) && c.y == old(c.y*c.y); method Square(c: Cell) requires acc(c.y); ensures acc(c.y) && c.y == old(c.y*c.y); method Square(c: Cell, ghost K: int) requires acc(c.y) && K == c.y; ensures acc(c.y) && c.y == K*K; method Square(c: Cell, ghost K: int) requires acc(c.y) && K == c.y; ensures acc(c.y) && c.y == K*K; call Square(c, c.y); call Square(c, *);

36. method Square(c: Cell) returns (r: int) requires acc(c.y, ε ); ensures acc(c.y, ε ) && r == c.y*c.y; method Square(c: Cell) returns (r: int) requires acc(c.y, ε ); ensures acc(c.y, ε ) && r == c.y*c.y; method Square(c: Cell) returns (r: int) requires acc(c.y); ensures acc(c.y) && r == c.y*c.y; method Square(c: Cell) returns (r: int) requires acc(c.y); ensures acc(c.y) && r == c.y*c.y; method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y; method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y; loses procedural abstraction ε loses procedural abstraction

37. A better notation? rd(c.y) ? Does that pick one K for each method activation? Can these 3 kinds of “parameters” (real, ghost, permission) be treated more uniformly? Which kinds should be indicated explicitly by the programmer and which should be figured out by the compiler? method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y; method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y;

38. Permissions guide what memory locations are allowed to be accessed Activation records can hold permissions Permissions can also be stored in various “boxes” (monitors, predicates, channels) Permissions can be transferred between activation records and boxes Locks grant mutually exclusive access to monitors

39. Chalice (and Boogie) available as open source: http://boogie.codeplex.com http://boogie.codeplex.com Tutorial and other papers available from: http://research.microsoft.com/~leino http://research.microsoft.com/~leino