1. A Calculus of Atomic Actions Tayfun Elmas, Shaz Qadeer and Serdar Tasiran POPL ‘09 236825 – Seminar in Distributed Algorithms Cynthia Disenfeld 27/05/2013 *Some slides are based on those of the authors

2. Goal Statically verifying (partial) correctness of shared-memory multithreaded programs Difficult: understand thread-interaction + shared memory Single-thread programs: pre/post conditions, loop invariants Multithreaded programs: consider the effect of all thread-interleavings (e.g. Owicki-Gries)

3. Approach (Sound) program transformations ◦ Abstraction ◦ Reduction Invariant strengthening

4. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

5. Motivating examples void inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } void inc() { int t; [havoc t; x := x+1;] } Lock-based atomic increment

6. Motivating examples void inc() { int t; while(true){ t := x; if (CAS(x,t,t+1) break; } void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } Lock-free atomic increment * Transformation from Flanagan et al.[2005]

7. Motivating examples void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } Lock-free atomic increment

8. Motivating examples void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } void inc() { int t; havoc t; havoc t; [assume x==t; x := t+1]; } Lock-free atomic increment void inc() { int t; [havoc t; x := x+1]; }

9. Motivating examples void add(int n){ while (0<n) { inc() n := n-1; } void add(int n){ while (0<n){ [x := x+1] n := n-1; } Client of inc void add(int n){ [assert 0<=n; x := x+n; n := 0]; }

10. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

11. QED approach ProgramInvariant Fine-grained concurrency Hard to prove! Invariant = True Coarse-grained concurrency Easy to prove Proof step If the original program may fail the new program may fail

12. Soundness For all If then exists or

13. Soundness For each proof step Proof steps: Invariant strengthening Reduction: build more coarse-grained atomic blocks Abstraction: add (possibly failing) behaviors

14. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

15. Reduction inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } main() { x := 0; inc(); || inc(); assert(x==2); }

16. Reduction inc() { int t; acquire(lock); R t := x; B t := t+1; B x := t; B release(lock); L } inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } inc() x := x+1 REDUCE-SEQUENTIAL Right Mover

17. Reduction main() { x := 0; inc(); || inc(); assert(x==2); } main() { x := 0; B x := x+1; || x := x+1; B assert(x==2); } main() { x := 0; x := x+1; assert(x==2); } INLINE-CALL REDUCE-PARALLEL

18. Static mover check Check using the current invariant if they access different variables, or are not enabled at the same time Each statement consists of: when can it be applied? how is the state changed?

19. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

20. Abstraction rule Replace with if For all If then exists or

21. Abstraction void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } SIMULATE Then, we can reduce havoc t + skip while (*){…}  havoc t

22. Abstraction Adding non-determinism ◦ Guards  if(*) ◦ t := x  havoc t ◦ assume …  skip Adding behaviors that may go wrong ◦ x := x+1  if (x==0) fail; x := x+1 ◦ y := y-x  assert (y>x); y := y-x

23. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

24. Example – Sorted linked list Hand-over-hand locking Find, insert, delete Local assertions Class invariant Atomic  easy! But… implementation with fine-grained locking

25. Insert(5)

26. Insert(x) p := find(x); //locks p n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); //they are linked as they should and their values have increasing order UNLOCK(p);

27. Insert(x) invariant: list is sorted p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); p.val and p.next are not affected by other threads t.val and t.next are not affected by other threads

28. Proof p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L

29. Proof find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); how to continue?

30. Apparent interference p.next := t; n := p.next; Thread AThread B

31. Apparent interference p.next := t; n := p.next; Thread AThread B But: both p’s are locked!

32. Ruling out interference - 1 assert owner[p]==A p.next := t; assert owner[p]==A p.next := t; assert owner[p]==B n := p.next; assert owner[p]==B n := p.next; Thread AThread B

33. Ruling out interference - 2 assert !inList[t] t.next := n; assert !inList[t] t.next := n; assert inList[p] n := p.next; assert inList[p] n := p.next; Thread AThread B

34. Reduction after abstraction find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

35. Borrowed assertions find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

36. Completing the proof find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); Invariant : List is sorted

37. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

38. Tactics High-level strategies  multiple rule proofs ◦ abstract from a read, write ◦ add invariants ◦ synchronization mechanisms

39. inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } Example acquire(lock) { assume lock==false; lock := true; } release(lock) { lock := false; } mutex P, x 1, …, x n mutex (lock==true), x

40. Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } AUX-ANNOTATE Invariant: lock==true iff a !=0

41. Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } SIMULATE inc() { int t; acquire(lock); a=tid; assert a==tid;t := x; t := t+1; assert a==tid;x := t; assert a==tid; release(lock); a=0; }

42. Tactic - mutex inc() { int t; acquire(lock); a=tid; R assert a==tid;t := x; B t := t+1; B assert a==tid;x := t; B assert a==tid; release(lock); a=0; L }

43. Tactic - mutex REDUCE & RELAX inc() { int t; acquire(lock); a=tid; assert a==tid; t := x; t := t+1; assert a==tid; x := t; assert a==tid; release(lock); a=0; }

44. Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

45. Experience Implementation ◦ Boogie + parallel composition ◦ Verification conditions for validity of each step: Z3 SMT Solver Benchmarks  without complicated global invariants Fine-grained locking ◦ Multiset ◦ Hand-over-hand locking Non-blocking algorithms

46. Future work More tactics More synchronization mechanisms C / Spec# Larger verification problems

47. Conclusions A formal and sound proof calculus for atomicity was presented Abstraction helps applying reduction and the other way around Assertions can be added and checked only later The program is simplified by obtaining coarser atomic actions Tactics can be defined to represent different synchronization mechanisms