A Calculus of Atomic Actions Tayfun Elmas, Shaz Qadeer and Serdar Tasiran POPL ‘ – Seminar in Distributed Algorithms Cynthia Disenfeld 27/05/2013 *Some slides are based on those of the authors

Goal Statically verifying (partial) correctness of shared-memory multithreaded programs Difficult: understand thread-interaction + shared memory Single-thread programs: pre/post conditions, loop invariants Multithreaded programs: consider the effect of all thread-interleavings (e.g. Owicki-Gries)

Approach (Sound) program transformations ◦ Abstraction ◦ Reduction Invariant strengthening

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Motivating examples void inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } void inc() { int t; [havoc t; x := x+1;] } Lock-based atomic increment

Motivating examples void inc() { int t; while(true){ t := x; if (CAS(x,t,t+1) break; } void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } Lock-free atomic increment * Transformation from Flanagan et al.[2005]

Motivating examples void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } Lock-free atomic increment

Motivating examples void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } void inc() { int t; havoc t; havoc t; [assume x==t; x := t+1]; } Lock-free atomic increment void inc() { int t; [havoc t; x := x+1]; }

Motivating examples void add(int n){ while (0<n) { inc() n := n-1; } void add(int n){ while (0<n){ [x := x+1] n := n-1; } Client of inc void add(int n){ [assert 0<=n; x := x+n; n := 0]; }

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

QED approach ProgramInvariant Fine-grained concurrency Hard to prove! Invariant = True Coarse-grained concurrency Easy to prove Proof step If the original program may fail the new program may fail

Soundness For all If then exists or

Soundness For each proof step Proof steps: Invariant strengthening Reduction: build more coarse-grained atomic blocks Abstraction: add (possibly failing) behaviors

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Reduction inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } main() { x := 0; inc(); || inc(); assert(x==2); }

Reduction inc() { int t; acquire(lock); R t := x; B t := t+1; B x := t; B release(lock); L } inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } inc() x := x+1 REDUCE-SEQUENTIAL Right Mover

Reduction main() { x := 0; inc(); || inc(); assert(x==2); } main() { x := 0; B x := x+1; || x := x+1; B assert(x==2); } main() { x := 0; x := x+1; assert(x==2); } INLINE-CALL REDUCE-PARALLEL

Static mover check Check using the current invariant if they access different variables, or are not enabled at the same time Each statement consists of: when can it be applied? how is the state changed?

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Abstraction rule Replace with if For all If then exists or

Abstraction void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } SIMULATE Then, we can reduce havoc t + skip while (*){…}  havoc t

Abstraction Adding non-determinism ◦ Guards  if(*) ◦ t := x  havoc t ◦ assume …  skip Adding behaviors that may go wrong ◦ x := x+1  if (x==0) fail; x := x+1 ◦ y := y-x  assert (y>x); y := y-x

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Example – Sorted linked list Hand-over-hand locking Find, insert, delete Local assertions Class invariant Atomic  easy! But… implementation with fine-grained locking

Insert(5)

Insert(x) p := find(x); //locks p n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); //they are linked as they should and their values have increasing order UNLOCK(p);

Insert(x) invariant: list is sorted p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); p.val and p.next are not affected by other threads t.val and t.next are not affected by other threads

Proof p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L

Proof find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); how to continue?

Apparent interference p.next := t; n := p.next; Thread AThread B

Apparent interference p.next := t; n := p.next; Thread AThread B But: both p’s are locked!

Ruling out interference - 1 assert owner[p]==A p.next := t; assert owner[p]==A p.next := t; assert owner[p]==B n := p.next; assert owner[p]==B n := p.next; Thread AThread B

Ruling out interference - 2 assert !inList[t] t.next := n; assert !inList[t] t.next := n; assert inList[p] n := p.next; assert inList[p] n := p.next; Thread AThread B

Reduction after abstraction find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

Borrowed assertions find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

Completing the proof find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); Invariant : List is sorted

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Tactics High-level strategies  multiple rule proofs ◦ abstract from a read, write ◦ add invariants ◦ synchronization mechanisms

inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } Example acquire(lock) { assume lock==false; lock := true; } release(lock) { lock := false; } mutex P, x 1, …, x n mutex (lock==true), x

Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } AUX-ANNOTATE Invariant: lock==true iff a !=0

Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } SIMULATE inc() { int t; acquire(lock); a=tid; assert a==tid;t := x; t := t+1; assert a==tid;x := t; assert a==tid; release(lock); a=0; }

Tactic - mutex inc() { int t; acquire(lock); a=tid; R assert a==tid;t := x; B t := t+1; B assert a==tid;x := t; B assert a==tid; release(lock); a=0; L }

Tactic - mutex REDUCE & RELAX inc() { int t; acquire(lock); a=tid; assert a==tid; t := x; t := t+1; assert a==tid; x := t; assert a==tid; release(lock); a=0; }

Outline Motivating examples Approach – Soundness Reduction Abstraction Borrowing assertions Tactics Experience / Conclusions

Experience Implementation ◦ Boogie + parallel composition ◦ Verification conditions for validity of each step: Z3 SMT Solver Benchmarks  without complicated global invariants Fine-grained locking ◦ Multiset ◦ Hand-over-hand locking Non-blocking algorithms

Future work More tactics More synchronization mechanisms C / Spec# Larger verification problems

Conclusions A formal and sound proof calculus for atomicity was presented Abstraction helps applying reduction and the other way around Assertions can be added and checked only later The program is simplified by obtaining coarser atomic actions Tactics can be defined to represent different synchronization mechanisms