1. Sequential reductions for verifying serializability Hagit Attiya Technion & EPFL G. RamalingamMSR India Noam Rinetzky University of London

2. The goal Verify concurrent data structures Pre-execution static analysis E.g., linked list with hand-over-hand locking memory safety (no memory leaks) shape invariants (it’s a list) “atomicity” (serializability) Find sequential reductions  Consider only sequential executions  But conclude that properties hold in all executions

3. Back-of-envelop estimate of gain Static analysis of a linked-list algorithm [Amit, Rinetzky, Reps, Sagiv, Yahav, CAV 2007] –Verifies e.g., memory safety, sortedness, pointed-to by a variable, heap sharing One thread (sequential) 10s3.6MB Two threads (interleaved)~4h886MB Three threads (interleaved)> 8h----

4. Serializability operation interleaved execution complete non-interleaved execution ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ thread-local views [Papadimitriou ‘79]

5. Serializability assists verification Concurrent code M Π = all executions of M cni-Π = all complete non-interleaved executions of M  Π φ = some thread-local property If M is serializable Then Π ⊨ φ  cni-Π ⊨ φ

6. How do we know that M is serializable, w/o considering all executions? I.e., by checking only cni-executions If M is serializable Then Π ⊨ φ  cni-Π ⊨ φ

7. Guard access to data with locks Access data only after getting a lock for it (well-lockedness) Follow a locking protocol that guarantees conflict serializability E.g., two-phase locking (2PL) or tree locking (TL) Special (and common) case: Disciplined programming with locks

8. Two-phase locking [Papadimitriou `79] Locks acquire (grow) phase followed by locks release (shrink) phase  No lock is acquired after some lock is released t1t1 H t1t1 t1t1 t2t2 t1t1

9. Tree (hand-over-hand) locking [Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76] Except for the first lock, acquire a lock only when holding the lock on its parent No lock is acquired after being released t1t1 H t1t1 t1t1 t2t2

10. Tree (hand-over-hand) locking [Bayer & Scholnick ‘77] [Kedem & Sliberschatz ‘76] [Smadi ‘76] Except for the first lock, acquire a lock only when holding the lock on its parent No lock is acquired after being released t1t1 t2t2 t2t2 H t1t1

11. Yes! –for databases –concurrency control monitor ensures that M follows the locking policy at run-time  M is serializable No! –for static analysis –no central monitor void p() { acquire(B) B = 0 int b = B release(B) if (b) Y = 2 acquire(A) } void q() { acquire(B) B = 1 release(B) } Not two-phase locked But only in interleaved executions How to (statically) verify that M follows a locking policy?

12. Our approach Applies for local conflict-serializable (LCS) locking protocols –Use only thread-local information (see next) E.g., two phase locking, tree locking, (dynamic) DAG locking… But not protocols that rely on a concurrency control monitor!

13. Thread-local properties A thread-owned view contains the values of thread’s local variables & global variables locked by it A property φ is thread-local if it –Can be expressed in terms of thread-owned views –Is prefix closed A thread-local property of an execution holds in every execution indistinguishable from it

14. Our contribution: Easy step cni-Π = all complete non-interleaved executions of M  Π For any LCS locking policy LP Π ⊨ LP  ni-Π ⊨ LP non-interleaved execution For any thread-local property φ Π ⊨ φ  ni-Π ⊨ φ Two phase locking Tree locking Dynamic tree locking Dynamic DAG locking

15. If all ni-executions follow LP  all executions follow LP σ is the shortest execution that does not follow LP  σ’ follows LP, guarantees conflict-serializability  there is a ni-execution that is “equivalent” to σ’ Ni-reduction: Proof sketch σ (t,e) σ’

16. Ni-reduction: Proof sketch (cont.)  there is a ni-execution that is “equivalent” to σ’  there is a ni-execution that is “equivalent” to σ where LP is violated σ’ σ’ ni σ ni (t,e)

17. Ni-reduction: Proof sketch  There is a ni-execution σ ni with the same conflicts as in σ  t can execute e also after σ ni Write σ ni = σ 1 σ t σ 2, σ t is the sub-exeuction by thread t  t can execute e also after σ 1 σ t σ 1 σ t (t,e) is a ni-execution and it follows the locking protocol Since σ 1 σ t (t,e) and σ (t,e) are conflict equivalent, σ (t,e) follows the locking protocol

18. acni-Π = all almost-complete non-interleaved executions of M  Π For any LCS locking policy LP Π ⊨ LP  acni-Π ⊨ LP almost complete non-interleaved execution Further reduction

19. Need to argue about termination Acni-reduction: A complication int X=0, Y=0 void p() { acquire(Y) y = Y release(Y); if (y ≠ 0) acquire(X) X = 3 release(X) } void q() { if (random(5) == 3){ acquire(Y) Y = 1 release(Y) while (true) nop } Y is set to 1 & the method enters an infinite loop Observe Y == 1 & violates 2PL

20.  Can use sequential reduction to verify termination For any “terminating” LCS locking policy LP Π ⊨ LP  acni-Π ⊨ LP Acni-reduction: Termination

21. Acni-reduction: Proof ideas Start from a ni-execution (and rely on the previous, ni-reduction to get there) Create its equivalent completion, if possible Not always possible, e.g., Does not access variables accessed by later threads t 1 :lock(v),t 1 :lock(u),t 2 :lock(u) u v

22. Implications for statis analysis Pessimistic analysis (over approximate) –Analyze a module from every possible state Semi-optimistic analysis –Analyze a module only from states that occur after a sequence of modules ran one after the other Optimistic analysis (precise) –Analyze a module only from states that occur after a sequence of modules ran to completion (one after the other) Acni-reduction Ni-reduction

23. Initial analysis results Shape analysis of hand-over-hand lists *Does not verify sortedness of list and fails to verify linearizability in some cases Shape analysis of hand-over-hand trees (for the first time) Our method 3.5s4.0MB TVLA prior596.1s90.3MB Separation logic* 0.4s0.2MB Our method 124.6s90.6MB

24. What’s next? Further extensions, esp., for software transactional memory –aborted transactions –non-locking non-conflict based serializability (e.g., using timestamps) Combine with other reductions [Guerraoui, Henzinger, Jobstmann, Singh]

25. And beyond… The cost of verifying adherence to a locking policy in an acni- / ni-execution Automatic insertion of lock acquire / release commands … Or fences?

26. Thank you!