O(log n / log log n) RMRs Randomized Mutual Exclusion Danny Hendler Philipp Woelfel PODC 2009 Ben-Gurion University University of Calgary

Distributed Shared Memory (DSM) Model network mem mem mem mem mem n asynchronous fault-free processes, each with its own locally-accessible memory segment. atomic operations: read / write local memory accesses: cheap Remote Memory References (RMRs): expensive

Cach-Coherent (CC) Model Main Memory Shared bus cache cache cache cache cache accesses served from cache: cheap Remote Memory References (RMRs): expensive

Talk outline Model Problem statement Prior art and our results Basic Algorithm (CC) Enhanced Algorithm (CC) Pseudo-code Open questions

The mutual exclusion problem (Dijkstra, 1965) We need to devise a protocol that guarantees mutually exclusive access by processes to a shared resource (such as a file, printer, etc.) Remainder code Requirements Mutual exclusion Starvation freedom Entry code Critical Section Exit Section

The RMR metric: motivation Theorem [Alur and Taubenfeld, 92] The winning process in any mutual exclusion algorithm using reads and writes may have to busy-wait for another process Even the winning process may perform an unbounded number of memory accesses # of memory accesses not a meaningful metric Metric used is: worst-case # of RMRs

Local-spin algorithms Definition In a local-spin algorithm all busy-waiting is done on locally-accessible variables. To incur a bounded number of RMRs, a mutual exclusion algorithm must be local-spin await (w = 0) Locally accessible memory w

Talk outline Model Problem statement Prior art and our results Basic Algorithm (CC) Enhanced Algorithm (CC) Pseudo-code Open questions

Most Relevant Prior Art Best upper bound for mutual exclusion: O(log n) RMRs (Yang and Anderson, Distributed Computing '96). A tight Θ(n log n) RMRs lower bound for deterministic mutex (Attiya, Hendler and Woelfel, STOC '08) Compare-and-swap (CAS) is equivalent to read/write for RMR complexity (Golab, Hadzilacos, Hendler and Woelfel, PODC '07)

Our Results Randomized mutual exclusion algorithms (for both CC/DSM) that has: O(log N / log log N) expected RMR complexity against a strong adversary, and O(log N) deterministic worst-case RMR complexity Separation in terms of RMR complexity between deterministic/randomized mutual exclusion algorithms

Shared-memory scheduling adversary types Oblivious adversary: Makes all scheduling decisions in advance Weak adversary: Sees a process' coin-flip only after the process takes the following step, can change future scheduling based on history Strong adversary: Can change future scheduling after each coin-flip / step based on history

Talk outline Model Problem statement Prior art and our results Basic algorithm (CC model) Enhanced Algorithm (CC model) Pseudo-code Open questions

Basic Algorithm – Data Structures Δ-1 Δ 1 Δ=Θ(log n / log log n) Key idea: Processes apply randomized promotion 1 2 Δ 1 2 n

Basic Algorithm – Data Structures (cont'd) pi1 pi2 pik Promotion Queue notified[1…n] Δ-1 Δ 1 2 n Per-node structure lock{P,} apply: <v1,v2, …,vΔ>

Basic Algorithm – Entry Section Δ-1 Δ 1 Lock= apply: <v1, , …,vΔ> i CAS(, i)  i i

Basic Algorithm – Entry Section: scenario #2 Δ-1 Δ 1 Lock=q apply: <v1, , …,vΔ> Failure CAS(,i) i i

Basic Algorithm – Entry Section: scenario #2 Δ-1 Δ 1 Lock=q apply: <v1, , …,vΔ> i await (n.lock=) || apply[ch]=) i

Basic Algorithm – Entry Section: scenario #2 Δ-1 Δ 1 Lock=q apply: <v1, , …,vΔ> i await (n.lock=) || apply[ch]=) i

Basic Algorithm – Entry Section: scenario #2 Δ-1 Δ 1 await (notified[i) =true) CS i

Basic Algorithm – Exit Section Δ-1 Δ 1 Lock=p apply: <v1, q, …,vΔ> Climb up from leaf until last node captured in entry section  Lottery i

Basic Algorithm – Exit Section Perform a lottery on the root Δ-1 Δ 1 Lock=p apply: <v1, , …,vΔ>  Promotion Queue q s t p

Basic Algorithm – Exit Section CS Δ-1 Δ 1 await (notified[i) =true) Promotion Queue q s t i

Basic Algorithm – Exit Section (scenario #2) Free Root Lock Δ-1 Δ 1 Promotion Queue EMPTY i

Basic Algorithm – Properties Lemma: mutual exclusion is satisfied Proof intuition: when a process exits, it either signals a single process without releasing the root's lock, or if the promoted-processes queue is empty, releases the lock.

Basic Algorithm – Properties (cont'd) Lemma: Expected RMR complexity is Θ(log N / log log N) await (n.lock=) || apply[ch]=) A waiting process participates in a lottery every constant number of RMRs incurred here Expected #RMRs incurred before promotion is Θ(log N / log log N)

Basic Algorithm – Properties (cont'd) Mutual Exclusion Expected RMR complexity:Θ(log N / log log N) Non-optimal worst-case complexity and (even worse) starvation possible.

Talk outline Model Problem statement Prior art and our results Basic algorithm (CC) Enhanced Algorithm (CC) Pseudo-code Open questions

The enhanced algorithm. Key idea Quit randomized algorithm after incurring ‘'too many’’ RMRS and then execute a deterministic algorithm. Problems How do we count the number of RMRs incurred? How do we “quit” the randomized algorithm?

Enhanced algorithm: counting RMRs problem await (n.lock=) || apply[ch]=) The problem: A process may incur here an unbounded number of RMRs without being aware of it.

Counting RMRs: solution Key idea Perform both randomized and deterministic promotion Lock=p apply: <v1, q, …,vΔ> token: Increment promotion token whenever releasing a node Perform deterministic promotion according to promotion index in addition to randomized promotion

Waiting processes may incur RMRs without participating in lotteries! The enhanced algorithm: quitting problem Upon exceeding allowed number of RMRs, why can't a process simply release captured locks and revert to a deterministic algorithm? ? 1 2 Δ N Waiting processes may incur RMRs without participating in lotteries!

Δ Quitting problem: solution Add a deterministic Δ-process mutex object to each node Δ-1 Δ 1 2 n Per-node structure lock{P,} apply: <v1,v2, …,vΔ> token: MX: Δ-process mutex

Quitting problem: solution (cont'd) Per-node structure lock{P,} apply: <v1,v2, …,vΔ> token: MX: Δ-process mutex After incurring O(log Δ) RMRs on a node, compete for the MX lock. Then spin trying to capture node lock. In addition to randomized and deterministic promotion, an exiting process promotes also the process that holds the MX lock, if any.

Worst-case number of RMRs = O(Δ log Δ)=O(log n) Quitting problem: solution (cont'd) After incurring O(log Δ) RMRs on a node, compete for the MX lock. Then spin trying to capture node lock. Worst-case number of RMRs = O(Δ log Δ)=O(log n)

Talk outline Model Problem statement Prior art and our results Basic algorithm (CC) Enhanced Algorithm (CC) Pseudo-code Open questions

Data-structures the i'th leaf i'th

The entry section i'th

The exit section i'th

Open Problems Is this best possible? For strong adversary? For weak adversary? For oblivious adversary? Is there an abortable randomized algorithm? Is there an adaptive one?

Thanks you! Any questions? 1 2 Δ 1 2 Δ