Concurrent Objects Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

Concurrent Computation memory object object Art of Multiprocessor Programming

Art of Multiprocessor Programming Objectivism What is a concurrent object? In this lecture we will focus on the first and last items. Art of Multiprocessor Programming

Art of Multiprocessor Programming Objectivism What is a concurrent object? How do we describe one? How do we implement one? How do we tell if we’re right? In this lecture we will focus on the first and last items. Art of Multiprocessor Programming

Art of Multiprocessor Programming Objectivism What is a concurrent object? How do we describe one? How do we tell if we’re right? Art of Multiprocessor Programming

FIFO Queue: Enqueue Method q.enq( ) Consider a FIFO queue. Art of Multiprocessor Programming

FIFO Queue: Dequeue Method q.deq()/ Art of Multiprocessor Programming

Art of Multiprocessor Programming A Lock-Based Queue class LockBasedQueue<T> { int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } Here is code for implementing a concurrent FIFO queue. Explain it IN DETAIL to the students. It’s the first algorithm they see which is not a mutual exclusion algorithm. The queue is implemented in an array. Initially the Head and Tail fields are equal and the queue is empty. All operations are synchronized Via the objects lock. If the Head and Tail differ by the queue size, then the queue is full. The Enq() method reads the Head() field, and if the queue is full, it waits until the queue is no longer full. It then stores the object in the array, and increments the Tail() field. Art of Multiprocessor Programming

Art of Multiprocessor Programming A Lock-Based Queue 1 capacity-1 2 head tail y z class LockBasedQueue<T> { int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } Here is code for implementing a concurrent FIFO queue. Explain it IN DETAIL to the students. It’s the first algorithm they see which is not a mutual exclusion algorithm. The queue is implemented in an array. Initially the Head and Tail fields are equal and the queue is empty. All operations are synchronized Via the objects lock. If the Head and Tail differ by the queue size, then the queue is full. The Enq() method reads the Head() field, and if the queue is full, it waits until the queue is no longer full. It then stores the object in the array, and increments the Tail() field. Queue fields protected by single shared lock Art of Multiprocessor Programming

Art of Multiprocessor Programming A Lock-Based Queue 1 capacity-1 2 head tail y z class LockBasedQueue<T> { int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } Here is code for implementing a concurrent FIFO queue. Explain it IN DETAIL to the students. It’s the first algorithm they see which is not a mutual exclusion algorithm. The queue is implemented in an array. Initially the Head and Tail fields are equal and the queue is empty. All operations are synchronized Via the objects lock. If the Head and Tail differ by the queue size, then the queue is full. The Enq() method reads the Head() field, and if the queue is full, it waits until the queue is no longer full. It then stores the object in the array, and increments the Tail() field. Initially head = tail Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Method calls mutually exclusive Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } If queue empty throw exception Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Queue not empty: remove item and update head Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Return result Art of Multiprocessor Programming

Implementation: Deq Release lock no matter what! 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Release lock no matter what! Art of Multiprocessor Programming

Art of Multiprocessor Programming Implementation: Deq public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Should be correct because modifications are mutually exclusive… Art of Multiprocessor Programming

Now consider the following implementation The same thing without mutual exclusion For simplicity, only two threads One thread enq only The other deq only This is sometimes called a producer-consumer channel or buffer. Art of Multiprocessor Programming

Wait-free 2-Thread Queue public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} 1 capacity-1 2 head tail y z Art of Multiprocessor Programming

Wait-free 2-Thread Queue public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z Art of Multiprocessor Programming

Wait-free 2-Thread Queue public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} 1 capacity-1 2 head tail y z How do we define “correct” when modifications are not mutually exclusive? Queue is updated without a lock! Art of Multiprocessor Programming

Defining concurrent queue implementations Need a way to specify a concurrent queue object Need a way to prove that an algorithm implements the object’s specification Lets talk about object specifications … In general, to make our intuitive understanding that the algorithm is correct we need a way to specify a concurrent queue object. Need a way to prove that the algorithm implements the object’s specification. We saw two types of object implementations, one that used locks and intuitively felt like a queue, and one that did not use locks, it was infact, wait-free. What properties are they actually providing, are they both implementations of a concurrent queue? In what ways are they similar and in what ways do they differ? Lets try and answer these questions. Art of Multiprocessor Programming

Correctness and Progress In a concurrent setting, we need to specify both the safety and the liveness properties of an object Need a way to define when an implementation is correct the conditions under which it guarantees progress There are two elements in which a concurrent specification imposes terms on the implementation: safety and liveness, or, in our case, correctness and progress. Lets begin with correctness Art of Multiprocessor Programming

Art of Multiprocessor Programming Sequential Objects Each object has a state Usually given by a set of fields Queue example: sequence of items Each object has a set of methods Only way to manipulate state Queue example: enq and deq methods Keep going back to the queue example throughout this lecture. Later, if you need to talk about multiple objects, give as an example a program that uses a stack and a queue. Art of Multiprocessor Programming

Sequential Specifications If (precondition) the object is in such-and-such a state before you call the method, Then (postcondition) the method will return a particular value or throw a particular exception. and (postcondition, con’t) the object will be in some other state when the method returns, Art of Multiprocessor Programming

Pre and PostConditions for Dequeue Precondition: Queue is non-empty Postcondition: Returns first item in queue Removes first item in queue Art of Multiprocessor Programming

Pre and PostConditions for Dequeue Precondition: Queue is empty Postcondition: Throws Empty exception Queue state unchanged Art of Multiprocessor Programming

Why Sequential Specifications Totally Rock Interactions among methods captured by side-effects on object state State meaningful between method calls Documentation size linear in number of methods Each method described in isolation Can add new methods Without changing descriptions of old methods Art of Multiprocessor Programming

What About Concurrent Specifications ? Methods? Documentation? Adding new methods? Art of Multiprocessor Programming

Art of Multiprocessor Programming Methods Take Time In the sequential world, we treat method calls as if they were instantaneous, but in fact, they take time. time time Art of Multiprocessor Programming

Art of Multiprocessor Programming Methods Take Time invocation 12:00 q.enq(...) In the sequential world, we treat method calls as if they were instantaneous, but in fact, they take time. time time Art of Multiprocessor Programming

Art of Multiprocessor Programming Methods Take Time invocation 12:00 q.enq(...) Method call In the sequential world, we treat method calls as if they were instantaneous, but in fact, they take time. time time Art of Multiprocessor Programming

Art of Multiprocessor Programming Methods Take Time invocation 12:00 q.enq(...) Method call In the sequential world, we treat method calls as if they were instantaneous, but in fact, they take time. time time Art of Multiprocessor Programming

Art of Multiprocessor Programming Methods Take Time invocation 12:00 void response 12:01 q.enq(...) Method call In the sequential world, we treat method calls as if they were instantaneous, but in fact, they take time. time time Art of Multiprocessor Programming

Sequential vs Concurrent Methods take time? Who knew? Concurrent Method call is not an event Method call is an interval. Art of Multiprocessor Programming

Concurrent Methods Take Overlapping Time Art of Multiprocessor Programming

Concurrent Methods Take Overlapping Time Method call time time Art of Multiprocessor Programming

Concurrent Methods Take Overlapping Time Method call Method call time time Art of Multiprocessor Programming

Concurrent Methods Take Overlapping Time Method call Method call Method call time time Art of Multiprocessor Programming

Sequential vs Concurrent Object needs meaningful state only between method calls Concurrent Because method calls overlap, object might never be between method calls Art of Multiprocessor Programming

Sequential vs Concurrent Each method described in isolation Concurrent Must characterize all possible interactions with concurrent calls What if two enqs overlap? Two deqs? enq and deq? … Art of Multiprocessor Programming

Sequential vs Concurrent Can add new methods without affecting older methods Concurrent: Everything can potentially interact with everything else Art of Multiprocessor Programming

Sequential vs Concurrent Can add new methods without affecting older methods Concurrent: Everything can potentially interact with everything else Panic! Art of Multiprocessor Programming

Art of Multiprocessor Programming The Big Question What does it mean for a concurrent object to be correct? What is a concurrent FIFO queue? FIFO means strict temporal order Concurrent means ambiguous temporal order Art of Multiprocessor Programming

Art of Multiprocessor Programming Intuitively… public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Art of Multiprocessor Programming

Art of Multiprocessor Programming Intuitively… public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } All modifications of queue are done mutually exclusive Art of Multiprocessor Programming

Art of Multiprocessor Programming Intuitively Lets capture the idea of describing the concurrent via the sequential q.deq lock() unlock() enq deq deq q.enq enq Behavior is “Sequential” On an intuitive level, since we use mutual exclusion locks, then each of the actual updates happens in a non-overlapping interval so the behavior as a whole looks sequential. This fits our intuition, and the question is if we can generalize this intuition. Stress the fact that we understand the concurrent execution because we have a way of mapping it to the sequential one, in which we know how a FIFO queue behaves. time Art of Multiprocessor Programming

Art of Multiprocessor Programming Linearizability Each method should “take effect” Instantaneously Between invocation and response events Object is correct if this “sequential” behavior is correct Any such concurrent object is Linearizable™ Linearizability is the generalization of this intuition to general objects, with our without mutual exclusion. Art of Multiprocessor Programming

Is it really about the object? Each method should “take effect” Instantaneously Between invocation and response events Sounds like a property of an execution… Art of Multiprocessor Programming

Is it really about the object? Each method should “take effect” Instantaneously Between invocation and response events Sounds like a property of an execution… A linearizable object: one all of whose possible executions are linearizable Art of Multiprocessor Programming

Art of Multiprocessor Programming Example Take you time with this example…don’t let them guess this one, show them, and let them guess the linearizability of the subsequent ones. time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(y) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(y) q.deq(x) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) q.enq(y) q.deq(x) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example linearizable q.enq(x) q.enq(x) q.enq(y) q.deq(x) q.deq(y) q.deq(y) q.enq(y) q.deq(x) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(x) Valid? q.deq(y) q.deq(y) q.enq(y) q.enq(y) q.deq(x) q.deq(x) time time (6) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(y) q.enq(x) q.deq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example not linearizable q.enq(x) q.enq(x) q.deq(y) q.enq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example time time (4) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) time time (4) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(x) time time (4) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(x) q.enq(x) q.deq(x) time time (4) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example linearizable q.enq(x) q.enq(x) q.deq(x) q.deq(x) time time (4) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) time time (8) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(y) time time (8) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) q.enq(y) time time (8) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) q.enq(y) q.deq(x) time time (8) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example Comme ci Comme ça multiple orders OK q.enq(x) q.deq(y) linearizable q.enq(y) q.deq(x) time Art of Multiprocessor Programming

Read/Write Register Example time time (4) Art of Multiprocessor Programming

Read/Write Register Example write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example not linearizable write(0) read(1) write(2) write(1) write(1) read(0) write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example not linearizable write(0) read(1) write(2) write(2) write(1) write(1) read(1) write(1) already happened time time (4) Art of Multiprocessor Programming

Read/Write Register Example time time (4) Art of Multiprocessor Programming

Read/Write Register Example time time (4) Art of Multiprocessor Programming

Read/Write Register Example linearizable write(0) write(1) write(2) write(2) write(1) read(1) time time (4) Art of Multiprocessor Programming

Reasoning About Linearizability: Locking 1 capacity-1 2 head tail y z public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Art of Multiprocessor Programming

Reasoning About Linearizability: Locking public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Linearization points are when locks are released Art of Multiprocessor Programming

More Reasoning: Wait-free public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} 1 capacity-1 2 head tail y z 1 capacity-1 2 head tail y z Art of Multiprocessor Programming

More Reasoning: Wait-free public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} 1 capacity-1 2 head tail y z Linearization order is order head and tail fields modified Remember that there is only one enqueuer and only one dequeuer Art of Multiprocessor Programming

Art of Multiprocessor Programming Strategy Identify one atomic step where method “happens” Critical section Machine instruction Doesn’t always work Might need to define several different steps for a given method We will see this phenomena that you might need different steps for different executions in lists chapter. Art of Multiprocessor Programming

Linearizability: Summary Powerful specification tool for shared objects Allows us to capture the notion of objects being “atomic” Don’t leave home without it We will see this phenomena that you might need different steps for different executions in lists chapter. Art of Multiprocessor Programming

Sequential Consistency No need to preserve real-time order Cannot re-order operations done by the same thread Can re-order non-overlapping operations done by different threads Often used to describe multiprocessor memory architectures Art of Multiprocessor Programming

Art of Multiprocessor Programming Example time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.deq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example q.enq(x) q.enq(y) q.enq(x) q.deq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Example not linearizable q.enq(x) q.enq(x) q.deq(y) q.enq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Yet Sequentially Consistent Example Yet Sequentially Consistent q.enq(x) q.enq(x) q.deq(y) q.enq(y) q.enq(y) time (5) Art of Multiprocessor Programming

Art of Multiprocessor Programming Progress We saw an implementation whose methods were lock-based (deadlock-free) We saw an implementation whose methods did not use locks (lock-free) How do they relate? So far we have talked about correctness. Now its time to discuss progress. Art of Multiprocessor Programming

Art of Multiprocessor Programming Progress Conditions Deadlock-free: some thread trying to acquire the lock eventually succeeds. Starvation-free: every thread trying to acquire the lock eventually succeeds. Lock-free: some thread calling a method eventually returns. Wait-free: every thread calling a method eventually returns. The above are definitions of progress conditions we have used and will use in the coming lectures. We give here informal definitions of progress conditions…formal ones need to talk about fair histories which is beyond the scope of this lecture…for the above conditions: A method implementation is deadlock-free if it guarantees min- imal progress in every fair history, and maximal progress in some fair history. A method implementation is starvation-free if it guarantees maximal progress in every fair history. A method implementation is lock-free if it guarantees minimal progress in every history and maximal progress in some history. A method implementation is wait-free if it guarantees maximal progress in every history. Art of Multiprocessor Programming

Art of Multiprocessor Programming Progress Conditions Non-Blocking Blocking Everyone makes progress Wait-free Starvation-free Someone makes progress Lock-free Deadlock-free Art of Multiprocessor Programming

Art of Multiprocessor Programming Summary We will look at linearizable blocking and non-blocking implementations of objects. Art of Multiprocessor Programming

Art of Multiprocessor Programming           This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License. You are free: to Share — to copy, distribute and transmit the work to Remix — to adapt the work Under the following conditions: Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Art of Multiprocessor Programming