Universality of Consensus and The Nature of Progress Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Turing Computability A mathematical model of computation Computable = Computable on a T-Machine

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Shared-Memory Computability Model of asynchronous concurrent computation Computable = Wait-free/Lock-free computable on a multiprocessor cache shared memory cache

Art of Multiprocessor Programming© Copyright Herlihy-Shavit The Consensus Hierarchy 1 Read/Write Registers, Snapshots… 2 getAndSet, getAndIncrement, … ∞ compareAndSet,… FIFO Queue, LIFO Stack Multiple Assignment Can we implement them from any other object that has consensus number ∞? Like compareAndSet()…

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Theorem: Universality Consensus is universal From n-thread consensus build a –Wait-free –Linearizable –n-threaded implementation –Of any sequentially specified object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Proof Outline A universal construction –From n-consensus objects –And atomic registers Any wait-free linearizable object –Not a practical construction –But we know where to start looking …

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Like a Turing Machine This construction –Illustrates what needs to be done –Optimization fodder Correctness, not efficiency –Why does it work? (Asks the scientist) –How does it work? (Asks the engineer) –Would you like fries with that? (Asks the liberal arts major)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit A Generic Sequential Object public interface SeqObject { public abstract Response apply(Invocation invoc); }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit A Generic Sequential Object public interface SeqObject { public abstract Response apply(Invocation invoc); } Push:5, Pop:null

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Invocation public class Invoc { public String method; public Object[] args; }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Invocation public class Invoc { public String method; public Object[] args; } Method name

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Invocation public class Invoc { public String method; public Object[] args; } Arguments

Art of Multiprocessor Programming© Copyright Herlihy-Shavit A Generic Sequential Object public interface SeqObject { public abstract Response apply(Invocation invoc); } OK, 4

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Response public class Response { public Object value; } Return value

Art of Multiprocessor Programming© Copyright Herlihy-Shavit A Universal Concurrent Object public interface SeqObject { public abstract Response apply(Invocation invoc); } A universal concurrent object is a concurrent object that is linearizable to the generic sequential object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Start with Lock-Free Universal Construction First Lock-free: infinitely often some method call finishes. Then Wait-Free: each method call takes a finite number of steps to finish

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Lock-free Construction: Naïve Idea Use consensus object to store pointer to cell with current state Each thread creates new cell –computes outcome, –and tries to switch pointer to its outcome Unfortunately not… –consensus objects can be used once only

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Naïve Idea enq deq

Art of Multiprocessor Programming© Copyright Herlihy-Shavit head Naïve Idea deq Concurrent Object enq ? Decide which to apply using consensus No good. Each thread can use consensus object only once

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Why only once? Why is consensus object not readable? (1) public decide(object value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } Solved one time 2-consensus. Not clear how to allow reuse of object or reading its state… Queue based consensus

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Improved Idea: Linked-List Representation enq tail deq Each node contains a fresh consensus object used to decide on next operation

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Construction Object represented as –Initial Object State –A Log: a linked list of the method calls New method call –Find end of list –Atomically append call –Compute response by traversing the log upto the call

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Basic Idea Use one-time consensus object to decide next pointer All threads update actual next pointer based on decision –OK because they all write the same value Challenges –Lock-free means we need to worry what happens if a thread stops in the middle

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures Standard interface for class whose objects are totally ordered

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures the invocation

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures Decide on next node (next method applied to object)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures Traversable pointer to next node (needed because you cannot repeatedly read a consensus object)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures Seq number

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Node implements java.lang.Comparable { public Invoc invoc; public Consensus decideNext; public Node next; public int seq; public Node(Invoc invoc) { invoc = invoc; decideNext = new Consensus () seq = 0; } Basic Data Structures Create a new node for a given method invocation

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object head 123 decideNext (Consensus Object) Ptr to cell w/highest Seq Num Seq number, Invoc tail node next 4

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object head 123 All threads repeatedly modify head…back to where we started? tail node 4

Art of Multiprocessor Programming© Copyright Herlihy-Shavit … The Solution head 123 tail node i 4 Make head an array Ptr to node at front Thread i updates location i Threads find head by finding Max of nodes pointed to by head array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object public class Universal { private Node[] head; private Node tail = new Node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object public class Universal { private Node[] head; private Node tail = new Node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail } Head Pointers Array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object public class Universal { private Node[] head; private Node tail = new Node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail } Tail is a sentinel node with sequence number 1

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object public class Universal { private Node[] head; private Node tail = new Node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail } Tail is a sentinel node with sequence number 1

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Object public class Universal { private Node[] head; private Node tail = new Node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail } Initially head points to tail

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public static Node max(Node[] array) { Node max = array[0]; for (int i = 1; i < array.length; i++) if (max.seq < array[i].seq) max = array[i]; return max; } Find Max Head Value

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public static Node max(Node[] array) { Node max = array[0]; for (int i = 0; i < array.length; i++) if (max.seq < array[i].seq) max = array[i]; return max; } Find Max Head Value Traverse the array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public static Node max(Node[] array) { Node max = array[0]; for (int i = 0; i < array.length; i++) if (max.seq < array[i].seq) max = array[i]; return max; } Find Max Head Value Compare the seq nums of nodes pointed to by the array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public static Node max(Node[] array) { Node max = array[0]; for (int i = 0; i < array.length; i++) if (max.seq < array[i].seq) max = array[i]; return max; } Find Max Head Value Return the node with max seq num

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } …

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … Apply will have invocation as input and return the appropriate response

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … My id

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … My method call

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … As long as I have not been threaded into list

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … Node at head of list that I will try to append to

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } Decide winning node; could have already been decided

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } Set next pointer based on decision Could have already been set by winner…in which case no affect

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … Set seq number indicating node was appended

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part I public Response apply(Invoc invoc) { int i = ThreadID.get(); Node prefer = new node(invoc); while (prefer.seq == 0) { Node before = Node.max(head); Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } … add to head array so new head will be found

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Part II – Compute Response nullenq( ) tail Red’s method call … deq() enq( ) Return Private copy of object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } Compute the result by sequentially applying the method calls in the list to a private copy of the object starting from the initial state

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } Start with initialized copy of the sequential object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } First new method call is appended after the tail

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } While not reached my own method call

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } Apply the current nodes method to object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } Return the result after applying my own method call

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Correctness List defines linearized sequential history Thread returns its response based on list order

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Lock-freedom Lock-free because –A thread moves forward in list –Can repeatedly fail to win consensus on “real” head only if another succeeds –Consensus winner adds node and completes within a finite number of steps

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Wait-free Construction Lock-free construction + announce array Stores (pointer to) node in announce –If a thread doesn’t append its node –Another thread will see it in array and help append it

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Helping “Announcing” my intention –Guarantees progress –Even if the scheduler hates me –My method call will complete Makes protocol wait-free Otherwise starvation possible

Art of Multiprocessor Programming© Copyright Herlihy-Shavit … … Wait-free Construction head 123 tail i 4 announce Ptr to the cell thread i wants to append i

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Universal { private Node[] announce; private Node[] head; private Node tail = new node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail; announce[j] = tail }; The Announce Array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Universal { private Node[] announce; private Node[] head; private Node tail = new node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail; announce[j] = tail }; The Announce Array Announce array

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class Universal { private Node[] announce; private Node[] head; private Node tail = new node(); tail.seq = 1; for (int j=0; j < n; j++){ head[j] = tail; announce[j] = tail }; The Announce Array All entries initially point to tail

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public Response apply(Invoc invoc) { int i = ThreadID.get(); announce[i] = new Node(invoc); head[i] = Node.max(head); while (announce[i].seq == 0) { … // while node not appended to list … } A Cry For Help

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public Response apply(Invoc invoc) { int i = ThreadID.get(); announce[i] = new Node(invoc); head[i] = Node.max(head); while (announce[i].seq == 0) { … // while node not appended to list … } A Cry For Help Announce new method call (node), asking help from others

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public Response apply(Invoc invoc) { int i = ThreadID.get(); announce[i] = new Node(invoc); head[i] = Node.max(head); while (announce[i].seq == 0) { … // while node not appended to list … } A Cry For Help Look for end of list

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public Response apply(Invoc invoc) { int i = ThreadID.get(); announce[i] = new Node(invoc); head[i] = Node.max(head); while (announce[i].seq == 0) { … // while node not appended to list … } A Cry For Help Main loop, while node not appended (either by me or some thread helping me)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Main Loop Non-zero sequence number indicates success Thread keeps helping append nodes Until its own node is appended

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Main Loop

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Main Loop Keep trying until my cell gets a sequence number

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Main Loop Possible end of list

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Main Loop Who do I help?

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Altruism Choose a thread to “help” If that thread needs help –Try to append its node –Otherwise append your own Worst case –Everyone tries to help same pitiful loser –Someone succeeds

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Help! When last node in list has sequence number k All threads check … –Whether thread k+1 mod n wants help –If so, try to append her node first

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Help! First time after thread k+1 announces –No guarantees After n more nodes appended –Everyone sees that thread k+1 wants help –Everyone tries to append that node –Someone succeeds

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Sliding Window Lemma After thread A announces its node No more than n other calls –Can start and finish –Without appending A’s node

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Helping head Max head +1 = N+4 N+2 N+3 … announce Thread 4: Help me! 4 So all see and help append 4 tail

Art of Multiprocessor Programming© Copyright Herlihy-Shavit The Sliding Help Window head N+2 N+3 … announce 4 tail Help 3 Help 4 3

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Sliding Help Window In each main loop iteration pick another thread to help

Art of Multiprocessor Programming© Copyright Herlihy-Shavit while (announce[i].seq == 0) { Node before = head[i]; Node help = announce[(before.seq + 1 % n)]; if (help.seq == 0) prefer = help; else prefer = announce[i]; … Sliding Help Window Help if help required, but otherwise it’s all about me!

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Rest is Same as Lock-free while (prefer.seq == 0) { … Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Rest is Same as Lock-free while (prefer.seq == 0) { … Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } Decide next node to be appended

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Rest is Same as Lock-free while (prefer.seq == 0) { … Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } Update next based on decision

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Rest is Same as Lock-free while (prefer.seq == 0) { … Node after = before.decideNext.decide(prefer); before.next = after; after.seq = before.seq + 1; head[i] = after; } Tell world that node is appended

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Finishing the Job Once thread’s node is linked The rest is again the same as in lock- free alg Compute the result by sequentially applying the method calls in the list to a private copy of the object starting from the initial state

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Then Same Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); }

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Universal Application Part II... //compute my response SeqObject MyObject = new SeqObject(); current = tail.next; while (current != prefer){ MyObject.apply(current.invoc); current = current.next; } return MyObject.apply(current.invoc); } Return the result after applying my own method call

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Shared-Memory Computability Wait-free/Lock-free computable = Threads with methods that solve n- consensus Universal Object

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean getAndSet(int update) { int prior = this.value; this.value = update; return prior; } GetAndSet is not Universal (1)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean getAndSet(int update) { int prior = this.value; this.value = update; return prior; } GetAndSet is not Universal (1) Consensus number 2

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean getAndSet(int update) { int prior = this.value; this.value = update; return prior; } GetAndSet is not Universal (1) Not universal for ≥ 3 threads

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean compareAndSet(int expected, int update) { int prior = this.value; if (this.value == expected) { this.value = update; return true; } return false; }} CompareAndSet is Universal (1)

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean compareAndSet(int expected, int update) { int prior = this.value; if (this.value == expected) { this.value = update; return true; } return false; }} CompareAndSet is Universal (1) Consensus number ∞

Art of Multiprocessor Programming© Copyright Herlihy-Shavit public class RMWRegister { private int value; public boolean compareAndSet(int expected, int update) { int prior = this.value; if (this.value == expected) { this.value = update; return true; } return false; }} CompareAndSet is Universal (1) Universal for any number of threads

Art of Multiprocessor Programming© Copyright Herlihy-Shavit On Older Architectures IBM 360 –testAndSet (getAndSet) NYU UltraComputer –getAndAdd Neither universal –Except for 2 threads

Art of Multiprocessor Programming© Copyright Herlihy-Shavit On Newer Architectures Intel x86, Itanium, SPARC –compareAndSet Alpha AXP, PowerPC –Load-locked/store-conditional All universal –For any number of threads Trend is clear …

Art of Multiprocessor Programming© Copyright Herlihy-Shavit Practical Implications Any architecture that does not provide a universal primitive has inherent limitations You cannot avoid locking for concurrent data structures … But why do we care?

Locking and Schedeuling What are the practical implications of locking? Locking affects the assumptions we need to make on the operating system in order to guarantee progress Lets understand how… Art of Multiprocessor Programming© Copyright Herlihy-Shavit

Schedeuling The scheduler is a part of the OS that determines –Which thread gets to run on which processor –How long it runs for A given thread can thus be active, that is, executing instructions, or suspended Art of Multiprocessor Programming© Copyright Herlihy-Shavit

Review Progress Conditions Deadlock-free: some thread trying to acquire the locks eventually succeeds. Starvation-free: every thread trying to acquire the locks eventually succeeds. Lock-free: some thread calling the method eventually returns. Wait-free: every thread calling the method eventually returns. Obstruction-free: every thread calling the method returns if it executes in isolation for long enough. 105

© 2007 Herlihy & Shavit 106 The Simple Snapshot is Obstruction-Free Put increasing labels on each entry Collect twice If both agree, –We’re done Otherwise, –Try again = Collect2 Collect

Obstruction-freedom In the simple snapshot alg: The update method is wait-free But the scan is obstruction-free: will complete only if it executes for long enough without concurrent updates. Art of Multiprocessor Programming© Copyright Herlihy-Shavit

Progress of Methods Some of the above defs refer to locks (part of implementation) or method calls And they ignore the scheduler Lets refine our progress definitions so that they apply to methods, and Take scheduling into account Art of Multiprocessor Programming© Copyright Herlihy-Shavit

A “Periodic Table” of Progress Conditions 109 Everyone makes progress Non-BlockingBlocking Someone makes progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free

A bit more formally Standard notion of abstract object Progress conditions relate to method calls of an object Threads on a multiprocessor never fail: –A thread is active if it takes an infinite number of concrete (machine level) steps –And is suspended if not. 110

Maximal vs. Minimal For a given history H: Minimal progress: in every suffix of H, some method call eventually completes. Maximal progress: in every suffix of H, every method call eventually completes. 111

The “Periodic Table” of Progress Conditions 112 Maximal progress Non-BlockingBlocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free

The Scheduler’s Role On a multiprocessor progress properties: Are not about the guarantees a method's implementation provides. They are about the scheduling assumptions needed in order to provide minimal or maximal progress. 113

Fair Scheduling A history is fair if each thread takes an infinite number of steps A method implementation is deadlock-free if it guarantees minimal progress in every fair history, and maximal progress in some fair history. 114

Starvation Freedom A method implementation is starvation-free if it guarantees maximal progress in every fair history. 115

Dependent Progress A progress condition is dependent if it does not guarantee minimal progress in every history, and is independent if it does. The blocking progress conditions (deadlock-freedom, Starvation- freedom) are dependent 116

Non-blocking Independent Conditions A method implementation is lock- free if it guarantees minimal progress in every history, and maximal progress in some history. A method implementation is wait- free if it guarantees maximal progress in every history. 117

The “Periodic Table” of Progress Conditions 118 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free Independent Dependent

Uniformly Isolating Schedules A history is uniformly isolating if, for every k > 0, any thread that takes an infinite number of steps has an interval where it takes at least k contiguous steps Modern systems provide ways of providing isolation…later we will learn about “backoff” and “yeild”. 119

A Non-blocking Dependent Condition A method implementation is obstruction-free if it guarantees maximal progress in every uniformly isolating history. 120

The “Periodic Table” of Progress Conditions 121 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent Uniform iso scheduler Fair scheduler

The “Periodic Table” of Progress Conditions 122 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free

Clash-Freedom: the “Einsteinium” of Progress A method implementation is clash-free if it guarantees minimal progress in every uniformly isolating history. Thm: clash-freedom strictly weaker than obstruction-freedom 123

Getting from Minimal to Maximal 124 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free ? Helping But helping is expensive

Universal Constructions Our lock-free universal construction provides minimal progress A scheduler is benevolent for that algorithm if it guarantees maximal progress in every allowable history. Many real-world operating system schedulers are benevolent They do not single out any indiviudual thread 125

Getting from Minimal to Maximal 126 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free ? Helping Universal Lock-free Construction Universal Wait-free Construction Use Wait-free/Lock-free Consensus Objects

Getting from Minimal to Maximal 127 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free Universal Wait-free Construction Universal Lock-free Construction If we use Starvation-free/Deadlock-free Consensus Objects result is respectively Starvation-free/Deadlock-free

Benevolent Schedulers Consider an algorithm that guarantees minimal progress. A scheduler is benevolent for that algorithm if it guarantees maximal progress in every allowable history. Many real-world operating system schedulers are benevolent They do not single out any indiviudual thread 128

In Practice On a multiprocessor we will write code expecting maximal progress. Progress conditions will then define the scheduling assumptions needed in order to provide it. s129

This Means We will mostly write lock-free and lock-based deadlock-free algorithms… and expect them to behave as if they are wait-free… because modern schedulers can be made benevolent and fair. s130

Principles to Practice We learned how to define the safety (correctness) and liveness (progress) of concurrent programs and objects We are ready to start the practice of implementing them Next lecture: implementing spin locks on multiprocesor machines… Art of Multiprocessor Programming© Copyright Herlihy-Shavit

Art of Multiprocessor Programming© Copyright Herlihy-Shavit This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to – Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.