1. Nir Shavit Multiprocessor Synchronization Spring 2003
Universality
Nir Shavit
Multiprocessor Synchronization
Spring 2003

2. Turing Computability A mathematical model of computation
T-Machine f r # a zr w 4 p t l s
1 g k 3
A mathematical model of computation
Complexity irrelevant to real machines
Computable = Computable on a T-Machine
29-Aug-19
© 2003 Herlihy and Shavit

3. Asynchronous Computability
Universal
Object
10011
Shared Memory
A model of concurrent computation
Complexity irrelevant to real machines
Wait-free/Lock-free computable = ????
29-Aug-19
© 2003 Herlihy and Shavit

4. Theorem: Universality
Consensus is universal
From n-thread consensus build a
Wait-free
Linearizable
n-threaded implementation
Of any sequentially specified object
29-Aug-19
© 2003 Herlihy and Shavit

5. Proof Outline Specify a universal object
Object implements any sequentially specified object canonically
Construct the universal object from read/write registers and objects that solve n-process consensus
29-Aug-19
© 2003 Herlihy and Shavit

6. Like a Turing Machine This construction Correctness, not efficiency
Not intended to be practical
But enlightening
Correctness, not efficiency
Why does it work? (The scientist)
How does it work? (The engineer)
Would you like fries with that? (The liberal arts major)
29-Aug-19
© 2003 Herlihy and Shavit

7. Universal Construction
Object implemented as a
List of method calls
Method call
Find end of list
Atomically append your call
29-Aug-19
© 2003 Herlihy and Shavit

8. Naive Idea
Use consensus object to store pointer to cell with current state
Each thread creates new cell, computes outcome, and tries to switch pointer to its outcome
NO GO…we only know how to build a one-time consensus object (where each thread accesses object once)
29-Aug-19
© 2003 Herlihy and Shavit

9. Basic Idea: Linked-List Representation
deq
enq
enq
enq
root
29-Aug-19
© 2003 Herlihy and Shavit

10. Basic Idea Use one-time consensus object as pointer Challenges
How to avoid starvation?
What if a thread stops in the middle?
29-Aug-19
© 2003 Herlihy and Shavit

11. Stylized Sequential Objects
Object itself is immutable
Method call produces
New copy of object
Return value (or exception)
29-Aug-19
© 2003 Herlihy and Shavit

12. Stylized Sequential Objects
Invocation
Method name
Arguments
Response
Copy of modified object
Return value
29-Aug-19
© 2003 Herlihy and Shavit

13. Sequential Objects public class SeqObject { public class Invoc {
public String method;
public Object[] args; }
public class Response {
public SeqObject object;
public Object value;
public Result apply(Invoc invoc) { … }}
29-Aug-19
© 2003 Herlihy and Shavit

14. Inner class: method name + arguments
Sequential Objects
public class SeqObject {
public class Invoc {
public String method;
public Object[] args; }
public class Result {
public SeqObject object;
public Object value;
public Result apply(Invoc invoc) { … }}
Inner class: method name + arguments
29-Aug-19
© 2003 Herlihy and Shavit

15. Inner class: new object state + return value
Sequential Objects
public class SeqObject {
public class Invoc {
public String method;
public Object[] args; }
public class Response {
public SeqObject object;
public Object value;
public Result apply(Invoc invoc) { … }}
Inner class: new object state + return value
29-Aug-19
© 2003 Herlihy and Shavit

16. Applies method: returns new state + return value
Sequential Objects
public class SeqObject {
public class Invoc {
public String method;
public Object[] args; }
public class Response {
public SeqObject object;
public Object response;
public Result apply(Invoc invoc) { … }}
Applies method: returns new state + return value
29-Aug-19
© 2003 Herlihy and Shavit

17. Basic Data Structures public class Cell { int seq; Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
29-Aug-19
© 2003 Herlihy and Shavit

18. Sequence number: zero while in play
Basic Data Structures
public class Cell {
int seq;
Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
Sequence number: zero while in play
29-Aug-19
© 2003 Herlihy and Shavit

19. Basic Data Structures Method name + args public class Cell { int seq;
Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
Method name + args
29-Aug-19
© 2003 Herlihy and Shavit

20. New object state + return value
Basic Data Structures
public class Cell {
int seq;
Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
New object state + return value
29-Aug-19
© 2003 Herlihy and Shavit

21. State resulting from invocation
Basic Data Structures
public class Cell {
int seq;
Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
State resulting from invocation
29-Aug-19
© 2003 Herlihy and Shavit

22. Consensus object that indicates next cell in list
Basic Data Structures
public class Cell {
int seq;
Invoc invoc;
Response response;
SeqObject nextState;
Consensus nextCell;
Consensus object that indicates next cell in list
29-Aug-19
© 2003 Herlihy and Shavit

23. Cell Constructor public Cell(Invoc invoc) { this.seq = 0;
this.invoc = invoc;
this.response = null;
this.nextState = null;
this.nextCell = new Consensus(); }
29-Aug-19
© 2003 Herlihy and Shavit

24. Sequence number zero means call incomplete
Cell Constructor
public Cell(Invoc invoc) {
this.seq = 0;
this.invoc = invoc;
this.response = null;
this.nextState = null;
this.nextCell = new Consensus(); }
Sequence number zero means call incomplete
29-Aug-19
© 2003 Herlihy and Shavit

25. Record method name + args
Cell Constructor
public Cell(Invoc invoc) {
this.seq = 0;
this.invoc = invoc;
this.response = null;
this.nextState = null;
this.nextCell = new Consensus(); }
Record method name + args
29-Aug-19
© 2003 Herlihy and Shavit

26. Cell Constructor First thread decides public Cell(Invoc invoc) {
this.seq = 0;
this.invoc = invoc;
this.response = null;
this.nextState = null;
this.nextCell = new Consensus(); }
First thread decides
29-Aug-19
© 2003 Herlihy and Shavit

27. Cell Comparison Cell max(Cell, aCell) { if (this.seq > aCell.seq)
return this;
else
return aCell; }}
29-Aug-19
© 2003 Herlihy and Shavit

28. Universal Object public class Universal { private Cell[] announce;
private Cell[] head; … }
29-Aug-19
© 2003 Herlihy and Shavit

29. If this thread does not succeed, another thread will help out
Universal Object
public class Universal {
private Cell[] announce;
private Cell[] head; … }
If this thread does not succeed, another thread will help out
29-Aug-19
© 2003 Herlihy and Shavit

30. Threads need a fast way to find the end of the list
Universal Object
public class Universal {
private Cell[] announce;
private Cell[] head; … }
Threads need a fast way to find the end of the list
29-Aug-19
© 2003 Herlihy and Shavit

31. Seq, Invoc, Resp, NextState, Cell i wishes to thread on
Universal Object
Seq, Invoc, Resp, NextState,
cell
Consensus Object
anchor 1 2 3
Highest Seq Num
head 2 2 3 1 1 3 1 1
Cell i wishes to thread on
announce
29-Aug-19
© 2003 Herlihy and Shavit

32. Thread A Allocates cell to represent call
Stores (pointer to) cell in announce
If A doesn’t execute it
Another thread will
Looks for thread near end of list
By scanning head array
Choosing cell with largest sequence num
29-Aug-19
© 2003 Herlihy and Shavit

33. Helping “Announcing” my intention Makes protocol wait-free
Guarantees progress
Even if the scheduler hates me
My method call will complete
Makes protocol wait-free
Otherwise starvation possible
29-Aug-19
© 2003 Herlihy and Shavit

34. Ask For Help public class Universal {
public Object apply(Invoc invoc) {
int i = Thread.myIndex();
this.announce[i] = new Cell(invoc);
for (int j = 0; j < N; j++)
this.head[i] =
max(this.head[i], this.head[j]); …
29-Aug-19
© 2003 Herlihy and Shavit

35. Announce my intention to append cell to list
Ask For Help
public class Universal {
public Object apply(Invoc invoc) {
int i = Thread.myIndex();
this.announce[i] = new Cell(invoc);
for (int j = 0; j < N; j++)
this.head[i] =
max(this.head[i], this.head[j]); …
Announce my intention to append cell to list
29-Aug-19
© 2003 Herlihy and Shavit

36. Where does it End? Need to find end of list
Can’t start from anchor
Starvation for slow threads
Each thread records last cell seen
If all scan that array
Some thread has last cell
29-Aug-19
© 2003 Herlihy and Shavit

37. Assume the Position max(this.head[i], this.head[j]);
public class Universal {
public Object apply(Invoc invoc) {
int i = Thread.myIndex();
this.announce[i] = new Cell(invoc);
for (int j = 0; j < N; j++)
this.head[i] =
max(this.head[i], this.head[j]); …
29-Aug-19
© 2003 Herlihy and Shavit

38. Assume the Position Search for end of list public class Universal {
public Object apply(Invoc invoc) {
int i = Thread.myIndex();
this.announce[i] = new Cell(invoc);
for (int j = 0; j < N; j++)
this.head[i] =
max(this.head[i], this.head[j]); …
Search for end of list
29-Aug-19
© 2003 Herlihy and Shavit

39. Are We Done Yet? Non-zero sequence number indicates success
Thread keeps appending cells
Until its own cell is done
29-Aug-19
© 2003 Herlihy and Shavit

40. Keep on Keeping on while (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq+1) % n];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; }
29-Aug-19
© 2003 Herlihy and Shavit

41. Loop until my cell gets a sequence number
Keep on Keeping on
while (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq+1) % n];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; }
Loop until my cell gets a sequence number
29-Aug-19
© 2003 Herlihy and Shavit

42. Keep on Keeping on Possible end of list
while (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq+1) % n];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; }
Possible end of list
29-Aug-19
© 2003 Herlihy and Shavit

43. Altruism Choose a thread to “help” If that thread needs help
Try to append its cell
Otherwise append your own
Worst-case
Everyone tries to help loser
Someone succeeds
29-Aug-19
© 2003 Herlihy and Shavit

44. Help! Last cell in list has sequence number k All threads check …
Whether thread k+1 mod n wants help
If so, try to append her cell first
29-Aug-19
© 2003 Herlihy and Shavit

45. Help! All threads try to help k+1 (mod n)
First time after k+1 announces
Some see announcement
Some may not (yikes! bad timing)
No guarantees
29-Aug-19
© 2003 Herlihy and Shavit

46. Help! First time after k+1 announces After n more cells appended
No guarantees
After n more cells appended
Everyone sees that k+1 wants help
Everyone tries to append that cell
Someone succeeds
29-Aug-19
© 2003 Herlihy and Shavit

47. Lemma After A announces cell No more than n other calls
Can start and finish
Without appending A’s cell
29-Aug-19
© 2003 Herlihy and Shavit

48. Helping … N+2 N+3 1 2 3 2 2 3 1 1 3 1 1 4 head announce
Max head +1 = N+4
Helping
N+2
N+3 … 1 2 3
So all see and help 4
Max head +1 = 4 2 2 3 1 1 3 1 1
head
Help me
announce 4
29-Aug-19
© 2003 Herlihy and Shavit

49. Pull Together while (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq+1) % n];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; …
29-Aug-19
© 2003 Herlihy and Shavit

50. Pick another thread to help
Pull Together
while (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq+1) % n];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; …
Pick another thread to help
29-Aug-19
© 2003 Herlihy and Shavit

51. Help if help requested, otherwise it’s all about me!
Pull Together
Help if help requested, otherwise it’s all about me!
While (this.announce[i].seq == 0) {
Cell before, help, prefer;
before = this.head[i];
help = announce[(before.seq % n) +1];
if (help.seq == 0)
prefer = help;
else
prefer = this.announce[i]; … }
29-Aug-19
© 2003 Herlihy and Shavit

52. Quality is Job 1.1 While (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState.propose(response);
after.response =
(Response)after.nextState.decide(); …}
29-Aug-19
© 2003 Herlihy and Shavit

53. Propose my favorite, but find out who actually won
Quality is Job 1.1
While (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState.propose(response);
after.response =
(Response)after.nextState.decide(); …}
Propose my favorite, but find out who actually won
29-Aug-19
© 2003 Herlihy and Shavit

54. Determine object’s prior state
Quality is Job 1.1
while (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState.propose(response);
after.response =
(Response)after.nextState.decide(); …}
Determine object’s prior state
29-Aug-19
© 2003 Herlihy and Shavit

55. Finishing the Job Once we have linked in a cell
Make sure remaining fields are filled in before moving on
New object state in response field
29-Aug-19
© 2003 Herlihy and Shavit

56. Quality is Job 1.1 while (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState = response.object;
after.response = response.value; …}
29-Aug-19
© 2003 Herlihy and Shavit

57. Quality is Job 1.1 Get prior Object state
while (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState = response.object;
after.response = response.value; …}
Get prior Object state
29-Aug-19
© 2003 Herlihy and Shavit

58. Quality is Job 1.1 Compute response
while (this.announce[i].seq == 0) { …
before.next.propose(prefer);
Cell after = before.next.decide();
SeqObject oldObject =
before.response.object;
Response response =
oldObject.apply(after.invoc);
after.nextState = response.object;
after.response = response.value; …}
Compute response
29-Aug-19
© 2003 Herlihy and Shavit

59. What Were We Thinking? while (this.announce[i].seq == 0) { …
after.nextState = response.object;
after.response = response.value;
this.head[i] = after;
after.seq = before.seq + 1; }
29-Aug-19
© 2003 Herlihy and Shavit

60. What Were We Thinking? Advance end of list
while (this.announce[i].seq == 0) { …
after.nextState = response.object;
after.response = response.value;
this.head[i] = after;
after.seq = before.seq + 1; }
Advance end of list
29-Aug-19
© 2003 Herlihy and Shavit

61. What Were We Thinking? Set my sequence number
while (this.announce[i].seq == 0) { …
after.nextState = response.object;
after.response = response.value;
this.head[i] = after;
after.seq = before.seq + 1; }
Set my sequence number
29-Aug-19
© 2003 Herlihy and Shavit

62. Asynchronous Computability
Universal
Object
10011
Wait-free/Lock-free computable =
Threads with methods that solve n-consensus
29-Aug-19
© 2003 Herlihy and Shavit

63. Compare&Swap is universal
public abstract class RMW {
private int value;
public void CAS(int old, int new) {
int prior = this.value;
if (this.value == old)
this.value = new;
return prior; }
29-Aug-19
(1)
© 2003 Herlihy and Shavit

64. Practical Implications
Better have a universal operation on your machine
Even if it costs more than a load or store to implement
Having it implies that you can implement anything
But still complexity may be an issue…
29-Aug-19
© 2003 Herlihy and Shavit

65. Till Now: Correctness Models Protocols Accurate (we never lied to you)
But idealized (so we forgot to mention a few things)
Protocols
Elegant
Important
But naïve
29-Aug-19
© 2003 Herlihy and Shavit

66. Next Time: Performance
Models
More complicated (not the same as complex!)
Still focus on principals (not soon obsolete)
Protocols
Elegant (in their fashion)
Important (why else would we pay attention)
And realistic (your mileage may vary)
29-Aug-19
© 2003 Herlihy and Shavit