1. Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2. Art of Multiprocessor Programming© Herlihy-Shavit 2007 2 The Five-Fold Path Coarse-grained locking Fine-grained locking Optimistic synchronization Lazy synchronization Lock-free synchronization

3. Art of Multiprocessor Programming© Herlihy-Shavit 2007 3 Another Fundamental Problem We told you about –Sets implemented using linked lists Next: queues Next: stacks

4. Art of Multiprocessor Programming© Herlihy-Shavit 2007 4 Queues & Stacks Both: pool of items Queue –enq() & deq() –First-in-first-out (FIFO) order Stack –push() & pop() –Last-in-first-out (LIFO) order

5. Art of Multiprocessor Programming© Herlihy-Shavit 2007 5 Bounded vs Unbounded Bounded –Fixed capacity –Good when resources an issue Unbounded –Holds any number of objects

6. Art of Multiprocessor Programming© Herlihy-Shavit 2007 6 Blocking vs Non-Blocking Problem cases: –Removing from empty pool –Adding to full (bounded) pool Blocking –Caller waits until state changes Non-Blocking –Method throws exception

7. Art of Multiprocessor Programming© Herlihy-Shavit 2007 7 This Lecture Bounded, Blocking, Lock-based Queue Unbounded, Non-Blocking, Lock-free Queue ABA problem Unbounded Non-Blocking Lock-free Stack Elimination-Backoff Stack

8. Art of Multiprocessor Programming© Herlihy-Shavit 2007 8 Queue: Concurrency enq(x) y=deq() enq() and deq() work at different ends of the object tailhead

9. Art of Multiprocessor Programming© Herlihy-Shavit 2007 9 Concurrency enq(x) Challenge: what if the queue is empty or full? y=deq() tail head

10. Art of Multiprocessor Programming© Herlihy-Shavit 2007 10 Bounded Queue Sentinel head tail

11. Art of Multiprocessor Programming© Herlihy-Shavit 2007 11 Bounded Queue head tail First actual item

12. Art of Multiprocessor Programming© Herlihy-Shavit 2007 12 Bounded Queue head tail Lock out other deq() calls deqLock

13. Art of Multiprocessor Programming© Herlihy-Shavit 2007 13 Bounded Queue head tail Lock out other enq() calls deqLock enqLock

14. Art of Multiprocessor Programming© Herlihy-Shavit 2007 14 Not Done Yet head tail deqLock enqLock Need to tell whether queue is full or empty

15. Art of Multiprocessor Programming© Herlihy-Shavit 2007 15 Not Done Yet head tail deqLock enqLock Permission to enqueue 8 items permits 8

16. Art of Multiprocessor Programming© Herlihy-Shavit 2007 16 Not Done Yet head tail deqLock enqLock Incremented by deq() Decremented by enq() permits 8

17. Art of Multiprocessor Programming© Herlihy-Shavit 2007 17 Enqueuer head tail deqLock enqLock permits 8 Lock enqLock

18. Art of Multiprocessor Programming© Herlihy-Shavit 2007 18 Enqueuer head tail deqLock enqLock permits 8 Read permits OK

19. Art of Multiprocessor Programming© Herlihy-Shavit 2007 19 Enqueuer head tail deqLock enqLock permits 8 No need to lock tail

20. Art of Multiprocessor Programming© Herlihy-Shavit 2007 20 Enqueuer head tail deqLock enqLock permits 8 Enqueue Node

21. Art of Multiprocessor Programming© Herlihy-Shavit 2007 21 Enqueuer head tail deqLock enqLock permits 8 7 getAndDecrement()

22. Art of Multiprocessor Programming© Herlihy-Shavit 2007 22 Enqueuer head tail deqLock enqLock permits 8 Release lock 7

23. Art of Multiprocessor Programming© Herlihy-Shavit 2007 23 Enqueuer head tail deqLock enqLock permits 7 If queue was empty, notify/signal waiting dequeuers

24. Art of Multiprocessor Programming© Herlihy-Shavit 2007 24 Unsuccesful Enqueuer head tail deqLock enqLock permits 0 Uh-oh Read permits

25. Art of Multiprocessor Programming© Herlihy-Shavit 2007 25 Dequeuer head tail deqLock enqLock permits 8 Lock deqLock

26. Art of Multiprocessor Programming© Herlihy-Shavit 2007 26 Dequeuer head tail deqLock enqLock permits 7 Read sentinel’s next field OK

27. Art of Multiprocessor Programming© Herlihy-Shavit 2007 27 Dequeuer head tail deqLock enqLock permits 7 Read value

28. Art of Multiprocessor Programming© Herlihy-Shavit 2007 28 Dequeuer head tail deqLock enqLock permits 7 Make first Node new sentinel

29. Art of Multiprocessor Programming© Herlihy-Shavit 2007 29 Dequeuer head tail deqLock enqLock permits 8 Increment permits

30. Art of Multiprocessor Programming© Herlihy-Shavit 2007 30 Dequeuer head tail deqLock enqLock permits 7 Release deqLock

31. Art of Multiprocessor Programming© Herlihy-Shavit 2007 31 Unsuccesful Dequeuer head tail deqLock enqLock permits 8 Read sentinel’s next field uh-oh

32. Art of Multiprocessor Programming© Herlihy-Shavit 2007 32 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

33. Art of Multiprocessor Programming© Herlihy-Shavit 2007 33 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

34. Art of Multiprocessor Programming© Herlihy-Shavit 2007 34 (Pop!) The Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

35. Art of Multiprocessor Programming© Herlihy-Shavit 2007 35 Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

36. Art of Multiprocessor Programming© Herlihy-Shavit 2007 36 Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq lock’s associated condition

37. Art of Multiprocessor Programming© Herlihy-Shavit 2007 37 Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Num permits: 0 to capacity

38. Art of Multiprocessor Programming© Herlihy-Shavit 2007 38 Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Head and Tail

39. Art of Multiprocessor Programming© Herlihy-Shavit 2007 39 Enq Method Part One public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … }

40. Art of Multiprocessor Programming© Herlihy-Shavit 2007 40 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One Lock and unlock enq lock

41. Art of Multiprocessor Programming© Herlihy-Shavit 2007 41 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One If queue is full, patiently await further instructions …

42. Art of Multiprocessor Programming© Herlihy-Shavit 2007 42 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Be Afraid How do we know the permits field won’t change?

43. Art of Multiprocessor Programming© Herlihy-Shavit 2007 43 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One Add new node

44. Art of Multiprocessor Programming© Herlihy-Shavit 2007 44 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One If queue was empty, wake frustrated dequeuers

45. Art of Multiprocessor Programming© Herlihy-Shavit 2007 45 Enq Method Part Deux public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); }

46. Art of Multiprocessor Programming© Herlihy-Shavit 2007 46 Enq Method Part Deux public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Are there dequeuers to be signaled?

47. Art of Multiprocessor Programming© Herlihy-Shavit 2007 47 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Enq Method Part Deux Lock and unlock deq lock

48. Art of Multiprocessor Programming© Herlihy-Shavit 2007 48 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Enq Method Part Deux Signal dequeuers that queue no longer empty

49. Art of Multiprocessor Programming© Herlihy-Shavit 2007 49 The Enq() & Deq() Methods Share no locks –That’s good But do share an atomic counter –Accessed on every method call –That’s not so good Can we alleviate this bottleneck?

50. Art of Multiprocessor Programming© Herlihy-Shavit 2007 50 Split the Counter The enq() method –Decrements only –Cares only if value is zero The deq() method –Increments only –Cares only if value is capacity

51. Art of Multiprocessor Programming© Herlihy-Shavit 2007 51 Split Counter Enqueuer decrements enqSidePermits Dequeuer increments deqSidePermits When enqueuer runs out –Locks deqLock –Transfers permits Intermittent synchronization –Not with each method call –Need both locks! (careful …)

52. Art of Multiprocessor Programming© Herlihy-Shavit 2007 52 A Lock-Free Queue Sentinel head tail

53. Art of Multiprocessor Programming© Herlihy-Shavit 2007 53 Compare and Set CAS

54. Art of Multiprocessor Programming© Herlihy-Shavit 2007 54 Enqueue head tail Enq( )

55. Art of Multiprocessor Programming© Herlihy-Shavit 2007 55 Enqueue head tail

56. Art of Multiprocessor Programming© Herlihy-Shavit 2007 56 Logical Enqueue head tail CAS

57. Art of Multiprocessor Programming© Herlihy-Shavit 2007 57 Physical Enqueue head tail Enqueue Node CAS

58. Art of Multiprocessor Programming© Herlihy-Shavit 2007 58 Enqueue These two steps are not atomic The tail field refers to either –Actual last Node (good) –Penultimate Node (not so good) Be prepared!

59. Art of Multiprocessor Programming© Herlihy-Shavit 2007 59 Enqueue What do you do if you find –A trailing tail? Stop and help fix it –If tail node has non-null next field –CAS the queue’s tail field to tail.next As in the universal construction

60. Art of Multiprocessor Programming© Herlihy-Shavit 2007 60 When CASs Fail During logical enqueue –Abandon hope, restart –Still lock-free (why?) During physical enqueue –Ignore it (why?)

61. Art of Multiprocessor Programming© Herlihy-Shavit 2007 61 Dequeuer head tail Read value

62. Art of Multiprocessor Programming© Herlihy-Shavit 2007 62 Dequeuer head tail Make first Node new sentinel CAS

63. Art of Multiprocessor Programming© Herlihy-Shavit 2007 63 Memory Reuse? What do we do with nodes after we dequeue them? Java: let garbage collector deal? Suppose there is no GC, or we prefer not to use it?

64. Art of Multiprocessor Programming© Herlihy-Shavit 2007 64 Dequeuer head tail CAS Can recycle

65. Art of Multiprocessor Programming© Herlihy-Shavit 2007 65 Simple Solution Each thread has a free list of unused queue nodes Allocate node: pop from list Free node: push onto list Deal with underflow somehow …

66. Art of Multiprocessor Programming© Herlihy-Shavit 2007 66 Why Recycling is Hard Free pool headtail Redirect head from grey to red zzz…

67. Art of Multiprocessor Programming© Herlihy-Shavit 2007 67 Both Nodes Reclaimed Free pool zzz headtail

68. Art of Multiprocessor Programming© Herlihy-Shavit 2007 68 One Node Recycled Free pool Yawn! headtail

69. Art of Multiprocessor Programming© Herlihy-Shavit 2007 69 Why Recycling is Hard Free pool CAS headtail OK, here I go!

70. Art of Multiprocessor Programming© Herlihy-Shavit 2007 70 Final State Free pool what went wrong? headtail Bad news

71. Art of Multiprocessor Programming© Herlihy-Shavit 2007 71 The Dreaded ABA Problem Head pointer has value A Thread reads value A headtail

72. Art of Multiprocessor Programming© Herlihy-Shavit 2007 72 Dreaded ABA continued zzz Head pointer has value B Node A freed headtail

73. Art of Multiprocessor Programming© Herlihy-Shavit 2007 73 Dreaded ABA continued Yawn! Head pointer has value A again Node A recycled & reinitialized headtail

74. Art of Multiprocessor Programming© Herlihy-Shavit 2007 74 Dreaded ABA continued CAS succeeds because pointer matches even though pointer’s meaning has changed CAS headtail

75. Art of Multiprocessor Programming© Herlihy-Shavit 2007 75 The Dreaded ABA Problem Is a result of CAS() semantics –I blame Sun, Intel, AMD, … Not with Load-Locked/Store- Conditional –Good for IBM?

76. Art of Multiprocessor Programming© Herlihy-Shavit 2007 76 Dreaded ABA – A Solution Tag each pointer with a counter Unique over lifetime of node Pointer size vs word size issues Overflow? –Don’t worry be happy? –Bounded tags? AtomicStampedReference class

77. Art of Multiprocessor Programming© Herlihy-Shavit 2007 77 Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package address S Stamp Reference Can get reference and stamp atomically

78. Art of Multiprocessor Programming© Herlihy-Shavit 2007 78 Concurrent Stack Methods –push(x) –pop() Last-in, First-out (LIFO) order Lock-Free!

79. Art of Multiprocessor Programming© Herlihy-Shavit 2007 79 Empty Stack Top

80. Art of Multiprocessor Programming© Herlihy-Shavit 2007 80 Push Top

81. Art of Multiprocessor Programming© Herlihy-Shavit 2007 81 Push Top CAS

82. Art of Multiprocessor Programming© Herlihy-Shavit 2007 82 Push Top

83. Art of Multiprocessor Programming© Herlihy-Shavit 2007 83 Push Top

84. Art of Multiprocessor Programming© Herlihy-Shavit 2007 84 Push Top

85. Art of Multiprocessor Programming© Herlihy-Shavit 2007 85 Push Top CAS

86. Art of Multiprocessor Programming© Herlihy-Shavit 2007 86 Push Top

87. Art of Multiprocessor Programming© Herlihy-Shavit 2007 87 Pop Top

88. Art of Multiprocessor Programming© Herlihy-Shavit 2007 88 Pop Top CAS

89. Art of Multiprocessor Programming© Herlihy-Shavit 2007 89 Pop Top CAS mine!

90. Art of Multiprocessor Programming© Herlihy-Shavit 2007 90 Pop Top CAS

91. Art of Multiprocessor Programming© Herlihy-Shavit 2007 91 Pop Top

92. Art of Multiprocessor Programming© Herlihy-Shavit 2007 92 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }} Lock-free Stack

93. Art of Multiprocessor Programming© Herlihy-Shavit 2007 93 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack tryPush attempts to push a node

94. Art of Multiprocessor Programming© Herlihy-Shavit 2007 94 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Read top value

95. Art of Multiprocessor Programming© Herlihy-Shavit 2007 95 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack current top will be new node’s successor

96. Art of Multiprocessor Programming© Herlihy-Shavit 2007 96 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Try to swing top, return success or failure

97. Art of Multiprocessor Programming© Herlihy-Shavit 2007 97 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Push calls tryPush

98. Art of Multiprocessor Programming© Herlihy-Shavit 2007 98 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Create new node

99. Art of Multiprocessor Programming© Herlihy-Shavit 2007 99 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack If tryPush() fails, back off before retrying Makes scheduling benevolent so Method is effectively wait- free

100. Art of Multiprocessor Programming© Herlihy-Shavit 2007 100 Lock-free Stack Good –No locking Bad –Without GC, fear ABA –Without backoff, huge contention at top –In any case, no parallelism

101. Art of Multiprocessor Programming© Herlihy-Shavit 2007 101 Big Question Are stacks inherently sequential? Reasons why –Every pop() call fights for top item Reasons why not –Stay tuned …

102. Art of Multiprocessor Programming© Herlihy-Shavit 2007 102 Elimination-Backoff Stack How to –“turn contention into parallelism” Replace familiar –exponential backoff With alternative –elimination-backoff

103. Art of Multiprocessor Programming© Herlihy-Shavit 2007 103 Observation Push( ) Pop() linearizable stack After an equal number of pushes and pops, stack stays the same Yes!

104. Art of Multiprocessor Programming© Herlihy-Shavit 2007 104 Idea: Elimination Array Push( ) Pop() stack Pick at random Pick at random Elimination Array

105. Art of Multiprocessor Programming© Herlihy-Shavit 2007 105 Push Collides With Pop Push( ) Pop() stack continue No need to access stack Yes!

106. Art of Multiprocessor Programming© Herlihy-Shavit 2007 106 No Collision Push( ) Pop() stack If no collision, access stack If pushes collide or pops collide access stack

107. Art of Multiprocessor Programming© Herlihy-Shavit 2007 107 Elimination-Backoff Stack Lock-free stack + elimination array Access Lock-free stack, –If uncontended, apply operation –if contended, back off to elimination array and attempt elimination

108. Art of Multiprocessor Programming© Herlihy-Shavit 2007 108 Elimination-Backoff Stack Push( ) Pop() Top CAS If CAS fails, back off

109. Art of Multiprocessor Programming© Herlihy-Shavit 2007 109 Dynamic Range and Delay Push( ) Pick random range and max time to wait for collision based on level of contention encountered

110. Art of Multiprocessor Programming© Herlihy-Shavit 2007 110 Linearizability Un-eliminated calls –linearized as before Eliminated calls: –linearize pop() immediately after matching push() Combination is a linearizable stack

111. Art of Multiprocessor Programming© Herlihy-Shavit 2007 111 Un-Eliminated Linearizability push(v 1 ) time linearizable push(v 1 ) pop(v 1 )

112. Art of Multiprocessor Programming© Herlihy-Shavit 2007 112 Eliminated Linearizability pop(v 2 )push(v 1 ) push(v 2 ) time push(v 2 ) linearizable pop(v 2 ) push(v 1 ) pop(v 1 ) Collision Point Red calls are eliminated

113. Art of Multiprocessor Programming© Herlihy-Shavit 2007 113 Backoff Has Dual Effect Elimination introduces parallelism Backoff onto array cuts contention on lock-free stack Elimination in array cuts down total number of threads ever accessing lock-free stack

114. Art of Multiprocessor Programming© Herlihy-Shavit 2007 114 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array

115. Art of Multiprocessor Programming© Herlihy-Shavit 2007 115 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array An array of exchangers

116. Art of Multiprocessor Programming© Herlihy-Shavit 2007 116 public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger

117. Art of Multiprocessor Programming© Herlihy-Shavit 2007 117 public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger Atomically modifiable reference + time stamp

118. Art of Multiprocessor Programming© Herlihy-Shavit 2007 118 Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package In C or C++: address S Stamp Reference

119. Art of Multiprocessor Programming© Herlihy-Shavit 2007 119 Extracting Reference & Stamp Public T get(int[] stampHolder);

120. Art of Multiprocessor Programming© Herlihy-Shavit 2007 120 Extracting Reference & Stamp Public T get(int[] stampHolder); Returns reference to object of type T Returns stamp at array index 0!

121. Art of Multiprocessor Programming© Herlihy-Shavit 2007 121 Exchanger Status enum Status {EMPTY, WAITING, BUSY};

122. Art of Multiprocessor Programming© Herlihy-Shavit 2007 122 Exchanger Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet

123. Art of Multiprocessor Programming© Herlihy-Shavit 2007 123 Exchange Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet One thread is waiting for rendez-vous

124. Art of Multiprocessor Programming© Herlihy-Shavit 2007 124 Exchange Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet One thread is waiting for rendez-vous Other threads busy with rendez-vous

125. Art of Multiprocessor Programming© Herlihy-Shavit 2007 125 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange

126. Art of Multiprocessor Programming© Herlihy-Shavit 2007 126 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Item & timeout

127. Art of Multiprocessor Programming© Herlihy-Shavit 2007 127 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Array to hold timestamp

128. Art of Multiprocessor Programming© Herlihy-Shavit 2007 128 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Loop until timeout

129. Art of Multiprocessor Programming© Herlihy-Shavit 2007 129 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Get other’s item and timestamp

130. Art of Multiprocessor Programming© Herlihy-Shavit 2007 130 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }} The Exchange Exchanger slot has three states

131. Art of Multiprocessor Programming© Herlihy-Shavit 2007 131 Lock-free Exchanger EMPTY

132. Art of Multiprocessor Programming© Herlihy-Shavit 2007 132 EMPTY Lock-free Exchanger CAS

133. Art of Multiprocessor Programming© Herlihy-Shavit 2007 133 WAITING Lock-free Exchanger

134. Art of Multiprocessor Programming© Herlihy-Shavit 2007 134 Lock-free Exchanger In search of partner … WAITING

135. Art of Multiprocessor Programming© Herlihy-Shavit 2007 135 WAITING Lock-free Exchanger Slot Still waiting … Try to exchange item and set state to BUSY CAS

136. Art of Multiprocessor Programming© Herlihy-Shavit 2007 136 BUSY Lock-free Exchanger Slot Partner showed up, take item and reset to EMPTY item stamp/state

137. Art of Multiprocessor Programming© Herlihy-Shavit 2007 137 EMPTYBUSY Lock-free Exchanger Slot item stamp/state Partner showed up, take item and reset to EMPTY

138. Art of Multiprocessor Programming© Herlihy-Shavit 2007 138 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY

139. Art of Multiprocessor Programming© Herlihy-Shavit 2007 139 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Slot is free, try to insert myItem and change state to WAITING

140. Art of Multiprocessor Programming© Herlihy-Shavit 2007 140 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Loop while still time left to attempt exchange

141. Art of Multiprocessor Programming© Herlihy-Shavit 2007 141 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Get item and stamp in slot and check if state changed to BUSY

142. Art of Multiprocessor Programming© Herlihy-Shavit 2007 142 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If successful reset slot state to EMPTY

143. Art of Multiprocessor Programming© Herlihy-Shavit 2007 143 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY and return item found in slot

144. Art of Multiprocessor Programming© Herlihy-Shavit 2007 144 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Otherwise we ran out of time, try to reset state to EMPTY, if successful time out

145. Art of Multiprocessor Programming© Herlihy-Shavit 2007 145 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If reset failed, someone showed up after all, take that item

146. Art of Multiprocessor Programming© Herlihy-Shavit 2007 146 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Set slot to EMPTY with new timestamp and return item found

147. Art of Multiprocessor Programming© Herlihy-Shavit 2007 147 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If initial CAS failed then someone else changed state from EMPTY to WAITING so retry from start

148. Art of Multiprocessor Programming© Herlihy-Shavit 2007 148 case WAITING: // someone waiting for me if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY

149. Art of Multiprocessor Programming© Herlihy-Shavit 2007 149 case WAITING: // someone waiting for me if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY state WAITING means someone is waiting for an exchange, so attempt to CAS my item in and change state to BUSY

150. Art of Multiprocessor Programming© Herlihy-Shavit 2007 150 case WAITING: // someone waiting for me if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY If successful return her item, state is now BUSY, otherwise someone else took her item so try again from start

151. Art of Multiprocessor Programming© Herlihy-Shavit 2007 151 case WAITING: // someone waiting for me if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY If state is BUSY then some other threads are using slot to exchange so start again

152. Art of Multiprocessor Programming© Herlihy-Shavit 2007 152 The Exchanger Slot Exchanger is lock-free Because the only way an exchange can fail is if others repeatedly succeeded or no-one showed up The slot we need does not require symmetric exchange

153. Art of Multiprocessor Programming© Herlihy-Shavit 2007 153 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array

154. Art of Multiprocessor Programming© Herlihy-Shavit 2007 154 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array visit the elimination array with a value and a range (duration to wait is not dynamic)

155. Art of Multiprocessor Programming© Herlihy-Shavit 2007 155 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Pick a random array entry

156. Art of Multiprocessor Programming© Herlihy-Shavit 2007 156 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Exchange value or time out

157. Art of Multiprocessor Programming© Herlihy-Shavit 2007 157 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push

158. Art of Multiprocessor Programming© Herlihy-Shavit 2007 158 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push First try to push

159. Art of Multiprocessor Programming© Herlihy-Shavit 2007 159 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push If failed back-off to try to eliminate

160. Art of Multiprocessor Programming© Herlihy-Shavit 2007 160 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Value being pushed and range to try

161. Art of Multiprocessor Programming© Herlihy-Shavit 2007 161 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Only a pop has null value so elimination was successful

162. Art of Multiprocessor Programming© Herlihy-Shavit 2007 162 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Else retry push on lock-free stack

163. Art of Multiprocessor Programming© Herlihy-Shavit 2007 163 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if otherValue != null) { return otherValue; } }} Elimination Stack Pop

164. Art of Multiprocessor Programming© Herlihy-Shavit 2007 164 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if ( otherValue != null) { return otherValue; } }} Elimination Stack Pop Same as push, if non-null other thread must have pushed so elimnation succeeds

165. Art of Multiprocessor Programming© Herlihy-Shavit 2007 165 Summary We saw both lock-based and lock- free implementations of queues and stacks Don’t be quick to declare a data structure inherently sequential –Linearizable stack is not inherently sequential ABA is a real problem, pay attention

166. Art of Multiprocessor Programming© Herlihy-Shavit 2007 166 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.