1. Concurrent Queues and Stacks The Art of Multiprocessor Programming Spring 2007

2. © Herlihy-Shavit 20072 Last Lecture Five approaches to concurrent data structure design: –Coarse-grained locking –Fine-grained locking –Optimistic synchronization –Lazy synchronization –Lock-free synchronization

3. © Herlihy-Shavit 20073 List-based Set We used an ordered list to implement a Set: –an unordered collection of objects –No duplicates –Methods Add() a new object Remove() an object Test if set Contains() object

4. © Herlihy-Shavit 20074 Course Grained Locking a b d c Simple but hotspot + bottleneck

5. © Herlihy-Shavit 20075 Fine Grained (Lock-Coupling) a b d Allows concurency but everyone always delayed by front guy = hotspots + bottleneck

6. © Herlihy-Shavit 20076 Optimistic List b c e a 1.Limited Hotspots (Only at locked Add(), Remove(), Find() destination locations, not traversals) 2.But two traversals 3.Yet traversals are wait-free!

7. © Herlihy-Shavit 20077 Lazy List a 0 0 0 a b c 0 e 1 d Lazy Add() and Remove() + Wait-free Contains()

8. © Herlihy-Shavit 20078 Lock-free List a 0 0 0 a b c 0 e 1 c 1. Add() and Remove() physically remove marked nodes 2. Wait-free find() traverses both marked and removed nodes

9. © Herlihy-Shavit 20079 Today: Another Fundamental Problem We told you about –Sets implemented using linked lists Next: queues –Ubiquitous data structure –Often used to buffer requests …

10. © Herlihy-Shavit 200710 Shared Pools Queue belongs to broader pool class Pool: similar to Set but –Allows duplicates (it’s a Multiset) –No membership test (no Contains())

11. © Herlihy-Shavit 200711 Pool Flavors Bounded –Fixed capacity –Good when resources an issue Unbounded –Holds any number of objects

12. © Herlihy-Shavit 200712 Pool Flavors Problem cases: –Removing from empty pool –Adding to full (bounded) pool Blocking –Caller waits until state changes Non-Blocking –Method throws exception

13. © Herlihy-Shavit 200713 Queues & Stacks Add() and Remove(): Queue enqueue (Enq ()) and dequeue (Deq ()) Stack push and pop A Queue is a pool with FIFO order on enqueues and dequeues A Stack is a pool with LIFO order on pushes and pops

14. © Herlihy-Shavit 200714 This Lecture Bounded, Blocking, Lock-based Queue Unbounded, Non-Blocking, Lock-free Queue Examine effects of ABA problem Unbounded Non-Blocking Lock-free Stack Elimination-Backoff Stack

15. © Herlihy-Shavit 200715 Queue: Concurrency enq(x) y=deq() enq() and deq() work at different ends of the object tailhead

16. © Herlihy-Shavit 200716 Concurrency enq(x) Challenge: what if the queue is empty or full? y=deq() tail head

17. © Herlihy-Shavit 200717 Bounded Queue Sentinel head tail

18. © Herlihy-Shavit 200718 Bounded Queue head tail First actual item

19. © Herlihy-Shavit 200719 Bounded Queue head tail Lock out other deq() calls deqLock

20. © Herlihy-Shavit 200720 Bounded Queue head tail Lock out other enq() calls deqLock enqLock

21. © Herlihy-Shavit 200721 Not Done Yet head tail deqLock enqLock Need to tell whether queue is full or empty

22. © Herlihy-Shavit 200722 Not Done Yet head tail deqLock enqLock Permission to enqueue 8 items permits 8

23. © Herlihy-Shavit 200723 Not Done Yet head tail deqLock enqLock Incremented by deq() Decremented by enq() permits 8

24. © Herlihy-Shavit 200724 Enqueuer head tail deqLock enqLock permits 8 Lock enqLock

25. © Herlihy-Shavit 200725 Enqueuer head tail deqLock enqLock permits 8 Read permits OK

26. © Herlihy-Shavit 200726 Enqueuer head tail deqLock enqLock permits 8 No need to lock tail

27. © Herlihy-Shavit 200727 Enqueuer head tail deqLock enqLock permits 8 Enqueue Node

28. © Herlihy-Shavit 200728 Enqueuer head tail deqLock enqLock permits 8 7 getAndDecrement()

29. © Herlihy-Shavit 200729 Enqueuer head tail deqLock enqLock permits 8 Release lock 7

30. © Herlihy-Shavit 200730 Enqueuer head tail deqLock enqLock permits 7 If queue was empty, notify waiting dequeuers

31. © Herlihy-Shavit 200731 Unsuccesful Enqueuer head tail deqLock enqLock permits 0 Uh-oh Read permits

32. © Herlihy-Shavit 200732 Dequeuer head tail deqLock enqLock permits 8 Lock deqLock

33. © Herlihy-Shavit 200733 Dequeuer head tail deqLock enqLock permits 7 Read sentinel’s next field OK

34. © Herlihy-Shavit 200734 Dequeuer head tail deqLock enqLock permits 7 Read value

35. © Herlihy-Shavit 200735 Dequeuer head tail deqLock enqLock permits 7 Make first Node new sentinel

36. © Herlihy-Shavit 200736 Dequeuer head tail deqLock enqLock permits 7 Release deqLock

37. © Herlihy-Shavit 200737 Dequeuer head tail deqLock enqLock permits 8 Increment permits

38. © Herlihy-Shavit 200738 Unsuccesful Dequeuer head tail deqLock enqLock permits 8 Read sentinel’s next field uh-oh

39. © Herlihy-Shavit 200739 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

40. © Herlihy-Shavit 200740 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

41. © Herlihy-Shavit 200741 Monitor Locks The Reentrant Lock is a monitor Allows blocking on a condition rather than spinning Threads: –acquire and release lock –wait on a condition

42. © Herlihy-Shavit 200742 Java Monitor Locks public interface Lock { void lock(); void lockInterruptibly() throw InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; }

43. © Herlihy-Shavit 200743 Java Locks public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } Acquire lock

44. © Herlihy-Shavit 200744 Java Locks public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } Release lock

45. © Herlihy-Shavit 200745 Java Locks public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } Conditions to wait on

46. © Herlihy-Shavit 200746 Lock Conditions public interface Condition { void await() throws InterruptedException; boolean await(long time, TimeUnit unit) throws InterruptedException; … void signal(); void signalAll(); }

47. © Herlihy-Shavit 200747 Lock Conditions public interface Condition { void await() throws InterruptedException; boolean await(long time, TimeUnit unit) throws InterruptedException; … void signal(); void signalAll(); } Release lock and wait on condition

48. © Herlihy-Shavit 200748 Lock Conditions public interface Condition { void await() throws InterruptedException; boolean await(long time, TimeUnit unit) throws InterruptedException; … void signal(); void signalAll(); } Signal release of next thread in line or all awaiting threads

49. © Herlihy-Shavit 200749 The await() Method Releases lock on q Sleeps (gives up processor) Awakens (resumes running) Reacquires lock & returns q.await()

50. © Herlihy-Shavit 200750 The signal() Method Awakens one waiting thread Which will reacquire lock Then returns q.signal();

51. © Herlihy-Shavit 200751 The signalAll() Method Awakens all waiting threads Which will reacquire lock Then returns q.signalAll();

52. © Herlihy-Shavit 200752 A Monitor Lock Critical Section waiting room Lock() unLock()

53. © Herlihy-Shavit 200753 Awaiting a Condition Critical Section waiting room Lock() await()

54. © Herlihy-Shavit 200754 Monitor Signalling Critical Section waiting room Lock() unLock() Signal() I will try to enter Notice, woken thread might still loose lock to outside contender…

55. © Herlihy-Shavit 200755 Monitor Signaling All Critical Section waiting room SignalAll() Any one of us can try to enter

56. © Herlihy-Shavit 200756 Java Synchronized Monitor await() is wait() signal() is notify() signalAll() is notifyAll()

57. © Herlihy-Shavit 200757 Back to our Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

58. © Herlihy-Shavit 200758 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

59. © Herlihy-Shavit 200759 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Reentrant lock can have a condition for threads to wait on

60. © Herlihy-Shavit 200760 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Num of permits ranges from 0 to capacity

61. © Herlihy-Shavit 200761 Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Head and Tail

62. © Herlihy-Shavit 200762 Bounded Queue Enq Part 1 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } …

63. © Herlihy-Shavit 200763 Bounded Queue Enq Part 1 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } … Lock enq lock

64. © Herlihy-Shavit 200764 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } … Bounded Queue Enq Part 1 If permits = 0 wait till notFullCondition becomes true then check permits again…

65. © Herlihy-Shavit 200765 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } … Bounded Queue Enq Part 1 Add a new node

66. © Herlihy-Shavit 200766 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } … Bounded Queue Enq Part 1 If I was the enqueuer that changed queue state from empty to full will need to wake dequeuers

67. © Herlihy-Shavit 200767 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0){ try {notFullCondition.await} } Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } … Bounded Queue Enq Part 1 Release the enq lock

68. © Herlihy-Shavit 200768 Bounded Queue Enq Part 2 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); }

69. © Herlihy-Shavit 200769 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Bounded Queue Enq Part 2 To let the dequeuers know that the queue is non-empty, acquire deqLock

70. © Herlihy-Shavit 200770 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Bounded Queue Enq Part 2 Signal all dequeuer waiting that they can attempt to re-acquire deqLock

71. © Herlihy-Shavit 200771 public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Bounded Queue Enq Part 2 Release deqLock

72. © Herlihy-Shavit 200772 The Shared Counter The enq() and deq() methods –Don’t access the same lock concurrently –But they still share a counter –Which they both increment or decrement on every method call –Can we get rid of this bottleneck?

73. © Herlihy-Shavit 200773 Split the Counter The enq() method –Decrements only –Cares only if value is zero The deq() method –Increments only –Cares only if value is capacity

74. © Herlihy-Shavit 200774 Split Counter Enqueuer decrements enqSidePermits Dequeuer increments deqSidePermits When enqueuer runs out –Locks deqLock –Transfers permits Intermittent synchronization –Not with each method call –Need both locks! (careful …)

75. © Herlihy-Shavit 200775 A Lock-Free Queue Sentinel head tail

76. © Herlihy-Shavit 200776 Compare and Set CAS

77. © Herlihy-Shavit 200777 Enqueue Step One head tail Enqueue Node CAS

78. © Herlihy-Shavit 200778 Enqueue Step Two head tail Enqueue Node CAS

79. © Herlihy-Shavit 200779 Enqueue These two steps are not atomic The tail field refers to either –Actual last Node (good) –Penultimate Node (not so good) Be prepared!

80. © Herlihy-Shavit 200780 Enqueue What do you do if you find –A trailing tail? Stop and fix it –If node pointed to by tail has non-null next field –CAS the queue’s tail field to tail.next Like in the universal construction

81. © Herlihy-Shavit 200781 When CASs Fail In Step One –Retry loop –Method still lock-free (why?) In Step Two –Ignore it (why?)

82. © Herlihy-Shavit 200782 Dequeuer head tail Read value

83. © Herlihy-Shavit 200783 Dequeuer head tail Make first Node new sentinel CAS

84. © Herlihy-Shavit 200784 Memory Reuse? What do we do with nodes after we dequeue them? Java: let garbage collector deal? Suppose there isn’t a GC, or we don’t want to use it?

85. © Herlihy-Shavit 200785 Dequeuer head tail CAS Can recycle

86. © Herlihy-Shavit 200786 Simple Solution Each thread has a free list of unused queue nodes Allocate node: pop from list Free node: push onto list Deal with underflow somehow …

87. © Herlihy-Shavit 200787 Why Recycling is Hard Free pool headtail Want to rediret tail from grey to red zzz…

88. © Herlihy-Shavit 200788 Why Recycling is Hard Free pool zzz headtail

89. © Herlihy-Shavit 200789 Why Recycling is Hard Free pool Yawn! headtail

90. © Herlihy-Shavit 200790 Why Recycling is Hard Free pool CAS headtail OK, here I go!

91. © Herlihy-Shavit 200791 Final State Free pool What went wrong? headtail

92. © Herlihy-Shavit 200792 The Dreaded ABA Problem Head pointer has value A Thread reads value A headtail

93. © Herlihy-Shavit 200793 Dreaded ABA continued zzz Head pointer has value B Node A freed headtail

94. © Herlihy-Shavit 200794 Dreaded ABA continued Yawn! Head pointer has value A again Node A recycled & reinitialized headtail

95. © Herlihy-Shavit 200795 Dreaded ABA continued CAS succeeds because pointer matches even though pointer’s meaning has changed CAS headtail

96. © Herlihy-Shavit 200796 The Dreaded ABA Problem Is a result of CAS() semantics (Sun, Intel, AMD) Does not arise with Load- Locked/Store-Conditional (IBM)

97. © Herlihy-Shavit 200797 Dreaded ABA – A Solution Tag each pointer with a counter Unique over lifetime of node Pointer size vs word size issues Overflow? –Don’t worry be happy? –Bounded tags? AtomicStampedReference class

98. © Herlihy-Shavit 200798 A Concurrent Stack Add() and Remove() of Stack are called push() and pop() A Stack is a pool with LIFO order on pushes and pops

99. © Herlihy-Shavit 200799 Unbounded Lock-free Stack Top

100. © Herlihy-Shavit 2007100 Unbounded Lock-free Stack Top

101. © Herlihy-Shavit 2007101 Push Top CAS

102. © Herlihy-Shavit 2007102 Push Top

103. © Herlihy-Shavit 2007103 Push Top CAS

104. © Herlihy-Shavit 2007104 Push Top

105. © Herlihy-Shavit 2007105 Push Top

106. © Herlihy-Shavit 2007106 Push Top

107. © Herlihy-Shavit 2007107 Push Top CAS

108. © Herlihy-Shavit 2007108 Push Top

109. © Herlihy-Shavit 2007109 Push Top

110. © Herlihy-Shavit 2007110 Pop Top CAS

111. © Herlihy-Shavit 2007111 Pop Top CAS

112. © Herlihy-Shavit 2007112 Pop Top CAS

113. © Herlihy-Shavit 2007113 Pop Top

114. © Herlihy-Shavit 2007114 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack

115. © Herlihy-Shavit 2007115 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Push uses tryPush method

116. © Herlihy-Shavit 2007116 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Create a new node

117. © Herlihy-Shavit 2007117 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Then try to push: if tryPush fails back-off before retrying

118. © Herlihy-Shavit 2007118 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack tryPush attempts to push a node at top

119. © Herlihy-Shavit 2007119 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Read top value

120. © Herlihy-Shavit 2007120 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack current top will be new node’s successor

121. © Herlihy-Shavit 2007121 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public void push(T value) { public void tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Try to swing top to point at my new node

122. © Herlihy-Shavit 2007122 Lock-free Stack Good: No locking Bad: if no GC then ABA as in queue (add time stamps) Bad: Contention on top (add backoff) Bad: No parallelism Is a stack inherently sequential?

123. © Herlihy-Shavit 2007123 Elimination-Backoff Stack How to “turn contention into parallelism” Replace regular exponential-backoff with an alternative elimination- backoff mechanism

124. © Herlihy-Shavit 2007124 Observation Push( ) Pop() linearizable stack After any equal number of pushes and pops, stack stays the same

125. © Herlihy-Shavit 2007125 Idea: Elimination Array Push( ) Pop() stack Pick at random Pick at random Elimination Array

126. © Herlihy-Shavit 2007126 Push Collides With Pop Push( ) Pop() stack continue No need to access stack

127. © Herlihy-Shavit 2007127 No Collision Push( ) Pop() stack If no collision, access stack Pop() If pushes collide or pops collide access stack

128. © Herlihy-Shavit 2007128 Elimination-Backoff Stack Lock-free stack + elimination array Access Lock-free stack, –If uncontended, apply operation –if contended, back off to elimination array and attempt elimination

129. © Herlihy-Shavit 2007129 Elimination-Backoff Stack Push( ) Pop() Top CAS If failed CAS back-off

130. © Herlihy-Shavit 2007130 Dynamic Range and Delay Push( ) Pick random range and max time to wait for collision based on level of contention encountered

131. © Herlihy-Shavit 2007131 Linearizability Un-eliminated Lock-free stack calls: linearized as before Eliminated calls: linearize push immediately after the pop at the collision point Combination is a linearizable stack

132. © Herlihy-Shavit 2007132 Backoff Has Dual Affect Elimination introduces parallelism Backoff onto array cuts contention on lock-free stack Elimination in array cuts down total number of threads ever accessing lock-free stack

133. © Herlihy-Shavit 2007133 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; Random random; public EliminationArray(int capacity) { exchanger = (Exchanger []) new Exchanger[capacity]; for (int i = 0; i < capacity; i++) { exchanger[i] = new Exchanger (); } random = new Random(); } … } Elimination Array

134. © Herlihy-Shavit 2007134 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; Random random; public EliminationArray(int capacity) { exchanger = (Exchanger []) new Exchanger[capacity]; for (int i = 0; i < capacity; i++) { exchanger[i] = new Exchanger (); } random = new Random(); } … } Elimination Array An array of exchangers

135. © Herlihy-Shavit 2007135 public class NBExchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger

136. © Herlihy-Shavit 2007136 public class NBExchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger Slot holds atomically modifiable reference and time stamp

137. © Herlihy-Shavit 2007137 Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package address S Stamp Reference

138. © Herlihy-Shavit 2007138 Extracting Reference & Stamp Public T get(int[] stampHolder);

139. © Herlihy-Shavit 2007139 Extracting Reference & Stamp Public T get(int[] stampHolder); Returns reference to object of type T Returns stamp at array index 0!

140. © Herlihy-Shavit 2007140 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange

141. © Herlihy-Shavit 2007141 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange Input item and max time to wait for exchange before timing out

142. © Herlihy-Shavit 2007142 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange Array to hold extracted timestamp

143. © Herlihy-Shavit 2007143 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange Loop as long as time to attempt exchange does not run out

144. © Herlihy-Shavit 2007144 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange Get others item and time- stamp

145. © Herlihy-Shavit 2007145 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp % 3) { case 0: // slot is free case 1: // someone waiting for me case 2: // others exchanging } }} The Exchange Exchanger slot has three states determined by the timestamp mod 3

146. © Herlihy-Shavit 2007146 Lock-free Exchanger Slot State = 0 item stamp/state 0 CAS

147. © Herlihy-Shavit 2007147 Lock-free Exchanger Slot State changed to 1 wait for someone to appear… item stamp/state 1

148. © Herlihy-Shavit 2007148 Lock-free Exchanger Slot Still waiting for someone to appear… item stamp/state 2 CAS Try to exchange item and set state to 2

149. © Herlihy-Shavit 2007149 Lock-free Exchanger Slot 2 means someone showed up, take item and reset to 0 item stamp/state 2

150. © Herlihy-Shavit 2007150 Lock-free Exchanger Slot Read item and increment timestamp to 0 mod 3 item stamp/state 20

151. © Herlihy-Shavit 2007151 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0

152. © Herlihy-Shavit 2007152 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 Slot is free, try and insert myItem and change state to 1

153. © Herlihy-Shavit 2007153 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 Loop while still time left to try and exchange

154. © Herlihy-Shavit 2007154 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 Get item and stamp in slot and check if state changed to 2

155. © Herlihy-Shavit 2007155 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 If successful reset slot state to 0

156. © Herlihy-Shavit 2007156 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 and return item found in slot

157. © Herlihy-Shavit 2007157 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 Otherwise we ran out of time, try and reset state to 0, if successful time out

158. © Herlihy-Shavit 2007158 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 If reset failed can only be that someone showed up after all, take her item

159. © Herlihy-Shavit 2007159 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 Set slot to 0 with new time stamp and return the item found

160. © Herlihy-Shavit 2007160 case 0: // slot is free if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == stamp + 2) { slot.set(null, stamp + 3); return herItem; }} if (slot.compareAndSet(myItem, null, stamp + 1, stamp)) {throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, stamp + 3); return herItem; } } break; Exchanger State 0 If initial CAS failed then someone else changed slot from 0 to 1 so retry from start

161. © Herlihy-Shavit 2007161 case 1: // someone waiting for me if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) return herItem; break; case 2: // others in middle of exchanging break; default: // impossible break; } Exchanger States 1 and 2

162. © Herlihy-Shavit 2007162 case 1: // someone waiting for me if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) return herItem; break; case 2: // others in middle of exchanging break; default: // impossible break; } Exchanger States 1 and 2 state 1 means someone is waiting for an exchange, so attempt to CAS my Item in and change state to 2

163. © Herlihy-Shavit 2007163 case 1: // someone waiting for me if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) return herItem; break; case 2: // others in middle of exchanging break; default: // impossible break; } Exchanger States 1 and 2 If successful return her item, state is now 2, otherwise someone else took her item so try again from start

164. © Herlihy-Shavit 2007164 case 1: // someone waiting for me if (slot.compareAndSet(herItem, myItem, stamp, stamp + 1)) return herItem; break; case 2: // others in middle of exchanging break; default: // impossible break; } Exchanger States 1 and 2 If state is 2 then some other threads are using slot to exchange so start again

165. © Herlihy-Shavit 2007165 Our Exchanger Slot Notice that we showed a general lock- free exchanger Its lock-free because the only way an exchange can fail is if others repeatedly succeeded or no-one showed up The slot we need does not require symmetric exchange

166. © Herlihy-Shavit 2007166 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range return (exchanger[slot].exchange(value, duration, timeUnit)) }} Elimination Array

167. © Herlihy-Shavit 2007167 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range) return (exchanger[slot].exchange(value, duration)) }} Elimination Array visit the elimination array with a value and a range (duration to wait is not dynamic)

168. © Herlihy-Shavit 2007168 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range return (exchanger[slot].exchange(value, duration)) }} Elimination Array Pick a random array entry

169. © Herlihy-Shavit 2007169 public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range return (exchanger[slot].exchange(value, duration)) }} Elimination Array Exchange value or time out

170. © Herlihy-Shavit 2007170 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push

171. © Herlihy-Shavit 2007171 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push First try to push

172. © Herlihy-Shavit 2007172 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push If failed back-off to try and eliminate

173. © Herlihy-Shavit 2007173 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Value being pushed and range to try

174. © Herlihy-Shavit 2007174 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Only a pop has null value so elimination was successful

175. © Herlihy-Shavit 2007175 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Else retry push on lock-free stack

176. © Herlihy-Shavit 2007176 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if ( otherValue != null) { return otherValue; } }} Elimination Stack Pop

177. © Herlihy-Shavit 2007177 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if ( otherValue != null) { return otherValue; } }} Elimination Stack Pop Same as push, if non-null other thread must have pushed so elimnation succeeds

178. © Herlihy-Shavit 2007178 Summary We saw both lock-based and lock- free implementations of queues and stacks Don’t be quick to declare a data structure inherently sequential –Linearizable stack is not inherently sequential ABA is a real problem, pay attention