1. Concurrent Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2. Art of Multiprocessor Programming22 Another Fundamental Problem We told you about –Sets implemented by linked lists –Queues Next: stacks

3. Art of Multiprocessor Programming33 Concurrent Stack Methods –push(x) –pop() Last-in, First-out (LIFO) order Lock-Free!

4. Art of Multiprocessor Programming44 Empty Stack Top

5. Art of Multiprocessor Programming55 Push Top

6. Art of Multiprocessor Programming66 Push Top CAS

7. Art of Multiprocessor Programming77 Push Top

8. Art of Multiprocessor Programming88 Push Top

9. Art of Multiprocessor Programming99 Push Top

10. Art of Multiprocessor Programming10 Push Top CAS

11. Art of Multiprocessor Programming11 Push Top

12. Art of Multiprocessor Programming12 Pop Top

13. Art of Multiprocessor Programming13 Pop Top CAS

14. Art of Multiprocessor Programming14 Pop Top CAS mine!

15. Art of Multiprocessor Programming15 Pop Top CAS

16. Art of Multiprocessor Programming16 Pop Top

17. Art of Multiprocessor Programming17 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }} Lock-free Stack

18. Art of Multiprocessor Programming18 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack tryPush attempts to push a node

19. Art of Multiprocessor Programming19 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Read top value

20. Art of Multiprocessor Programming20 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack current top will be new node’s successor

21. Art of Multiprocessor Programming21 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Try to swing top, return success or failure

22. Art of Multiprocessor Programming22 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Push calls tryPush

23. Art of Multiprocessor Programming23 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Create new node

24. Art of Multiprocessor Programming24 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack If tryPush() fails, back off before retrying

25. Art of Multiprocessor Programming25 Lock-free Stack Good –No locking Bad –Without GC, fear ABA –Without backoff, huge contention at top –In any case, no parallelism

26. Art of Multiprocessor Programming26 Big Question Are stacks inherently sequential? Reasons why –Every pop() call fights for top item Reasons why not –Stay tuned …

27. Art of Multiprocessor Programming27 Elimination-Backoff Stack How to –“turn contention into parallelism” Replace familiar –exponential backoff With alternative –elimination-backoff

28. Art of Multiprocessor Programming28 Observation Push( ) Pop() linearizable stack After an equal number of pushes and pops, stack stays the same After an equal number of pushes and pops, stack stays the same Yes!

29. Art of Multiprocessor Programming29 Idea: Elimination Array Push( ) Pop() stack Pick at random Pick at random Elimination Array

30. Art of Multiprocessor Programming30 Push Collides With Pop Push( ) Pop() stack continue No need to access stack No need to access stack Yes!

31. Art of Multiprocessor Programming31 No Collision Push( ) Pop() stack If no collision, access stack If no collision, access stack If pushes collide or pops collide access stack

32. Art of Multiprocessor Programming32 Elimination-Backoff Stack Lock-free stack + elimination array Access Lock-free stack, –If uncontended, apply operation –if contended, back off to elimination array and attempt elimination

33. Art of Multiprocessor Programming33 Elimination-Backoff Stack Push( ) Pop() Top CAS If CAS fails, back off

34. Art of Multiprocessor Programming34 Dynamic Range and Delay Push( ) Pick random range and max waiting time based on level of contention encountered

35. Art of Multiprocessor Programming35 Linearizability Un-eliminated calls –linearized as before Eliminated calls: –linearize pop() immediately after matching push() Combination is a linearizable stack

36. Art of Multiprocessor Programming36 Backoff Has Dual Effect Elimination introduces parallelism Backoff to array cuts contention on lock- free stack Elimination in array cuts down number of threads accessing lock-free stack

37. Art of Multiprocessor Programming37 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array

38. Art of Multiprocessor Programming38 public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array An array of Exchangers

39. Art of Multiprocessor Programming39 public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); Digression: A Lock-Free Exchanger

40. Art of Multiprocessor Programming40 public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger Atomically modifiable reference + status

41. Art of Multiprocessor Programming41 Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package In C or C++: address S reference stamp

42. Art of Multiprocessor Programming42 Extracting Reference & Stamp public T get(int[] stampHolder);

43. Art of Multiprocessor Programming43 Extracting Reference & Stamp public T get(int[] stampHolder); Returns reference to object of type T Returns stamp at array index 0!

44. Art of Multiprocessor Programming44 Exchanger Status enum Status {EMPTY, WAITING, BUSY};

45. Art of Multiprocessor Programming45 Exchanger Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet

46. enum Status {EMPTY, WAITING, BUSY}; Art of Multiprocessor Programming46 Exchange Status Nothing yet One thread is waiting for rendez-vous

47. Art of Multiprocessor Programming47 Exchange Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet One thread is waiting for rendez-vous Other threads busy with rendez-vous

48. Art of Multiprocessor Programming48 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange

49. Art of Multiprocessor Programming49 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Item and timeout

50. Art of Multiprocessor Programming50 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Array holds status

51. Art of Multiprocessor Programming51 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Loop until timeout

52. Art of Multiprocessor Programming52 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Get other’s item and status

53. Art of Multiprocessor Programming53 public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }} The Exchange An Exchanger has three possible states

54. Art of Multiprocessor Programming54 Lock-free Exchanger

55. Art of Multiprocessor Programming55 Lock-free Exchanger CAS

56. Art of Multiprocessor Programming56 Lock-free Exchanger

57. Art of Multiprocessor Programming57 Lock-free Exchanger In search of partner …

58. Art of Multiprocessor Programming58 Lock-free Exchanger Slot Still waiting … Try to exchange item and set status to BUSY CAS

59. Art of Multiprocessor Programming59 Lock-free Exchanger Slot Partner showed up, take item and reset to EMPTY item status

60. Art of Multiprocessor Programming60 EMPTY Lock-free Exchanger Slot item status

61. Art of Multiprocessor Programming61 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY

62. Art of Multiprocessor Programming62 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Try to insert myItem and change state to WAITING

63. Art of Multiprocessor Programming63 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Spin until either myItem is taken or timeout Spin until either myItem is taken or timeout

64. Art of Multiprocessor Programming64 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY myItem was taken, so return herItem that was put in its place myItem was taken, so return herItem that was put in its place

65. Art of Multiprocessor Programming65 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Otherwise we ran out of time, try to reset status to EMPTY and time out

66. Art of Multiprocessor Programming66Art of Multiprocessor Programming© Herlihy-Shavit 2007 66 case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If reset failed, someone showed up after all, so take that item If reset failed, someone showed up after all, so take that item

67. Art of Multiprocessor Programming67 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Clear slot and take that item

68. Art of Multiprocessor Programming68 case EMPTY: // slot is free if (slot.CAS(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.CAS(myItem, null, WAITING, EMPTY)){ throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If initial CAS failed, then someone else changed status from EMPTY to WAITING, so retry from start If initial CAS failed, then someone else changed status from EMPTY to WAITING, so retry from start

69. Art of Multiprocessor Programming69 case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY

70. Art of Multiprocessor Programming70 case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY someone is waiting to exchange, so try to CAS my item in and change state to BUSY someone is waiting to exchange, so try to CAS my item in and change state to BUSY

71. Art of Multiprocessor Programming71 case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY If successful, return other’s item, otherwise someone else took it, so try again from start If successful, return other’s item, otherwise someone else took it, so try again from start

72. Art of Multiprocessor Programming72 case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY If BUSY, other threads exchanging, so start again If BUSY, other threads exchanging, so start again

73. Art of Multiprocessor Programming73 The Exchanger Slot Exchanger is lock-free Because the only way an exchange can fail is if others repeatedly succeeded or no-one showed up The slot we need does not require symmetric exchange

74. Art of Multiprocessor Programming74 public class EliminationArray { … public T visit(T value, int range) throws TimeoutException { int slot = random.nextInt(range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Back to the Stack: the Elimination Array

75. Art of Multiprocessor Programming75 public class EliminationArray { … public T visit(T value, int range) throws TimeoutException { int slot = random.nextInt(range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array visit the elimination array with fixed value and range visit the elimination array with fixed value and range

76. Art of Multiprocessor Programming76 public class EliminationArray { … public T visit(T value, int range) throws TimeoutException { int slot = random.nextInt(range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Pick a random array entry

77. Art of Multiprocessor Programming77 public class EliminationArray { … public T visit(T value, int range) throws TimeoutException { int slot = random.nextInt(range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Exchange value or time out

78. Art of Multiprocessor Programming78 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push

79. Art of Multiprocessor Programming79 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push First, try to push

80. Art of Multiprocessor Programming80 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push If I failed, backoff & try to eliminate

81. Art of Multiprocessor Programming81 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push Value pushed and range to try

82. Art of Multiprocessor Programming82 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push Only pop() leaves null, so elimination was successful Only pop() leaves null, so elimination was successful

83. Art of Multiprocessor Programming83 public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.range); if (otherValue == null) { return; } Elimination Stack Push Otherwise, retry push() on lock-free stack

84. Art of Multiprocessor Programming84 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.range; if (otherValue != null) { return otherValue; } }} Elimination Stack Pop

85. Art of Multiprocessor Programming85 public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.range; if (otherValue != null) { return otherValue; } }} Elimination Stack Pop If value not null, other thread is a push(), so elimination succeeded If value not null, other thread is a push(), so elimination succeeded

86. Art of Multiprocessor Programming86 Summary We saw both lock-based and lock-free implementations of queues and stacks Don’t be quick to declare a data structure inherently sequential –Linearizable stack is not inherently sequential (though it is in worst case) ABA is a real problem, pay attention

87. Art of Multiprocessor Programming87 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.