1. Linked Lists: Locking, Lock- Free, and Beyond … Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2. Art of Multiprocessor Programming© Herlihy-Shavit 2007 2 Last Lecture: Spin-Locks CS Resets lock upon exit spin lock critical section...

3. Art of Multiprocessor Programming© Herlihy-Shavit 2007 3 Today: Concurrent Objects Adding threads should not lower throughput –Contention effects –Mostly fixed by Queue locks Should increase throughput –Not possible if inherently sequential –Surprising things are parallelizable

4. Art of Multiprocessor Programming© Herlihy-Shavit 2007 4 Coarse-Grained Synchronization Each method locks the object –Avoid contention using queue locks –Easy to reason about In simple cases –Standard Java model Synchronized blocks and methods So, are we done?

5. Art of Multiprocessor Programming© Herlihy-Shavit 2007 5 Coarse-Grained Synchronization Sequential bottleneck –Threads “stand in line” Adding more threads –Does not improve throughput –Struggle to keep it from getting worse So why even use a multiprocessor? –Well, some apps inherently parallel …

6. Art of Multiprocessor Programming© Herlihy-Shavit 2007 6 This Lecture Introduce four “patterns” –Bag of tricks … –Methods that work more than once … For highly-concurrent objects Goal: –Concurrent access –More threads, more throughput

7. Art of Multiprocessor Programming© Herlihy-Shavit 2007 7 First: Fine-Grained Synchronization Instead of using a single lock.. Split object into –Independently-synchronized components Methods conflict when they access –The same component … –At the same time

8. Art of Multiprocessor Programming© Herlihy-Shavit 2007 8 Second: Optimistic Synchronization Search without locking … If you find it, lock and check … –OK: we are done –Oops: start over Evaluation –Usually cheaper than locking –Mistakes are expensive

9. Art of Multiprocessor Programming© Herlihy-Shavit 2007 9 Third: Lazy Synchronization Postpone hard work Removing components is tricky –Logical removal Mark component to be deleted –Physical removal Do what needs to be done

10. Art of Multiprocessor Programming© Herlihy-Shavit 2007 10 Fourth: Lock-Free Synchronization Don’t use locks at all –Use compareAndSet() & relatives … Advantages –Robust against asynchrony Disadvantages –Complex –Sometimes high overhead

11. Art of Multiprocessor Programming© Herlihy-Shavit 2007 11 Linked List Illustrate these patterns … Using a list-based Set –Common application –Building block for other apps

12. Art of Multiprocessor Programming© Herlihy-Shavit 2007 12 Set Interface Unordered collection of items No duplicates Methods –add(x) put x in set –remove(x) take x out of set –contains(x) tests if x in set

13. Art of Multiprocessor Programming© Herlihy-Shavit 2007 13 List-Based Sets public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); }

14. Art of Multiprocessor Programming© Herlihy-Shavit 2007 14 List-Based Sets public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Add item to set

15. Art of Multiprocessor Programming© Herlihy-Shavit 2007 15 List-Based Sets public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(Tt x); } Remove item from set

16. Art of Multiprocessor Programming© Herlihy-Shavit 2007 16 List-Based Sets public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Is item in set?

17. Art of Multiprocessor Programming© Herlihy-Shavit 2007 17 List Node public class Node { public T item; public int key; public Node next; }

18. Art of Multiprocessor Programming© Herlihy-Shavit 2007 18 List Node public class Node { public T item; public int key; public Node next; } item of interest

19. Art of Multiprocessor Programming© Herlihy-Shavit 2007 19 List Node public class Node { public T item; public int key; public Node next; } Usually hash code

20. Art of Multiprocessor Programming© Herlihy-Shavit 2007 20 List Node public class Node { public T item; public int key; public Node next; } Reference to next node

21. Art of Multiprocessor Programming© Herlihy-Shavit 2007 21 The List-Based Set abc Sorted with Sentinel nodes (min & max possible keys) -∞ +∞

22. Art of Multiprocessor Programming© Herlihy-Shavit 2007 22 Reasoning about Concurrent Objects Invariant –Property that always holds Established by –True when object is created –Truth preserved by each method Each step of each method

23. Art of Multiprocessor Programming© Herlihy-Shavit 2007 23 Specifically … Invariants preserved by –add() –remove() –contains() Most steps are trivial –Usually one step tricky –Often linearization point

24. Art of Multiprocessor Programming© Herlihy-Shavit 2007 24 Interference Invariants make sense only if –methods considered –are the only modifiers Language encapsulation helps –List nodes not visible outside class

25. Art of Multiprocessor Programming© Herlihy-Shavit 2007 25 Interference Freedom from interference needed even for removed nodes –Some algorithms traverse removed nodes –Careful with malloc() & free() ! Garbage-collection helps here

26. Art of Multiprocessor Programming© Herlihy-Shavit 2007 26 Abstract Data Types Concrete representation Abstract Type –{a, b} ab

27. Art of Multiprocessor Programming© Herlihy-Shavit 2007 27 Abstract Data Types Meaning of rep given by abstraction map –S( ) = {a,b} a b

28. Art of Multiprocessor Programming© Herlihy-Shavit 2007 28 Rep Invariant Which concrete values meaningful? –Sorted? –Duplicates? Rep invariant –Characterizes legal concrete reps –Preserved by methods –Relied on by methods

29. Art of Multiprocessor Programming© Herlihy-Shavit 2007 29 Blame Game Rep invariant is a contract Suppose –add() leaves behind 2 copies of x –remove() removes only 1 Which one is incorrect?

30. Art of Multiprocessor Programming© Herlihy-Shavit 2007 30 Blame Game Suppose –add() leaves behind 2 copies of x –remove() removes only 1 Which one is incorrect? –If rep invariant says no duplicates add() is incorrect –Otherwise remove() is incorrect

31. Art of Multiprocessor Programming© Herlihy-Shavit 2007 31 Rep Invariant (partly) Sentinel nodes –tail reachable from head Sorted No duplicates

32. Art of Multiprocessor Programming© Herlihy-Shavit 2007 32 Abstraction Map S(head) = –{ x | there exists a such that a reachable from head and a.item = x –}

33. Art of Multiprocessor Programming© Herlihy-Shavit 2007 33 Sequential List Based Set a c d a b c Add() Remove()

34. Art of Multiprocessor Programming© Herlihy-Shavit 2007 34 Sequential List Based Set a c d b a b c Add() Remove()

35. Art of Multiprocessor Programming© Herlihy-Shavit 2007 35 Course Grained Locking a b d

36. Art of Multiprocessor Programming© Herlihy-Shavit 2007 36 Course Grained Locking a b d c

37. Art of Multiprocessor Programming© Herlihy-Shavit 2007 37 honk! Course Grained Locking a b d c Simple but hotspot + bottleneck honk!

38. Art of Multiprocessor Programming© Herlihy-Shavit 2007 38 Coarse-Grained Locking Easy, same as synchronized methods –“One lock to rule them all …” Simple, clearly correct –Deserves respect! Works poorly with contention –Queue locks help –But bottleneck still an issue

39. Art of Multiprocessor Programming© Herlihy-Shavit 2007 39 Fine-grained Locking Requires careful thought –“Do not meddle in the affairs of wizards, for they are subtle and quick to anger” Split object into pieces –Each piece has own lock –Methods that work on disjoint pieces need not exclude each other

40. Art of Multiprocessor Programming© Herlihy-Shavit 2007 40 Hand-over-Hand locking abc

41. Art of Multiprocessor Programming© Herlihy-Shavit 2007 41 Hand-over-Hand locking abc

42. Art of Multiprocessor Programming© Herlihy-Shavit 2007 42 Hand-over-Hand locking abc

43. Art of Multiprocessor Programming© Herlihy-Shavit 2007 43 Hand-over-Hand locking abc

44. Art of Multiprocessor Programming© Herlihy-Shavit 2007 44 Hand-over-Hand locking abc

45. Art of Multiprocessor Programming© Herlihy-Shavit 2007 45 Removing a Node abcd remove(b)

46. Art of Multiprocessor Programming© Herlihy-Shavit 2007 46 Removing a Node abcd remove(b)

47. Art of Multiprocessor Programming© Herlihy-Shavit 2007 47 Removing a Node abcd remove(b)

48. Art of Multiprocessor Programming© Herlihy-Shavit 2007 48 Removing a Node abcd remove(b)

49. Art of Multiprocessor Programming© Herlihy-Shavit 2007 49 Removing a Node acd remove(b)

50. Art of Multiprocessor Programming© Herlihy-Shavit 2007 50 Removing a Node abcd remove(c) remove(b)

51. Art of Multiprocessor Programming© Herlihy-Shavit 2007 51 Removing a Node abcd remove(b) remove(c)

52. Art of Multiprocessor Programming© Herlihy-Shavit 2007 52 Removing a Node abcd remove(b) remove(c)

53. Art of Multiprocessor Programming© Herlihy-Shavit 2007 53 Removing a Node abcd remove(b) remove(c)

54. Art of Multiprocessor Programming© Herlihy-Shavit 2007 54 Removing a Node abcd remove(b) remove(c)

55. Art of Multiprocessor Programming© Herlihy-Shavit 2007 55 Removing a Node abcd remove(b) remove(c)

56. Art of Multiprocessor Programming© Herlihy-Shavit 2007 56 Removing a Node abcd remove(b) remove(c)

57. Art of Multiprocessor Programming© Herlihy-Shavit 2007 57 Removing a Node abcd remove(b) remove(c)

58. Art of Multiprocessor Programming© Herlihy-Shavit 2007 58 Uh, Oh acd remove(b) remove(c)

59. Art of Multiprocessor Programming© Herlihy-Shavit 2007 59 Uh, Oh acd Bad news remove(b) remove(c)

60. Art of Multiprocessor Programming© Herlihy-Shavit 2007 60 Problem To delete node b –Swing node a’s next field to c Problem is, –Someone could delete c concurrently ba cbac

61. Art of Multiprocessor Programming© Herlihy-Shavit 2007 61 Insight If a node is locked –No one can delete node’s successor If a thread locks –Node to be deleted –And its predecessor –Then it works

62. Art of Multiprocessor Programming© Herlihy-Shavit 2007 62 Hand-Over-Hand Again abcd remove(b)

63. Art of Multiprocessor Programming© Herlihy-Shavit 2007 63 Hand-Over-Hand Again abcd remove(b)

64. Art of Multiprocessor Programming© Herlihy-Shavit 2007 64 Hand-Over-Hand Again abcd remove(b)

65. Art of Multiprocessor Programming© Herlihy-Shavit 2007 65 Hand-Over-Hand Again abcd remove(b) Found it!

66. Art of Multiprocessor Programming© Herlihy-Shavit 2007 66 Hand-Over-Hand Again abcd remove(b) Found it!

67. Art of Multiprocessor Programming© Herlihy-Shavit 2007 67 Hand-Over-Hand Again acd remove(b)

68. Art of Multiprocessor Programming© Herlihy-Shavit 2007 68 Removing a Node abcd remove(b) remove(c)

69. Art of Multiprocessor Programming© Herlihy-Shavit 2007 69 Removing a Node abcd remove(b) remove(c)

70. Art of Multiprocessor Programming© Herlihy-Shavit 2007 70 Removing a Node abcd remove(b) remove(c)

71. Art of Multiprocessor Programming© Herlihy-Shavit 2007 71 Removing a Node abcd remove(b) remove(c)

72. Art of Multiprocessor Programming© Herlihy-Shavit 2007 72 Removing a Node abcd remove(b) remove(c)

73. Art of Multiprocessor Programming© Herlihy-Shavit 2007 73 Removing a Node abcd remove(b) remove(c)

74. Art of Multiprocessor Programming© Herlihy-Shavit 2007 74 Removing a Node abcd remove(b) remove(c)

75. Art of Multiprocessor Programming© Herlihy-Shavit 2007 75 Removing a Node abcd remove(b) remove(c)

76. Art of Multiprocessor Programming© Herlihy-Shavit 2007 76 Removing a Node abcd remove(b) remove(c)

77. Art of Multiprocessor Programming© Herlihy-Shavit 2007 77 Removing a Node abcd remove(b) remove(c)

78. Art of Multiprocessor Programming© Herlihy-Shavit 2007 78 Removing a Node abd remove(b)

79. Art of Multiprocessor Programming© Herlihy-Shavit 2007 79 Removing a Node abd remove(b)

80. Art of Multiprocessor Programming© Herlihy-Shavit 2007 80 Removing a Node abd remove(b)

81. Art of Multiprocessor Programming© Herlihy-Shavit 2007 81 Removing a Node ad remove(b) remove(c)

82. Art of Multiprocessor Programming© Herlihy-Shavit 2007 82 Removing a Node ad

83. Art of Multiprocessor Programming© Herlihy-Shavit 2007 83 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}

84. Art of Multiprocessor Programming© Herlihy-Shavit 2007 84 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Key used to order node

85. Art of Multiprocessor Programming© Herlihy-Shavit 2007 85 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }} Predecessor and current nodes

86. Art of Multiprocessor Programming© Herlihy-Shavit 2007 86 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Make sure locks released

87. Art of Multiprocessor Programming© Herlihy-Shavit 2007 87 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Everything else

88. Art of Multiprocessor Programming© Herlihy-Shavit 2007 88 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … }

89. Art of Multiprocessor Programming© Herlihy-Shavit 2007 89 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } lock pred == head

90. Art of Multiprocessor Programming© Herlihy-Shavit 2007 90 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Lock current

91. Art of Multiprocessor Programming© Herlihy-Shavit 2007 91 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Traversing list

92. Art of Multiprocessor Programming© Herlihy-Shavit 2007 92 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;

93. Art of Multiprocessor Programming© Herlihy-Shavit 2007 93 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Search key range

94. Art of Multiprocessor Programming© Herlihy-Shavit 2007 94 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; At start of each loop: curr and pred locked

95. Art of Multiprocessor Programming© Herlihy-Shavit 2007 95 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If item found, remove node

96. Art of Multiprocessor Programming© Herlihy-Shavit 2007 96 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If node found, remove it

97. Art of Multiprocessor Programming© Herlihy-Shavit 2007 97 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Unlock predecessor

98. Art of Multiprocessor Programming© Herlihy-Shavit 2007 98 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Only one node locked!

99. Art of Multiprocessor Programming© Herlihy-Shavit 2007 99 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; demote current

100. Art of Multiprocessor Programming© Herlihy-Shavit 2007 100 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Find and lock new current

101. Art of Multiprocessor Programming© Herlihy-Shavit 2007 101 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Lock invariant restored

102. Art of Multiprocessor Programming© Herlihy-Shavit 2007 102 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Otherwise, not present

103. Art of Multiprocessor Programming© Herlihy-Shavit 2007 103 Why does this work? To remove node e –Must lock e –Must lock e’s predecessor Therefore, if you lock a node –It can’t be removed –And neither can its successor

104. Art of Multiprocessor Programming© Herlihy-Shavit 2007 104 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable pred reachable from head curr is pred.next So curr.item is in the set

105. Art of Multiprocessor Programming© Herlihy-Shavit 2007 105 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Linearization point if item is present

106. Art of Multiprocessor Programming© Herlihy-Shavit 2007 106 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Node locked, so no other thread can remove it ….

107. Art of Multiprocessor Programming© Herlihy-Shavit 2007 107 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Item not present

108. Art of Multiprocessor Programming© Herlihy-Shavit 2007 108 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable pred reachable from head curr is pred.next pred.key < key key < curr.key

109. Art of Multiprocessor Programming© Herlihy-Shavit 2007 109 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Linearization point

110. Art of Multiprocessor Programming© Herlihy-Shavit 2007 110 Adding Nodes To add node e –Must lock predecessor –Must lock successor Neither can be deleted –(Is successor lock actually required?)

111. Art of Multiprocessor Programming© Herlihy-Shavit 2007 111 Same Abstraction Map S(head) = –{ x | there exists a such that a reachable from head and a.item = x –}

112. Art of Multiprocessor Programming© Herlihy-Shavit 2007 112 Rep Invariant Easy to check that –tail always reachable from head –Nodes sorted, no duplicates

113. Art of Multiprocessor Programming© Herlihy-Shavit 2007 113 Drawbacks Better than coarse-grained lock –Threads can traverse in parallel Still not ideal –Long chain of acquire/release –Inefficient

114. Art of Multiprocessor Programming© Herlihy-Shavit 2007 114 Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK

115. Art of Multiprocessor Programming© Herlihy-Shavit 2007 115 Optimistic: Traverse without Locking b d e a add(c) Aha!

116. Art of Multiprocessor Programming© Herlihy-Shavit 2007 116 Optimistic: Lock and Load b d e a add(c)

117. Art of Multiprocessor Programming© Herlihy-Shavit 2007 117 What Can Possibly Go Wrong? b d e a add(c)

118. Art of Multiprocessor Programming© Herlihy-Shavit 2007 118 What Can Possibly Go Wrong? b d e a add(c) remove(b )

119. Art of Multiprocessor Programming© Herlihy-Shavit 2007 119 What Can Possibly Go Wrong? b d e a add(c)

120. Art of Multiprocessor Programming© Herlihy-Shavit 2007 120 Validate (1) b d e a add(c) Yes, b still reachable from head

121. Art of Multiprocessor Programming© Herlihy-Shavit 2007 121 What Else Can Go Wrong? b d e a add(c)

122. Art of Multiprocessor Programming© Herlihy-Shavit 2007 122 What Else Can Go Wrong? b d e a add(c) add(b’) b’

123. Art of Multiprocessor Programming© Herlihy-Shavit 2007 123 What Else Can Go Wrong? b d e a add(c) b’

124. Art of Multiprocessor Programming© Herlihy-Shavit 2007 124 Optimistic: Validate(2) b d e a add(c) Yes, b still points to d

125. Art of Multiprocessor Programming© Herlihy-Shavit 2007 125 Optimistic: Linearization Point b d e a add(c) c

126. Art of Multiprocessor Programming© Herlihy-Shavit 2007 126 Same Abstraction Map S(head) = –{ x | there exists a such that a reachable from head and a.item = x –}

127. Art of Multiprocessor Programming© Herlihy-Shavit 2007 127 Invariants Careful: we may traverse deleted nodes But we establish properties by –Validation –After we lock target nodes

128. Art of Multiprocessor Programming© Herlihy-Shavit 2007 128 Correctness If –Nodes b and c both locked –Node b still accessible –Node c still successor to b Then –Neither will be deleted –OK to delete and return true

129. Art of Multiprocessor Programming© Herlihy-Shavit 2007 129 Removing an Absent Node abde remove(c ) Aha!

130. Art of Multiprocessor Programming© Herlihy-Shavit 2007 130 Validate (1) abde Yes, b still reachable from head remove(c )

131. Art of Multiprocessor Programming© Herlihy-Shavit 2007 131 Validate (2) abde remove(c ) Yes, b still points to d

132. Art of Multiprocessor Programming© Herlihy-Shavit 2007 132 OK Computer abde remove(c ) return true

133. Art of Multiprocessor Programming© Herlihy-Shavit 2007 133 Correctness If –Nodes b and d both locked –Node b still accessible –Node d still successor to b Then –Neither will be deleted –No thread can add c after b –OK to return false

134. Art of Multiprocessor Programming© Herlihy-Shavit 2007 134 Validation private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; }

135. Art of Multiprocessor Programming© Herlihy-Shavit 2007 135 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor & current nodes

136. Art of Multiprocessor Programming© Herlihy-Shavit 2007 136 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Begin at the beginning

137. Art of Multiprocessor Programming© Herlihy-Shavit 2007 137 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Search range of keys

138. Art of Multiprocessor Programming© Herlihy-Shavit 2007 138 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor reachable

139. Art of Multiprocessor Programming© Herlihy-Shavit 2007 139 private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Is current node next?

140. Art of Multiprocessor Programming© Herlihy-Shavit 2007 140 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Otherwise move on

141. Art of Multiprocessor Programming© Herlihy-Shavit 2007 141 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor not reachable

142. Art of Multiprocessor Programming© Herlihy-Shavit 2007 142 Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …

143. Art of Multiprocessor Programming© Herlihy-Shavit 2007 143 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Search key

144. Art of Multiprocessor Programming© Herlihy-Shavit 2007 144 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Retry on synchronization conflict

145. Art of Multiprocessor Programming© Herlihy-Shavit 2007 145 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Examine predecessor and current nodes

146. Art of Multiprocessor Programming© Herlihy-Shavit 2007 146 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Search by key

147. Art of Multiprocessor Programming© Herlihy-Shavit 2007 147 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Stop if we find item

148. Art of Multiprocessor Programming© Herlihy-Shavit 2007 148 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Move along

149. Art of Multiprocessor Programming© Herlihy-Shavit 2007 149 On Exit from Loop If item is present –curr holds item –pred just before curr If item is absent –curr has first higher key –pred just before curr Assuming no synchronization problems

150. Art of Multiprocessor Programming© Herlihy-Shavit 2007 150 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

151. Art of Multiprocessor Programming© Herlihy-Shavit 2007 151 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Always unlock

152. Art of Multiprocessor Programming© Herlihy-Shavit 2007 152 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Lock both nodes

153. Art of Multiprocessor Programming© Herlihy-Shavit 2007 153 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Check for synchronization conflicts

154. Art of Multiprocessor Programming© Herlihy-Shavit 2007 154 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method target found, remove node

155. Art of Multiprocessor Programming© Herlihy-Shavit 2007 155 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method target not found

156. Art of Multiprocessor Programming© Herlihy-Shavit 2007 156 Optimistic List Limited hot-spots –Targets of add(), remove(), contains() –No contention on traversals Moreover –Traversals are wait-free –Food for thought …

157. Art of Multiprocessor Programming© Herlihy-Shavit 2007 157 So Far, So Good Much less lock acquisition/release –Performance –Concurrency Problems –Need to traverse list twice –contains() method acquires locks Most common method call

158. Art of Multiprocessor Programming© Herlihy-Shavit 2007 158 Evaluation Optimistic is effective if –cost of scanning twice without locks Less than –cost of scanning once with locks Drawback –contains() acquires locks –90% of calls in many apps

159. Art of Multiprocessor Programming© Herlihy-Shavit 2007 159 Lazy List Like optimistic, except –Scan once –contains(x) never locks … Key insight –Removing nodes causes trouble –Do it “lazily”

160. Art of Multiprocessor Programming© Herlihy-Shavit 2007 160 Lazy List remove() –Scans list (as before) –Locks predecessor & current (as before) Logical delete –Marks current node as removed (new!) Physical delete –Redirects predecessor’s next (as before)

161. Art of Multiprocessor Programming© Herlihy-Shavit 2007 161 Lazy Removal aa b c d

162. Art of Multiprocessor Programming© Herlihy-Shavit 2007 162 Lazy Removal aa b c d Present in list

163. Art of Multiprocessor Programming© Herlihy-Shavit 2007 163 Lazy Removal aa b c d Logically deleted

164. Art of Multiprocessor Programming© Herlihy-Shavit 2007 164 Lazy Removal aa b c d Physically deleted

165. Art of Multiprocessor Programming© Herlihy-Shavit 2007 165 Lazy Removal aa b d Physically deleted

166. Art of Multiprocessor Programming© Herlihy-Shavit 2007 166 Lazy List All Methods –Scan through locked and marked nodes –Removing a node doesn’t slow down other method calls … Must still lock pred and curr nodes.

167. Art of Multiprocessor Programming© Herlihy-Shavit 2007 167 Validation No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr

168. Art of Multiprocessor Programming© Herlihy-Shavit 2007 168 Business as Usual abc

169. Art of Multiprocessor Programming© Herlihy-Shavit 2007 169 Business as Usual abc

170. Art of Multiprocessor Programming© Herlihy-Shavit 2007 170 Business as Usual abc

171. Art of Multiprocessor Programming© Herlihy-Shavit 2007 171 Business as Usual abc remove(b)

172. Art of Multiprocessor Programming© Herlihy-Shavit 2007 172 Business as Usual abc a not marked

173. Art of Multiprocessor Programming© Herlihy-Shavit 2007 173 Business as Usual abc a still points to b

174. Art of Multiprocessor Programming© Herlihy-Shavit 2007 174 Business as Usual a bc Logical delete

175. Art of Multiprocessor Programming© Herlihy-Shavit 2007 175 Business as Usual a bc physical delete

176. Art of Multiprocessor Programming© Herlihy-Shavit 2007 176 Business as Usual a bc

177. Art of Multiprocessor Programming© Herlihy-Shavit 2007 177 New Abstraction Map S(head) = –{ x | there exists node a such that a reachable from head and a.item = x and a is unmarked –}

178. Art of Multiprocessor Programming© Herlihy-Shavit 2007 178 Invariant If not marked then item in the set and reachable from head and if not yet traversed it is reachable from pred

179. Art of Multiprocessor Programming© Herlihy-Shavit 2007 179 Validation private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }

180. Art of Multiprocessor Programming© Herlihy-Shavit 2007 180 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Predecessor not Logically removed

181. Art of Multiprocessor Programming© Herlihy-Shavit 2007 181 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Current not Logically removed

182. Art of Multiprocessor Programming© Herlihy-Shavit 2007 182 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Predecessor still Points to current

183. Art of Multiprocessor Programming© Herlihy-Shavit 2007 183 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

184. Art of Multiprocessor Programming© Herlihy-Shavit 2007 184 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Validate as before

185. Art of Multiprocessor Programming© Herlihy-Shavit 2007 185 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Key found

186. Art of Multiprocessor Programming© Herlihy-Shavit 2007 186 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Logical remove

187. Art of Multiprocessor Programming© Herlihy-Shavit 2007 187 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} physical remove

188. Art of Multiprocessor Programming© Herlihy-Shavit 2007 188 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; }

189. Art of Multiprocessor Programming© Herlihy-Shavit 2007 189 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Start at the head

190. Art of Multiprocessor Programming© Herlihy-Shavit 2007 190 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Search key range

191. Art of Multiprocessor Programming© Herlihy-Shavit 2007 191 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Traverse without locking (nodes may have been removed)

192. Art of Multiprocessor Programming© Herlihy-Shavit 2007 192 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Present and undeleted?

193. Art of Multiprocessor Programming© Herlihy-Shavit 2007 193 Summary: Wait-free Contains a 0 0 0 a b c 0 e 1 d Use Mark bit + Fact that List is ordered 1.Not marked  in the set 2.Marked or missing  not in the set

194. Art of Multiprocessor Programming© Herlihy-Shavit 2007 194 Lazy List a 0 0 0 a b c 0 e 1 d Lazy add() and remove() + Wait-free contains()

195. Art of Multiprocessor Programming© Herlihy-Shavit 2007 195 Evaluation Good: –contains() doesn’t lock –In fact, its wait-free! –Good because typically high % contains() –Uncontended calls don’t re-traverse Bad –Contended calls do re-traverse –Traffic jam if one thread delays

196. Art of Multiprocessor Programming© Herlihy-Shavit 2007 196 Traffic Jam Any concurrent data structure based on mutual exclusion has a weakness If one thread –Enters critical section –And “eats the big muffin” Cache miss, page fault, descheduled … Software error, … –Everyone else using that lock is stuck!

197. Art of Multiprocessor Programming© Herlihy-Shavit 2007 197 Reminder: Lock-Free Data Structures No matter what … –Some thread will complete method call –Even if others halt at malicious times –Weaker than wait-free, yet Implies that –You can’t use locks (why?) –Um, that’s why they call it lock-free

198. Art of Multiprocessor Programming© Herlihy-Shavit 2007 198 Lock-free Lists Next logical step Eliminate locking entirely contains() wait-free and add() and remove() lock-free Use only compareAndSet() What could go wrong?

199. Art of Multiprocessor Programming© Herlihy-Shavit 2007 199 Adding a Node abc

200. Art of Multiprocessor Programming© Herlihy-Shavit 2007 200 Adding a Node abcb

201. Art of Multiprocessor Programming© Herlihy-Shavit 2007 201 Adding a Node abcb CAS

202. Art of Multiprocessor Programming© Herlihy-Shavit 2007 202 Adding a Node abcb

203. Art of Multiprocessor Programming© Herlihy-Shavit 2007 203 Adding a Node abcb

204. Art of Multiprocessor Programming© Herlihy-Shavit 2007 204 Removing a Node abcd remov e b remov e c CAS

205. Art of Multiprocessor Programming© Herlihy-Shavit 2007 205 Bad news Look Familiar? abcd remov e b remov e c

206. Art of Multiprocessor Programming© Herlihy-Shavit 2007 206 Problem Method updates node’s next field After node has been removed

207. Art of Multiprocessor Programming© Herlihy-Shavit 2007 207 Solution Use AtomicMarkableReference Atomically –Swing reference and –Update flag Remove in two steps –Set mark bit in next field –Redirect predecessor’s pointer

208. Art of Multiprocessor Programming© Herlihy-Shavit 2007 208 Marking a Node AtomicMarkableReference class –Java.util.concurrent.atomic package address F mark bit Reference

209. Art of Multiprocessor Programming© Herlihy-Shavit 2007 209 Extracting Reference & Mark Public Object get(boolean[] marked);

210. Art of Multiprocessor Programming© Herlihy-Shavit 2007 210 Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0!

211. Art of Multiprocessor Programming© Herlihy-Shavit 2007 211 Extracting Reference Only public boolean isMarked(); Value of mark

212. Art of Multiprocessor Programming© Herlihy-Shavit 2007 212 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark);

213. Art of Multiprocessor Programming© Herlihy-Shavit 2007 213 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); If this is the current reference … And this is the current mark …

214. Art of Multiprocessor Programming© Herlihy-Shavit 2007 214 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); …then change to this new reference … … and this new mark

215. Art of Multiprocessor Programming© Herlihy-Shavit 2007 215 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark);

216. Art of Multiprocessor Programming© Herlihy-Shavit 2007 216 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); If this is the current reference …

217. Art of Multiprocessor Programming© Herlihy-Shavit 2007 217 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark);.. then change to this new mark.

218. Art of Multiprocessor Programming© Herlihy-Shavit 2007 218 Removing a Node abcd remov e c CAS

219. Art of Multiprocessor Programming© Herlihy-Shavit 2007 219 Removing a Node abd remov e b remov e c c CAS failed

220. Art of Multiprocessor Programming© Herlihy-Shavit 2007 220 Removing a Node abd remov e b remov e c c

221. Art of Multiprocessor Programming© Herlihy-Shavit 2007 221 Removing a Node ad remov e b remov e c

222. Art of Multiprocessor Programming© Herlihy-Shavit 2007 222 Traversing the List Q: what do you do when you find a “logically” deleted node in your path? A: finish the job. –CAS the predecessor’s next field –Proceed (repeat as needed)

223. Art of Multiprocessor Programming© Herlihy-Shavit 2007 223 Lock-Free Traversal abcd CAS Uh-oh

224. Art of Multiprocessor Programming© Herlihy-Shavit 2007 224 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; }

225. Art of Multiprocessor Programming© Herlihy-Shavit 2007 225 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } A container for pred and current values

226. Art of Multiprocessor Programming© Herlihy-Shavit 2007 226 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr;

227. Art of Multiprocessor Programming© Herlihy-Shavit 2007 227 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window

228. Art of Multiprocessor Programming© Herlihy-Shavit 2007 228 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr

229. Art of Multiprocessor Programming© Herlihy-Shavit 2007 229 The Find Method Window window = find(item); At some instant, predcurrsucc item or …

230. Art of Multiprocessor Programming© Herlihy-Shavit 2007 230 The Find Method Window window = find(item); At some instant, pred curr= null succ item not in list

231. Art of Multiprocessor Programming© Herlihy-Shavit 2007 231 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}}

232. Art of Multiprocessor Programming© Herlihy-Shavit 2007 232 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Keep trying

233. Art of Multiprocessor Programming© Herlihy-Shavit 2007 233 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Find neighbors

234. Art of Multiprocessor Programming© Herlihy-Shavit 2007 234 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} She’s not there …

235. Art of Multiprocessor Programming© Herlihy-Shavit 2007 235 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to mark node as deleted

236. Art of Multiprocessor Programming© Herlihy-Shavit 2007 236 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} If it doesn’t work, just retry, if it does, job essentially done

237. Art of Multiprocessor Programming© Herlihy-Shavit 2007 237 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to advance reference (if we don’t succeed, someone else did or will). a

238. Art of Multiprocessor Programming© Herlihy-Shavit 2007 238 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}}

239. Art of Multiprocessor Programming© Herlihy-Shavit 2007 239 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}} Item already there.

240. Art of Multiprocessor Programming© Herlihy-Shavit 2007 240 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}} create new node

241. Art of Multiprocessor Programming© Herlihy-Shavit 2007 241 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}} Install new node, else retry loop

242. Art of Multiprocessor Programming© Herlihy-Shavit 2007 242 Wait-free Contains public boolean contains(Tt item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) }

243. Art of Multiprocessor Programming© Herlihy-Shavit 2007 243 Wait-free Contains public boolean contains(T item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Only diff is that we get and check marked

244. Art of Multiprocessor Programming© Herlihy-Shavit 2007 244 Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }}

245. Art of Multiprocessor Programming© Herlihy-Shavit 2007 245 Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} If list changes while traversed, start over Lock-Free because we start over only if someone else makes progress

246. Art of Multiprocessor Programming© Herlihy-Shavit 2007 246 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Start looking from head

247. Art of Multiprocessor Programming© Herlihy-Shavit 2007 247 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Move down the list

248. Art of Multiprocessor Programming© Herlihy-Shavit 2007 248 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Get ref to successor and current deleted bit

249. Art of Multiprocessor Programming© Herlihy-Shavit 2007 249 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Try to remove deleted nodes in path…code details soon

250. Art of Multiprocessor Programming© Herlihy-Shavit 2007 250 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find If curr key that is greater or equal, return pred and curr

251. Art of Multiprocessor Programming© Herlihy-Shavit 2007 251 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Otherwise advance window and loop again

252. Art of Multiprocessor Programming© Herlihy-Shavit 2007 252 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } …

253. Art of Multiprocessor Programming© Herlihy-Shavit 2007 253 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Try to snip out node

254. Art of Multiprocessor Programming© Herlihy-Shavit 2007 254 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … if predecessor’s next field changed must retry whole traversal

255. Art of Multiprocessor Programming© Herlihy-Shavit 2007 255 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Otherwise move on to check if next node deleted

256. Art of Multiprocessor Programming© Herlihy-Shavit 2007 256 Summary: Lock-free Removal a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS pointer Use CAS to verify pointer is correct Not enough!

257. Art of Multiprocessor Programming© Herlihy-Shavit 2007 257 Lock-free Removal a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Problem: d not added to list… Must Prevent manipulation of removed node’s pointer Node added Before Physical Removal CAS

258. Art of Multiprocessor Programming© Herlihy-Shavit 2007 258 Our Solution: Combine Bit and Pointer a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Mark-Bit and Pointer are CASed together Fail CAS: Node not added after logical Removal

259. Art of Multiprocessor Programming© Herlihy-Shavit 2007 259 A Lock-free Algorithm a 0 0 0 a b c 0 e 1 c 1. add() and remove() physically remove marked nodes 2. Wait-free find() traverses both marked and removed nodes

260. Art of Multiprocessor Programming© Herlihy-Shavit 2007 260 Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls.

261. Art of Multiprocessor Programming© Herlihy-Shavit 2007 261 High Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling

262. Art of Multiprocessor Programming© Herlihy-Shavit 2007 262 Low Contains Ratio

263. Art of Multiprocessor Programming© Herlihy-Shavit 2007 263 As Contains Ratio Increases % Contains()

264. Art of Multiprocessor Programming© Herlihy-Shavit 2007 264 Summary Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization

265. Art of Multiprocessor Programming© Herlihy-Shavit 2007 265 “To Lock or Not to Lock” Locking vs. Non-blocking: Extremist views on both sides The answer: nobler to compromise, combine locking and non-blocking –Example: Lazy list combines blocking add() and remove() and a wait-free contains() –Blocking/non-blocking is a property of a method

266. Art of Multiprocessor Programming© Herlihy-Shavit 2007 266 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.