1. Linked Lists: Optimistic, Lock-Free, … Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Pavol Černý, Programming Paradigms for Concurrency, Fall 2010, IST Austria

2. Art of Multiprocessor Programming2 Set Interface Unordered collection of items No duplicates Methods –add(x) put x in set –remove(x) take x out of set –contains(x) tests if x in set

3. Art of Multiprocessor Programming3 List-Based Sets public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); }

4. Art of Multiprocessor Programming4 List Node public class Node { public T item; public int key; public Node next; }

5. Art of Multiprocessor Programming5 The List-Based Set abc Sorted with Sentinel nodes (min & max possible keys) -∞ +∞

6. Art of Multiprocessor Programming6 Invariants Sentinel nodes –tail reachable from head Sorted No duplicates

7. Art of Multiprocessor Programming7 This Lecture Introduce three “patterns” –Bag of tricks … –Methods that work more than once …

8. Art of Multiprocessor Programming8 This Lecture Introduce three “patterns” –Bag of tricks … –Methods that work more than once … For highly-concurrent objects –Concurrent access –More threads, more throughput

9. Art of Multiprocessor Programming9 First: Optimistic Synchronization Search without locking … If you find it, lock and check … –OK: we are done –Oops: start over Evaluation –Usually cheaper than locking, but –Mistakes are expensive

10. Art of Multiprocessor Programming10 Second: Lazy Synchronization Postpone hard work Removing components is tricky –Logical removal Mark component to be deleted –Physical removal Do what needs to be done

11. Art of Multiprocessor Programming11 Third: Lock-Free Synchronization Don’t use locks at all –Use compareAndSet() & relatives … Advantages –No Scheduler Assumptions/Support Disadvantages –Complex –Sometimes high overhead

12. Art of Multiprocessor Programming12 Drawbacks of fine-grained locking Better than coarse-grained lock –Threads can traverse in parallel Still not ideal –Long chain of acquire/release –Inefficient

13. Art of Multiprocessor Programming13 Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK

14. Art of Multiprocessor Programming14 Optimistic: Traverse without Locking b d e a add(c) Aha!

15. Art of Multiprocessor Programming15 Optimistic: Lock and Load b d e a add(c)

16. Art of Multiprocessor Programming16 Optimistic: Lock and Load b d e a add(c) c

17. Art of Multiprocessor Programming17 What could go wrong? b d e a add(c) Aha!

18. Art of Multiprocessor Programming18 What could go wrong? b d e a add(c)

19. Art of Multiprocessor Programming19 What could go wrong? b d e a remove(b )

20. Art of Multiprocessor Programming20 What could go wrong? b d e a remove(b )

21. Art of Multiprocessor Programming21 What could go wrong? b d e a add(c)

22. Art of Multiprocessor Programming22 What could go wrong? b d e a add(c) c

23. Art of Multiprocessor Programming23 What could go wrong? d e a add(c) Uh-oh

24. Art of Multiprocessor Programming24 Validate – Part 1 b d e a add(c) Yes, b still reachable from head

25. Art of Multiprocessor Programming25 What Else Could Go Wrong? b d e a add(c) Aha!

26. Art of Multiprocessor Programming26 What Else Coould Go Wrong? b d e a add(c) add(b’)

27. Art of Multiprocessor Programming27 What Else Coould Go Wrong? b d e a add(c) add(b’) b’

28. Art of Multiprocessor Programming28 What Else Could Go Wrong? b d e a add(c) b’

29. Art of Multiprocessor Programming29 What Else Could Go Wrong? b d e a add(c) c

30. Art of Multiprocessor Programming30 Validate Part 2 (while holding locks) b d e a add(c) Yes, b still points to d

31. Art of Multiprocessor Programming31 Optimistic: Linearization Point b d e a add(c) c

32. Art of Multiprocessor Programming32 Correctness If –Nodes b and c both locked –Node b still accessible –Node c still successor to b Then –Neither will be deleted –OK to delete and return true

33. Art of Multiprocessor Programming33 Unsuccessful Remove abde remove(c) Aha!

34. Art of Multiprocessor Programming34 Validate (1) abde Yes, b still reachable from head remove(c)

35. Art of Multiprocessor Programming35 Validate (2) abde remove(c) Yes, b still points to d

36. Art of Multiprocessor Programming36 OK Computer abde remove(c) return false

37. Art of Multiprocessor Programming37 Correctness If –Nodes b and d both locked –Node b still accessible –Node d still successor to b Then –Neither will be deleted –No thread can add c after b –OK to return false

38. Art of Multiprocessor Programming38 Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; }

39. Art of Multiprocessor Programming39 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor & current nodes

40. Art of Multiprocessor Programming40 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Begin at the beginning

41. Art of Multiprocessor Programming41 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Search range of keys

42. Art of Multiprocessor Programming42 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor reachable

43. Art of Multiprocessor Programming43 private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Is current node next?

44. Art of Multiprocessor Programming44 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Otherwise move on

45. Art of Multiprocessor Programming45 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor not reachable

46. Art of Multiprocessor Programming46 Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } …

47. Art of Multiprocessor Programming47 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Search key

48. Art of Multiprocessor Programming48 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Retry on synchronization conflict

49. Art of Multiprocessor Programming49 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Examine predecessor and current nodes

50. Art of Multiprocessor Programming50 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Search by key

51. Art of Multiprocessor Programming51 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Stop if we find item

52. Art of Multiprocessor Programming52 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Remove: searching Move along

53. Art of Multiprocessor Programming53 On Exit from Loop If item is present –curr holds item –pred just before curr If item is absent –curr has first higher key –pred just before curr Assuming no synchronization problems

54. Art of Multiprocessor Programming54 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

55. Art of Multiprocessor Programming55 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Always unlock

56. Art of Multiprocessor Programming56 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Lock both nodes

57. Art of Multiprocessor Programming57 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Check for synchronization conflicts

58. Art of Multiprocessor Programming58 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method target found, remove node

59. Art of Multiprocessor Programming59 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method target not found

60. Art of Multiprocessor Programming60 Optimistic List Limited hot-spots –Targets of add(), remove(), contains() –No contention on traversals Moreover –Traversals are wait-free –Food for thought …

61. Art of Multiprocessor Programming61 So Far, So Good Much less lock acquisition/release –Performance –Concurrency Problems –Need to traverse list twice –contains() method acquires locks

62. Art of Multiprocessor Programming62 Evaluation Optimistic is effective if –cost of scanning twice without locks is less than –cost of scanning once with locks Drawback –contains() acquires locks –90% of calls in many apps

63. Art of Multiprocessor Programming63 Lazy List Like optimistic, except –Scan once –contains(x) never locks … Key insight –Removing nodes causes trouble –Do it “lazily”

64. Art of Multiprocessor Programming64 Lazy List remove() –Scans list (as before) –Locks predecessor & current (as before) Logical delete –Marks current node as removed (new!) Physical delete –Redirects predecessor’s next (as before)

65. Art of Multiprocessor Programming65 Lazy Removal aa b c d

66. c Art of Multiprocessor Programming66 Lazy Removal aa b d Present in list

67. c Art of Multiprocessor Programming67 Lazy Removal aa b d Logically deleted

68. Art of Multiprocessor Programming68 Lazy Removal aa b c d Physically deleted

69. Art of Multiprocessor Programming69 Lazy Removal aa b d Physically deleted

70. Art of Multiprocessor Programming70 Lazy List All Methods –Scan through locked and marked nodes –Removing a node doesn’t slow down other method calls … Must still lock pred and curr nodes.

71. Art of Multiprocessor Programming71 Validation No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr

72. Art of Multiprocessor Programming72 Business as Usual abc

73. Art of Multiprocessor Programming73 Business as Usual abc

74. Art of Multiprocessor Programming74 Business as Usual abc

75. Art of Multiprocessor Programming75 Business as Usual abc remove(b)

76. Art of Multiprocessor Programming76 Business as Usual abc a not marked

77. Art of Multiprocessor Programming77 Business as Usual abc a still points to b

78. Art of Multiprocessor Programming78 Business as Usual a bc Logical delete

79. Art of Multiprocessor Programming79 Business as Usual a bc physical delete

80. Art of Multiprocessor Programming80 Business as Usual a bc

81. Art of Multiprocessor Programming81 Validation private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); }

82. Art of Multiprocessor Programming82 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Predecessor not Logically removed

83. Art of Multiprocessor Programming83 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Current not Logically removed

84. Art of Multiprocessor Programming84 private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Predecessor still Points to current

85. Art of Multiprocessor Programming85 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

86. Art of Multiprocessor Programming86 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Validate as before

87. Art of Multiprocessor Programming87 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Key found

88. Art of Multiprocessor Programming88 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Logical remove

89. Art of Multiprocessor Programming89 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} physical remove

90. Art of Multiprocessor Programming90 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; }

91. Art of Multiprocessor Programming91 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Start at the head

92. Art of Multiprocessor Programming92 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Search key range

93. Art of Multiprocessor Programming93 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Traverse without locking (nodes may have been removed)

94. Art of Multiprocessor Programming94 Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Present and undeleted?

95. Art of Multiprocessor Programming95 Summary: Wait-free Contains a 0 0 0 a b c 0 e 1 d Use Mark bit + list ordering 1.Not marked  in the set 2.Marked or missing  not in the set

96. Art of Multiprocessor Programming96 Lazy List a 0 0 0 a b c 0 e 1 d Lazy add() and remove() + Wait-free contains()

97. Art of Multiprocessor Programming97 Evaluation Good: –contains() doesn’t lock –In fact, its wait-free! –Good because typically high % contains() –Uncontended calls don’t re-traverse Bad –Contended add() and remove() calls do re-traverse –Traffic jam if one thread delays

98. Art of Multiprocessor Programming98 Traffic Jam Any concurrent data structure based on mutual exclusion has a weakness If one thread –Enters critical section –And “eats the big muffin” Cache miss, page fault, descheduled … –Everyone else using that lock is stuck! –Need to trust the scheduler….

99. Art of Multiprocessor Programming99 Reminder: Lock-Free Data Structures No matter what … –Guarantees minimal progress in any execution –i.e. Some thread will always complete a method call –Even if others halt at malicious times –Implies that implementation can’t use locks

100. Art of Multiprocessor Programming100 Lock-free Lists Next logical step –Wait-free contains() –lock-free add() and remove() Use only compareAndSet() –What could go wrong?

101. Art of Multiprocessor Programming101 Remove Using CAS a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS pointer Use CAS to verify pointer is correct Not enough!

102. Art of Multiprocessor Programming102 Problem… a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Problem: d not added to list… Must Prevent manipulation of removed node’s pointer Node added Before Physical Removal CAS

103. Art of Multiprocessor Programming103 The Solution: Combine Bit and Pointer a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Mark-Bit and Pointer are CASed together (AtomicMarkableReference) Fail CAS: Node not added after logical Removal

104. Art of Multiprocessor Programming104 Solution Use AtomicMarkableReference Atomically –Swing reference and –Update flag Remove in two steps –Set mark bit in next field –Redirect predecessor’s pointer

105. Art of Multiprocessor Programming105 Marking a Node AtomicMarkableReference class –Java.util.concurrent.atomic package address F mark bit Reference

106. Art of Multiprocessor Programming106 Extracting Reference & Mark Public Object get(boolean[] marked);

107. Art of Multiprocessor Programming107 Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0!

108. Art of Multiprocessor Programming108 Extracting Mark Only public boolean isMarked(); Value of mark

109. Art of Multiprocessor Programming109 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark);

110. Art of Multiprocessor Programming110 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); If this is the current reference … And this is the current mark …

111. Art of Multiprocessor Programming111 Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); …then change to this new reference … … and this new mark

112. Art of Multiprocessor Programming112 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark);

113. Art of Multiprocessor Programming113 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); If this is the current reference …

114. Art of Multiprocessor Programming114 Changing State public boolean attemptMark( Object expectedRef, boolean updateMark);.. then change to this new mark.

115. Art of Multiprocessor Programming115 Removing a Node abcd remov e c CAS

116. Art of Multiprocessor Programming116 Removing a Node abd remov e b remov e c c CAS failed

117. Art of Multiprocessor Programming117 Removing a Node abd remov e b remov e c c

118. Art of Multiprocessor Programming118 Removing a Node ad remov e b remov e c

119. Art of Multiprocessor Programming119 Traversing the List Q: what do you do when you find a “logically” deleted node in your path? A: finish the job. –CAS the predecessor’s next field –Proceed (repeat as needed)

120. Art of Multiprocessor Programming120 Lock-Free Traversal (only Add and Remove) abcd CAS Uh-oh pred curr pred curr

121. Art of Multiprocessor Programming121 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; }

122. Art of Multiprocessor Programming122 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } A container for pred and current values

123. Art of Multiprocessor Programming123 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr;

124. Art of Multiprocessor Programming124 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window

125. Art of Multiprocessor Programming125 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr

126. Art of Multiprocessor Programming© Herlihy-Shavit 2007 126 The Find Method Window window = find(item); At some instant, predcurrsucc item or …

127. Art of Multiprocessor Programming© Herlihy-Shavit 2007 127 The Find Method Window window = find(item); At some instant, pred curr= null succ item not in list

128. Art of Multiprocessor Programming128 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}}

129. Art of Multiprocessor Programming129 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Keep trying

130. Art of Multiprocessor Programming130 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Find neighbors

131. Art of Multiprocessor Programming131 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} She’s not there …

132. Art of Multiprocessor Programming132 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to mark node as deleted

133. Art of Multiprocessor Programming133 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} If it doesn’t work, just retry, if it does, job essentially done

134. Art of Multiprocessor Programming134 Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to advance reference (if we don’t succeed, someone else did or will). a

135. Art of Multiprocessor Programming135 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}}

136. Art of Multiprocessor Programming136 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Item already there.

137. Art of Multiprocessor Programming137 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} create new node

138. Art of Multiprocessor Programming138 Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Install new node, else retry loop

139. Art of Multiprocessor Programming139 Wait-free Contains public boolean contains(Tt item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) }

140. Art of Multiprocessor Programming140 Wait-free Contains public boolean contains(T item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Only diff is that we get and check marked

141. Art of Multiprocessor Programming141 Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }}

142. Art of Multiprocessor Programming142 Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} If list changes while traversed, start over Lock-Free because we start over only if someone else makes progress

143. Art of Multiprocessor Programming143 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Start looking from head

144. Art of Multiprocessor Programming144 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Move down the list

145. Art of Multiprocessor Programming145 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Get ref to successor and current deleted bit

146. Art of Multiprocessor Programming146 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Try to remove deleted nodes in path…code details soon

147. Art of Multiprocessor Programming147 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find If curr key that is greater or equal, return pred and curr

148. Art of Multiprocessor Programming148 public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Otherwise advance window and loop again

149. Art of Multiprocessor Programming149 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } …

150. Art of Multiprocessor Programming150 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Try to snip out node

151. Art of Multiprocessor Programming151 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … if predecessor’s next field changed must retry whole traversal

152. Art of Multiprocessor Programming152 Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Otherwise move on to check if next node deleted

153. Art of Multiprocessor Programming153 Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls.

154. Art of Multiprocessor Programming154 High Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling

155. Art of Multiprocessor Programming155 Low Contains Ratio

156. Art of Multiprocessor Programming156 As Contains Ratio Increases % Contains()

157. Art of Multiprocessor Programming157 Summary Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization

158. Art of Multiprocessor Programming158 Homework

159. Projects Topic: good: choose from among our suggestions better: define your own project On your own or in groups of two Pick a project by: before Christmas Progress report 1: January 15 th (2 pages) Presentation and final report: January 27 th and February 3 rd (final report: 4 pages)

160. Project 1: Irregular data parallelism Cavity Example: Delaunay mesh refinement Effects of updates are local, but not bounded statically (“irregular”). Can we still exploit locality for parallelism?

161. Project 1: Irregular data parallelism http://iss.ices.utexas.edu/lonestar/index.html Locality of effects: Mesh retriangulation

162. Project 1: Irregular data parallelism Lonestar benchmark suite: http://iss.ices.utexas.edu/lonestar/index.html 1.Barnes-Hut N-body Simulation 2.Delaunay Mesh refinement 3.Focused communities 4.Delaunay triangulation 5.… Project: 1. pick one of these applications, 2. find a good (possibly novel) way of parallelizing it, 3. implement it (by modifying the sequential implementation provided in Lonestar benchmarks), 4. confirm improvement in running time by experimentation.

163. Project 2: Deductive verification: proving a concurrent data structure correct 1.Pick an implementation of a concurrent data structure: a stack, a queue, a set,.. 2.Pick a theorem prover or a verification tool: for example: PVS or QED 3.Prove that the implementation is linearizable 3 5 7 9 P1: remove(7) P2: remove(5)

164. Project 2: Deductive verification: proving a concurrent data structure correct References: 1.R. Colvin, L. Groves, V. Luchangco, M. Moir: Formal Verification of a Lazy Concurrent List-Based Set Algorithm. CAV 2006 2.T. Elmas, S. Qadeer, A. Sezgin, O. Subasi, S. Tasiran: Simplifying Linearizability Proofs with Reduction and Abstraction. TACAS 2010. 3 5 7 9 P1: remove(7) P2: remove(5)

165. Project 3:Performance measurement/ performance model for concurrent programs 1.Pick a problem with at least three-four different solutions 1.Lock implementations 2.Data structures: queues, stacks, sets… 2.Examine the performance of the solutions in different settings: 1.small number of threads vs large number of threads 2.2 cores, small amount of memory (laptop) vs 8 cores, large memory/cache (server) 3.different usage models 4.input that generates little vs input that generates lots of contention 3a. Find a hybrid solution that works well in a particular setting or 3b. Find a performance model that explains the data

166. Project 4: Your own!

167. Art of Multiprocessor Programming167 Summary Mutual Exclusion Algorithms (Peterson, Filter, Bakery), Lower Bounds (Covering proof) Correcteness conditions: 1. Linearizability, 2. Sequential consistency Proving Linearizability: 1. Rely-Guarantee, 2. Reductions (left movers, right movers) Case study: Concurrent Lists (coarse-grained, fine-grained, optimistic, lazy, lock-free implementations)