1. Barrier Synchronization Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2. Art of Multiprocessor Programming 2 Simple Video Game Prepare frame for display –By graphics coprocessor “soft real-time” application –Need at least 35 frames/second –OK to mess up rarely

3. Art of Multiprocessor Programming 3 Simple Video Game while (true) { frame.prepare(); frame.display(); }

4. Art of Multiprocessor Programming 4 Simple Video Game while (true) { frame.prepare(); frame.display(); } What about overlapping work? –1 st thread displays frame –2 nd prepares next frame

5. Art of Multiprocessor Programming 5 Two-Phase Rendering while (true) { if (phase) { frame[0].display(); } else { frame[1].display(); } phase = !phase; } while (true) { if (phase) { frame[1].prepare(); } else { frame[0].prepare(); } phase = !phase; }

6. Art of Multiprocessor Programming 6 Two-Phase Rendering while (true) { if (phase) { frame[0].display(); } else { frame[1].display(); } phase = !phase; } while (true) { if (phase) { frame[1].prepare(); } else { frame[0].prepare(); } phase = !phase; } even phases

7. Art of Multiprocessor Programming 7 Two-Phase Rendering while (true) { if (phase) { frame[0].display(); } else { frame[1].display(); } phase = !phase; } while (true) { if (phase) { frame[1].prepare(); } else { frame[0].prepare(); } phase = !phase; } odd phases

8. Art of Multiprocessor Programming 8 Synchronization Problems How do threads stay in phase? Too early? –“we render no frame before its time” Too late? –Recycle memory before frame is displayed

9. Art of Multiprocessor Programming 9 Ideal Parallel Computation 0 0 0 1 1 1

10. Art of Multiprocessor Programming 10 Ideal Parallel Computation 2 2 2 1 1 1

11. Art of Multiprocessor Programming 11 Real-Life Parallel Computation 0 0 0 1 1 zzz…

12. Art of Multiprocessor Programming 12 Real-Life Parallel Computation 2 1 1 zzz… Uh, oh 0

13. Art of Multiprocessor Programming 13 Barrier Synchronization 0 0 0 barrier

14. Art of Multiprocessor Programming 14 Barrier Synchronization 1 1 1 barrier

15. Art of Multiprocessor Programming 15 Barrier Synchronization No thread enters here Until every thread has left here barrier

16. Art of Multiprocessor Programming 16 Why Do We Care? Mostly of interest to –Scientific & numeric computation Elsewhere –Garbage collection –Less common in systems programming –Still important topic

17. Art of Multiprocessor Programming 17 Duality Dual to mutual exclusion –Include others, not exclude them Same implementation issues –Interaction with caches … Invalidation? Local spinning?

18. Art of Multiprocessor Programming 18 Example: Parallel Prefix abcd aa+ba+b+c +d before after

19. Art of Multiprocessor Programming 19 Parallel Prefix abcd One thread Per entry

20. Art of Multiprocessor Programming 20 Parallel Prefix: Phase 1 abcd aa+bb+cc+d

21. Art of Multiprocessor Programming 21 Parallel Prefix: Phase 2 abcd aa+ba+b+c +d

22. Art of Multiprocessor Programming 22 Parallel Prefix N threads can compute –Parallel prefix –Of N entries –In log 2 N rounds What if system is asynchronous? –Why we need barriers

23. Art of Multiprocessor Programming 23 Prefix class Prefix extends Thread { int[] a; int i; Barrier b; void Prefix(int[] a, Barrier b, int i) { a = a; b = b; i = i; }

24. class Prefix extends Thread { int[] a; int i; Barrier b; void Prefix(int[] a, Barrier b, int i) { a = a; b = b; i = i; } Art of Multiprocessor Programming 24 Prefix Array of input values

25. class Prefix extends Thread { int[] a; int i; Barrier b; void Prefix(int[] a, Barrier b, int i) { a = a; b = b; i = i; } Art of Multiprocessor Programming 25 Prefix Thread index

26. class Prefix extends Thread { int[] a; int i; Barrier b; void Prefix(int[] a, Barrier b, int i) { a = a; b = b; i = i; } Art of Multiprocessor Programming 26 Prefix Shared barrier

27. class Prefix extends Thread { int[] a; int i; Barrier b; void Prefix(int[] a, Barrier b, int i) { a = a; b = b; i = i; } Art of Multiprocessor Programming 27 Prefix Initialize fields

28. Art of Multiprocessor Programming 28 Where Do the Barriers Go? public void run() { int d = 1, sum = 0; while (d < N) { if (i >= d) sum = a[i-d]; if (i >= d) a[i] += sum; d = d * 2; }}}

29. Art of Multiprocessor Programming 29 Where Do the Barriers Go? public void run() { int d = 1, sum = 0; while (d < N) { if (i >= d) sum = a[i-d]; b.await(); if (i >= d) a[i] += sum; d = d * 2; }}} Make sure everyone reads before anyone writes

30. Art of Multiprocessor Programming 30 Where Do the Barriers Go? public void run() { int d = 1, sum = 0; while (d < N) { if (i >= d) sum = a[i-d]; b.await(); if (i >= d) a[i] += sum; b.await(); d = d * 2; }}} Make sure everyone writes before anyone reads Make sure everyone reads before anyone writes

31. Art of Multiprocessor Programming 31 Barrier Implementations Cache coherence –Spin on locally-cached locations? –Spin on statically-defined locations? Latency –How many steps? Symmetry –Do all threads do the same thing?

32. Art of Multiprocessor Programming 32 Barriers public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}}

33. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 33 Barriers Number of threads not yet arrived

34. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 34 Barriers Number of threads participating

35. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 35 Barriers Initialization

36. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 36 Barriers Principal method

37. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 37 Barriers If I’m last, reset fields for next time

38. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 38 Barriers Otherwise, wait for everyone else

39. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 39 Barriers What’s wrong with this protocol?

40. Art of Multiprocessor Programming 40 Reuse Barrier b = new Barrier(n); while ( mumble() ) { work(); b.await() } do work synchronize repeat

41. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 41 Barriers

42. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 42 Barriers Waiting for Phase 1 to finish

43. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 43 Barriers Waiting for Phase 1 to finish Phase 1 is so over

44. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming 44 Barriers ZZZZZ…. Prepare for phase 2

45. public class Barrier { AtomicInteger count; int size; public Barrier(int n){ count = AtomicInteger(n); size = n; } public void await() { if (count.getAndDecrement()==1) { count.set(size); } else { while (count.get() != 0); }}}} Art of Multiprocessor Programming Uh-Oh Waiting for Phase 1 to finish Waiting for Phase 2 to finish

46. Art of Multiprocessor Programming 46 Basic Problem One thread “wraps around” to start phase 2 While another thread is still waiting for phase 1 One solution: –Always use two barriers

47. Art of Multiprocessor Programming 47 Sense-Reversing Barriers public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}}

48. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 48 Sense-Reversing Barriers Completed odd or even-numbered phase?

49. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 49 Sense-Reversing Barriers Store sense for next phase

50. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 50 Sense-Reversing Barriers Get new sense determined by last phase

51. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 51 Sense-Reversing Barriers If I’m last, reverse sense for next time

52. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 52 Sense-Reversing Barriers Otherwise, wait for sense to flip

53. public class Barrier { AtomicInteger count; int size; volatile boolean sense = false; threadSense = new ThreadLocal … public void await { boolean mySense = threadSense.get(); if (count.getAndDecrement()==1) { count.set(size); sense = mySense } else { while (sense != mySense) {} } threadSense.set(!mySense)}}} Art of Multiprocessor Programming 53 Sense-Reversing Barriers Prepare sense for next phase

54. Art of Multiprocessor Programming 54 2-barrier Combining Tree Barriers

55. Art of Multiprocessor Programming 55 2-barrier Combining Tree Barriers

56. Art of Multiprocessor Programming 56 Combining Tree Barrier public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}}

57. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 57 Combining Tree Barrier Parent barrier in tree

58. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 58 Combining Tree Barrier Am I last?

59. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await(); count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 59 Combining Tree Barrier Proceed to parent barrier

60. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 60 Combining Tree Barrier Prepare for next phase

61. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 61 Combining Tree Barrier Notify others at this node

62. public class Node{ AtomicInteger count; int size; Node parent; volatile boolean sense; public void await() {… if (count.getAndDecrement()==1) { if (parent != null) { parent.await() count.set(size); sense = mySense } else { while (sense != mySense) {} }…}}} Art of Multiprocessor Programming 62 Combining Tree Barrier I’m not last, so wait for notification

63. Art of Multiprocessor Programming 63 Combining Tree Barrier No sequential bottleneck –Parallel getAndDecrement() calls Low memory contention –Same reason Cache behavior –Local spinning on bus-based architecture –Not so good for NUMA

64. Art of Multiprocessor Programming 64 Remarks Everyone spins on sense field –Local spinning on bus-based (good) –Network hot-spot on distributed architecture (bad) Not really scalable

65. Art of Multiprocessor Programming 65 Tournament Tree Barrier If tree nodes have fan-in 2 –Don’t need to call getAndDecrement() –Winner chosen statically At level i –If i-th bit of id is 0, move up –Otherwise keep back

66. Art of Multiprocessor Programming 66 Tournament Tree Barriers winnerloser winnerloser winnerloser root

67. Art of Multiprocessor Programming 67 Tournament Tree Barriers All flags blue

68. Art of Multiprocessor Programming 68 Tournament Tree Barriers Loser thread sets winner’s flag

69. Art of Multiprocessor Programming 69 Tournament Tree Barriers Loser spins on own flag

70. Art of Multiprocessor Programming 70 Tournament Tree Barriers Winner spins on own flag

71. Art of Multiprocessor Programming 71 Tournament Tree Barriers Winner sees own flag, moves up, spins

72. Art of Multiprocessor Programming 72 Tournament Tree Barriers Bingo!

73. Art of Multiprocessor Programming 73 Tournament Tree Barriers Sense-reversing: next time use blue flags

74. Art of Multiprocessor Programming 74 Tournament Barrier class TBarrier { volatile boolean flag; TBarrier partner; TBarrier parent; boolean top; … }

75. Art of Multiprocessor Programming 75 Tournament Barrier class TBarrier { volatile boolean flag; TBarrier partner; TBarrier parent; boolean top; … } Notifications delivered here

76. Art of Multiprocessor Programming 76 Tournament Barrier class TBarrier { volatile boolean flag; TBarrier partner; TBarrier parent; boolean top; … } Other thead at same level

77. Art of Multiprocessor Programming 77 Tournament Barrier class TBarrier { volatile boolean flag; TBarrier partner; TBarrier parent; boolean top; … } Parent (winner) or null (loser)

78. Art of Multiprocessor Programming 78 Tournament Barrier class TBarrier { volatile boolean flag; TBarrier partner; TBarrier parent; boolean top; … } Am I the root?

79. Art of Multiprocessor Programming 79 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}}

80. Art of Multiprocessor Programming 80 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Le root, c’est moi Current sense

81. Art of Multiprocessor Programming 81 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} I am already a winner

82. Art of Multiprocessor Programming 82 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Wait for partner

83. Art of Multiprocessor Programming 83 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Synchronize upstairs

84. Art of Multiprocessor Programming 84 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Inform partner

85. Art of Multiprocessor Programming 85 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Inform partner Order is important (why?)

86. Art of Multiprocessor Programming 86 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Natural-born loser

87. Art of Multiprocessor Programming 87 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Tell partner I’m here

88. Art of Multiprocessor Programming 88 Tournament Barrier void await(boolean mySense) { if (top) { return; } else if (parent != null) { while (flag != mySense) {}; parent.await(mySense); partner.flag = mySense; } else { partner.flag = mySense; while (flag != mySense) {}; }}} Wait for notification from partner

89. Art of Multiprocessor Programming 89 Remarks No need for read-modify-write calls Each thread spins on fixed location –Good for bus-based architectures –Good for NUMA architectures

90. Art of Multiprocessor Programming 90 Dissemination Barrier At round i –Thread A notifies thread A+2 i (mod n) Requires log n rounds

91. Art of Multiprocessor Programming 91 Dissemination Barrier +1 +2+4

92. Art of Multiprocessor Programming 92 Remarks Elegant Good source of homework problems Not cache-friendly

93. Art of Multiprocessor Programming 93 Ideas So Far Sense-reversing –Reuse without reinitializing Combining tree –Like counters, locks … Tournament tree –Optimized combining tree Dissemination barrier –Intellectually Pleasing (matter of taste)

94. Art of Multiprocessor Programming 94 Which is best for Multicore? On a cache coherent multicore chip: perhaps none of the above… Here is another (arguably) better algorithm …

95. Art of Multiprocessor Programming 95 Static Tree Barrier One node per thread, statically assigned

96. Art of Multiprocessor Programming 96 Static Tree Barrier Sense-reversing flag

97. Art of Multiprocessor Programming 97 2 0 Static Tree Barrier 2 0 0 Node has count of missing children

98. Art of Multiprocessor Programming 98 2 0 Static Tree Barrier 2 0 0 Spin until zero …

99. 21 Art of Multiprocessor Programming 99 2 0 Static Tree Barrier 0 0 My counter is zero, decrement parent

100. 21 Art of Multiprocessor Programming 100 2 0 Static Tree Barrier 0 0 Spin on done flag

101. 2 1 21 Art of Multiprocessor Programming 101 0 Static Tree Barrier 0 0

102. 2 1 21 Art of Multiprocessor Programming 102 0 Static Tree Barrier 0 0

103. 2 1 21 Art of Multiprocessor Programming 103 0 Static Tree Barrier 0 0

104. 10 2 1 Art of Multiprocessor Programming 104 0 Static Tree Barrier 0 0

105. 10 2 1 Art of Multiprocessor Programming 105 0 Static Tree Barrier 0 0

106. 1 0 10 Art of Multiprocessor Programming 106 0 Static Tree Barrier 0 0 yowzah!

107. 1 0 10 Art of Multiprocessor Programming 107 0 Static Tree Barrier 0 0 yowzah!

108. 1 0 10 Art of Multiprocessor Programming 108 0 Static Tree Barrier 0 0 yowzah!

109. 1 0 10 Art of Multiprocessor Programming 109 0 Static Tree Barrier 0 0 yes!

110. 1 2 12 Art of Multiprocessor Programming 110 0 Static Tree Barrier 0 0

111. Art of Multiprocessor Programming 111 Remarks Very little cache traffic Minimal space overhead On message-passing architecture –Send notification & sense down tree

112. The Nature of Progress* Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

113. Concurrent Programming Many real-word data structures –blocking (lock-based) implementations & –non-blocking (no locks) implementations For example: –linked lists, queues, stacks, hash maps,… 113 Does this make sense?

114. Concurrent Programming Many data structures combine blocking & non-blocking methods Java™ concurrency package –skiplists, hash tables, exchangers –on 10 million desktops. 114 Can seemingly contradictory conditions co-exist in same alg?…

115. Progress Conditions Deadlock-free: –Some thread eventually acquires lock. Starvation-free: –Every thread eventually acquires lock. Lock-free: –Some method call returns. Wait-free: –Every method call returns. Obstruction-free: –Every method call returns if it executes in isolation 115 We will show an example shortly

116. 116 List-Based Sets Unordered collection of elements No duplicates Methods –Add() a new element –Remove() an element –Contains() if element is present

117. 117 Coarse Grained Locking a b d Lock is starvation-free: every attempt to acquire the lock eventually succeeds. c

118. 118 Fine Grained (Lock Coupling) a b d Overlapping locks detect overlapping operations c Deadlock-free: some thread eventually acquires lock.

119. 119 Optimistic Fine Grained b c e a add(), remove(), contains() lock destination nodes in order Deadlock-free: some thread trying to acquire the locks eventually succeeds.

120. 120 Obstruction-free contains() b c d a Obstruction-free: the method returns if it executes in isolation for long enough. Snapshot: if all nodes traversed twice are the same

121. 121 The Simple Snapshot is Obstruction-Free Put increasing labels on each entry Collect twice If both agree, –We’re done Otherwise, –Try again 1 22 1 7 13 18 12 = Collect2 Collect1 1 22 1 7 13 18 12

122. Obstruction-freedom In the simple snapshot alg: –The update method is wait-free –But scan is obstruction-free Completes if it executes in isolation (no concurrent updates). 122

123. 123 Wait-free contains() a 0 0 0 a b c 0 e 1 d Use mark bit + list ordering 1.Not marked  in the set 2.Marked or missing  not in the set

124. 124 Lazy List-based Set Alg a 0 0 0 a b c 0 e 1 d Combine blocking and non-blocking: deadlock-free add () and remove() and wait-free contains()

125. 125 Lock-free List-Based Set a 0 0 0 a b c 0 e 1 c Logical Removal = Set Mark Bit 0 d mark and reference CASed together Lock-free add () and remove() and wait-free contains() CAS will fail

126. So how can this make sense? Why have methods with different progress conditions? Let us try to understand this… Art of Multiprocessor Programming© Copyright Herlihy- Shavit 2007 126

127. Progress Conditions Deadlock-free: –Some thread eventually acquires lock. Starvation-free: –Every thread eventually acquires lock. Lock-free: –Some method call returns. Wait-free: –Every method call returns. Obstruction-free: –Every method call returns if it executes in isolation 127

128. A “Periodic Table” of Progress Conditions 128 All make progress Non-BlockingBlocking Some make progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free

129. More Formally Standard notion of abstract object Progress conditions relate to method calls of an object –A thread is active if it takes an infinite number of concrete (machine level) steps –And is suspended if not. 129

130. Maximal vs. Minimal Minimal progress –some call eventually completes –System matters, not individuals Maximal progress –every call eventually completes. –Individuals matter 130 Flags courtesy of www.theodora.com/flags used with permission

131. The “Periodic Table” of Progress Conditions 131 Maximal progress Non-BlockingBlocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free

132. The Scheduler’s Role Multiprocessor progress properties: Are not about the guarantees a method's implementation provides. Are about scheduling needed to provide minimal or maximal progress. 132

133. Fair Scheduling A history is fair if each thread takes an infinite number of steps A method implementation is deadlock- free if it guarantees minimal progress in every fair history. 133

134. Starvation Freedom A method implementation is starvation- free if it guarantees maximal progress in every fair history. 134

135. Dependent Progress Dependent progress conditions – Do not guarantee minimal progress in every history Independent ones do. Blocking progress conditions –deadlock-freedom, Starvation-freedom –are dependent. 135

136. Non-blocking Independent Conditions A lock-free method guarantees –minimal progress –in every history. A wait-free method guarantees –maximal progress –in every history. 136

137. The “Periodic Table” of Progress Conditions 137 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free Independent Dependent

138. Uniformly Isolating Schedules A history is uniformly isolating if any thread eventually runs by itself for “long enough” Modern systems do this with backoff, yield, etc. 138

139. A Non-blocking Dependent Condition A method implementation is obstruction- free if it guarantees – maximal progress –in every uniformly isolating history. 139

140. The “Periodic Table” of Progress Conditions 140 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent Uniform iso scheduler Fair scheduler

141. The “Periodic Table” of Progress Conditions 141 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free

142. Clash-Freedom: the “Einsteinium” of Progress A method implementation is clash-free if it guarantees –minimal progress –in every uniformly isolating history. Thm: clash-freedom strictly weaker than obstruction-freedom 142

143. Getting from Minimal to Maximal 143 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free ? Helping But helping is expensive

144. Universal Constructions Lock-free universal construction –provides minimal progress A scheduler is benevolent –if it guarantees maximal progress anyway Real-world OS schedulers are benevolent –mostly They do not persecute any individual thread 144

145. Getting from Minimal to Maximal 145 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free ? Helping Use Wait-free/Lock-free Consensus Objects Universal Wait-free Construction Universal Lock-free Construction

146. Getting from Minimal to Maximal 146 Maximal progress Non-Blocking Blocking Minimal progress Lock- free Starvation- free Deadlock- free Wait- free Obstruction- free IndependentDependent ? Clash- free Universal Wait-free Construction Universal Lock-free Construction If we use Starvation-free/Deadlock-free Consensus Objects result is respectively Starvation-free/Deadlock-free

147. Maximal Progress Postulate Programmers want maximal progress. Methods’ progress conditions define –What we expect from the scheduler –For example Don’t halt in critical section Let me run in isolation long enough … s147

148. Why Lock-Free is OK We all want maximal progress –Wait-free Yet we often write lock-free or deadlock- free lock-based algorithms OK if we expect the scheduler to be benevolent –Often true (not always!) Art of Multiprocessor Programming 148

149. 149 Shared-Memory Computability What is (and is not) concurrently computable Wait-free Atomic Registers Lock-free/Wait-free Hierarchy and Universal Constructions 10011 Shared Memory

150. Troubling Intellectual Question… Why use non-blocking lock-free and wait-free conditions when most code uses locks? 150 I think I think, therefore I think I am (Ambrose Bierce)

151. The Answer Not about being non-blocking… About being independent! Do not rely on the good behavior of the scheduler. 151

152. 152 By Analogy to Church-Turing 011010 1 Reads and Writes Infinite tape Finite State Controller Using a dependent condition is like relying on an oracle to recognize languages… The dependency masks the true power of the concurrent object… aba n b n a n

153. 153 Shared-Memory Computability Independent progress: use Lock-free and Wait- free Memory Hierarchy and Universal Constructions 10011 Shared Memory

154. Programmers Expect the Best 154 Programmers expect maximal progress. Progress conditions define scheduler requirements necessary to achieve it.

155. This Concludes Our Course Material Principles: Mathematical foundations of multicore programming Practice: How multicore software and the architectures it runs on embody these principles Art of Multiprocessor Programming 155

156. Art of Multiprocessor Programming 156 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.