1. Chair of Software Engineering Concurrent Object-Oriented Programming Prof. Dr. Bertrand Meyer Lecture 5: Concurrent Objects

2. 2 Material From The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

3. 3 Concurrent Computation memory object

4. 4 Objectivism What is a concurrent object? How do we describe one? How do we implement one? How do we tell if we ’ re right?

5. 5 FIFO Queue: Enqueue Method q.enq ( )

6. 6 FIFO Queue: Dequeue Method q.deq()/

7. 7 A Lock-Based Queue class LockBasedQueue { int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } Queue fields protected by single shared lock 0 1 capacity-1 2 head tail y z Initially head = tail

8. 8 Implementation: Deq public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } 0 1 capacity-1 2 head tail y z Method calls mutually exclusive If queue empty throw exception Queue not empty: remove item and update head Return result Release lock no matter what! Should be correct because modifications are mutually exclusive…

9. 9 Now consider the following implementation The same thing without mutual exclusion For simplicity, only two threads One thread enq only The other deq only

10. 10 Wait-free 2-Thread Queue public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { while (tail-head == capacity); // busy-wait items[tail % capacity] = x; tail++; } public Item deq() { while (tail == head); // busy-wait Item item = items[head % capacity]; head++; return item; } 0 1 capacity-1 2 head tail y z Queue is updated without a lock! How do we define “correct” when modifications are not mutually exclusive?

11. 11 Defining concurrent queue implementation Need a way to specify a concurrent queue object Need a way to prove that an algorithm implements the object ’ s specification Lets talk about object specifications

12. 12 Correctness and Progress In a concurrent setting, we need to specify both the safety and the liveness properties of an object Need a way to define when an implementation is correct the conditions under which it guarantees progress

13. 13 Sequential Objects Each object has a state Usually given by a set of fields Queue example: sequence of items Each object has a set of methods Only way to manipulate state Queue example: enq and deq methods

14. 14 Sequential Specifications If (precondition) the object is in such-and-such a state before you call the method, Then (postcondition) the method will return a particular value or throw a particular exception. and (postcondition, con ’ t) the object will be in some other state when the method returns,

15. 15 Pre- and Postcondition for Dequeue Precondition: Queue is non-empty Postcondition: Returns first item in queue Postcondition: Removes first item in queue

16. 16 Pre- and Postcondition for Dequeue Precondition: Queue is empty Postcondition: Throws Empty exception Postcondition: Queue state unchanged

17. 17 Why sequential specifications totally rock Interactions among methods captured by side- effects on object state State meaningful between method calls Documentation size linear in number of methods Each method described in isolation Can add new methods Without changing descriptions of old methods

18. 18 What about concurrent specifications ? Methods? Documentation? Adding new methods?

19. 19 Methods take time time invocation 12:00 q.enq(. ) Method call void response 12:01

20. 20 Sequential vs. Concurrent Sequential Methods take time? Who knew? Concurrent Method call is not an event Method call is an interval.

21. 21 Concurrent methods take overlapping time time Method call

22. 22 Sequential vs. Concurrent Sequential: Object needs meaningful state only between method calls Concurrent Because method calls overlap, object might never be between method calls

23. 23 Sequential vs. Concurrent Sequential: Each method described in isolation Concurrent: Must characterize all possible interactions with concurrent calls What if two enq s overlap? Two deq s? enq and deq ? …

24. 24 Sequential vs. Concurrent Sequential: Can add new methods without affecting older methods Concurrent: Everything can potentially interact with everything else Panic!

25. 25 The Big Question What does it mean for a concurrent object to be correct? What is a concurrent FIFO queue? FIFO means strict temporal order Concurrent means ambiguous temporal order

26. 26 Intuitively… public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } All modifications of queue are done mutually exclusive

27. 27 Intuitively… time q.deq q.enq enq deq lock() unlock() lock() unlock() Behavior is “Sequential” enq deq Lets capture the idea of describing the concurrent via the sequential

28. 28 Linearizability Each method should “ take effect ” Instantaneously Between invocation and response events Object is correct if this “ sequential ” behavior is correct Any such concurrent object is Linearizable ™

29. 29 Is it really about the object? Each method should “ take effect ” Instantaneously Between invocation and response events Sounds like a property of an execution … A linearizable object: one all of whose possible executions are linearizable

30. 30 Example time q.enq(x) q.enq(y)q.deq(x) q.deq(y)

31. 31 Example time q.enq(x) q.enq(y)q.deq(x) q.deq(y) linearizable q.enq(x) q.enq(y)q.deq(x) q.deq(y) time

32. 32 Example time q.enq(x) q.enq(y)q.deq(x) q.deq(y) Valid? q.enq(x) q.enq(y)q.deq(x) q.deq(y) time

33. 33 Example time q.enq(x)q.deq(y) q.enq(y)

34. 34 Example time q.enq(x) q.enq(y) q.deq(y) q.enq(x) q.enq(y) not linearizable

35. 35 Example time q.enq(x) q.deq(x)

36. 36 Example time q.enq(x) q.deq(x) q.enq(x) q.deq(x) time linearizable

37. 37 Example time q.enq(x) time q.enq(y) q.deq(y) q.deq(x)

38. 38 Example q.enq(x) q.enq(y) q.deq(y) q.deq(x) Comme ci time Comme ça linearizable

39. 39 Read/Write Register Example time read(1)write(0) write(1) write(2) time read(0) write(1) already happened

40. 40 Read/Write Register Example time read(1)write(0) write(1) write(2) time read(0) write(1) write(1) already happened not linearizable

41. 41 Read/Write Register Example time read(1)write(0) write(1) write(2) time read(1) write(1) already happened

42. 42 Read/Write Register Example time read(1)write(0) write(1) write(2) time read(1) write(1)write(2) write(1) already happened not linearizable

43. 43 Read/Write Register Example time write(0) write(1) write(2) time read(1)

44. 44 Read/Write Register Example time write(0) write(1) write(2) time read(1) write(1) write(2) linearizable

45. 45 Talking About Executions Why? Can ’ t we specify the linearization point of each operation without describing an execution? Not Always In some cases, linearization point depends on the execution

46. 46 Formal Model of Executions Define precisely what we mean Ambiguity is bad when intuition is weak Allow reasoning Formal But mostly informal In the long run, actually more important

47. 47 Invocation Notation A q.enq(x) thread method object arguments

48. 48 Response Notation A q: void thread result object Method is implicit

49. 49 Response Notation A q: empty() thread object exception Method is implicit

50. 50 History - Describing an Execution Sequence of invocations and responses A q.enq(3) A q:void A q.enq(5) B p.enq(4) B p:void B q.deq() B q:3 H =

51. 51 Matching Invocation & response match if A q.enq(3) A q:void Thread names agree Object names agree Method call

52. 52 Object Projections H |q = A q.enq(3) A q:void B q.deq() B q:3 B p.enq(4) B p:void

53. 53 Thread Projections H |B = B p.enq(4) B p:void B q.deq() B q:3 A q.enq(3) A q:void

54. 54 Complete Subhistory A q.enq(3) A q:void B p.enq(4) B p:void B q.deq() B q:3 no matching response H An invocation is pending if it has no matching response. The call may or may not have taken effect. Discard the pending invocation to get the complete subhistory. complete( ) = A q.enq(5)

55. 55 Sequential Histories In sequential histories method calls of different threads do not interleave. A q.enq(3) A q:void B p.enq(4) B p:void B q.deq() B q:3 A q:enq(5) match Final pending invocation OK

56. 56 Equivalent Histories Two histories are equivalent if different threads see the same thing in both histories. In case of two histories H and G with threads A and B: H|A = G|A and H|B = G|B. H= A q.enq(3) B p.enq(4) B p:void B q.deq() A q:void B q:3 A q.enq(3) A q:void B p.enq(4) B p:void B q.deq() B q:3 G=

57. 57 Sequential Specifications A sequential specification is some way of telling whether a Single-thread, single-object history Is legal For example: Pre and post-conditions But plenty of other techniques exist …

58. 58 Legal Histories A sequential (multi-object) history H is legal if For every object x H|x is in the sequential spec for x

59. 59 Precedence A method call precedes another if response event precedes invocation event. A q.enq(3) B p.enq(4) B p.void A q:void B q.deq() B q:3 Method call

60. 60 Non-Precedence We have non-precedence when some method calls overlap one another. A q.enq(3) B p.enq(4) B p.void B q.deq() A q:void B q:3 Method call

61. 61 Notation Given History H method executions m 0 and m 1 in H We say m 0  H  m 1, if m 0 precedes m 1 Relation m 0  H  m 1 is a Partial order Total order if H is sequential

62. 62 Linearizability History H is linearizable if it can be extended to G by Appending zero or more responses to pending invocations Discarding other pending invocations So that G is equivalent to Legal sequential history S where  G   S

63. 63 What is  G   S It is a limitation on the choice of S! time a b GG SS c GG  G = {a  c, b  c}  S = {a  b, a  c, b  c}

64. 64 Remarks Some pending invocations Took effect, so keep them Discard the rest Condition  G   S Means that S respects “real-time order” of G

65. 65 Example A q.enq(3) B q.enq(4) B q:void B q.deq() B q:4 time B.q.enq(4 ) A.q.enq(3) B.q.deq(4 ) B. q.enq(6) Complete this pending invocation A.q.enq(3 ) A q:void discard this one B q:enq(6) B q.enq(4) B q:void A q.enq(3) A q:void B q.deq() B q:4 Equivalent sequential history

66. 66 Concurrency How much concurrency does linearizability allow? When must a method invocation block?

67. 67 Concurrency Focus on total methods Defined in every state Example: deq() that throws Empty exception Versus deq() that waits … Why? Otherwise, blocking unrelated to synchronization

68. 68 Concurrency Question: When does linearizability require a method invocation to block? Answer: never. Linearizability is non-blocking

69. 69 Non-Blocking Theorem If method invocation A q.inv(…) is pending in history H, then there exists a response A q:res(…) such that H + A q:res(…) is linearizable.

70. 70 Proof Pick linearization S of H If S already contains Invocation A q.inv(…) and response, Then we are done. Otherwise, pick a response such that S + A q.inv(…) + A q:res(…) Possible because object is total.

71. 71 Composability Theorem History H is linearizable if and only if For every object x H|x is linearizable We care about objects only! (Materialism?)

72. 72 Why does composability matter? Modularity Can prove linearizability of objects in isolation Can compose independently implemented objects

73. 73 Reasoning About Linearizability: Locking public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Linearization points are when locks are released

74. 74 More Reasoning: Lock-free public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(Item x) { while (tail-head == capacity); // busy-wait items[tail % capacity] = x; tail++; } public Item deq() { while (tail == head); // busy-wait Item item = items[head % capacity]; head++; return item; } Linearization order is order head and tail fields modified

75. 75 Strategy Identify one atomic step where method “ happens ” Critical section Machine instruction Doesn ’ t always work Might need to define several different steps for a given method

76. 76 Linearizability: Summary Powerful specification tool for shared objects Allows us to capture the notion of objects being “ atomic ” Don ’ t leave home without it

77. 77 Alternative: Sequential Consistency History H is Sequentially Consistent if it can be extended to G by Appending zero or more responses to pending invocations Discarding other pending invocations So that G is equivalent to a Legal sequential history S where  G   S Differs from linearizability

78. 78 Alternative: Sequential Consistency No need to preserve real-time order Cannot re-order operations done by the same thread Can re-order non-overlapping operations done by different threads Often used to describe multiprocessor memory architectures

79. 79 Example time q.enq(x)q.deq(y) q.enq(y)

80. 80 Example sequentially consisent time q.enq(x) q.enq(y) q.deq(y) q.enq(x) q.enq(y) not linearizable

81. 81 Theorem Sequential Consistency is not a local property (and thus we lose composability).

82. 82 FIFO Queue Example time p.enq(x)p.deq(y)q.enq(x) time q.enq(y)q.deq(x)p.enq(y) History H

83. 83 H|p Sequentially Consistent time p.enq(x)p.deq(y) p.enq(y) q.enq(x) q.enq(y)q.deq(x) time

84. 84 H|q Sequentially Consistent time p.enq(x)p.deq(y)q.enq(x) q.enq(y)q.deq(x)p.enq(y) time

85. 85 Ordering imposed by p time p.enq(x)p.deq(y)q.enq(x) q.enq(y)q.deq(x)p.enq(y) time

86. 86 Ordering imposed by q time p.enq(x)p.deq(y)q.enq(x) q.enq(y)q.deq(x)p.enq(y) time

87. 87 Ordering imposed by both p.enq(x) time q.enq(x) q.enq(y)q.deq(x) p.deq(y) p.enq(y) time

88. 88 Fact Most hardware architectures don ’ t support sequential consistency Because they think it ’ s too strong Here ’ s another story…

89. 89 The Flag Example time x.write(1) y.read(0) y.write(1) x.read(0) time Each thread’s view is sequentially consistent It went first Entire history isn’t sequentially consistent Can’t both go first Is this behavior really so wrong? We can argue either way…

90. 90 Opinion 1: It ’ s Wrong This pattern Write mine, read yours Is exactly the flag principle Beloved of Alice and Bob Heart of mutual exclusion Peterson Bakery, etc. It ’ s non-negotiable!

91. 91 Opinion 2: But It Feels So Right Many hardware architects think that sequential consistency is too strong Too expensive to implement in modern hardware OK if flag principle violated by default Honored by explicit request

92. 92 Memory Hierarchy On modern multiprocessors, processors do not read and write directly to memory. Memory accesses are very slow compared to processor speeds. Instead, each processor reads and writes directly to a cache.

93. 93 Memory Operations To read a memory location, load data into cache. To write a memory location update cached copy, Lazily write cached data back to memory

94. 94 While Writing to Memory A processor can execute hundreds, or even thousands of instructions Why delay on every memory write? Instead, write back in parallel with rest of the program.

95. 95 Revisionist History Flag violation history is actually OK processors delay writing to memory Until after reads have been issued. Otherwise unacceptable delay between read and write instructions. Who knew you wanted to synchronize?

96. 96 Who knew you wanted to synchronize? Writing to memory = mailing a letter Vast majority of reads & writes Not for synchronization No need to idle waiting for post office If you want to synchronize Announce it explicitly Pay for it only when you need it

97. 97 Explicit Synchronization Memory barrier instruction Flush unwritten caches Bring caches up to date Compilers often do this for you Entering and leaving critical sections Expensive

98. 98 Volatile In Java, can ask compiler to keep a variable up-to-date with volatile keyword Also inhibits reordering, removing from loops, & other “ optimizations ”

99. 99 Real-World Hardware Memory Weaker than sequential consistency But you can get sequential consistency at a price OK for expert, tricky stuff assembly language, device drivers, etc. Linearizability more appropriate for high-level software

100. 100 Critical Sections Easy way to implement linearizability Take sequential object Make each method a critical section Problems Blocking No concurrency

101. 101 Linearizability Operation takes effect instantaneously between invocation and response Uses sequential specification, locality implies composablity Good for high level objects

102. 102 Correctness: Linearizability Sequential Consistency Not composable Harder to work with Good way to think about hardware models We will use linearizability as in the remainder of this course unless stated otherwise.

103. 103 Progress We saw an implementation whose methods were lock- based (deadlock-free) We saw an implementation whose methods did not use locks (lock-free) How do they relate?

104. 104 Maximal vs. Minimal Minimal progress: in some suffix of H, some pending active invocation has a matching response (some method call eventually completes ). Maximal progress: in every suffix of H, every pending active invocation has a matching response (every method call always completes).

105. 105 Progress Conditions Deadlock-free: some thread trying to acquire the lock eventually succeeds. Starvation-free: every thread trying to acquire the lock eventually succeeds. Lock-free: some thread calling a method eventually returns. Wait-free: every thread calling a method eventually returns.

106. 106 Progress Conditions Wait-free Lock-free Starvation-free Deadlock-free Everyone makes progress Non-Blocking Blocking Someone makes progress

107. 107 Fair Histories A history is fair if each thread always continues takes steps. On multiprocessors this is controlled by the operating system. So fair histories are ones in which the operating system guarantees each thread continues to take steps