1. Distributed Algorithms (22903) Lecturer: Danny Hendler Shared objects: linearizability, wait-freedom and simulations Most of this presentation is based on the book “Distributed Computing” by Hagit attiya & Jennifer Welch. Some slides are based on presentations by Maurice Herlihy & Nir Shavit.

2. 2

3. 3 Shared Objects (cont’d) Each object has a state –Usually given by a set of shared memory fields Objects may be implemented from simpler base objects Each object supports a set of operations –Only way to manipulate state –E.g. – a shared counter supports the fetch&increment operation

4. 4 Shared Objects Correctness Correctness of a sequential counter fetch&increment, applied to a counter with value v, returns v and increments the counter’s value to (v+1). Values returned by consecutive operations: 0, 1, 2, … But how shall we define the correctness of a shared counter?

5. 5 time q.enq(x) q.enq(y) q.deq(x) q.deq(y) fetch&inc time Shared Objects Correctness (cont’d) There is only a partial order between operations! InvocationResponse

6. 6 Shared Objects Correctness (cont’d) An invocation calls an operation on an object. c. f&I () object method arguments

7. 7 Shared Objects Correctness (cont’d) An object returns the response of the operation. c: 12 object response

8. 8 Shared Objects Correctness (cont’d) A sequential object history is a sequence of matching invocations and responses on the object. Example: a sequential history of a FIFO queue q.enq(3) q:void q.enq(7) q:void q.deq() q:3 q.deq() q:7

9. 9 Shared Objects Correctness (cont’d) Sequential specification The correct behavior of the object in the absence of concurrency: a set of legal sequential object histories. Example: the sequential spec of a counter H 0 : H 1 : c.f&i() c:0 H 2 : c.f&i() c:0 c.f&i() c:1 H 3 : c.f&i() c:0 c.f&i() c:1 c.f&i() c:2 H 4 : c.f&i() c:0 c.f&i() c:1 c.f&i() c:2 c.f&i() c:3...

10. 10 Shared Objects Correctness (cont’d) Linearizability An execution is linearizable if there exists a permutation of the operations on each object o, , such that:  is a sequential history of o  preserves the partial order of the execution.

11. 11 Example time q.enq(x) q.enq(y)q.deq(x) q.deq(y) linearizable q.enq(x) q.enq(y)q.deq(x) q.deq(y) time (6)

12. 12 Example time q.enq(x) q.enq(y) q.deq(y) not linearizable q.enq(x) q.enq(y) (5)

13. 13 Example time q.enq(x) q.deq(x) q.enq(x) q.deq(x) time (4) linearizable

14. 14 Example time q.enq(x) q.enq(y) q.deq(y) linearizable q.deq(x) time q.enq(x) q.enq(y) q.deq(y) q.deq(x) q.enq(x) q.enq(y) q.deq(y) q.deq(x) multiple orders OK (8)

15. 15 Wait freedom Wait-freedom An algorithm is wait-free if every operation terminates after performing some finite number of events. Wait-freedom implies that there is no use of locks (no mutual exclusion). Thus the problems inherent to locks are avoided: Deadlock Priority inversion

16. 16 Wait-free linearizable implementations Example: the sequential spec of a register H 0 : H 1 : r.read() r:init H 2 : r.write(v1) r:ack H 3 : r.write(v1) r:ack r.read() r:v1 r.read() r:v1 H 4 : r.write(v1) r:ack r.write(v2) r:ack r.read() r:v2... Read returns the value written by last Write (or initial value if there were no preceding writes)

17. 17 Wait-free (linearizable) register simulations Binary single-reader/single-writer register (Multi-valued) single-reader/single-writer register multi-reader/single-writer register multi-reader/multi-writer register

18. 18 A wait-free (linearizable) implementation of a single-writer-single-reader (SRSW) multi- valued register from binary SRSW registers Would the above implementation of a k-valued register (initialized to i) work? Initially B[0]…B[k-1]=0, B[i]=1 (i is the initial value of R) Read(R) Return the index of the highest entry of B that equals 1 Write(R, v) Write 1 to B[v], write 0 to the entry corresponding to the previous value (if other than v). No!

19. 19 An example of a non-linearizable execution Initially B[0]…B[2]=0, B[3]=1 Read Write(1) Write(2) Read B[0] Return 0 Read B[1] Return 0 Write 1 to B[1] Ack Write 0 to B[3] Ack Write 1 to B[2] Ack Read B[2] Return 1 Ack Return 2 Read Read B[0] Return 0 Read B[1] Return 1 Write 0 to B[1] Ack Return 1 = linearization point Write(1) precedes Write(2) AND Read(2) precedes Read(1). This is not linearizable!

20. 20 A Wait-free Linearizable Implementation Initially B[v]=1 and all other entries equal 0, where v is the initial value of R. Read(R) 1.i:=0 2.while B[i]=0 do i:=i+1 // Search upwards till first set bit 3.up:= i, v:=i 4.for i=up –1 downto 0 do // Search downwards for lowest set bit 5. if B[i]=1 then v:=i 6.return v Write(R,v) 1.B[v]:=1 2.For i:=v-1 downto 0 do B[i]:=0 // Reset lower bits 3.return ack

21. 21 The linearization order Write 1 (R,1) Write 2 (R,4)Write 3 (R,3) Write 4 (R,1) Read 1 (R, init) Read 2 (R, 4) Read 4 (R, 3) Write 1 (R, 1) Write 2 (R, 4) Write 3 (R, 3) Write 4 (R, 1) Read 1 (R, init) Read 2 (R, 4) Read 3 (R, 4) Read 4 (R, 3) Read 5 (R, 1) Writes linearized first All reads from a specific write linearized after it, in their real- time order. Read 3 (R, 4) Read 5 (R, 1)

22. 22 SRSW multi-valued register correctness proof Wait freedom Trivial from the code (no loops) Linearizability Proof partitioned to the following 4 cases: 1.The order between write operations is maintained in  (follows from construction) 2.If R precedes W in E, then it precedes it in  (triv.) 3.The order between read operations is maintained in  4.If W precedes R in E, then it precedes it in  We'll now see the proof for case 4.

23. 23 Illustration for Lemma 1 B 0 1 v u 1 0 v1v1 Written by W v1v1 Written by W 1

24. 24 Illustration for Lemma 1 B 0 1 v u 1 0 v1v1 0 v2v2 Written by W Written by W 1 v2v2 Written by W 2

25. 25 Illustration for Lemma 2 (case 4) W(v) E: R R π:π: W(v) (v’) Case 1: v’ ≤ v v v’ 1 Written by W’ 1 Written by W 0 0 0 0 0 W’(v’)

26. 26 W(v) E: R R π:π: W(v) W’(v’) (v’) Case 2: v’ > v v’ 1 Written by W’ v 1 Written by W Written by W’’ 0 From Lemma 1, R returns a value written by an operation that follows W’’. W’(v’) W’’(x) Illustration for Lemma 2 (case 4)

27. 27 Illustration for Lemma 3 E: R1R1 R2R2 π:π: R2R2 R1R1 W 1 (v 1 ) W 2 (v 2 ) Case 1: v 1 = v 2 v 1 =v 2 1 Written by W 2 1 Written by W 1

28. 28 Illustration for Lemma 3 (cont’d) E: R1R1 R2R2 π:π: R2R2 R1R1 W 1 (v 1 ) W 2 (v 2 ) Case 2: v 1 > v 2 v1v1 1 Written by W 1 v2v2 1 Written by W 2 Since R 1 precedes R 2 and R 2 reads from W 2, R 1 must see 1 in v 2 when scanning down

29. 29 Illustration for Lemma 3 (cont’d) E: R1R1 R2R2 π:π: R2R2 R1R1 W 1 (v 1 ) W 2 (v 2 ) Case 3: v 1 < v 2 v2v2 1 Written by W 2 v1v1 1 Written by W 1 From Lemma 1, R 2 returns a value written by an operation no sooner than W 3 ! 0 Written by W 3 W 3 (v 3 )

30. 30 A wait-free Implementation of a (muti-valued) multi-reader register from (multi-valued) SRSW registers.

31. 31 Would this work? Read(R) by p i 1.return Val[i] Write(R,v) 1.For i:=0 to n-1 do Val[i]:=v 2.return ack SRSW Val[i]: The value written by the writer for reader p i Is the algorithm wait-free? Is the algorithm linearziable? Yes Nope

32. 32 An example of a non-linearizable execution Initially Val[0]=Val[1]=0 Read Read Val[0] Return 1 = linearization point Read(1) precedes Read(0). This is not linearizable! Write(1) Write 1 to Val[0] Ack Write 1 to Val[1] Ack P w: P 0: P 1: Return 1 Read Read Val[1] Return 0

33. 33 An example of a non-linearizable execution Initially Val[0]=Val[1]=0 Read: Read Val[0] Read(1) precedes Read(0). This is not linearizable! Write(1): Write 1 to Val[0] P w: P 0: P 1: Read Val[1] Return 0 Return 1 Read: Write 1 to Val[1] Ack

34. 34 A proof that: no such simulation is possible, unless some readers…write!

35. 35 A wait-free implementation of a (muti-valued) multi- reader register from (multi-valued) SRSW registers. Data structures used Values are pairs of the form:. Sequence-numbers are ever increasing. Val[i]: The value written by p w for reader p i, for 1 ≤ i ≤ n Report[i,j]: The value returned by the most recent read operation performed by p i ; written by p i and read by p j, 1 ≤ i,j ≤ n.

36. 36 A wait-free implementation of a multi-reader register from SRSW registers (cont’d). Initially Report[i,j]=Val[i]=(v 0, 0), where v 0 is R’s initial value. Read(R) ; performed by process p r 1.(v[0],s[0]):=Val[r] ; most recent value written by writer 2.for (i:=1 to n do) (v[i],s[i])=Report[i,r] ; most recent value reported to p r by reader p i 3.Let j be such that s[j]=max{s[0], s[1], …, s[n]} 4.for i:=1 to n do Report[r,i]=(v[j],s[j]) ; p r reports to all readers 5.Return (v[j]) Write(R,v) ; performed by the single writer 1.seq:=seq+1 2.for i=1 to n do Val[i]=(v,seq) 3.return ack

37. 37 The linearization order Write(v1, 1) Write(v2,2)Write(v3,3) Write(v4,4) Read 1 (init, 0) Read 2 (v1, 1) Read 4 (v2, 2) Read 3 (v2, 2) Read 5 (v4, 4) Write(v1, 1) Write(v2, 2) Write(v3, 3) Write(v4, 4) Read 1 (init, 0) Read 2 (v1, 1) Read 3 (v2, 2) Read 4 (v2, 2) Read 5 (v4, 4) Writes linearized first Reads considered according to increasing order of response, and put after the write with same sequence ID.

38. 38 A wait-free Implementation of a multi- reader-multi-writer register from multi-reader-single-writer registers

39. 39 A wait-free implementation of a MRMW register from MRSW registers. Data structures used Values are pairs of the form:. Sequence-numbers are ever increasing. TS[i]: The vector timestamp of writer p i, for 0 ≤ i ≤ m-1. Written by p i and read by all writers. Val[i]: The latest value written by writer p i, for 0 ≤ i ≤ m-1, together with the vector timestamp associated with that value. Written by p i and read by all n readers.

40. 40 Concurrent timestamps Provide a total order for write operations The total order respects the partial order of write operations Timestamp implemented as vectors Ordered by lexicographic order Each writer increments its vector entry

41. 41 Concurrent timestamps example Writer 1 Writer 2 Writer 3 TS[1] TS[2] TS[3] Order: 0 1 00

42. 42 Concurrent timestamps example Writer 1 Writer 2 Writer 3 TS[1] TS[2] TS[3] Order: 1 00 110

43. 43 Concurrent timestamps example Writer 1 Writer 2 Writer 3 TS[1] TS[2] TS[3] Order: 1 00 110 1 1 1 1 2 1

44. 44 A wait-free Implementation of a MRMW register from MRSW registers. Initially TS[i]= and Val[i] equals the initial value of R Read(R) ; performed by reader p r 1.for i:=0 to m-1 do (v[i], t[i]):=Val[i] ; v and t are local 2.Let j be such that t[j]=max{t[0],…,t[m-1]} ; Lexicographic max 3.Return v[j] Write(R,v) ; performed by the writer p w 1.ts=NewCTS() ; Writer p w obtains a new vector timestamp 2.Val[w]:=(v,ts) 3.return ack Procedure NewCTS() ; called by writer p w 1.for i:=0 to m-1 do 2. lts[i]:=TS[i].i ; extract the i’th entry from TS of the i’th writer 3.lts[w]=lts[w]+1 ; Increment own entry 4.TS[w]=lts ; write p w ’s new timestamp 5.return lts

45. 45 The linearization order Write(v1, ) Write(v4, ) Read 1 (init, ) Read 2 (init, ) Read 4 (v2, ) Read 3 (v2, ) Read 5 (v4, ) Write(v1, ) Write(v2, ) Write(v3, ) Write(v4, ) Read 1 (init, ) Read 2 (init, ) Read 3 (v2, ) Read 4 (v2, ) Read 5 (v4, ) Writes linearized first by timestamp order Reads considered according to increasing order of response, and put after the write with same timestamped Writer 1 Writer 2 Write(v2, ) Write(v3, ) Reader 1 Reader 2