1. Synchronization Chapter 6 Part I Clock Synchronization & Logical clocks Part II Mutual Exclusion Part III Election Algorithms Part IV Transactions

2. –Most of the lecture notes are based on slides by Prof. Jalal Y. Kawash at Univ. of Calgary –Some slides are from Prof. Steve Goddard at University Nebraska, Lincoln, Prof. Harandi, Prof. Hou, Prof. Gupta and Prof. Vaidya from University of Illinois, Prof. Kulkarni from Princeton University –I have modified them and added new slides Giving credit where credit is due: CSCE455/855 Distributed Operating Systems

3. Motivation Synchronization is important if we want to –control access to a single, shared resource –agree on the ordering of events Synchronization in Distributed Systems is much more difficult than in uniprocessor systems We will study: 1.Synchronization based on “Actual Time”. 2.Synchronization based on “Relative Time”. 3.Synchronization based on Co-ordination (with Election Algorithms). 4.Distributed Mutual Exclusion. 5.Distributed Transactions.

4. Synchronization Chapter 6 Part I Clock Synchronization & Logical clocks

5. Lack of Global Time in DS It is impossible to guarantee that physical clocks run at the same frequency Lack of global time, can cause problems Example: UNIX make –Edit output.c at a client –output.o is at a server (compile at server) –Client machine clock can be lagging behind the server machine clock

6. Lack of Global Time – Example When each machine has its own clock, an event that occurred after another event may nevertheless be assigned an earlier time.

7. A Note on Real Time Universal Coordinated Time (UTC): basis of all modern civil timekeeping Radio stations can broadcast UTC for receivers to collect it –WWV SW radio station in Colorado

8. Physical Clock Synchronization (1) External: synchronize with an external resource, UTC source |S(t) – C i (t)| < D Internal: synchronize without access to an external resource |C i (t) – C j (t)| < D

9. Cristian’s Algorithm – External Synch External source S Denote clock value at process X by C(X) Periodically, a process P: 1.send message to S, requesting time 2.Receive message from S, containing time C(S) 3.Adjust C at P, should we set C(P) = C(S)? Reply takes time When P adjusts C(P) to C(S), C(S) > C(P)

10. Cristian’s Algorithm – External Synch How to estimate machine A’s time offset relative to B? B A T2T2 T1T1 T3T3 T4T4

11. Global Positioning System Computing a position in a two-dimensional space

12. Berkeley Algorithm – Internal Synch Internal: synchronize without access to an external resource |C i (t) – C j (t)| < D Periodically, S: send C(S) to each client P P: calculate  P = C(P) – C(S) send  P to S S: receive all  P ’s compute an average  send  -  P to client P P: apply  -  P to C(P)

13. The Berkeley Algorithm a)The time daemon asks all the other machines for their clock values b)The machines answer c)The time daemon tells everyone how to adjust their clock Propagation time? Time server fails?

14. Importance of Synchronized Clocks New H/W and S/W for synchronizing clocks is easily available Nowadays, it is possible to keep millions of clocks synchronized to within few milliseconds of UTC New algorithms can benefit

15. Event Ordering in Centralized Systems A process makes a kernel call to get the time Process which tries to get the time later will always get a higher (or equal) time value  no ambiguity in the order of events and their time

16. Logical Clocks For many DS algorithms, associating an event to an absolute real time is not essential, we only need to know an unambiguous order of events Lamport's timestamps Vector timestamps

17. Logical Clocks (Cont.) Synchronization based on “relative time”. –Example: Unix make (Is output.c updated after the generation of output.o?) “relative time” may not relate to the “real time”. What’s important is that the processes in the Distributed System agree on the ordering in which certain events occur. Such “clocks” are referred to as Logical Clocks.

18. Example: Why Order Matters? –Replicated accounts in New York(NY) and San Francisco(SF) –Two updates occur at the same time Current balance: $1,000 Update1: Add $100 at SF; Update2: Add interest of 1% at NY Whoops, inconsistent states!

19. Lamport Algorithm Clock synchronization does not have to be exact –Synchronization not needed if there is no interaction between machines –Synchronization only needed when machines communicate –i.e. must only agree on ordering of interacting events

20. Lamport’s “Happens-Before” Partial Order Given two events e & e’, e < e’ if: 1.Same process: e < i e’, for some process P i 2.Same message: e = send(m) and e’=receive(m) for some message m 3.Transitivity: there is an event e* such that e < e* and e* < e’

21. Concurrent Events Given two events e & e’: If neither e < e’ nor e’< e, then e || e’ P1P1 P2P2 P3P3 Real Time a b c f d e m1m1 m2m2

22. Lamport Logical Clocks Substitute synchronized clocks with a global ordering of events –e i < e j  LC(e i ) < LC(e j ) –LC i is a local clock: contains increasing values each process i has own LC i –Increment LC i on each event occurrence –within same process i, if e j occurs before e k LC i (e j ) < LC i (e k ) –If e s is a send event and e r receives that send, then LC i (e s ) < LC j (e r )

23. Lamport Algorithm Each process increments local clock between any two successive events Message contains a timestamp Upon receiving a message, if received timestamp is ahead, receiver fast forward its clock to be one more than sending time

24. Lamport Algorithm (cont.) Timestamp –Each event is given a timestamp t –If e s is a send message m from p i, then t=LC i (e s ) –When p j receives m, set LC j value as follows If t < LC j, increment LC j by one –Message regarded as next event on j If t ≥ LC j, set LC j to t+1

25. Lamport’s Algorithm Analysis (1) Claim: e i < e j  LC(e i ) < LC(e j ) Proof: by induction on the length of the sequence of events relating to e i and e j P1P1 P2P2 P3P3 Real Time a b c f d e m1m1 m2m2 1 2 3 5 4 1 g 5

26. Lamport’s Algorithm Analysis (2) LC(e i ) < LC(e j )  e i < e j ? Claim: if LC(e i ) < LC(e j ), then it is not necessarily true that e i < e j P1P1 P2P2 P3P3 Real Time a b c f d e m1m1 m2m2 1 2 3 5 4 1 g 2

27. Total Ordering of Events Happens before is only a partial order Make the timestamp of an event e of process P i be: (LC(e),i) (a,b) < (c,d) iff a < c, or a = c and b < d

28. Application: Totally-Ordered Multicasting –Message is timestamped with sender ’ s logical time –Message is multicast (including sender itself) –When message is received It is put into local queue Ordered according to timestamp Multicast acknowledgement –Message is delivered to applications only when It is at head of queue It has been acknowledged by all involved processes

29. Application: Totally-Ordered Multicasting Update 1 is time-stamped and multicast. Added to local queues. Update 2 is time-stamped and multicast. Added to local queues. Acknowledgements for Update 2 sent/received. Update 2 can now be processed. Acknowledgements for Update 1 sent/received. Update 1 can now be processed. (Note: all queues are the same, as the timestamps have been used to ensure the “happens-before” relation holds.)

30. Limitation of Lamport’s Algorithm e i < e j  LC(e i ) < LC(e j ) However, LC(e i ) < LC(e j ) does not imply e i < e j –for instance, (1,1) < (1,3), but events a and e are concurrent P1P1 P2P2 P3P3 Real Time a b c f d e m1m1 m2m2 (1,1) (2,1) (3,2) (5,3) (4,2) (1,3) g (2,3)

31. Vector Timestamps P i ’s clock is a vector VT i [] VT i [i] = number of events P i has stamped VT i [j] = what P i thinks number of events P j has stamped (i  j)

32. Vector Timestamps (cont.) Initialization –the vector timestamp for each process is initialized to (0,0,…,0) Local event –when an event occurs on process P i, VT i [i]  VT i [i] + 1 e.g., on processor 3, (1,2,1,3)  (1,2,2,3)

33. Message passing –when P i sends a message to P j, the message has timestamp t[]=VT i [] –when P j receives the message, it sets VT j [k] to max (VT j [k],t[k]), for k = 1, 2, …, N e.g., P 2 receives a message with timestamp (3,2,4) and P 2 ’s timestamp is (3,4,3), then P 2 adjusts its timestamp to (3,4,4) Vector Timestamps (cont.)

34. Comparing Vectors VT 1 = VT 2 iff VT 1 [i] = VT 2 [i] for all i VT 1  VT 2 iff VT 1 [i]  VT 2 [i] for all i VT 1 < VT 2 iff VT 1  VT 2 & VT 1  VT 2 –for instance, (1, 2, 2) < (1, 3, 2)

35. Vector Timestamp Analysis Claim: e < e’ iff e.VT < e’.VT P1P1 P2P2 P3P3 Real Time a b c f d e m1m1 m2m2 [1,0,0] [2,0,0] [2,1,0] [2,2,3] [2,2,0] [0,0,1] g [0,0,2]

36. Application: Causally-Ordered Multicasting For ordered delivery of related messages –V i [i] is only incremented when sending –When k gets a msg from j, with timestamp ts, the msg is buffered until: 1: ts[j] = V k [j] + 1 –(timestamp indicates this is the next msg that k is expecting from j) 2: ts[i] ≤ V k [i] for all i ≠ j –(k has seen all msgs that were seen by j when j sent the msg)

37. Causally-Ordered Multicasting P2 a P1 c d P3 e g [1,0,0] [1,0,1] Post a r: Reply a Two messages: message a from P1; message r from P3 replying to message a; Scenario1: at P2, message a arrives before the reply r b [1,0,1] [0,0,0]

38. Causally-Ordered Multicasting (cont.) P2 a P1P3 d g [1,0,0] [1,0,1] Post a r: Reply a Buffered c [1,0,0] Scenario2: at P2, message a arrives after the reply r. The reply is not delivered right away. b [1,0,1] [0,0,0] Deliver r

39. Ordered Communication Totally ordered multicast –Use Lamport timestamps Causally ordered multicast –Use vector timestamps