1. EEC-681/781 Distributed Computing Systems Lecture 10 Wenbing Zhao wenbing@ieee.org Cleveland State University

2. 2 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Outline Clock Synchronization issues Clock Synchronization Algorithms –Centralized –Distributed Event ordering and logical clocks Due date for project progress report –11/20 Monday mid-night –No extension!

3. 3 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Motivation for Clock Synchronization In everyday life, we are relying on clocks to coordinate our activities –For example, in EEC681, we meet every Monday and Wednesday between 6-7:50pm In computer systems, it is also convenient to use clock as a way to coordinate different activities –For example, the “make” program relies on files’ timestamp to decide if a recompilation is necessary

4. 4 30-Jun-15Wenbing Zhao Motivation for Clock Synchronization Stock market buy and sell orders Secure document timestamps (with cryptographic certification) Aviation traffic control and position reporting Radio and TV programming launch and monitoring Intruder detection, location and reporting Multimedia synchronization for real-time teleconferencing Network monitoring, measurement and control Differentiated services traffic engineering

5. 5 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Universal Coordinated Time Universal Coordinated Time (UTC): –Based on the number of transitions per second of the cesium 133 atom (pretty accurate) –At present, the real time is taken as the average of some 50 cesium-clocks around the world –Introduces a leap second from time to time to compensate that days are getting longer UTC is broadcast through short wave radio and satellite. –Satellites can give an accuracy of about 0.5 ms

6. 6 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Physical Clocks in Computer Systems Every machine has a timer that generates an interrupt H times per second There is a clock in machine p that ticks on each timer interrupt –Denote the value of that clock by C p (t), where t is UTC time –Ideally, for each machine p, C p (t) = t, or, in other words, dC/dt = 1

7. 7 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Clock Time and UTC The relation between clock time and UTC when clocks tick at different rates In practice: 1 -  < dC/dt < 1 +  Maximum drift rate:  Clock time, C

8. 8 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Terms Clock drift rate (or clock accuracy) –The amount of deviation from UTC per unit of time (a day or a week, etc.) Clock precision: –Resolution of the clock, e.g., 1  s, or 1ns Clock skew –The difference in time values of two clocks is called clock skew –Maximum clock skew of a group of clocks is determined by the two clocks that have the largest clock difference

9. 9 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Clock Synchronization Two clocks are said to be synchronized at a particular instance of time if the clock skew of the two clocks is less than some specified constant δ A set of clocks are said to be synchronized if the clock skew of any two clocks in this set is less than δ

10. 10 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Clock Synchronization Issues A distributed system requires: –External Synchronization Synchronize with an external time source –Internal Synchronization Clocks within the same network synchronize with each other Clock Synchronization requires: –Each node can read the other nodes’ clock values Must consider unpredicted communication delay –Time must never run backward Smooth adjustments => must maintain the order of the events

11. 11 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Message Propagation Time Estimate of message propagation time: (T 1 -T 0 -I)/2

12. 12 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Message Delay Distribution T1T1 T3T3 T2T2 T4T4 Server Client x 00

13. 13 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Clock Synchronization Algorithms Centralized –Passive Time Server Centralized Algorithm Cristian’s algorithm –Active Time Server Centralized Algorithm Berkeley Algorithm Distributed –Global Averaging Distributed Algorithms –Localized Averaging Distributed Algorithms

14. 14 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Passive Timer Server Each node periodically sends a message (time=?) to the time server at the current local clock time, T 0 The server responds with a message (time = T), T is the current time of the server The client receives the message at the local clock time T 1, and adjusts its local clock time to T+(T 1 -T 0 -I)/2 –The time taken by the server to handle the request message is I, T+(T 1 -T 0 -I)/2 –Several measurements of T 1 -T 0, discard the unreliable ones

15. 15 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Active Time Server The time server periodically broadcasts its clock time (T) Other nodes receive the message to correct their own clocks –Each node has the knowledge of the approximate time (T a ) required for the propagation of the message, T+T a –Each nodes replies with the local clock time The server –Knows the approximate propagation time from each node –Takes fault-tolerant average of clock values as current time –The server adjusts it own, and sends the amount by which each node clock requires adjustment to each node

16. 16 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Centralized Algorithms – Drawbacks Single-point failure Scalability

17. 17 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Global Averaging Distributed Algorithms Each node broadcasts its local clock time periodically Each node waits for time T –The node collects the messages broadcast by other nodes –For each message received, the node keeps the local time –At the end of T, the node estimates the skew of its clock with respect to each of the other nodes on the basis of the times at which it received –The node computes a fault-tolerant average of the estimated skews and uses it to adjust its local clock

18. 18 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Localized Averaging Distributed Algorithms Each node exchanges its clock time with its neighbors Then sets its clock time to the average of its own clock and the clock times of its neighbors

19. 19 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Exercise Consider the behavior of two machines in a distributed system. Both have clocks that are supposed to tick 1000 times per millisecond. One of them actually does, but the other ticks only 990 times per millisecond. If UTC updates come in once a minute, what is the maximum clock skew that will occur?

20. 20 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Event Ordering “Time, Clocks, and the Ordering of Events in a Distributed System”, by Leslie Lamport, Communications of the ACM, July 1978, Volume 21, Number 7, pp.558-565 He showed that it is possible to synchronize all the clocks to produce a single, unambiguous time standard He pointed out the clock synchronization need not to be absolute –What usually matters is not that all processes agree on exactly what time it is, but rather, that they agree on the order in which events occur

21. 21 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Happens-Before Relation With perfectly accurate physical time –An event a happened before an event b if a happened at an earlier time than b Without using the physical clocks –Assume that the system is composed of a collection of processes, each process consists of a sequence of events –The events of a process form a sequence, where a occurs before b in this sequence if a happens before b –Assume sending and receiving a message is an event in a process

22. 22 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Happens-before Relation “Happens-before” relation, denoted by “→”, is defined as follows: –The relation “→” on the set of events of a system is the relation satisfying the following three conditions: If a and b are events in the same process, and a comes before b, then a → b If a is the sending of a message by one process and b is the receipt of the same message by another process, then a → b If a → b and b → c, then a → c –Event a causally affects event b

23. 23 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Partial Ordering Two distinct events a and b are said to be concurrent if a → b and b → a –Neither event can causally affect the other –This introduces a partial ordering of events in a system with concurrently operating processes

24. 24 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Logical Clocks Logical clocks: Use the clock just as a way of assigning a number to an event, where the number is the time at which the event occurs –Define a clock C i for each process P i Assigns a number C i (a) to any event a in that process The entire system of clocks is represented by the function C which assigns to any event b the number C(b), where C(b) =C j (b) if b is an event in process P j The clocks C i are logical clocks rather than physical clocks

25. 25 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Implementation of Logical Clocks The logical clocks is correct if the events of the system that are related to each other by the happens-before relation can be properly ordered using these clocks Clock condition: –For any event a, b, if a  b then C(a) <C(b)

26. 26 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Implementation of Logical Clocks According to our definition of the happens- before relation, the clock condition is satisfied if the following two conditions hold: –C1: if a and b are events in process P i, and a comes before b, then C i (a) < C i (b) –C2: if a is the sending of a message by process P i and b is the receipt of that message by process P j, then C i (a) < C j (b)

27. 27 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Implementation of Logical Clock To meet C1: –Each process P i increments C i between any two successive events To meet C2: –(a) if event a is the sending of a message m by process P i, then the message m contains a timestamp T m = C i (a). –(b) Upon receiving a message m, process P j sets C j greater than or equal to its present value and greater than T m

28. 28 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Implementation of Logical Clocks by Counters A Lamport logical clock is a monotonically increasing software counter Each process P i keeps its own logical clock C i which is used to apply Lamport timestamps to events To capture the happens-before relation →, processes update their logical clocks and transmit the values of their logical clocks in messages as follows: –Before each event at P i : C i := C i +1 –When P i sends a message m, it piggybacks t = C i –When P j receives (m,t): C j := max(C j,t) + 1 e → e’ => C(e) < C(e’)

29. 29 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Implementation of Logical Clocks by Counters Question: Which two events are concurrent?

30. 30 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Total Ordering of Events We can use the logical clocks satisfying the Clock Condition to place a total ordering on the set of all system events –Simply order the events by the times at which occur –To break the ties, Lamport proposed the use of any arbitrary total ordering of the processes, i.e. process id

31. 31 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Total Ordering of Events Using this method, we can assign a unique timestamp to each event in a distributed system to provide a total ordering of all events Very useful in distributed system –Solving the mutual exclusion problem –Totally ordered reliable multicast => needed to build fault tolerant systems

32. 32 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Vector Timestamps Lamport timestamps do not guarantee that if C(a) < C(b) then a indeed happened before b We need vector timestamps for that –Each process P i has an array V i [1..n], where V i [j] denotes the number of events that process P i knows have taken place at process P j –When P i sends a message m, it adds 1 to V i [i], and sends V i along with m as vector timestamp vt(m) => upon arrival, each other process knows P i ’s timestamp V i (a) < V j (b)  –V i (a)[k] <= V j (b)[k] for every k, and –V i (a)[i] < V j (b)[j]

33. 33 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Vector Timestamps When a process P j receives a message m from P i with vector timestamp vt(m), it (1) updates each V j [k] to max(V j [k],vt(m)[k]), and (2) increments V j [j] by 1. Book is wrong! (not increment V j [i] by 1)

34. 34 Fall Semester 2006EEC-681: Distributed Computing SystemsWenbing Zhao Vector Timestamps Example: