2. Distributed Systems Fall 2010 Time and synchronization

3. Fall 20105DV0203 Outline Introduction Basic definitions Synchronization algorithms –Synchronous systems –Cristian's algorithm –Berkeley algorithm –Network Time Protocol Summary

4. Fall 20105DV0204 Time, and the lack thereof A global notion of the correct time would be tremendously useful. Why? –Consistency of distributed data, transactions, authenticity checks (ticket lifetimes), duplication detection, distributed debugging and garbage detection, etc.

5. Fall 20105DV0205 Time, and the lack thereof Why do we not have global time? –Clocks drift, are inaccurate, may fail arbitrarily, etc. –Time is relative, and depends on the observer of the timed events Causal relationships (cause and effect) may not be violated

6. Fall 20105DV0206 Basic definitions Distributed system is P, consisting of N processes: p i, i = 1, 2, …, N Each process has state s i Processes communicate only via message passing (network) Events e occur in processes –Internal events –Send events –Receive events

7. Fall 20105DV0207 Basic definitions Events are ordered within a process by the relation → i e 0 → i e 1 → i e 2 Define a history of p i as the events as described by → i history(p i ) = h i =

8. Fall 20105DV0208 Basic definitions Clock skew –Instantaneous difference between readings of any two clocks Clock drift –Variations in how clocks count time (oscillations in a crystal), which cause divergence between clocks

9. Basic definitions Clock drift rate –Change in offset between clock and a perfect clock Consumer level clocks 10 -6 seconds/second, roughly 1 second for each 11.6 days Fall 201095DV020

10. Fall 20105DV02010 Computer clocks Hardware clock H(t) –Gives “raw” time reading Software clock –C(t) = αH(t) + β –Scaled by OS to give accurate time –Used for timestamps

11. Fall 20105DV02011 Time sources Coordinated Universal Time (abbreviated UTC, thanks to the French) –Atomic clocks –Used for synchronization of all kinds of equipment (e.g. your computer, GPS, fancy radio-controlled clocks, etc.)

12. Fall 20105DV02012 Synchronization types External synchronization –Processes are synchronized to external time source (e.g. UTC) Internal synchronization –“Correct time” exists only within a group of processes –Must not be synchronized to external source

13. Fall 20105DV02013 Correctness and monotonicity Correctness (drift is bounded): (1 – p)(t' – t) ≤ H(t') – H(t) ≤ (1 + p)(t' – t) –Forbids “jumps” in hardware clocks to the bound p Monotonicity (ever-increasing) t' > t ⇒ C(t') > C(t) –Note: only deals with software clock –Simpler, and often sufficient

14. Fall 20105DV02014 Synchronization algorithms Internal synchronization –In synchronous systems (trivial case) –Berkeley algorithm External synchronization –Cristian's algorithm –Network Time Protocol (NTP)

15. Fall 20105DV02015 Clock synchronization in synchronous systems Synchronous systems define bounds on all relevant parts –Clock drift –Message transmission delays –Process execution step requirements Send request, get response back Internal

16. Clock synchronization in synchronous systems Only uncertainty is actual current transmission delay u = (max – min) –Set time to (time in response) + u/2 –For N processes, optimum bound is u(1 - 1/N) Internal Fall 2010165DV020

17. Fall 20105DV02017 Cristian's algorithm S is connected to time source p requests (m r ) and receives (m t ) time –S records time as soon before transmitting message as possible –p knows total round-trip-time T round –Simply set time to (t + T round / 2)? External

18. Fall 20105DV02018 Cristian's algorithm Only if at same LAN! But then, if minimum transmit time (t min ) is known: –Earliest time S could have placed time in m t was t min after p dispatched mr, and t min before p received m t –[t + t min, t + T round – t min ] –Width of range is (T round – 2 t min ), so accuracy is +-(T round /2 - t min ) External

19. Fall 20105DV02019 Cristian's algorithm Single point of failure! Crashing server? –Multicast to group of servers Fake servers? –Establish cryptographic authentication Arbitrarily failing servers? –Have enough correct ones to achieve agreement External

20. Fall 20105DV02020 Berkeley algorithm Uses Cristian's methods Master/Slave relationship Master polls slaves –Gets current time in each slave –Sends the offset from own time to each slave Master fails? –Crash: elect a new one! –Arbitrary failure? Oops… Internal

21. Fall 20105DV02021 Network Time Protocol Unlike the others, designed for WAN rather than LAN use –Time servers close to the time source are more trusted –Redundant paths → survives disconnects –Massively scalable –Authentication of time servers to avoid propagation of arbitrary failures External

22. Fall 20105DV02022 Network Time Protocol Synchronization subnets –Primary level (stratum) is directly connected to time source –Secondary level syncs to primary, tertiary to secondary, etc. High strata number means less reliable –Dynamically reconfigurable: if time source goes down, primary level becomes secondary level External

23. Fall 20105DV02023 Network Time Protocol Multicast mode –“Time is X” between LAN nodes Only as accurate as LAN allows Used only for unimportant nodes Procedure-call mode –Similar to Cristian's algorithm –More accurate than multicast mode Symmetric mode –Pairs of messages –Used in lower strata External

24. Fall 20105DV02024 Network Time Protocol All messages sent over UDP For procedure-call and symmetric mode, messages contain –Local time of previous NTP messages between the nodes were sent and received –Local time of current message transmission Receiver notes local time when message is received External

25. Fall 20105DV02025 Network Time Protocol Delay in Server B may be non- negligible Messages may be lost along the way External

26. Fall 20105DV02026 Network Time Protocol For each message pair calculate o i estimated offset between clocks d i total transmission time (delay) True offset is denoted o (without the index) Denote transmission time of m as t, and that of m' as t' External

27. Fall 20105DV02027 Network Time Protocol T i-2 = T i-3 + t + o T i = T i-1 + t' – o leads to d i = t + t' = T i-2 – T i-3 + T i – T i-1 also o = o i + (t' – t)/2, where o i = (T i-2 – T i-3 + T i-1 - T i )/2 External

28. Fall 20105DV02028 Network Time Protocol Since t, t' ≥ 0, we know that o i – d i /2 ≤ o ≤ o i + d i /2 Or, in English: o i is an estimate of the offset, and d i is a measure of its accuracy External

29. Fall 20105DV02029 Network Time Protocol Pairs are retained for quality calculations NTP peers communicate with many other peers, to decrease error External

30. Fall 20105DV02030 Summary We do not have universal time –But we can synchronize clocks “reasonably well” anyway Internal vs. external synchronization Real-time systems must use more sophisticated algorithms than what we have seen during this lecture!

31. Fall 20105DV02031 Summary Algorithms –Synchronous system (trivial) –Cristian's algorithm Used in many others –Berkeley algorithm Master/Slave application of Cristian's for internal synchronization –Network Time Protocol Suitable for WANs Message pairs

32. Fall 20105DV02032 Next lecture Logical time Global states Distributed debugging