1. Time and Global States Chapter 11

2. Why time? Time is an Important and interesting issue in distributes systems. One we can measure accurately. Can use as a metric. Example: e-commerce transaction timestamp for auditing and accountability purposes. Need time for maintaining consistency of databases. According to Einstein’s theory of relativity a stationary observer on earth and an observer moving away from earth, if they observed an event at the same time, the event may “happen” at different times for them Relative order of two events can be reversed unless one caused the other. Thus the notion of physical time is problematic in distributed systems. We will examine synchronization of clocks using message passing; We will study logical clocks: vector clocks. We will also look at algorithms to capture global states of distributed systems as they execute.

3. Clocks, event and process states History (pi) = hi =  i represents relation A processor clock is derived from a hardware clock: C i (t) = αH i (t) + β May result in clock skew and drift Coordinated universal time: most accurate clock use atomic oscillators with very low drift rates of 1 in 10 13 In use since 1967: standard second is defined as 9,192,631,770 periods of transitions between the two hyperfine levels of the ground state of Caesium-133 (Cs 133 ) Days, hours, years are rooted in astronomical time. Japanese earthquake should have caused Earth to rotate a bit faster, shortening the length of the day by about 1.8 microseconds (a microsecond is one millionth of a second). Currents within earth’s core also affects decreases and increases.

4. Coordinated Universal Time Abbreviated UTC (is equivalent to Greenwich Mean Time (GMT)). This is based on atomic time ( leap second is inserted, rarely leap second is deleted to keep up astronomical time). UTC signal are regularly synchronized and broadcast. In USA the radio station WWV broadcasts time signals on several shortwave frequencies. Satellite sources for time include Global Positioning Systems (GPS). NIST is another source. …

5. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.1 Skew between computer clocks in a distributed system

6. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.2 Clock synchronization using a time server: Critian’s method {1989} m r m t p Time server,S The clock requests time and synchronizes only if the round-trip is within a certain bound. Very unlikely in a busy network! Single server problem/ single point failure. The Berkeley algorithm works on master slave mode, polling all the nodes in a system and synchronizing them.

7. NTP: The Network Time Protocol Mills {1995} To provide a service enabling clients across the internet to be synchronized accurately to UTC – NTP employs statistical techniques for filtering of timing data and it discriminates between the quality of timing data from different servers. To provide a reliable service that can survive lengthy losses of connectivity: – There are redundant servers and redundant paths between servers; server can reconfigure in case of failures. To enable clients to re-synchronize frequently to offset the rates of drift found in most computers; this service will scale to large number of clients and servers To provide protection against interference with the time service whether accidental or malicious Hierarchical structure for the time servers: primary and secondary at different strata. NTP servers synchronize with each other in one of three modes: multicast, procedure-call, symmetric-mode in the order of accuracy All messages use UDP and so deliver unreliably Messages between a pair is discusses next Each message carries three time stamps: sent time, recd time, time of this message.

8. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.3 An example synchronization subnet in an NTP implementation 1 2 3 2 33 Note: Arrows denote synchronization control, numbers denote strata.

9. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.4 Messages exchanged between a pair of NTP peers T i T i-1 T i-2 T i-3 Server B Server A Time mm' Time

10. Algorithm for Clock Synch Study and understand the algorithm in page 444. We will discuss this during lecture.

11. Logical Time and logical Clock From the point of view of a single process, events are ordered uniquely by times shown on the clock. Lamport used causality as means of defining the concept of “logical time” and “logical clock” A new relationship for partial ordering was defined: happened-before (HB) 

12. Happened-Before Relationship  HB1: There exists a process and two events e1 and e2 that occurred in that order then, e1  e2 HB2: For any message m, send(m)  receive (m) HB3: if there are events e1, e2, e3, e1  e2, e2  e3, then e1  e3

13. Logical clocks (LC) Lamport designed a simple mechanism by which HB ordering can be captured numerically, called logical clock. A logical clock is monotonically increasing software counter, with no relationship to physical clock. Rules for logical clocks: LC1: Li is incremented before each event in Pi LC2: – When a process Pi sends a message m, it piggy backs on m t = Li – On receiving the message process Pj computes Lj = max(Lj, t), applies LC1, and timestamps receive message event.

14. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.5 Events occurring at three processes

15. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.6 Lamport timestamps for the events shown in Figure 11.5

16. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 One more example LC2: Lamport timestamps for the events shown in Figure 11.5 2345 ?

17. Totally ordered logical clock Vector clocks: Look at the algorithm in page 447 Consider a system of N interacting processes. Then the logical clock is a vector of size N. (this is a disadvantage) Comparison of the clocks in this case a vector comparison (somewhat complex than the scalar)

18. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.7 Vector timestamps for the events shown in Figure 11.5

19. Global States Detecting global properties is very critical for successful implementation of distributed systems How to detect a set of consistent states among distributed processes? This set of local states for a global set called the “cut”. This is made complex by a message passed around among the processes. A cut C is consistent if for each event it contains all the events that happened-before(HB) that event. A consistent global state is one that corresponds to a consistent cut.

20. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.8 Detecting global properties

21. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.9 Cuts m 1 m 2 p 1 p 2 Physical time e 1 0 Consistent cut Inconsistent cut e 1 1 e 1 2 e 1 3 e 2 0 e 2 1 e 2 2

22. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.10 Chandy and Lamport’s ‘snapshot’ algorithm Marker receiving rule for process p i On p i ’s receipt of a marker message over channel c: if (p i has not yet recorded its state) it records its process state now; records the state of c as the empty set; turns on recording of messages arriving over other incoming channels; else p i records the state of c as the set of messages it has received over c since it saved its state. end if Marker sending rule for process p i After p i has recorded its state, for each outgoing channel c: p i sends one marker message over c (before it sends any other message over c).

23. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.11 Two processes and their initial states

24. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Figure 11.12 The execution of the processes in Figure 11.11