1. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 6/A Synchronization Modified by Dr. Gheith Abandah

2. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Overview Clock Synchronization Physical Clocks Global Positioning System Clock Synchronization Algorithms Logical Clocks Lamport’s Logical Clocks Vector Clocks Mutual Exclusion Overview Centralized Algorithm A Decentralized Algorithm A Distributed Algorithm A Token Ring Algorithm A Comparison of the Four Algorithms Global Positioning of Nodes Election Algorithms Traditional Election Algorithms Elections in Wireless Environments Elections in Large-scale Systems

3. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization Need for clock synchronization: Consider Unix make utility. When each machine has its own clock, an event that occurred after another event may nevertheless be assigned an earlier time.

4. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Physical Clocks (1) The mean solar day is the time between to sun transits. 1 second = 1 solar day / (24*60*60). Sun transit: reaching highest apparent point. 300 millions years ago, the year was 400 days.

5. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Physical Clocks (2) Problem: Sometimes we simply need the exact time, not just an ordering. Solution: Universal Coordinated Time (UTC): Based on the number of transitions per second of the cesium 133 atom (pretty accurate), started in 1958. At present, the real time is taken as the average of some 50 cesium-clocks around the world. Introduces a leap second from time to time to compensate that days are getting longer. Note: UTC is broadcast through short wave radio (WWV) and geostationary environment operational satellite (GEOS). Satellites can give an accuracy of about ± 0.5 ms.

6. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Physical Clocks (3) International atomic time (TAI) seconds are of constant length, unlike solar seconds. Leap seconds are introduced when necessary to keep in phase with the sun. More than 30 seconds have been added since 1958.

7. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Global Positioning System (1) You can get an accurate account of time as a side-effect of GPS. There are 29 satellites that broadcast their positions and time.

8. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Global Positioning System (2) Real world facts that complicate GPS 1.It takes a while before data on a satellite’s position reaches the receiver. 2.The receiver’s clock is generally not in synch with that of a satellite. Therefore, need 4 satellites → 4 equations in 4 unknowns.

9. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization Algorithms (1) Problem: Suppose we have a distributed system with a UTC-receiver somewhere in it → we still have to distribute its time to each machine. Basic principle Every machine has a timer that generates an interrupt H times per second. There is a clock in machine p that ticks on each timer interrupt. Denote the value of that clock by C p (t), where t is UTC time. Ideally, we have that for each machine p, C p (t) = t, or, in other words, dC/dt = 1.

10. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization Algorithms (2) Goal: Never let two clocks in any system differ by more than δ time units → synchronize at least every δ /(2ρ) seconds.

11. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization Algorithms (3) Principle: Every machine asks a time server for the accurate time at least once every δ /(2ρ) seconds (Network Time Protocol). Note: Okay, but you need an accurate measure of round trip delay, including interrupt handling and processing incoming messages. Get time form a time server of a higher stratum level, like stratum-1 (has WWV receiver or atomic clock).

12. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Network Time Protocol Getting the current time from a time server. The NTP does 8 rounds and adopt the minimum time round. Offset = T 3 + ((T 2 -T 1 )+(T 4 -T 3 ))/2 - T 4 : -ve slowdown clock, +ve run faster, no jumbs.

13. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Berkeley Algorithm (1) The time server is proactive. (a) The time daemon asks all the other machines for their clock values.

14. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Berkeley Algorithm (2) (b) The machines answer.

15. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Berkeley Algorithm (3) (c) The time daemon tells everyone how to adjust their clock.

16. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization in Wireless Networks (1) Need to minimize power consumption, e.g., in WSN. (a) The usual critical path in determining network delays.

17. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Clock Synchronization in Wireless Networks (2) (b) The critical path in the case of Reference Broadcast Synchronization (RBS) where only receivers synchronize their clocks.

18. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Overview Clock Synchronization Physical Clocks Global Positioning System Clock Synchronization Algorithms Logical Clocks Lamport’s Logical Clocks Vector Clocks Mutual Exclusion Overview Centralized Algorithm A Decentralized Algorithm A Distributed Algorithm A Token Ring Algorithm A Comparison of the Four Algorithms Global Positioning of Nodes Election Algorithms Traditional Election Algorithms Elections in Wireless Environments Elections in Large-scale Systems

19. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Lamport’s Logical Clocks (1) The notion of ordering The "happens-before" relation → can be observed directly in two situations: If a and b are events in the same process, and a occurs before b, then a → b is true. If a is the event of a message being sent by one process, and b is the event of the message being received by another process, then a → b If a → b and b → c, then a → c Note: This introduces a partial ordering of events in a system with concurrently operating processes.

20. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Lamport’s Logical Clocks (2) Each process P i maintains a local counter C i and adjusts this counter according to the following rules: 1.For any two successive events that take place within P i, C i is incremented by 1. 2.Each time a message m is sent by process P i, the message receives a timestamp ts(m) = C i. 3.Whenever a message m is received by a process P j, P j adjusts its local counter C j to max{C j, ts(m)}; then executes step 1 before passing m to the application.

21. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Lamport’s Logical Clocks (3) (a) Three processes, each with its own clock. The clocks run at different rates.

22. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Lamport’s Logical Clocks (4) (b) Lamport’s algorithm corrects the clocks.

23. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Lamport’s Logical Clocks (5) The positioning of Lamport’s logical clocks in distributed systems. Adjustments take place in the middleware layer.

24. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Example: Totally Ordered Multicasting (1) Updating a replicated database and leaving it in an inconsistent state. P1 adds $100 to an account (initial value: $1000) P2 increments account by 1% There are two replicas

25. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Example: Totally Ordered Multicasting (2) Solution Process P i sends timestamped message msg i to all others. The message itself is put in a local queue queue i. Any incoming message at P j is queued in queue j, according to its timestamp, and acknowledged to every other process. P j passes a message msg i to its application if: 1.msg i is at the head of queue j 2.for each process P k, there is a message msg k in queue j with a larger timestamp. Note: We are assuming that communication is reliable and FIFO ordered.

26. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Vector Clocks (1) We cannot conclude that a causally precedes b. Lamport’s clocks do not guarantee that if C(a) < C(b) that a causally preceded b Event a: m 1 is received at T = 16; Event b: m 2 is sent at T = 20.

27. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Vector Clocks (2) Vector clocks are constructed by letting each process P i maintain a vector VC i with the following two properties: 1.VC i [ i ] is the number of events that have occurred so far at P i. In other words, VC i [ i ] is the local logical clock at process P i. 2.If VC i [ j ] = k then P i knows that k events have occurred at P j. It is thus P i ’s knowledge of the local time at P j.

28. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Vector Clocks (3) Steps carried out to accomplish property 2 of previous slide: 1.Before executing an event P i executes VC i [ i ] ← VC i [i ] + 1. 2.When process P i sends a message m to P j, it sets m’s (vector) timestamp ts (m) equal to VC i after having executed the previous step. 3.Upon the receipt of a message m, process P j adjusts its own vector by setting VC j [k ] ← max{VC j [k ], ts (m)[k ]} for each k, after which it executes the first step and delivers the message to the application.

29. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Vector Clocks (4) Observation We can now ensure that a message is delivered only if all causally preceding messages have already been delivered. Adjustment P i increments VC i [i] only when sending a message, and P j “adjusts” VC j when receiving a message (i.e., effectively does not change VC j [j]). Pj postpones delivery of m until: 1.ts(m)[i] = VC j [i]+1 (next expected msg from P i ). 2.ts(m)[k] ≤ VC j [k] for all k ≠ i (P j has seen all msgs seen by P i ).

30. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Enforcing Causal Communication Enforcing causal communication using vector clocks. m* delivery to the application at P 2 is delayed because ts(m)[0] is not ≤ VC 2 [0]