1. Lecture 5 Time and synchronization
Distributed Systems
Lecture 5
Time and synchronization

2. Previous lecture
Failure detection
Approaches
Metrics
Gossip protocols

3. Motivation
You want both audio and video to be synchronized when watching a movie
You want to catch the 6:05 train but your clock is off my 7 minutes
What if it is late by 7 minutes?
What if it is fast by 7 minutes?

4. Motivation Cloud airline reservation system
Server A receives a client request to purchase last ticket on flight 123.
Server A timestamps purchase using local clock 9h:15m:32.45s, and logs it. Replies ok to client.
That was the last seat. Server A sends message to Server B saying “flight full.”
B enters “Flight ABC 123 full” + local clock value (which reads 9h:10m:10.11s) into its log.
Server C queries A’s and B’s logs. Is confused that a client purchased a ticket after the flight became full.
May execute incorrect or unfair actions.

5. Synchronization
At any time the values of all non faulty clocks must be approximately equal (within Δmax).
There is a small bound on the amount by which a non faulty process is changed.

6. System definition A set of N distributed process
Every process has a local physical clock
No direct access to a shared global clock
Communication between processes is message based

7. Applications At most once message delivery Cache consistency
Active replication
Medium access control
GPS
Global system for mobile communications (2nd generation)

8. Time division multiple access
Requirement:
Real-time communication using a shared medium
Problem:
Collisions can arbitrarily delay messages
Solution:
Synchronize clocks to determine access slots

9. Time division multiple access
Synchronize clocks to determine access slots
Frame based data flow
Divide frames into slots
Scheduler assigns processes to slots
Clock synchronization: collision free schedule execution 1.
2-3. 4.

10. Problem: instable clocks
Quartz oscillators oscillate at slightly different frequencies

11. Keeping time Oscillator In a computer  Pendulum Quartz crystal
CMOS
Microwave
In a computer 

12. Clock drifts in distributed systems

13. Clock resynchronization
Should we set back time?

14. Synchronization algorithms

15. Internal vs. external synchronization
Synchronize Δmax w.r.t. external time reference
Internal
Synchronize Δmax w.r.t. other system members
Externally synchronized clocks are also internally synchronized. The converse is not true.

16. Hardware vs. software synchronization
Hardware (assisted) clock
Precise (e.g., phase locking)
Expensive (extra hardware needed)
Attach GPS receivers to every machine
Software clock
Less precise
More flexible
Cheap

17. Simple method Issue RPC to server to query for time
Does not account for network or processing latency
All machines should have ~same time not necessarily the correct time!

18. Cristian’s algorithm 1989 – Flaviu Cristian Probabilistic algorithm
Only achieves synchronization if RTT << accuracy
Used in low latency intranets
Algorithm
P requests time from S connected to a UTC time source
S timestamps message from P with T and responds immediately
P sets time to be T + RTT/2
If min is the minimum time for one way transmission
Then time at S when P receives message is [T + min, T + RTT - min]
Therefore the accuracy of the algorithm is RTT/2 – min
If RTT is long repeat request
Take request with smallest RTT
If non uniform RTTs  take weighted average to compute error

19. Berkeley’s algorithm 1989 – Gusella and Zatti @ Berkeley
Assumes no machine has an accurate time source
For intranets
Algorithm
Pick S using an election process
S polls Ps who reply with their time
S observes RTT and estimates its own and Ps’ times
S averages the clock time by ignoring outliers
S sends out adjustments (+/-) to each P
WHY not absolute times?
Method cancels individual time drifts
Initial experiments shown clocks to be 20-25ms in sync
P usually applies the - correction over a period of time by slowing down the clock
not to break the monotonic time property

20. Network Time Protocol (NTP)
1985 – one of the oldest Internet protocols in use
Designed by David Mills
Coordinates participating computers within few ms of UTC
Tens of ms over Internet
<1ms in LANs
Current version is NTPv4

21. NTP
Uses a network of time servers to synchronize all processes on a network.
Time servers are connected by a synchronization subnet tree. The root is in touch with UTC. Each node synchronizes its
children nodes.
Primary server, direct synch. 1
Main adv: f-t of av. and accuracy.
Synch: multicast mode or procedure call mode (~Cristian’s algorithm) or symmetric mode (next slide)
Secondary servers, synched by the primary server 2 2 2
Strata 3, synched by the secondary servers 3 3 3 3 3 3

22. NTP A typical client will poll 3 or more servers to sync its time
Time offset and roundtrip delay: T i
i-1 -2 - 3
Server B
Server A
Time m m'
t and t’: actual transmission times
for m and m’(unknown)
o: true offset of clock at B
relative to clock at A
T(B) – T(A)
oi: estimate of actual offset
between the two clocks
di: estimate of accuracy (delay) of oi ;
total transmission times for m
and m’; di=t+t’
T(ABA)

23. NTP accuracy problem Accurate offset from a suitable sample population
Solution:
Minimum filter
Order m readings according to RTT
Select the lowest RTT reading

24. Next lecture
Global states and snapshots