1. Chap 5 Distributed Coordination
Physical clock synchronization
Causality relation
Snapshot taking
Lamport’s logical clock
Vector logical clock
Multicast
Event notification
Leader election
Distributed mutual exclusion
01/06/2019

2. 5.1 Time and clock Two roles of time
- Defines temporal order among events
- Duration (measured by timer)
UTC (Coordinated Universal Time) is based
on Cesium-133 atom oscillation; located at over 200 labs in the world
Leap seconds: to take care of Earth’s slowing rotation
With satellites, 0.5ms accuracy is possible.
(100 MIPS  50,000 instructions in 0.5ms).
01/06/2019

3. Clock skew
Skew: Clock reading (from single clock) is location-dependent, e.g., distance from satellite or clock source on a circuit board
Drift: Multiple clocks.
t = the real time
Cp(t) = the reading of a clock p at time t (Cp(t)= t for ideal clock)
dCp(t) /dt = ticking rate (dCp(t) /dt = 1 for ideal clock)
max drift rate r
1- r  dCp(t) /dt  1+ r
In t (sec), two clocks with drift rate r may drift
apart by 2rt. 2rt < d  resynch every d/2r sec
01/06/2019

4. Clock synchronization methods
Perfect synchronization physically impossible.
Time server can receive UTC signals.
Each client has reasonably accurate quartz
clock, and periodically gets time from server.
Client
Time
server
UTC
Client
01/06/2019

5. Cristian’s method Tp T0 I Tp T1 t Server Client
Estimate Tp (propagation delay) from
T1 – T0 = 2 x Tp + I.
where I = processing time.
Current time = t (server’s time in message) + Tp.
01/06/2019

6. OSF DCE Time is an interval [t-e, t+e]. TS1 TS2 TS3 TS4 Reject
New time interval
Two intervals overlap  cannot say which time is earlier
(In case of overlap, Unix make should recompile).
01/06/2019

7. 1. All events in a process are naturally ordered.
e1: Get up
e2: Start breakfast
e3: End breakfast
e4: Leave for work
e5: Make flight
reservation
e6: ….
Process 2
e1: Get up
e2: Start breakfast
e3: End breakfast
e4: Leave for work
e5: Make flight
reservation
e6: ….
Observe:
1. All events in a process are naturally ordered.
2. Some events may causally affect an event in
another process.
01/06/2019

8. 5.2 Causality relation Time p1 p2 p3 p4 P q2 q3 q4 q5 Q q1 R r1 r2 r3
transitive
Event changes state of process.
State remains same till next event occurs.
01/06/2019

9. Formal definition a b defined by:
If a occurs earlier than b in a process, then ab.
If a is sending event and b is receiving event of same message, then a b.
If ab and bc, then ac. [Transitive]
If a b, then a causally precedes (or happened before) b; a and b are causally related
a and b are concurrent if neither ab nor ba.
01/06/2019

10. Message-related events
Sending event
Receiving event
Message arrival (at kernel) and delivery
(to user process): Kernel can control timing of delivery after arrival.
Previous diagram shows only delivery times.
01/06/2019

11. Snapshots (taken at 2:00pm by local clocks)
$100 $0 $0
$100
1:59pm
$100
In channel $0
$100
$100
2:01
$100
sum = $100
sum = $0
sum = $200
(a)
(b)
(c)
Snapshots taken at
01/06/2019

12. Census taking in ancient kingdom
Village
Village
Village
Village
Want to take census counting all people, some of whom may be traveling on highways.
01/06/2019

13. Census taking algorithm
Close all gates into/out of each village (process) and count people (record process state) in village; these actions need not be synched with other villages
Open each outgoing gate and send official with a red cap (special marker message).
Open each incoming gate and count all travelers (record channel state= messages sent but not received yet) who arrive ahead of official.
Tally the counts from all villages.
01/06/2019

14. Algorithm SNAPSHOT
All processes are initially white: Messages sent by white(red) processes are also white (red)
MSend [Marker sending rule for process P]
Suspend all other activities until done
Record P’s state
Turn red
Send one marker over each output channel of P.
MReceive [Marker receiving rule for P]
On receiving marker over channel C,
if P is white { Record state of channel C as empty;
Invoke MSend; }
else record the state of C as sequence of white messages received since P turned red.
Stop when marker is received on each incoming channel
01/06/2019

15. Property
If network is strongly connected and at least one process initiates MSend, then SNAPSHOT
will take consistent global snapshot (collection of process states and channel states).
01/06/2019

16. Snapshots taken by SNAPSHOT algorithm
msgs arriving
before maker
constitute
channel state
$100
$100 in
channel $0 $0 $0
sum = $100
sum = $100
(a)
(b) OK OK
Need not use time.
01/06/2019

17. Snapshots taken by SNAPSHOT
$100
$100 $0
marker
marker
$100
marker
$100
sum = $200
sum = $100
(c)
(d)
Cannot happen
Will be like this
01/06/2019

18. Cuts corresponding to snapshots
Note that
they intersect
Cuts corresponding to snapshots A B A B
$100
$100 in
channel $0 $0 $0
sum = $100
sum = $100
(a)
(b)
Cut is where past events end.
01/06/2019

19. Cuts
Cut C divides all events to PC (those which happened in the past relative to C) and FC (future events).
Cut C is consistent if there is no message whose sending event is in FC and whose receiving event is in PC.
01/06/2019

20. Progress shown by cuts Time p1 p2 p3 p4 P Q q1 q2 q35 7 8
There are 5*4 = 20 possible cuts.
01/06/2019

21. Inconsistent cuts Time p1 p2 p3 p4 P Q q1 q2 q3
2*3 + 1*3 = 9 are inconsistent, and 11 are
consistent.
Inconsistent cut cannot actually happen States in
Inconsistent cut could not have coexisted.
01/06/2019

22. More examples Time p1 p2 p3 p4 P q2 q3 q4 q5 Q M q1 R r1 r2 r4 r3
inconsistent
cut
consistent
cut
01/06/2019

23. State recorded by SNAPSHOT consistent cut
$100 M
State of A before
M was sent
$100
State of B after
M was received
sum = $200
Msg M goes from future to past 
SNOPSHOP never generates such cut
(see slide 17)
01/06/2019

24. More consistent cuts Time p1 p2 p3 p4 P q2 q3 q4 q5 Q q1 R r1 r2 r4 r3
01/06/2019

25. Checkpointing
Cut C is consistent  C doesn’t contradict sequence of events experienced by any site  can assume it did exist at the same time
Can use snapshot as checkpoint, from which activity in distributed system can be resumed after crash
01/06/2019