Synchronization CSCI 4900/6900

Importance of Clocks & Synchronization Avoiding simultaneous access of resources –Cooperate to grant exclusive access for small durations Process may need to agree upon ordering of events –First person to complete assignment will receive bonus –How to infer who completed first? Synchronization & ordering is difficult in distributed setting –We have seen this several times !!!

Chapter Outline Actual time-based synchronization –Physical clocks –Synchronizing multiple clocks Relative ordering-based synchronization –Logical clocks, Lamport’s algorithm, Vector time stamps Global state Leader election algorithms (time permitting) Mutual exclusion in distributed systems Distributed transactions

Clocks in Systems Notion of time in centralized system is unambiguous –Do a system call to get current time Important property of time – Always moves forward –If B executes the system call after A, B will never get a lower value Many system operations need some notion of time for correctness Example – “Make” program in Unix –Re-compiles only files that were changed since last compilation –What happens when there is no global agreement on time?

Clock Synchronization When each machine has its own clock, an event that occurred after another event may nevertheless be assigned an earlier time.

System Clocks & Clock Skew Almost all computers have clocks (timers) A precisely cut quartz crystal kept under tension –Oscillates at a well-defined frequency Two registers – Counter & holding register –Each oscillation of crystal decrements counter by 1 –Interrupt when counter is zero & reload from holding reg. –Each interrupt is a tick Clocks at multiple CPUs cannot be guaranteed to oscillate at exact same frequency Difference between various clocks – Clock Skew

How is Time Actually Measured? Solar time – Based on earth’s rotation –Transit of sun: Sun reaching the highest apparent in sky –Solar day: Time b/w two consecutive sun transits –Solar second: 1/86400 th of a solar day –Mean solar second: 1/86400 th of a mean solar day Atomic time –Second: Time for cesium-133 atom to make 9,192,631,770 transitions –International atomic time (TAI) Leap seconds to resolve difference b/w TAI & solar time (UTC) NIST broadcasts UTC on radio station (WWV)

Physical Clocks (1) Computation of the mean solar day.

Physical Clocks (2) TAI seconds are of constant length, unlike solar seconds. Leap seconds are introduced when necessary to keep in phase with the sun.

Clock Synchronization Algorithms Two related problems –If one machine has WWV receiver: synchronize all machines to machine with the WWV receiver –No WWV receiver: Keep all machines relatively synchronized Many algorithms with some key assumptions –Each machine has timer that causes interrupt H times/sec –Increments software clock on each interrupt –C p (t) indicates clock value when UTC time is t –In ideal world dC/dt = 1

Drift & Max. Drift Rate The relation between clock time and UTC when clocks tick at different rates. In real world dC/dt is not one Maximum drift rate: ρ such that 1- ρ ≤ dC/dt ≤ 1 + ρ –Specified by manufactures

Clock Synchronization Algorithms Two clocks drifting from UTC in opposite directions at rate of ρ need to be synchronized every δ/2ρ secs. Chritian’s Algorithm: –Suited when one machine has WWV receiver –Each machine sends a request to time server periodically (period < δ/2ρ ) seconds –Time server responds with its current time (C UTC ) Simple scheme –Set receivers time to C UTC –Two problems Clock might run backward !!! Doesn’t consider processing time

Cristian's Algorithm Introduce change gradually – Reduce time by a small amount Add T0 – T1/2 to C UTC

The Berkeley Algorithm a)The time daemon asks all the other machines for their clock values b)The machines answer c)The time daemon tells everyone how to adjust their clock

Logical Clocks For many applications it is sufficient if all machines agree upon some time –Synchronization with UTC not needed Logical clocks Lamport showed that in many cases clock synchronization is not needed –What actually is needed is agreement with regards to ordering of events –Example – Compilation occurred before file editing

Concepts in Logical Clocks Happens-before relation a->b if one of the following is true –a and b are events in same process and a occurs before b –If a is an event of sending a message and b is the event of receiving the same message is another process a->b. Implies that message cannot be received before it is sent Happens-before is transitive –If a->b and b->c then a->c If two events in x and y are in two processes that never exchange messages then x & y are concurrent

Lamport’s Timestamps Algorithm We need a way of measuring time such that for every event a we can assign a time C(a) on which all processes agree. If a->b than C(a) < C(b) Alternatively –If a and b are in same process and a precedes b C(a) < C(b) –If a representing sending a message and b represents receiving the same message C(a) < C(b) Clock C should always move forward

Lamport’s Algorithm Each process runs with its own clock –Clocks need not be synchronized Processes sends messages that are time-stamped with the local clock time When a process receives a message –If its local clock is more than time-stamp of message no need of any adjustment –If local clock less than message’s time-stamp, local clock incremented to one more than message’s time-stamp Between every two events the clock is incremented Use decimal point followed by process number for global uniqueness

Example: Totally-Ordered Multicasting Updating a replicated database and leaving it in an inconsistent state.

Globally Ordered Multicast Database is replicated at multiple locations Query is forwarded to nearest copy, where as update is sent to all replicas Suppose client deposits $100 to account (having $1000) and bank starts a process to add 1% interest Two replicas may get these update requests in different order –One replica shows $1110, where as the other $1111 What we need is globally ordered multicast

Lamport’s Algorithm for GOM Messages timestamped with sender’s local time Assumptions –When message is multicast it is also sent to sender –Messages from same sender arrive sequentially When a process receives a message, it is put into local queue Receiver multicasts acknowledgement to everyone Local times adjusted through Lamport’s algorithm Each process will have the exact same local queue Deliver the first message if all the ACKs are received

Causality Lamport’s algorithm leads to all events being ordered irrespective of whether they are dependent on each other Example – Electronic bulletin board –Users may post new articles or respond to existing ones –No dependence between different threads Lamport’s algorithm imposes order even between independent events CAUSALITY – Two events are ordered only if one is dependent on the other

Vector Timestamps for Causality Each event is assigned a vector timestamp –If VT(a) < VT (b) then a is said to causally precede b Each process maintains a vector P i maintains V i –V i [i] – Number of events occurred at P i –V i [j] = k then P i knows that k events have occurred at P j When P i sends message to P j, it piggybacks Vi P j knows about how many events have occurred at Pi P j also knows how many events have occurred at other processes –Pj adjusts its local vector by setting V j [k] = max{V j [k], vt[k]} –Increments V j [i] by 1 Delivers a message r only if vt(r) [j] = Vk[j] + 1 and vt(r) [i] j

Global State Local states of each processes together with messages in transit Global states is needed for detecting termination conditions/deadlocks Distributed snapshot algorithm (Chandy & Lamport) –Reflects state where distributed system might have been Always represents a consistent global state –If it is recorded that a process received a message we should have also recorded that the message has been sent Distributed snapshots via Cuts

Cuts for Global State a)A consistent cut b)An inconsistent cut

Distributed Snapshots Algorithm Assumption – Processes are connected via uni-directional point-to-point links (channels) Any process (p) may initiate algorithm P records its local state and sends a marker to along each outgoing channel –Marker indicates that receiver should participate in the snapshot algorithm When process Q receives a marker –It saves its local state, if it has already not done so, and sends marker along each outgoing channel –If Q had recorded local state, it records the state of the channel –DONE to indicate that process has done its part

Global Snapshots a)Organization of a process and channels for a distributed snapshot

Global State (3) b)Process Q receives a marker for the first time and records its local state c)Q records all incoming message d)Q receives a marker for its incoming channel and finishes recording the state of the incoming channel

Termination Detection Detecting the termination of a distributed computation Initiator starts the process by sending a marker to all processes Each process records its state On completing its part of global snapshots, it checks whether there are no messages in transit If no transit messages process sends DONE to initiator Else sends CONTINUE –Initiator has to restart the detection algorithm

Leader Election in Distributed Systems Many distributed systems need one process to act as a coordinator/initiator –“First among equals” not “one above the rest” Does not matter which process takes up responsibility –But all process should agree about who would be the leader Assumption: Each process has a unique identifier Bully Algorithm Ring Algorithm

Bully Algorithm When a process P notices that current coordinator has failed, it sends an ELECTION message to all processes with higher IDs If no one responds, P becomes the leader If a higher-up receives P’s message, it will send an OK message to P and execute the algorithm Process with highest ID takes over as coordinator by sending COORDINATOR message If a process with higher ID comes back, it takes over leadership by sending COORDINATOR message

Bully Algorithm - Example The bully election algorithm Process 4 holds an election Process 5 and 6 respond, telling 4 to stop Now 5 and 6 each hold an election

Bully Algorithm - Example (2) d)Process 6 tells 5 to stop e)Process 6 wins and tells everyone

Ring Algorithm When a process P notices current coordinator’s failure, it builds an ELECTION message –Message contains the P’s ID P sends message to it successor –Skip if the successor is down Each process adds its ID to the ELECTION message When election message reaches the initiator, it converts message to COORDINATOR –The process ID with highest ID is declared as new coordinator

Ring Algorithm - Example