Time, Clocks, and the Ordering of Events in a Distributed System Leslie Lamport Massachusetts Computer Associates,Inc. Presented by Xiaofeng Xiao.

Slides:



Advertisements
Similar presentations
Time, Clocks, and the Ordering of Events in a Distributed System
Advertisements

Virtual Time “Virtual Time and Global States of Distributed Systems” Friedmann Mattern, 1989 The Model: An asynchronous distributed system = a set of processes.
Synchronization Chapter clock synchronization * 5.2 logical clocks * 5.3 global state * 5.4 election algorithm * 5.5 mutual exclusion * 5.6 distributed.
Dr. Kalpakis CMSC 621, Advanced Operating Systems. Logical Clocks and Global State.
Distributed Systems Dinesh Bhat - Advanced Systems (Some slides from 2009 class) CS 6410 – Fall 2010 Time Clocks and Ordering of events Distributed Snapshots.
Distributed Systems Spring 2009
Ordering and Consistent Cuts Presented By Biswanath Panda.
1. Explain why synchronization is so important in distributed systems by giving an appropriate example When each machine has its own clock, an event that.
Synchronization Clock Synchronization Logical Clocks Global State Election Algorithms Mutual Exclusion.
Ordering and Consistent Cuts
20101 Synchronization in distributed systems A collection of independent computers that appears to its users as a single coherent system.
Ordering and Consistent Cuts Presented by Chi H. Ho.
Time, Clocks and the Ordering of Events in a Distributed System - by Leslie Lamport.
EEC-681/781 Distributed Computing Systems Lecture 10 Wenbing Zhao Cleveland State University.
EEC-681/781 Distributed Computing Systems Lecture 10 Wenbing Zhao Cleveland State University.
Lecture 12 Synchronization. EECE 411: Design of Distributed Software Applications Summary so far … A distributed system is: a collection of independent.
Time, Clocks, and the Ordering of Events in a Distributed System Leslie Lamport (1978) Presented by: Yoav Kantor.
Distributed Systems Foundations Lecture 1. Main Characteristics of Distributed Systems Independent processors, sites, processes Message passing No shared.
Dr. Kalpakis CMSC 621, Advanced Operating Systems. Fall 2003 URL: Logical Clocks and Global State.
TIME, CLOCKS AND THE ORDERING OF EVENTS IN A DISTRIBUTED SYSTEM L. Lamport Computer Science Laboratory SRI International.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved DISTRIBUTED SYSTEMS.
CIS 720 Distributed algorithms. “Paint on the forehead” problem Each of you can see other’s forehead but not your own. I announce “some of you have paint.
EECS 262a Advanced Topics in Computer Systems Lecture 14 Lamport Clocks and OCC October 20 th, 2014 John Kubiatowicz Electrical Engineering and Computer.
© Oxford University Press 2011 DISTRIBUTED COMPUTING Sunita Mahajan Sunita Mahajan, Principal, Institute of Computer Science, MET League of Colleges, Mumbai.
DISTRIBUTED ALGORITHMS By Nancy.A.Lynch Chapter 18 LOGICAL TIME By Sudha Elavarti.
Synchronization. Why we need synchronization? It is important that multiple processes do not access shared resources simultaneously. Synchronization in.
Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps.
Lamport’s Logical Clocks & Totally Ordered Multicasting.
“Virtual Time and Global States of Distributed Systems”
Synchronization Chapter 5.
Communication & Synchronization Why do processes communicate in DS? –To exchange messages –To synchronize processes Why do processes synchronize in DS?
Distributed Systems Principles and Paradigms Chapter 05 Synchronization.
Event Ordering Greg Bilodeau CS 5204 November 3, 2009.
CIS825 Lecture 2. Model Processors Communication medium.
9/14/20051 Time, Clocks, and the Ordering of Events in a Distributed System by L. Lamport CS 5204 Operating Systems Vladimir Glina Fall 2005.
Synchronization CSCI 4900/6900. Importance of Clocks & Synchronization Avoiding simultaneous access of resources –Cooperate to grant exclusive access.
D u k e S y s t e m s Asynchronous Replicated State Machines (Causal Multicast and All That) Jeff Chase Duke University.
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S
CS603 Clock Synchronization February 4, What is Clock Synchronization? All nodes agree on time What do we mean by time? –Monotonic –Any observation.
CS 3471 CS 347: Parallel and Distributed Data Management Notes13: Time and Clocks.
CS 347Notes 121 CS 347: Parallel and Distributed Data Management Notes12: Time and Clocks Hector Garcia-Molina.
Feb 15, 2001CSCI {4,6}900: Ubiquitous Computing1 Announcements.
Logical Clocks. Topics r Logical clocks r Totally-Ordered Multicasting.
Event Ordering. CS 5204 – Operating Systems2 Time and Ordering The two critical differences between centralized and distributed systems are: absence of.
Synchronization Chapter 5. Clock Synchronization When each machine has its own clock, an event that occurred after another event may nevertheless be assigned.
Lecture 1: Logical and Physical Time with some Applications Anish Arora CSE 6333 Notes include material from Dr. Jeff Brumfield.
Ordering of Events in Distributed Systems UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department CS 739 Distributed Systems Andrea C. Arpaci-Dusseau.
CS533 Concepts of Operating Systems Class 8 Time, Clocks, and the Ordering of Events in a Distributed System By Constantia Tryman.
COMP 655: Distributed/Operating Systems Summer 2011 Dr. Chunbo Chu Week 6: Synchronyzation 3/5/20161 Distributed Systems - COMP 655.
CIS 825 Review session. P1: Assume that processes are arranged in a ring topology. Consider the following modification of the Lamport’s mutual exclusion.
Mutual Exclusion Algorithms. Topics r Defining mutual exclusion r A centralized approach r A distributed approach r An approach assuming an organization.
CS3771 Today: Distributed Coordination  Previous class: Distributed File Systems Issues: Naming Strategies: Absolute Names, Mount Points (logical connection.
11-Jun-16CSE 542: Operating Systems1 Distributed systems Time, clocks, and the ordering of events in a distributed system Leslie Lamport. Communications.
Ordering of Events in Distributed Systems UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department CS 739 Distributed Systems Andrea C. Arpaci-Dusseau.
John Kubiatowicz and Anthony D. Joseph
SYNCHORNIZATION Logical Clocks.
Distributed Systems CS
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S
John Kubiatowicz Electrical Engineering and Computer Sciences
Outline Theoretical Foundations - continued Lab 1
Outline Theoretical Foundations
Chien-Liang Fok Distribution Seminar Chien-Liang Fok
TIME, CLOCKS AND THE ORDERING OF EVENTS IN A DISTRIBUTED SYSTEM
Distributed Systems CS
Chapter 5 (through section 5.4)
Basics of Distributed Systems
Distributed Systems CS
John Kubiatowicz Electrical Engineering and Computer Sciences
CSE 542: Operating Systems
Outline Theoretical Foundations
Presentation transcript:

Time, Clocks, and the Ordering of Events in a Distributed System Leslie Lamport Massachusetts Computer Associates,Inc. Presented by Xiaofeng Xiao

Abstract Concept of time and distributed system Concept of time and distributed system The Partial Ordering The Partial Ordering Logical Clocks Logical Clocks ---- Clock Conditions ---- Clock Conditions Total Ordering Total Ordering ---- Solve mutual exclusion problem ---- Solve mutual exclusion problem Anomalous Behavior Anomalous Behavior Physical Clocks Physical Clocks Conclusion Conclusion

Time One dimension. It can not move backward. It can not stop. One dimension. It can not move backward. It can not stop. It is derived from concept of the order in which events occur. It is derived from concept of the order in which events occur. The concepts “before” and “after” need to be reconsidered in a distributed system. The concepts “before” and “after” need to be reconsidered in a distributed system.

Distributed System A distributed system consists of collection of distinct processes which are spatially separated, and which communicate with one another by exchanging messages. A distributed system consists of collection of distinct processes which are spatially separated, and which communicate with one another by exchanging messages. It could be a network of interconnected computers, like ARPA ("Advanced Research Projects Agency" ) net, or just a single computer with separate processes. It could be a network of interconnected computers, like ARPA ("Advanced Research Projects Agency" ) net, or just a single computer with separate processes. It is sometimes impossible to say that one of two events occurred first in a distributed system. “happened before” is a partial ordering of the events in the system. It is sometimes impossible to say that one of two events occurred first in a distributed system. “happened before” is a partial ordering of the events in the system.

ARPA net

The Partial Ordering Our system: System is composed of a collection of processes. System is composed of a collection of processes. Each process consists of a sequence of events. Each process consists of a sequence of events. One event could be the execution of a subprogram on a computer or the execution of a single machine instruction. It depends upon the application. One event could be the execution of a subprogram on a computer or the execution of a single machine instruction. It depends upon the application. A single process is defined to be a set of events with an a priori total ordering. A single process is defined to be a set of events with an a priori total ordering.

Definition of “happened before” The relation “→” on the set of events of a system is the smallest relation satisfying the following three conditions: The relation “→” on the set of events of a system is the smallest relation satisfying the following three conditions: (1) If a and b are events in the same process, and a comes before b, then a→b. (1) If a and b are events in the same process, and a comes before b, then a→b. (2) If a is the sending of a message by one process and b is the receipt of the same message by another processes, then a→b. (2) If a is the sending of a message by one process and b is the receipt of the same message by another processes, then a→b. (3) If a→b and b→c then a→c. (3) If a→b and b→c then a→c. Two distinct events a and b are said to be concurrent if a — /→b and b —/→ a. Two distinct events a and b are said to be concurrent if a — /→b and b —/→ a. We assume that a —/→ a for any event a. We assume that a —/→ a for any event a.

Space-time diagram p 1 →r 4 since p 1 → q 2 and q 2 → q 4 and q 4 → r 3 and r 3 → r 4 p 1 →r 4 since p 1 → q 2 and q 2 → q 4 and q 4 → r 3 and r 3 → r 4 p 3 and q 3 are concurrent. p 3 and q 3 are concurrent.

Logical Clocks A clock is just a way of assigning a number to an event. Definition of logical clocks: A clock C i for each process P i is a function which assigns a number C i to any event a in that process. A clock C i for each process P i is a function which assigns a number C i to any event a in that process. The entire system of clocks is represented by the function C which assigns to any event b the number C = C j if b is an event b in process P j. The entire system of clocks is represented by the function C which assigns to any event b the number C = C j if b is an event b in process P j.

Clock Condition Clock Condition Clock Condition For any events a, b: if a→ b then C For any events a, b: if a→ b then C Clock Condition is satisfied if Clock Condition is satisfied if C1. If a and b are events in process P i, and a comes before b, then C i C1. If a and b are events in process P i, and a comes before b, then C i C2. If a is the sending of a message by process P i and b is the receipt of that message by process P j, then C i C2. If a is the sending of a message by process P i and b is the receipt of that message by process P j, then C i

Example of “ticks” C1 means that there must be a tick line between any two events on a process line C1 means that there must be a tick line between any two events on a process line C2 means that every message line must cross a tick line. C2 means that every message line must cross a tick line.

Redraw Figure 2

Clock Condition Now assume that the processes are algorithms, and the events represent certain actions during their execution. Process Pi’s clock is represented by a register Ci, so that Ci is the value contained by Ci during the event a. Now assume that the processes are algorithms, and the events represent certain actions during their execution. Process Pi’s clock is represented by a register Ci, so that Ci is the value contained by Ci during the event a. To meet condition C1 and C2, the processes need to obey the following rules: IR1. Each process Pi increments Ci between any two successive events. IR1. Each process Pi increments Ci between any two successive events. IR2. (a) If event a is the sending of a message m by process Pi, then the message m contains a timestamp Tm=Ci. (b) Upon receiving a message m, process Pj sets Cj greater than or equal to its present value and greater than Tm. IR2. (a) If event a is the sending of a message m by process Pi, then the message m contains a timestamp Tm=Ci. (b) Upon receiving a message m, process Pj sets Cj greater than or equal to its present value and greater than Tm.

Total Ordering We can use a system of clocks satisfying the Clock Condition to place a total ordering on the set of all system events. We can use a system of clocks satisfying the Clock Condition to place a total ordering on the set of all system events. We simply order the events by the times at which they occur We simply order the events by the times at which they occur To break ties, we use any arbitrary total ordering ~< of the processes. To break ties, we use any arbitrary total ordering ~< of the processes.

Definition of total ordering“ => ” If a is an event in process P i and b is an event in process P j, then a=>b if and only if either (i) C i (a) b if and only if either (i) C i (a)<C j (b) or (ii) C i (a) =C j (b) and P i ~< P j. Clock Condition implies that if a → b then a=>b. Clock Condition implies that if a → b then a=>b. In other words, the relation => is a way of completing the “happened before” partial ordering to a total ordering.

Unique → and not unique => The ordering => depends upon the clock systems and is not unique. The ordering => depends upon the clock systems and is not unique. Example: If we have C i (a) =C j (b) and choose P i ~ b. If we choose P j ~ a The partial ordering → which is uniquely determined by the system of events. The partial ordering → which is uniquely determined by the system of events.

Total Ordering Solve the mutual exclusion problem Mutual exclusion: A collection of processes which share a single resource. Only one process can use the resource at a time, the other processes will be excluded from doing the same thing Mutual exclusion: A collection of processes which share a single resource. Only one process can use the resource at a time, the other processes will be excluded from doing the same thing Requirements: (I) A process which has been granted the resource must release it before it can be granted to another process. (II) Different requests for the resource must be granted in the order in which they are made. (III) If every process which is granted the resource eventually release it, then every request is eventually granted. Requirements: (I) A process which has been granted the resource must release it before it can be granted to another process. (II) Different requests for the resource must be granted in the order in which they are made. (III) If every process which is granted the resource eventually release it, then every request is eventually granted.

Total Ordering Implement a system clocks with rules IR1 and IR2, and use them to define a total ordering => of all events. Implement a system clocks with rules IR1 and IR2, and use them to define a total ordering => of all events. Assumptions: 1. For any two processes P i and P j, the messages sent from P i to P j are received in the same order as they are sent. 2. Every message is eventually received. 3. A process can send messages directly to every other process. 4. Each process maintains its own request queue which is never seen by any other process. The request queues initially contain the single message T 0 :P 0 requests resource. Assumptions: 1. For any two processes P i and P j, the messages sent from P i to P j are received in the same order as they are sent. 2. Every message is eventually received. 3. A process can send messages directly to every other process. 4. Each process maintains its own request queue which is never seen by any other process. The request queues initially contain the single message T 0 :P 0 requests resource.

Total Ordering The algorithm: To request the resource, process Pi sends the message Tm:Pi requests resource to every other process, and puts that message on its request queue, where Tm is the timestamp of the message To request the resource, process Pi sends the message Tm:Pi requests resource to every other process, and puts that message on its request queue, where Tm is the timestamp of the message When process Pj receives the message Tm:Pi requests resource, it places it on its request queue and sends a (timestamped) acknowledgment message to Pi. When process Pj receives the message Tm:Pi requests resource, it places it on its request queue and sends a (timestamped) acknowledgment message to Pi. To release the resource, process Pi removes any Tm:Pi request resource message from its request queue and sends a (timstamped) Pi releases resource messages to every other process. To release the resource, process Pi removes any Tm:Pi request resource message from its request queue and sends a (timstamped) Pi releases resource messages to every other process.

Total Ordering The algorithm (cont’d) When process Pj receives a Pi release resource message, it removes any Tm:Pi requests resource message from its request queue. When process Pj receives a Pi release resource message, it removes any Tm:Pi requests resource message from its request queue. Process Pi is granted the resource when the following two conditions are satisfied: (i) There is a Tm:Pi request resource message in its request queue which is ordered before any other request in its queue by the relation =>. (ii) Pi has received a message from every other process timestamped later than Tm. Process Pi is granted the resource when the following two conditions are satisfied: (i) There is a Tm:Pi request resource message in its request queue which is ordered before any other request in its queue by the relation =>. (ii) Pi has received a message from every other process timestamped later than Tm.

Total Ordering Example Initial: P0 has the resource, P1 send its request first then P2 send its request Initial: P0 has the resource, P1 send its request first then P2 send its request At time T1,P1 send the message T1:P1 request resource to P0 and P2, and put this message on its request queue. (P1’s queue: T1:P1) At time T1,P1 send the message T1:P1 request resource to P0 and P2, and put this message on its request queue. (P1’s queue: T1:P1) At time T2, P2 send the message T2:P2 request resource to P0 and P1, and put this message on its request queue. (P2’s queue: T2:P2) At time T2, P2 send the message T2:P2 request resource to P0 and P1, and put this message on its request queue. (P2’s queue: T2:P2) At the time T3, P0 receives the message T2:P2, it place it on its request queue and sends a timestamped acknowledge message T3:M to P2 At the time T3, P0 receives the message T2:P2, it place it on its request queue and sends a timestamped acknowledge message T3:M to P2 At the time T4, P0 receives the message T1:P1, it place it on its request queue and sends a timestamped acknowledge message T4:M to P1 At the time T4, P0 receives the message T1:P1, it place it on its request queue and sends a timestamped acknowledge message T4:M to P1

Total Ordering Example (cont’d) At the time T5, P2 receives the message T1:P1, it place it on its request queue and sends a timestamped acknowledge message T5:M to P1 (P2’s queue: T2:P2, T1:P1) At the time T5, P2 receives the message T1:P1, it place it on its request queue and sends a timestamped acknowledge message T5:M to P1 (P2’s queue: T2:P2, T1:P1) At the time T6, P1 receives the message T2:P2, it place it on its request queue and sends a timestamped acknowledge message T6:M to P2 (P1’s queue: T1:P1, T2:P2) At the time T6, P1 receives the message T2:P2, it place it on its request queue and sends a timestamped acknowledge message T6:M to P2 (P1’s queue: T1:P1, T2:P2) At the time T7, P0 release the resource and sends a timestamped message T7:M to P1 and P2. From the condition (ii) of rule 5, P1 has received a message from every other process timestamped later than T1. But P2 does not satisfy this condition. Therefore, P1 is granted the resource. At the time T7, P0 release the resource and sends a timestamped message T7:M to P1 and P2. From the condition (ii) of rule 5, P1 has received a message from every other process timestamped later than T1. But P2 does not satisfy this condition. Therefore, P1 is granted the resource.

Anomalous Behavior Consider a nationwide system of interconnected computers. Suppose a person issues a request A on a computer A, and then telephones a friend in another city to have him issue a request B on a different computer B. It is quite possible for a request B to receive a lower timestamp and be ordered before request A. Consider a nationwide system of interconnected computers. Suppose a person issues a request A on a computer A, and then telephones a friend in another city to have him issue a request B on a different computer B. It is quite possible for a request B to receive a lower timestamp and be ordered before request A. Relevant external events may influence the ordering of system events. Relevant external events may influence the ordering of system events. Two possible solutions: 1. The user gives the timestamp T B is later than T A ; 2. Construct a system of clocks which satisfies the Strong Clock Condition: For any event a, b in φ: if a→b then C Two possible solutions: 1. The user gives the timestamp T B is later than T A ; 2. Construct a system of clocks which satisfies the Strong Clock Condition: For any event a, b in φ: if a→b then C

Physical Clocks We can construct a system of physical clocks which, running quite independently of one another, will satisfy the Strong Clock Condition. Then we can use physical clocks to eliminate anomalous behavior. We can construct a system of physical clocks which, running quite independently of one another, will satisfy the Strong Clock Condition. Then we can use physical clocks to eliminate anomalous behavior. Properties: Properties: 1. Clock runs continuously. 2. Clock runs at approximately the correct rate. i.e. dC i (t)/dt ≈1 for all t. PC1. There exists a constant ĸ<<1 such that for all i: | dC i (t)/dt -1| < ĸ 3. Clocks must be synchronized so that C i (t) ≈ C j (t) for all i, j, and t. PC2. For all i, j: | C i (t) - C j (t) | < ε ( ε is a sufficiently small constant )

Physical Clocks Clock synchronize algorithm Clock synchronize algorithm Let m be a message which sent at physical time t and received at time t’. We define ע m = t’ – t to be the total delay of the message m. This delay will not be known to the process which receives m. However, we assume that the receiving process knows some minimum delay 0 ≤ μ m ≤ ע m. We call ξm = ע m - μ m the unpredictable delay of the message. The physical clocks need to obey: IR1’. For each i, if P i does not receive a message at physical time t, then Ci is differentiable at t and dC i (t)/dt > 0. IR2’. (a) If P i sends a message m at physical time t, then m contains a timestamp T m =C i (t). (b) Upon receiving a message m at time t’, process P j sets C j (t’) equal to maximum (C j (t’-0), T m + μ m ).

Conclusion The concept of “happening before” defines an invariant partial ordering of the events in a distributed system. The concept of “happening before” defines an invariant partial ordering of the events in a distributed system. An algorithm for extending that partial ordering to a somewhat arbitrary total ordering. An algorithm for extending that partial ordering to a somewhat arbitrary total ordering. Total ordering is used to solve a simple synchronization problem. Total ordering is used to solve a simple synchronization problem. Total ordering can produce anomalous behavior but it can be prevented by the use of properly synchronized physical clocks. Total ordering can produce anomalous behavior but it can be prevented by the use of properly synchronized physical clocks.

Happy Thanksgiving

Question? Please rise your hand !