COMP 655: Distributed/Operating Systems Summer 2011 Dr. Chunbo Chu Week 7: Consistency 4/13/20151Distributed Systems - COMP 655

4/13/2015Distributed Systems - COMP 6552 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

4/13/2015Distributed Systems - COMP 6553 Transparency in a Distributed System TransparencyDescription Access Hide differences in data representation and how a resource is accessed LocationHide where a resource is located MigrationHide that a resource may move to another location Relocation Hide that a resource may be moved to another location while in use ReplicationHide that a resource is replicated Concurrency Hide that a resource may be shared by several competitive users FailureHide the failure and recovery of a resource

4/13/2015Distributed Systems - COMP 6554 What problems does replication solve? Some capacity and performance problems –Keep replicas on both sides of a bottleneck –Keep replicas on both sides of a connection with long delays Two kinds of incoherence: –Replication provides some location transparency –Replication provides some failure transparency ( aka fault tolerance)  Continue to work if one copy goes down  Continue to work if the network goes down

4/13/2015Distributed Systems - COMP 6555 What problems does replication cause? Consistency –To maintain concurrency transparency, system has to keep replicas updated Complexity –To maintain replication transparency, system has to be able to locate and select appropriate replicas Overhead can take back capacity and performance gains

4/13/2015Distributed Systems - COMP 6556 If you remember only one two thing(s) … 1.There are many types of consistency, known as “consistency models” 2.As the consistency model gets stronger The system gets easier to use. The system gets harder to implement. The system gets slower and consumes more resources.

4/13/2015Distributed Systems - COMP 6557 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

Consistency Models A Consistency Model is a contract between the software and the memory –it states that the memory will work correctly but only if the software obeys certain rules The issue is how we can state rules that are not too restrictive but allow fast execution in most common cases These models represent a more general view of sharing data than what we have seen so far! Conventions we will use: W(x)a means “a write to x with value a ” R(y)b means “a read from y that returned value b ” 4/13/20158Distributed Systems - COMP 655

Data-centric Consistency Models 4/13/2015Distributed Systems - COMP 6559 The general organization of a logical data store, physically distributed and replicated across multiple processes. Each process interacts with its local copy, which must be kept ‘consistent’ with the other copies.

4/13/2015Distributed Systems - COMP Types of (data-centric) consistency

Strict Consistency Strict consistency is the strictest model –a read returns the most recently written value (changes are instantaneous) –not well-defined unless the execution of commands is serialized centrally –otherwise the effects of a slow write may have not propagated to the site of the read –this is what uniprocessors support: a = 1; a = 2; print(a); always produces “2” –to exercise our notation: P1: W(x)1 P2: R(x)0 R(x)1 –is this strictly consistent? 4/13/201511Distributed Systems - COMP 655

4/13/2015Distributed Systems - COMP Strict consistency P1: P2: P1: P2: D=H D:DVD players, H:high, L:low, set:=, get:? D?H D=H D?LD?H Yes No

Sequential Consistency Sequential consistency ( serializability ): the results are the same as if operations from different processors are interleaved, but operations of a single processor appear in the order specified by the program Example of sequentially consistent execution: P1: W(x)1 P2: R(x)0 R(x)1 Sequential consistency is inefficient: we want to weaken the model further 4/13/201513Distributed Systems - COMP 655

4/13/2015Distributed Systems - COMP Sequential consistency 1 P1: P2: P3: P4: D=H D=L D?L D:DVD players, H:high, L:low, set:=, get:?

4/13/2015Distributed Systems - COMP Sequential consistency 2 P1: P2: P3: P4: D=H D=L D?L D?H D:DVD players, H:high, L:low, set:=, get:?

4/13/2015Distributed Systems - COMP Sequential consistency - not P1: P2: P3: P4: D=H D=L D?L D?HD?L D?H D:DVD players, H:high, L:low, set:=, get:?

Causal Consistency Causal consistency: writes that are potentially causally related must be seen by all processors in the same order. Concurrent writes may be seen in a different order on different machines –causally related writes: the write comes after a read that returned the value of the other write Examples (which one is causally consistent, if any?) P1: W(x)1W(x)3 P2: R(x)1 W(x)2 P3: R(x)1 R(x)3 R(x)2 P4: R(x)1 R(x)2 R(x)3 P1: W(x)1 P2: R(x)1 W(x)2 P3: R(x)2 R(x)1 P4: R(x)1 R(x)2 Implementation needs to keep dependencies 4/13/201517Distributed Systems - COMP 655

4/13/2015Distributed Systems - COMP Causal consistency P1: P2: P3: P4: D=L D?L D?HD?L D?H D=M D=H D?M No causal relationship Not sequentially consistent Potential causal relationships D:DVD players, H:high, M: medium, L:low, set:=, get:?

4/13/2015Distributed Systems - COMP Causal consistency - not P1: P2: P3: P4: D=L D?L D?H D=H D?L Potential causal relationship D:DVD players, H:high, L:low, set:=, get:?

4/13/2015Distributed Systems - COMP Causal consistency - ok P1: P2: P3: P4: D=L D?M D=M D?L No causal relationship D:DVD players, M:medium, L:low, set:=, get:?

Consider the following sequence of operations: P1: W(x)1W(x)3 P2: W(x)2 P3: R(x)3 R(x)2 P4: R(x)2 R(x)3 Is this execution causally consistent? Add or modify an event to change the answer. Exercise 4/13/201521Distributed Systems - COMP 655

4/13/2015Distributed Systems - COMP Types of (data-centric) consistency

4/13/2015Distributed Systems - COMP Client-centric consistency A mobile user may access different replicas of a distributed database at different times. This type of behavior implies the need for a view of consistency that provides guarantees for single client regarding accesses to the data store.

4/13/2015Distributed Systems - COMP Client-centric consistency models ModelThe idea Monotonic reads Each read by a process returns the same value as the previous read, or a more recent value Monotonic writes Each write by a process must complete before the next write of the data item by the process Read your writes Each write by a process will be visible in any subsequent read by that process Writes follow reads Each write by a process after a read will be done at all replicas on a value that is at least as recent as the value read

Distributed Systems - COMP Monotonic Reads A data store provides monotonic read consistency if when a process reads the value of a data item x, any successive read operations on x by that process will always return the same value or a more recent value. Example error: successive access to have ‘disappearing messages’ a)A monotonic-read consistent data store b)A data store that does not provide monotonic reads. indicates propagation of the earlier write L1 and L2 are two locations process moves from L1 to L2 No propagation guarantees 4/13/2015

Distributed Systems - COMP Monotonic Writes A write operation by a process on a data item x is completed before any successive write operation on x by the same process. Implies a copy must be up to date before performing a write on it. Example error: Library updated in wrong order. a)A monotonic-write consistent data store. b)A data store that does not provide monotonic-write consistency. In both examples, process performs a write at L1, moves and performs a write at L2 4/13/2015

Distributed Systems - COMP Read Your Writes The effect of a write operation by a process on data item x will always be seen by a successive read operation on x by the same process. Example error: deleted messages re-appear. (a) A data store that provides read-your-writes consistency. (b) A data store that does not. In both examples, process performs a write at L1, moves and performs a read at L2 4/13/2015

Distributed Systems - COMP Writes Follow Reads A write operation by a process on a data item x following a previous read operation on x by the same process is guaranteed to take place on the same or a more recent value of x that was read. Example error: Newsgroup displays responses to articles before original article has propagated there (a) A writes-follow-reads consistent data store (b) A data store that does not provide writes-follow-reads consistency In both examples, process performs a read at L1, moves and performs a write at L2 4/13/2015

Distributed Systems - COMP Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

4/13/2015Distributed Systems - COMP Types of replicas Any experience with server-initiated replicas?

4/13/2015Distributed Systems - COMP What to propagate? Notifications Updated data Update operations

4/13/2015Distributed Systems - COMP Who initiates propagation? Server (push-based protocol) Client (pull-based protocol)