12/17/2015Distributed Systems - Comp 6551 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

12/17/2015Distributed Systems - Comp 6552 Transparency in a Distributed System TransparencyDescription Access Hide differences in data representation and how a resource is accessed LocationHide where a resource is located MigrationHide that a resource may move to another location Relocation Hide that a resource may be moved to another location while in use ReplicationHide that a resource is replicated Concurrency Hide that a resource may be shared by several competitive users FailureHide the failure and recovery of a resource

12/17/2015Distributed Systems - Comp 6553 What problems does replication solve? Some capacity and performance problems –Keep replicas on both sides of a bottleneck –Keep replicas on both sides of a connection with long delays Two kinds of incoherence: –Replication provides some location transparency –Replication provides some failure transparency ( aka fault tolerance)  Continue to work if one copy goes down  Continue to work if the network goes down

12/17/2015Distributed Systems - Comp 6554 What problems does replication cause? Consistency –To maintain concurrency transparency, system has to keep replicas updated Complexity –To maintain replication transparency, system has to be able to locate and select appropriate replicas Overhead can take back capacity and performance gains

12/17/2015Distributed Systems - Comp 6555 If you remember only one two thing(s) … 1.There are many types of consistency, known as “consistency models” 2.As the consistency model gets stronger The system gets easier to use. The system gets harder to implement. The system gets slower and consumes more resources.

12/17/2015Distributed Systems - Comp 6556 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

12/17/2015Distributed Systems - Comp 6557 Types of (data-centric) consistency

12/17/2015Distributed Systems - Comp 6558 Ground rules for examples Examples deal with a company’s inventory data Inventory data is replicated at every location This is for illustrative purposes; we are NOT claiming this is a good architectural idea Assume each process in each example is at a different location Assume that invisible processes propagate each update to all other copies Time moves left to right

12/17/2015Distributed Systems - Comp 6559 Strict consistency P1: P2: P1: P2: D=H D:DVD players, H:high, L:low, set:=, get:? D?H D=H D?LD?H Yes No

12/17/2015Distributed Systems - Comp Sequential consistency 1 P1: P2: P3: P4: D=H D=L D?L D:DVD players, H:high, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Sequential consistency 2 P1: P2: P3: P4: D=H D=L D?L D?H D:DVD players, H:high, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Sequential consistency - not P1: P2: P3: P4: D=H D=L D?L D?HD?L D?H D:DVD players, H:high, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Cost of sequential consistency Assume we have a distributed system with a replicated store and a feature to maintain consistency across the replicas Let r be the expected read time Let w be the expected write time Let t be the minimal packet transfer time between nodes in the system Then r + w  t

12/17/2015Distributed Systems - Comp Causal consistency P1: P2: P3: P4: D=L D?L D?HD?L D?H D=M D=H D?M No causal relationship Not sequentially consistent Potential causal relationships D:DVD players, H:high, M: medium, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Causal consistency - not P1: P2: P3: P4: D=L D?L D?H D=H D?L Potential causal relationship D:DVD players, H:high, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Causal consistency - ok P1: P2: P3: P4: D=L D?M D=M D?L No causal relationship D:DVD players, M:medium, L:low, set:=, get:?

12/17/2015Distributed Systems - Comp Types of (data-centric) consistency

12/17/2015Distributed Systems - Comp Eventual consistency DNS –A domain can only be updated by its naming authority (therefore, no write-write conflicts) –Updates propagated on a schedule –Most accesses work from cache –Can refresh a cached value on failure Web –A page can only be updated by its owner (again, no write-write conflicts) –Most accesses work from cache –Page can be refreshed if user suspects a failure

12/17/2015Distributed Systems - Comp Client-centric consistency

12/17/2015Distributed Systems - Comp Client-centric consistency models ModelThe idea Monotonic reads Each read by a process returns the same value as the previous read, or a more recent value Monotonic writes Each write by a process must complete before the next write of the data item by the process Read your writes Each write by a process will be visible in any subsequent read by that process Writes follow reads Each write by a process after a read will be done at all replicas on a value that is at least as recent as the value read

12/17/2015Distributed Systems - Comp Client-centric examples Imagine a discussion group with replicas in Columbus and Nairobi Imagine these events –Alice starts thread A –Bob starts thread B –Alice edits A to make A’ –Alice adds attachment to make A’’ –Carol posts reply AC

12/17/2015Distributed Systems - Comp Monotonic reads ColumbusNairobi Carol reads again OK B AA Carol reads B Not mono read

12/17/2015Distributed Systems - Comp Monotonic reads ColumbusNairobi Carol reads again OK A AB Carol reads B Not mono read

12/17/2015Distributed Systems - Comp Monotonic writes ColumbusNairobi OK A’ AA A’’ A’ Not mono write

12/17/2015Distributed Systems - Comp Read your writes ColumbusNairobi Alice reads again OK A AB AC B Not read your writes Alice reads AC Read your writes, but Non-monotonic reads

12/17/2015Distributed Systems - Comp Writes follow reads ColumbusNairobi AC arrives OK A A Carol reads AC Not writes follow reads

12/17/2015Distributed Systems - Comp Writes follow reads to prevent this ColumbusNairobi A AC A Carol reads AC Bob readsDave reads Dave sees reply before original

12/17/2015Distributed Systems - Comp Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

12/17/2015Distributed Systems - Comp Types of replicas Any experience with server-initiated replicas?

12/17/2015Distributed Systems - Comp What to propagate? Notifications Updated data Update operations

12/17/2015Distributed Systems - Comp Who initiates propagation? Server (push-based protocol) Client (pull-based protocol)