1. 12/17/2015Distributed Systems - Comp 6551 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

2. 12/17/2015Distributed Systems - Comp 6552 Transparency in a Distributed System TransparencyDescription Access Hide differences in data representation and how a resource is accessed LocationHide where a resource is located MigrationHide that a resource may move to another location Relocation Hide that a resource may be moved to another location while in use ReplicationHide that a resource is replicated Concurrency Hide that a resource may be shared by several competitive users FailureHide the failure and recovery of a resource

3. 12/17/2015Distributed Systems - Comp 6553 What problems does replication solve? Some capacity and performance problems –Keep replicas on both sides of a bottleneck –Keep replicas on both sides of a connection with long delays Two kinds of incoherence: –Replication provides some location transparency –Replication provides some failure transparency ( aka fault tolerance)  Continue to work if one copy goes down  Continue to work if the network goes down

4. 12/17/2015Distributed Systems - Comp 6554 What problems does replication cause? Consistency –To maintain concurrency transparency, system has to keep replicas updated Complexity –To maintain replication transparency, system has to be able to locate and select appropriate replicas Overhead can take back capacity and performance gains

5. 12/17/2015Distributed Systems - Comp 6555 If you remember only one two thing(s) … 1.There are many types of consistency, known as “consistency models” 2.As the consistency model gets stronger The system gets easier to use. The system gets harder to implement. The system gets slower and consumes more resources.

6. 12/17/2015Distributed Systems - Comp 6556 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

7. 12/17/2015Distributed Systems - Comp 6557 Types of (data-centric) consistency

8. 12/17/2015Distributed Systems - Comp 6558 Ground rules for examples Examples deal with a company’s inventory data Inventory data is replicated at every location This is for illustrative purposes; we are NOT claiming this is a good architectural idea Assume each process in each example is at a different location Assume that invisible processes propagate each update to all other copies Time moves left to right

9. 12/17/2015Distributed Systems - Comp 6559 Strict consistency P1: P2: P1: P2: D=H D:DVD players, H:high, L:low, set:=, get:? D?H D=H D?LD?H Yes No

10. 12/17/2015Distributed Systems - Comp 65510 Sequential consistency 1 P1: P2: P3: P4: D=H D=L D?L D:DVD players, H:high, L:low, set:=, get:?

11. 12/17/2015Distributed Systems - Comp 65511 Sequential consistency 2 P1: P2: P3: P4: D=H D=L D?L D?H D:DVD players, H:high, L:low, set:=, get:?

12. 12/17/2015Distributed Systems - Comp 65512 Sequential consistency - not P1: P2: P3: P4: D=H D=L D?L D?HD?L D?H D:DVD players, H:high, L:low, set:=, get:?

13. 12/17/2015Distributed Systems - Comp 65513 Cost of sequential consistency Assume we have a distributed system with a replicated store and a feature to maintain consistency across the replicas Let r be the expected read time Let w be the expected write time Let t be the minimal packet transfer time between nodes in the system Then r + w  t

14. 12/17/2015Distributed Systems - Comp 65514 Causal consistency P1: P2: P3: P4: D=L D?L D?HD?L D?H D=M D=H D?M No causal relationship Not sequentially consistent Potential causal relationships D:DVD players, H:high, M: medium, L:low, set:=, get:?

15. 12/17/2015Distributed Systems - Comp 65515 Causal consistency - not P1: P2: P3: P4: D=L D?L D?H D=H D?L Potential causal relationship D:DVD players, H:high, L:low, set:=, get:?

16. 12/17/2015Distributed Systems - Comp 65516 Causal consistency - ok P1: P2: P3: P4: D=L D?M D=M D?L No causal relationship D:DVD players, M:medium, L:low, set:=, get:?

17. 12/17/2015Distributed Systems - Comp 65517 Types of (data-centric) consistency

18. 12/17/2015Distributed Systems - Comp 65518 Eventual consistency DNS –A domain can only be updated by its naming authority (therefore, no write-write conflicts) –Updates propagated on a schedule –Most accesses work from cache –Can refresh a cached value on failure Web –A page can only be updated by its owner (again, no write-write conflicts) –Most accesses work from cache –Page can be refreshed if user suspects a failure

19. 12/17/2015Distributed Systems - Comp 65519 Client-centric consistency

20. 12/17/2015Distributed Systems - Comp 65520 Client-centric consistency models ModelThe idea Monotonic reads Each read by a process returns the same value as the previous read, or a more recent value Monotonic writes Each write by a process must complete before the next write of the data item by the process Read your writes Each write by a process will be visible in any subsequent read by that process Writes follow reads Each write by a process after a read will be done at all replicas on a value that is at least as recent as the value read

21. 12/17/2015Distributed Systems - Comp 65521 Client-centric examples Imagine a discussion group with replicas in Columbus and Nairobi Imagine these events –Alice starts thread A –Bob starts thread B –Alice edits A to make A’ –Alice adds attachment to make A’’ –Carol posts reply AC

22. 12/17/2015Distributed Systems - Comp 65522 Monotonic reads ColumbusNairobi Carol reads again OK B AA Carol reads B Not mono read

23. 12/17/2015Distributed Systems - Comp 65523 Monotonic reads ColumbusNairobi Carol reads again OK A AB Carol reads B Not mono read

24. 12/17/2015Distributed Systems - Comp 65524 Monotonic writes ColumbusNairobi OK A’ AA A’’ A’ Not mono write

25. 12/17/2015Distributed Systems - Comp 65525 Read your writes ColumbusNairobi Alice reads again OK A AB AC B Not read your writes Alice reads AC Read your writes, but Non-monotonic reads

26. 12/17/2015Distributed Systems - Comp 65526 Writes follow reads ColumbusNairobi AC arrives OK A A Carol reads AC Not writes follow reads

27. 12/17/2015Distributed Systems - Comp 65527 Writes follow reads to prevent this ColumbusNairobi A AC A Carol reads AC Bob readsDave reads Dave sees reply before original

28. 12/17/2015Distributed Systems - Comp 65528 Consistency and Replication The problems we are trying to solve Types of consistency Approaches to propagation

29. 12/17/2015Distributed Systems - Comp 65529 Types of replicas Any experience with server-initiated replicas?

30. 12/17/2015Distributed Systems - Comp 65530 What to propagate? Notifications Updated data Update operations

31. 12/17/2015Distributed Systems - Comp 65531 Who initiates propagation? Server (push-based protocol) Client (pull-based protocol)