Consistency and Replication

Outline Introduction (what’s it all about) Data-centric consistency Client-centric consistency Replica management Consistency protocols

Why Replicate? Reliability –If one goes down, the others can stay up. –Corrupted data Performance –Divide the work (single server vs multiple servers) –Place data closer to place it is used. What is the challenge? –Consistency (when updates are needed) –Consider a web cache in your browser.

Costs As a scaling technique (for performance problem) Trade-off: keep copies up to date require bandwidth. P Access replica N times per second Update replica M times per second As a scaling technique, may not always be applicable. What if N << M?

Assume synchronous replication A dilemma: –Scalability can be alleviated by replication and caching. –Keep copies consistent is scaling problem (and require bandwidth too) E.g, consistency requires global synchronization! –real solution is to relax consistency requirements. WAN Withdraw $50

Recap on Synchronization Do some synch mechanisms apply here? –Prefect clock synch? –Message ordering? Total ordered multicast Causal relationship? –Mutual exclusion Centralized or decentrilized? Distributed A leader or multiple leaders

Outline Introduction (what’s it all about) Data-centric consistency Client-centric consistency Replica management Consistency protocols

Consistency Models Enforcing absolute ordering is too expensive, especially with replication and caching. So we need to allow for “mis-ordering”. –We could just do it casually. Tell programmers, “Well, you always see things in exact order”. –They would say, “What do you mean?” So we need an exact, very precise way of specifying the kinds of inconsistencies that the application might see. That is the purpose and point of having consistency models.

Data Stores Consistency is viewed as read/write ops on shared data. A consistency model is a contract between the processes and the data store (Shared memory, database, file systems). A read operation on a data item returns a value of last write. (nothing can claim as a best solution)

Example: A warehouse data item Distributed reads/writes. A middleware provide a function call that ensures the consistency model

Outline Data-centric consistency Continuous Consistency Sequential Consistency Causal Consistency Grouping Operations

Continuous Consistency (1) A 3 ops, B 2 ops A not see 1 B’s op, the value is 5 B not see 3 A’s op, the MAX value is 6

So application decides the consistency or tolerate level of inconsistency

Continuous Consistency (2) Choosing the appropriate granularity for a conit. e.g, two replica can differ in one item (a) Two updates lead to update propagation.

Continuous Consistency (3) But here (b), no update propagation is needed (yet).

Outline Data-centric consistency Continuous Consistency Sequential Consistency Causal Consistency Grouping Operations

Notation Processes execute to the right as time progresses. The notation W 1 (x)a means that the process wrote the value ‘a’ to the variable x. The notation R 2 (x)a means that the process read the value ‘a’ from the variable x. The subscript is often dropped.

Sequential Consistency The result of any execution is the same as if the (read and write) operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program. –There is some global order. – We’re talking about interleaved executions: there is some total ordering for all operations taken together. not time

Program A A-OP1 A-OP2 A-OP3 Program B B-OP1 B-OP2 B-OP3 Global Order 1 A-OP1 A-OP2 A-OP3 B-OP1 B-OP2 B-OP3 Global Order 2 A-OP1 B-OP1 A-OP2 B-OP2 B-OP3 A-OP3 Global Order 3 A-OP1 B-OP1 A-OP2 B-OP3 B-OP2 A-OP3 no

Sequential Consistency (3) Write b happened before write a -- sequential Write b happened before write a -- not sequential

Sequential Consistency (4) 6! = 720 possible execution sequence. 90 valid

Sequential Consistency (5) Figure 7-7. Four valid execution sequences for the processes of Fig The vertical axis is time. Four sample valid interleaving. Reorder with signature: p1, p2, p3 64 of them (not all are allowed) under sequence consistency The contract says: given the program, these many outputs are correct.

Causal Consistency For a data store to be considered causally consistent, it is necessary that the store obeys the following condition: Writes that are potentially causally related must be seen by all processes in the same order. Concurrent writes may be seen in a different order on different machines. –causally related: If event b is caused by event a, then a must be first, then b –Concurrent - not casually related –Weaker than sequential consistency –Deal with writes

Causal Consistency (2) W2(x)b and W1(x)c are concurrent This sequence is allowed with a causally-consistent store, but not with a sequentially consistent store.

Causal Consistency Causally consistent? a and b are related, so incorrect a and b are not related, so correct W1(x)a and W2(x)b causally related

Recap slides (next 3)

Consistency Models Consistency is viewed as read/write ops on shared data. An exact, very precise way of specifying the kinds of inconsistencies that the application might see –A consistency model is a contract between the processes and the data store (Shared memory, database, file systems). A middleware provide a function call that ensures the consistency model

What consistency models? W1a and W2b are related, not causal consistency W1a and W2b are not related, is causal consistency What about sequential consistency ?

Outline Data-centric consistency Continuous Consistency Sequential Consistency Causal Consistency Grouping Operations

Grouping Operations Instead of read and write ops, what about many operations that applications perform under control of syn –E.g, Mutual exclusion. –How do we handle consistency in Multi Thread programs? –Use locks. –E.g, ENTER_CS and LEAVE_CS As viewed by an external, data-centric process, what do locks do? –Many read and write: turn non-atomic operations into atomic ones (functionally). –In other words, they group them.

Synchronization Variables Operations are grouped via synchronization variables (locks). Each sync var protects an associated data. Each kind of sync var has some associated properties. Each sync var has a current owner, A non-owner needs to send msg to the owner for ownership (and data value) A sync var can be owned by many processors nonexclusively.

Grouping Operations Entry Consistency: Necessary criteria for correct synchronization: 1.An acquire access of a synchronization variable is not allowed to perform until all updates to guarded shared data have been performed with respect to that process. - Given up an ownership means finishing previous updates 2.Before exclusive mode access to synchronization variable by a process is allowed to perform with respect to that process, no other process may hold the synchronization variable, not even in nonexclusive mode. - Writing must be exclusive 3.After exclusive mode access to a synchronization variable has been performed, any other process’ next nonexclusive mode access to that synchronization variable may not be performed until it has performed with respect to that variable’s owner. - otherwise, no guarantee on consistency if one does a nonexclusive mode

Grouping Operations (2) A valid event sequence for entry consistency.

Summary Data-centric consistency –Continues consistency –Consistent ordering of operations Sequential consistency Causal consistency Grouping operations

Outline Introduction (what’s it all about) Data-centric consistency Client-centric consistency –Eventual consistency –Monotonic reads –Monotonic writes –Read your writes –Writes follow reads Replica management Consistency protocols

Weaker Models Sometimes strong models are needed, if the result of race conditions are very bad. –Banks Sometimes the result of races are just inefficiency, or inconvenience, etc. –DNS, web caches, How strong is Orbitz’s model? –If it shows a ticket available, is it really? –How does it prevent two people from reserving the same seat?

Eventual Consistency One kind of weaker model is eventual consistency –It eventually becomes consistent if updates are not frequent. Updates eventually propagate to all –Write-write conflicts are relatively easy to solve (infrequent, by a small portion of nodes) –Read-write conflicts are handled.

Mobile users (short time) How well does EC work for mobile clients? –If replica is location related Client-centric is for this. Consistent for a single client.

Outline Data-centric consistency Client-centric consistency Goal: perhaps avoid system wide consistency, by concentrating on what specific clients want, instead of what should be maintained by servers. Eventual Consistency Monotonic Reads Monotonic Writes Read Your Writes Writes Follow Reads

Data Stores Local read/write Eventually propagate to all

Notation x i [t] is the version of x at local copy L i at time t. Version x i [t] is the result of a series of write operations at L i that took place since initialization. This is WS(x i [t]). If operations in WS(x i [t]) have also been performed at local copy L j at a later time t 2, we write WS(x i [t 1 ];x j [t 2 ]).

Monotonic Reads A data store is said to provide monotonic- read consistency if the following condition holds: –If a process reads the value of a data item x any successive read operation on x by that process will always return that same value or a more recent value.

Monotonic Reads For one processor p WS(x1) sent to L2, is monotonic WS(x1) not sent to L2, not monotonic

Monotonic Writes In a monotonic-write consistent store, the following condition holds: –A write operation by a process on a data item x is completed before any successive write operation on x by the same process. if needed, a new write will wait for old ones to finish,

Monotonic Writes WS(x1) sent to L2, is monotonic WS(x1) not sent to L2, not monotonic

Read Your Writes A data store is said to provide read-your-writes consistency, if the following condition holds: The effect of a write operation by a process on data item x will always be seen by a successive read operation on x by the same process. –No matter where the location of the read is Suppose your web browser has a cache. –You update your web page on the server. –Before refresh your browser… –After refresh your browser. –Do you have read-your-writes consistency?

Read Your Writes (2) W(x1) is part of WS (x1,x2), is read your writes The read doesn’t include the W(x1), not R-Y-W i.e. updating your Web page and guaranteeing that your Web browser shows the newest version instead of its cached copy.

Writes Follow Reads (1) A data store is said to provide writes-follow-reads consistency, if the following holds: A write operation by a process on a data item x following a previous read operation on x by the same process is guaranteed to take place on the same or a more recent value of x that was read Example: See reactions to posted articles only if you have the original posting (a read “pulls in” the corresponding write operation). Respond to blog article, or chatting group.

Writes Follow Reads (2) is writes follow reads Not writes follow reads