1. Replication
Chapter 7

2. Reasons for Replication
Data replication is a common technique in distributed systems. There are two reasons for data replication:
It increases the reliability of a system.
If one replica is unavailable or crashes, use another
Protect against corrupted data
It improves the performance of a system.
Scale with size of the distributed system (replicated Web servers)
Scale in geographically distributed systems (Web proxies)
The key issue is the need to maintain consistency of replicated data.
If one copy is modified, others become inconsistent.

3. Object Replication There are two approaches for object sharing:
The object itself can handle concurrent invocation.
A Java object can be constructed as a monitor by declaring the object’s methods to be synchronized.
The object is completely unprotected against concurrent invocations, but the server in which the object resides is made responsible for concurrency control.
In particular, use an appropriate object adapter.

4. Object Replication
A remote object capable of handling concurrent invocations on its own.
A remote object for which an object adapter is required to handle concurrent invocations

5. Object Replication
Organization of a distributed remote object shared by two different clients.

6. Object Replication There are two approaches for object replication:
The application is responsible for replication.
Application needs to handle consistency issues.
The system (middleware) handles replication.
Consistency issues are handled by the middleware.
It simplifies application development but makes object-specific solutions harder.

7. Object Replication
A distributed system for replication-aware distributed objects.
A distributed system responsible for replica management

8. Replication and Scaling
Replication and caching are used for system scalability.
Multiple copies improve performance by reducing access latency but have higher network overheads of maintaining consistency.
Example: An object is replicated N times.
Consider the Read frequency R and the write frequency W
If R << W, high consistency overhead and wasted messages
Consistency maintenance is itself an issue
What semantics to provide?
Tight consistency requires globally synchronized clocks.

9. Replication and Scaling
The solution is to loosen consistency requirements.
Variety of consistency semantics possible
Consistency model (consistency semantics)
Contract between processes and the data store
If processes obey certain rules, data store will work correctly.
All models attempt to return the results of the last write for a read operation.
Differ in how last write is determined/defined

10. Data-Centric Consistency Models
The general organization of a logical data store, physically distributed and replicated across multiple processes.

11. Strict Consistency
Definition: Any read on a data item X returns a value corresponding to the result of the most recent write on X.
This definition implicitly assumes the existence of absolute global time. Naturally available in uni-processor systems, but impossible to implement in distributed systems.
W(x)a means written value to x is a.
Behavior of two processes, operating on the same data item.
A strictly consistent store.
A store that is not strictly consistent-because must recent written value to x is a but not nothing(NIL)

12. Sequential Consistency
Sequential consistency: The result of any execution is the same as if the (read and write) operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program.
All processes see the same interleaving of (write) operations.
A sequentially consistent data store.
A data store that is not sequentially consistent. ( w(x)b is latest write so R(x)b should be access in first read and then R(x)a )

13. Linearizability
Definition: The result of any execution is the same as if the (read and write) operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program. In addition, if TSop1(x) < TSop2(y), then operation OP1(x) should precede OP2(y) in this sequence.
Tsop1(x)-> Timestamp of operation1
In this model, operations are assumed to receive a timestamp using a globally available clock with finite precision.
A linearizable data store is also sequentially consistent, but it is more expensive to implement than sequential consistency
Linearizability is primarily used to assist formal verification of concurrent programs,

14. Sequential Consistency and Serializability
Definition: Sequential consistency is comparable to serializability in the case of transactions.
The difference is that of granularity: sequential consistency is defined in terms of read and write operations, whereas serializability is defined in terms of transactions, which aggregate such operations.
Sequential consistency is a programmer-friendly model, but it has serious performance problems. So other weaker consistency models have been proposed.

15. Causal Consistency
Causal consistency requires a total order of causally related write operations only.
A read is causally related to the write that provided the data the read got.
A write is causally related to a read that happened before this write in the same process.
If write1  read, and read  write2, then write1  write2.
Necessary condition for causal consistency: Writes that are potentially casually related must be seen by all processes in the same order. Concurrent writes may be seen in a different order on different machines.

16. Causal Consistency
This sequence is allowed with a causally-consistent store, but not with sequentially or strictly consistent store.
Note: W1(x)a  W2(x)b, but W2(x)b || W1(x)c
W1(x)a  W2(x)b because W2(x)b is obtained after reading the value of W1(x)a and hence they are causally related, so every process must see them in same order.

17. Causal Consistency A violation of a casually-consistent store.
A correct sequence of events in a casually-consistent store in this case R(x)a is removed so they are not causally related and can be viewed in any order

18. Fig: A valid sequence of events of FIFO consistency.
Necessary Condition: Writes done by a single process are seen by all other processes in the order in which they were issued, but writes from different processes may be seen in a different order by different processes.
Fig: A valid sequence of events of FIFO consistency.
->Here the order of R(x)a can be in any order but R(x)c must access after R(x)b because b and c both are on the same process.

19. Summary of Consistency Models
Description
Strict
Absolute time ordering of all shared accesses matters.
Linearizability
All processes must see all shared accesses in the same order. Accesses are furthermore ordered according to a (nonunique) global timestamp
Sequential
All processes see all shared accesses in the same order. Accesses are not ordered in time
Causal
All processes see causally-related shared accesses in the same order.
FIFO
All processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order
(a)
Weak
Shared data can be counted on to be consistent only after a synchronization is done
Release
Shared data are made consistent when a critical region is exited
Entry
Shared data pertaining to a critical region are made consistent when a critical region is entered.
(b)
Consistency models not using synchronization operations.
Models with synchronization operations.

20. System Model Assume replica manager applies operations to its replica
set of replica managers may be fixed or could change dynamically
requests are reads or writes (updates)

21. A basic architectural model for the management of replicated data

22. System Model-Replica manager
Five phases in performing a request
Front end issues the request
Either sends to a single (primary) replica or multicasts to all replica managers
Coordination
RMs coordinate in preparation for the execution of the request, i.e., agree if request is to be performed and the ordering of the request relative to others
FIFO ordering, Causal ordering, Total (sequential) ordering

23. System Model-Replica manager
Five phases in performing a request
Execution
May be tentative (not fully agreed upon).
Agreement
Reach consensus on effect of the request, e.g., agree to commit or abort in a transactional system
Response

24. Replication technique for fault tolerance (service provision technique)
Passive Replication
Active Replication
Both the technique provide the fault tolerance. In passive replication each client request is processed by all the replicas. In active replication each client request is processed by all the replicas

25. The Passive (Primary-Backup) Model
PRM( Primary Replica Manager) informs to all BRM(Backup Replica Manager) and provides the updated data BRM.
Only Primary Replica manager is directly connected to front end

26. Passive (Primary-Backup) Replication
Request: Front end issue client request with unique request ID to PRM(primary replica manager). Coordination: primary RM takes updates in FIFO. It is just preparation for execution.it maintains the order of request for execution. It may maintains the request in FIFO, causal and total ordering etc. Execution: primary RM executes the update and stores the response Agreement: primary sends the update to all backups. The backups send acknowledgements Response: primary responds to the FE, which hands the response back to the client

27. Passive (Primary-Backup) Replication
Implements SC(sequential consistency), since it sequences all updates (all reads can then be inserted to form the total order)

28. Passive (Primary-Backup) Replication
If primary fails,
It should be replaced with a unique backup
RMs that survive must agree upon which operations had been performed when the replacement primary takes over (will discuss this in next chapter on fault tolerance)

29. Active Replication Using Multicast
FE multicasts request to each replica

30. Active Replication Using Multicast
Sequence of Events:
Request: FE with unique identifier of client request multicast request to all RM in a totally order fashion.
Co-ordination: The Group command system deliver request to every RM in same(total) order.
Execution: Every RM execute request which are delivered in same (total-> if a process issue m then m’ then m->m’ ) order, correct RM process the request identically. Since the request is send in same order.
Agreement: This phase not required, every RM process request separately.
Response: Each RM send response to FE the no of replicas that the front end collects depends up on failure assumption and on the multicast algorithm. If , for example, the goal is to tolerate only crash failure and the multicast satisfies uniform agreement and ordering properties, then the FE passes first response to client discarding others response.
If request = n then ,
Response is <= n.
Drawbacks: Unnecessary response flow.

31. Fault tolerant services:
How replica systems provide service?
Concern is how to provide a service that is correct, despite up to n process failures by replicating the data and functionality at replica managers.

32. 7.5 High available services
The concern is to manage replica so that the service is highly available to clients.
The “Gossip” Architecture:
Framework for implementing highly available services by replicating data closed to the points where groups of clients need it.
RM (Replica manager) exchange “gossip” messages periodically to convey updates they have received from the clients.
Operations(Two basic types of operations):
Query: Read only operations
Updates: modify but do not read the state.
Front end sends the query/updates to any RM provide reasonable response time.

33. The “Gossip” Architecture:
Sequence of events:
Request: FE(front end) sends request to a single RM at a time, However it may talk with other RM if nearest one fail or heavily loaded.
Update response: If the request is update RM replies as soon as it receive the update.
Co-ordination: RM does not process request until it satisfies certain ‘ordering constraints’.
Execution: RM who receive request execution it .
Response: Reply on behalf of request.
Agreement: Replica manager updates one-another by exchanging the gossip messages it will occurs accordingly after several update occurs.
In order to control ordering of operation processing each FE keeps a vector timestamp
that track the latest data accessed by FE.

34. -> prev: is a vector timestamp for the latest version seen by the FE (and client) ->val,new:- new is the vector timestamp of the resulting value,val ->Update ID: update id is the vector timestamp of the update FE
Clients
Service
Vector
timestamps RM
gossip
Query, prev
Update ID
Query
Update
val
val,new

35. Replica Management
Where shall we place replica servers to minimize overall data transfer?
In its general form it is a classical optimisation problem, but in practice it is often a management/commercial issue! D 5 3 2 B 4 C A 2 E 1 1 4 3 1 1 1 1 H 1 I G 3 2 5 F 1 2 3 2 3 2 L M 2 K 2 J 1 2 2 4 1 P O 1 N 1
20-May-19
COMP28112 Lecture 15

36. A greedy heuristic to find locations (minimum k-median problem)
Find the total cost of accessing each site from all the other sites. Choose the site with the minimum total cost.
Repeat (1) above, taking also into account sites hosting replicas (i.e., recalculate costs).
(see for more algorithms: ‘on the placement of web server replicas’, INFOCOM2001)
Example: (using only nodes E, I, L, M, P from the graph in slide 14) E I L M P 1 5 2 4 3
Node M has the minimum cost!
Once we chose M, the next iteration may take into account the fact that sites attempt to access the nearest replica.

37. Step 2 (after we chose node M for replica hosting)
Can (naively) just remove node M and repeat to find another node.
Better, but more work, replace each value in the table by min(value, value-to-M)
Note that this destroys any symmetry it had initially: E I L M P 1
5 → 2 2
4 → 2
4 → 1
3 → 1
5 → 3
4 → 3 3
2 → 0
1 → 0
3 → 0
3 → 2
Remember the trade-off! Heuristics are useful when it is expensive to check exhaustively all possible combinations;
for n hosts & m replicas: this is n! / ( m! (n-m)! )