1. 6.4 Data and File Replication Gang Shen

2. Why replicate  Performance  Reliability  Resource sharing  Network resource saving

3. Challenge  Transparency Replication Concurrent control Failure recovery Serialization

4. Atomicity  In database systems, atomicity is one of the ACID transaction properties. An atomic transaction is a series of database operations which either all occur, or all do not occur[1].  All or nothing

5. Atomicity  In DFS (Distributed File System), replicated objects (data or file) should follow atomicity rules, i.e., all copies should be updated (synchronously or asynchronously) or none.

6. Goal  One-copy serializability: The effect of transactions performed by clients on replicated objects should be the same as if they had been performed one at a time on a single set of objects.[2]

7. Architecture  FSA, File service agent, client interface  RM, replica manager, provide replication functions [3]

8. Architecture[3]

9. Read operations [3]  Read-one-primary, FSA only read from a primary RM, consistency  Read-one, FSA may read from any RM, concurrency  Read-quorum, FSA must read from a quorum of RMs to decide the currency of data

10. Write Operations[3]  Write-one-primary, only write to primary RM, primary RM update all other RMs  Write-all, update to all RMs  Write-all- available, write to all functioning RMs. Faulty RM need to be synched before bring online.

11. Write Operations  Write-quorum, update to a predefined quorum of RMs  Write-gossip, update to any RM and lazily propagated to other RMs

12. Read one primary, write one primary  Other RMs are backups of primary RM  No concurrency  Easy serialized  Simple to implement  Achieve one-copy serializability  Primary RM is performance bottleneck

13. Read one, Write all  Provides concurrency  Concurrency control protocol needed to ensure consistency (serialization)  Achieve one-copy serializability  Difficult to implement (there will be failed TM to block any updates)

14. Read one, Write all available  Variation of Read one, Write all  May not guarantee one-copy serializability  Issue of loss conflict in transactions

15. Loss of Conflicts  Assume to RMs, (a,b), object X,Y replicated to both.  Two transactions  T1:R(X),W(Y),commit  T2:R(Y),W(X),commit

16. Loss of Conflict  If Xa,Yb failed, transaction as follows  T1:R(Xa),(Yb failed),W(Ya),commit  T2:R(Yb),(Xa failed),W(Xb),commit  There is no conflict since no object is shared. Thus loss conflict. This can introduce error.

17. Read quorum, Write quorum  Version number attached to replicated object  Highest version numbered object is the latest object in read.  Write operation advances version by 1  Write quorum > half of all object copies  Write quorum+read quorum > all object copies

18. Gossip Update  Applicable for frequent read, less update situations  Increased performance  Typical read one, write gossip  Use timestamp

19. Basic Gossip Update  Used for overwrite  Three operations, read, update, gossip arrive  Read, if TS fsa <=TS rm, RM has recent data, return it, otherwise wait for gossip, or try other RM  Update, if Ts fsa >TS rm, update. Update TS rm send gossip. Otherwise, process based on application, perform update or reject  Gossip arrive, update RM if gossip carries new updates.

20. Causal Order Gossip Protocol[3]  Used for read-modify  In a fixed RM configuration  Using vector timestamps  Using buffer to keep the order

21. Windows Server 2003[4]  Support DFS  “State based, multi master” scheduled replication  Use namespace for transparent file sharing  Use Remote Differential Compression to propagate change only to save bandwidth

22. Continued[5] If replication detects a conflict, last update wins. No merge files, but copies are kept for reference.

23. Reference [1] Wikipedia; http://en.wikipedia.org/wiki/Atomicity [2] M. T. Harandi;J. Hou (modified: I. Gupta);"Transactions with Replication";http://www.crhc.uiuc.edu/~nhv/428/slides/repl-trans.ppt [3] Randy Chow,Theodore Johnson, “Distributed Operating Systems & Algorithms”, 1998 [4] "Overview of the Distributed File System Solution in Microsoft Windows Server 2003 R2";http://technet2.microsoft.com/WindowsServer/en/library/d3afe6ee- 3083-4950-a093-8ab748651b761033.mspx?mfr=true [5] "Distributed File System Replication: Frequently Asked Questions";http://technet2.microsoft.com/WindowsServer/en/library/f9b 98a0f-c1ae-4a9f-9724-80c679596e6b1033.mspx?mfr=true