1. 6.4 Data and File Replication
Sreekanth Padidala

2. Why replicate Performance Reliability Resource sharing
Network resource saving

3. Challenge Transparency Replication Concurrent control Failure recovery
Serialization

4. Atomicity
In database systems, atomicity is one of the ACID transaction properties. An atomic transaction is a series of database operations which either all occur, or all do not occur[1].
All or nothing

5. Atomicity
In DFS (Distributed File System), replicated objects (data or file) should follow atomicity rules, i.e., all copies should be updated (synchronously or asynchronously) or none.

6. Goal
One-copy serializability: The effect of transactions performed by clients on replicated objects should be the same as if they had been performed one at a time on a single set of objects.[2]

7. Architecture FSA , File service agent, client interface
RM, replica manager, provide replication functions [3]

8. Architecture[3]

9. Options for Read/Write
Read-one-primary: read from a primary RM (consistency)
Read-one: read from any RM (concurrency)
Read-quorum: read from a quorum of RMs (currency)
WRITE
Write-one-primary: write to one primary replica
Primary RM propagates the updates to all other RMs
Write-all: atomic updates to all RMs (subsequent writes must wait)
Write-all-available: atomic updates to all available (non-faulty) RMs
Failure recovery
Write-quorum: atomic updates to a quorum of RMs
Write-gossip: updates to any RM and are lazily propagate to others

10. Read operations [3]
Read-one-primary, FSA only read from a primary RM, consistency
Read-one, FSA may read from any RM, concurrency
Read-quorum, FSA must read from a quorum of RMs to decide the currency of data

11. Write Operations[3]
Write-one-primary, only write to primary RM, primary RM update all other RMs
Write-all, update to all RMs
Write-all- available, write to all functioning RMs. Faulty RM need to be synched before bring online.

12. Write Operations Write-quorum, update to a predefined quorum of RMs
Write-gossip, update to any RM and lazily propagated to other RMs

13. Read one primary, write one primary
Other RMs are backups of primary RM
No concurrency
Easy serialized
Simple to implement
Achieve one-copy serializability
Primary RM is performance bottleneck

14. Read one, Write all Provides concurrency
Concurrency control protocol needed to ensure consistency (serialization)
Achieve one-copy serializability
Difficult to implement (there will be failed TM to block any updates)

15. Read one, Write all available
Variation of Read one, Write all
May not guarantee one-copy serializability
Issue of loss conflict in transactions

16. Read quorum, Write quorum
Version number attached to replicated object
Highest version numbered object is the latest object in read.
Write operation advances version by 1
2*write-quorum > number of replicas
Read-quorum+write_quorum > number of replicas
Usually read-quorum is chosen to be smaller than write-quorum
Voting by witnesses
Weighted voting schemes.

17. Gossip Update Applicable for frequent read, less update situations
Increased performance
Typical read one, write gossip
Use timestamp

18. Basic Gossip Update Used for overwrite
Three operations, read, update, gossip arrive
Read, if TSfsa<=TSrm, RM has recent data, return it, otherwise wait for gossip, or try other RM
Update, if Tsfsa>TSrm, update. Update TSrm send gossip. Otherwise, process based on application, perform update or reject
Gossip arrive, update RM if gossip carries new updates.

19. Causal Order Gossip Protocol[3]
Used for read-modify
In a fixed RM configuration
Using vector timestamps
Using buffer to keep the order

20. Windows Server 2003[4] Support DFS
“State based, multi master” scheduled replication
Use namespace for transparent file sharing
Use Remote Differential Compression to propagate change only to save bandwidth

21. Continued[5]
If replication detects a conflict, last update wins. No merge files, but copies are kept for reference.

22. Reference
[1] Wikipedia; [2] M. T. Harandi;J. Hou (modified: I. Gupta);"Transactions with Replication"; [3] Randy Chow,Theodore Johnson, “Distributed Operating Systems & Algorithms”, 1998 [4] "Overview of the Distributed File System Solution in Microsoft Windows Server 2003 R2"; [5] "Distributed File System Replication: Frequently Asked Questions";