More on Replication and Consistency CS-4513 D-term More on Replication and Consistency CS-4513 Distributed Computing Systems (Slides include materials from Operating System Concepts, 7 th ed., by Silbershatz, Galvin, & Gagne, Distributed Systems: Principles & Paradigms, 2 nd ed. By Tanenbaum and Van Steen, and Modern Operating Systems, 2 nd ed., by Tanenbaum)

More on Replication and Consistency CS-4513 D-term Replication Common in distributed systems –Reliability –Performance Maintaining consistency among replicas –Correctness in operation –Unsurprising behavior Many models for consistency

More on Replication and Consistency CS-4513 D-term A Mini-Taxonomy of Replication Techniques Copy entire database vs. Update individual items Single source of changes I.e., Primary-based protocols vs. Multiple, independent sources of changes I.e. Replicated-write protocols

More on Replication and Consistency CS-4513 D-term Copying Entire Database Single source of changes –Simply copy entire database –E.g., DNS domain Common practice for sub-domains at administrative & managerial layers Not at all practical at the global layer …

More on Replication and Consistency CS-4513 D-term Copying Entire Database (continued) … Multiple concurrent, sources of changes –Every item in database must have timestamp With suitable granularity –Merge copies, item-by-item –Question: what about deletions?

More on Replication and Consistency CS-4513 D-term Example of Merging Databases Replica A contains entry for “Smith” but not “Jones” Replica B contains entry for “Jones” but not “Smith” When merging, –Should A add “Jones” or delete “Smith”? –Should B add “Smith” or delete “Jones”?

More on Replication and Consistency CS-4513 D-term Solution Maintain deletions until all replicas have seen them. Special sublists of deleted items Timestamps on deletions, as with all other changes Question: can it be proven mathematically that replicas converge?

More on Replication and Consistency CS-4513 D-term Multiple Sources of Change Example Grapevine Registry In initial implementation, an entire registry was copied and merged after each update Each item has timestamp {local time, IP address} Local time is in terms of server making the update Registry’s own timestamp Latest timestamp of any of its members Each registry and group comprises two sublists Current members Deleted members Rule: an RName could only be a member of one sublist …

More on Replication and Consistency CS-4513 D-term Grapevine Registry Example (continued) … Merging item-by-item Later version of any item replaces earlier version of same item Item may move between current and deleted lists No provision for resolving inconsistent, concurrent updates by different servers Apply same rule recursively to groups I.e., an item may be a group, comprising other items To update group, merge with copied group by same rule

More on Replication and Consistency CS-4513 D-term What if Database is Too Big? Propagate updates to individual items Each item has timestamp Newer timestamp replaces older timestamp Each update is a single operation (or sequence) Add, Delete, or Change Apply recursively to groups Propagate only the element that has changed Preserve deletions until all updates have fully propagated

More on Replication and Consistency CS-4513 D-term Propagating Individual Updates Primary-based protocols Simple, since all updates are monotonically time- stamped Replicated write protocols Similar to merge protocols Later timestamp wins Deletions must be preserved until updates have fully propagated

More on Replication and Consistency CS-4513 D-term Question What can we say about update order as perceived by clients?

More on Replication and Consistency CS-4513 D-term Wrap-up — Replication and Consistency Much more to be said Lots of research Practical implementations in a number of large, distributed systems

More on Replication and Consistency CS-4513 D-term Questions?

More on Replication and Consistency CS-4513 D-term DNS Naming Hierarchy