Algorithm for Virtually Synchronous Group Communication Idit Keidar, Roger Khazan, Nancy Lynch MIT Lab for Computer Science Theory of Distributed Systems Group

Virtual Synchorny Application Virtual Synchrony Multicast Service Membership Service

Virtual Synchrony Synchronization of Messages and Views: Powerful abstraction for replication Semantics: VS [Birman, Joseph 87], EVS, SVS Procs that go together through same views, deliver same sets of messages.

Example: Virtual Synchrony

Project Goals High-quality design of a VS GCS for WANs Mathematical-quality (precise, formal, well-documented) Useful semantcis Efficient algorithm Scalable architecture Modular design Specification Algorithm Proof Performance Analysis

Publications ICDCS’00: Intern. Conf. On Distributed Comp. Submited to SICOMP (SIAM Journal of Comp.) ICSE’00: Intern. Conf. On Software Engineering Invited to ACM TOSEM

Virtual Synchrony: How To? Before moving into new view: Need to know which synch msgs to use, since there may be several view proposals Exchange synch messages (“flush”) to agree which msgs to deliver in old view.

Example: Synchronization Msgs

Problematic Scenario

Existing Solutions Limit Reconfiguration Do not allow joins during reconfiguration When someone wants to join: first, deliver view without joiner; then, start new reconfiguration. Use common id to identify synch msgs for same view proposal

Limited Reconfiguration

Problems with Existing Solutions Limited Reconfiguration Obsolete views delivered to application Creates overhead Limits usefulness of virtual synchrony Use of common id to identify synch msgs Pre-agreement or dissemination is required Costly, especially in WANs

Our Idea Don’t limit reconfiguration Issue locally unique id per process for each view proposal Tag synch msgs with these local ids View includes vector of latest local ids View is a triple: e.g., < 4, {p, q, r}, [8, 9, 3] > Procs use sync msgs identified by view Hence, procs use right sync msgs

Our Algorithm Allows Joiners

No Common Sync Ids Required

Transient Failure

Implementation VS library (C++), linked with application Use [KSMD,00] membership service implemented in C++, socket interface with members Reliable FIFO layer (made in Hebrew University), uses IP multicast and recovers lost messages, library --- linked with VS

Group Communication -- Useful “Building Block” Group Abstraction processes interact in a group dynamic: fail/join/partition/merge Reliable Group Multicast Group Membership -- generates “views” tell each process who it is connected to Systems: Ensemble, Horus, Isis, Newtop, Psync, Sphynx, Relacs, Totem, Transis

Example: Group Communication