Distributed Systems Lecture 7 Multicast 1

Previous lecture Global states – Cuts – Collecting state – Algorithms 2

Motivation Coordination is needed by distributed systems but hard to achieve: – Events happen concurrently – Communication links are not reliable – Computers can crash – New nodes can join the systems – Asynchronous environments  Need of an efficient way to coordinate a group of processes 3 Refer to slides of Andreas Chalmers University of Technology, Indranil IIT, and Insup University of Pennsylvania

A distributed system in war: synchronous example Friendly fighter Radar Missile Commander Enemy missile Friend Ready to Fire Cancel AskPrepare Friendly fighter 4

A distributed system in war: reality Radar Missile Commander Enemy missile Friend Ready to Fire Fire AskPrepare Friendly fighter Friendly fighter 5

Group communication What is a group? – A number of processes which cooperate to provide a service. – An abstract identity to name a collection of processes. Group Communication – For coordination among processes of a group.

Who needs group communication? Highly available servers (client-server) Database Replication Multimedia Conferencing Online Games Cluster management …

High availability Who needs group communication?

Online game Fault-tolerance, Order

Different comm. methods Unicast – Point-to-Point Communication – Multiple copies are sent Broadcast – One-to-All Communication – Abuse of Network Bandwidth Multicast  this lecture – One-to-multiple (group) Communication

Desired group communication Name Abstraction Efficiency Delivery Guarantees Dynamic Membership  Multicast  Reliability, Ordering  Group membership service

Properties of communication Ordering – Total ordering, causal ordering Failure behavior Reliability – Validity, integrity, agreement

Properties of Group Name of group Addresses of group members Dynamic group membership Options: – Peer group or client-server group – Closed or Open Group

Akamai’s Configuration Management System (called ACMS) uses a core group of 3-5 servers – Servers continuously multicast to each other the latest updates. They use reliable multicast – After an update is reliably multicast within this group, it is then sent out to all the (1000s of) servers Akamai has all over the world Air Traffic Control System – Orders by one ATC need to be ordered (and reliable) multicast out to other ATCs Newsgroup servers multicast to each other in a reliable and ordered manner Facebook servers multicast your updates to each other Examples of multicast using

So what is the big issue … More than 20 years since proposal but no wide area IP multicast deployment Scalability (with number of groups) -- Routers maintain per-group state IP Multicast: best-effort multi-point delivery service -- Providing higher level features such as reliability, congestion control, flow control, and security has shown to be more difficult than in the unicast case Can we achieve efficient multi-point delivery without IP-layer support?

Application layer multicast Stanford CMU Stan1 Stan2 Berk2 Overlay Tree Gatech Berk1 Berkeley Gatech Stan1 Stan2 Berk1 Berk2 CMU

Pros and cons Scalability – Routers do not maintain per-group state – End systems do, but they participate in very few groups Potentially simplify support for higher level functionality – Leverage computation and storage of end systems – Leverage solutions for unicast congestion, error and flow control Efficiency concerns – redundant traffic on physical links – increase in latency due to end-systems

System structure The overlay comprises of : A central source (may be replicated for fault tolerance) A number of overcast nodes (standard PCs with lot’s of storage) - organized into a distribution tree rooted at the source - bandwidth efficient trees Final Consumers – members of the multicast group - allows unmodified HTTP clients to join

Bandwidth Efficient Overlay Trees “…three ways of organizing the root and the nodes into a distribution tree.” 10 Mb/s 100 Mb/s R 1 2 R 1 2 R 12 R 12

The node addition algorithm R R Physical network substrate Overcast distribution tree

The client side: how to join a multicast group Clients join a multicast group through a typical HTTP GET request Root determines where to connect the client to the multicast tree using – Status of overcast nodes – Location of client Root selects “best” server and redirects the client to that server

Client Joins R1R R2R2 R3R3 Key: Content query (multicast join) Query redirect Content delivery

Application (at process p) MULTICAST PROTOCOL send multicast Incoming messages deliver multicast (upcall) One process p Multicast protocol

Let’s assume the all processes know the group membership A straightforward way to implement B-multicast is to use a reliable one-to-one send (unicast) operation: – B-multicast(group g, message m) for each process p in g, send (p, m). – receive(m) B-deliver(m) at p. A correct process = a non-faulty process A basic multicast primitive guarantees a correct process will eventually deliver the message, as long as the sender (multicasting process) does not crash. – Can we provide reliability even when the sender crashes (after it has sent the multicast)? Basic multicast (B-multicast)

Integrity – A correct process p delivers a message m at most once Validity: If a correct process multicasts (sends) message m, then it will eventually deliver m itself – Guarantees liveness to the sender Agreement: If some correct process delivers message m, then all other correct processes in group(m) will eventually deliver m – Property of all or nothing – Validity and agreement together ensure overall liveness: If some correct process multicasts a message m, then, all correct processes deliver m too. Reliable multicast

R-multicast B-multicast reliable unicast “USES” Reliable R-multicast algorithm Integrity Agreement if some correct process B-multicasts a message m, then, all correct processes R-deliver m too. If no correct process B-multicasts m, then no correct processes R-deliver m. Integrity, Validity

FIFO ordering – If a correct process issues multicast(g,m) and then multicast(g,m’) then every correct process that delivers m’ will have already delivered m – Sender ordered Causal ordering – If multicast(g,m)  multicast(g,m’) then any correct process that delivers m’ will have already delivered m – Happens before Total ordering – If a correct process delivers message m before m’ (independent of the senders), then any other correct process that delivers m’ will have already delivered m. Message ordering

Virtual synchrony 28

FIFO ordering 29 Without failures all we need is to run algorithm on FIFO channels (i.e., TCP) With failures we need to be careful about the order in which things are done

Causal ordering 30 Start from FIFO ordering It has been shown that from it we can achieve causal ordering by adding vector time

Total ordering 31 Centralized sequencer Token based implementation Token has a sequence number When p holds token it can send out new burst of multicasts

Asynchrony FIFO and causal ordering can be used asynchronously – Can always be delivered at the time the multicast is sent – We gain concurrency at sender side Helps only if remainder of group is idle or doing non conflicting tasks But: – Things pile up in output buffers – Much is lost in case of failures, – We consume much of sender-side buffering space Total ordering stutters – Ordering is not known before a round of message exchange has been completed 32

Next lecture RPC and marshalling 33