1. Distributed Systems Lecture 7 Multicast 1

2. Previous lecture Global states – Cuts – Collecting state – Algorithms 2

3. Motivation Coordination is needed by distributed systems but hard to achieve: – Events happen concurrently – Communication links are not reliable – Computers can crash – New nodes can join the systems – Asynchronous environments  Need of an efficient way to coordinate a group of processes 3 Refer to slides of Andreas Larsson @ Chalmers University of Technology, Indranil Gupta @ IIT, and Insup Lee @ University of Pennsylvania

4. A distributed system in war: synchronous example Friendly fighter Radar Missile Commander Enemy missile Friend Ready to Fire Cancel AskPrepare Friendly fighter 4

5. A distributed system in war: reality Radar Missile Commander Enemy missile Friend Ready to Fire Fire AskPrepare Friendly fighter Friendly fighter 5

6. Group communication What is a group? – A number of processes which cooperate to provide a service. – An abstract identity to name a collection of processes. Group Communication – For coordination among processes of a group.

7. Who needs group communication? Highly available servers (client-server) Database Replication Multimedia Conferencing Online Games Cluster management …

8. High availability Who needs group communication?

9. Online game Fault-tolerance, Order

10. Different comm. methods Unicast – Point-to-Point Communication – Multiple copies are sent Broadcast – One-to-All Communication – Abuse of Network Bandwidth Multicast  this lecture – One-to-multiple (group) Communication

11. Desired group communication Name Abstraction Efficiency Delivery Guarantees Dynamic Membership  Multicast  Reliability, Ordering  Group membership service

12. Properties of communication Ordering – Total ordering, causal ordering Failure behavior Reliability – Validity, integrity, agreement

13. Properties of Group Name of group Addresses of group members Dynamic group membership Options: – Peer group or client-server group – Closed or Open Group

14. Akamai’s Configuration Management System (called ACMS) uses a core group of 3-5 servers – Servers continuously multicast to each other the latest updates. They use reliable multicast – After an update is reliably multicast within this group, it is then sent out to all the (1000s of) servers Akamai has all over the world Air Traffic Control System – Orders by one ATC need to be ordered (and reliable) multicast out to other ATCs Newsgroup servers multicast to each other in a reliable and ordered manner Facebook servers multicast your updates to each other Examples of multicast using

15. So what is the big issue … More than 20 years since proposal but no wide area IP multicast deployment Scalability (with number of groups) -- Routers maintain per-group state IP Multicast: best-effort multi-point delivery service -- Providing higher level features such as reliability, congestion control, flow control, and security has shown to be more difficult than in the unicast case Can we achieve efficient multi-point delivery without IP-layer support?

16. Application layer multicast Stanford CMU Stan1 Stan2 Berk2 Overlay Tree Gatech Berk1 Berkeley Gatech Stan1 Stan2 Berk1 Berk2 CMU

17. Pros and cons Scalability – Routers do not maintain per-group state – End systems do, but they participate in very few groups Potentially simplify support for higher level functionality – Leverage computation and storage of end systems – Leverage solutions for unicast congestion, error and flow control Efficiency concerns – redundant traffic on physical links – increase in latency due to end-systems

18. System structure The overlay comprises of : A central source (may be replicated for fault tolerance) A number of overcast nodes (standard PCs with lot’s of storage) - organized into a distribution tree rooted at the source - bandwidth efficient trees Final Consumers – members of the multicast group - allows unmodified HTTP clients to join

19. Bandwidth Efficient Overlay Trees “…three ways of organizing the root and the nodes into a distribution tree.” 10 Mb/s 100 Mb/s R 1 2 R 1 2 R 12 R 12

20. The node addition algorithm R 5 5 7 1 10 2 3 8 R 1 2 3 Physical network substrate Overcast distribution tree

21. The client side: how to join a multicast group Clients join a multicast group through a typical HTTP GET request Root determines where to connect the client to the multicast tree using – Status of overcast nodes – Location of client Root selects “best” server and redirects the client to that server

22. Client Joins R1R1 1 2 3 456 R2R2 R3R3 Key: Content query (multicast join) Query redirect Content delivery

23. Application (at process p) MULTICAST PROTOCOL send multicast Incoming messages deliver multicast (upcall) One process p Multicast protocol

24. Let’s assume the all processes know the group membership A straightforward way to implement B-multicast is to use a reliable one-to-one send (unicast) operation: – B-multicast(group g, message m) for each process p in g, send (p, m). – receive(m) B-deliver(m) at p. A correct process = a non-faulty process A basic multicast primitive guarantees a correct process will eventually deliver the message, as long as the sender (multicasting process) does not crash. – Can we provide reliability even when the sender crashes (after it has sent the multicast)? Basic multicast (B-multicast)

25. Integrity – A correct process p delivers a message m at most once Validity: If a correct process multicasts (sends) message m, then it will eventually deliver m itself – Guarantees liveness to the sender Agreement: If some correct process delivers message m, then all other correct processes in group(m) will eventually deliver m – Property of all or nothing – Validity and agreement together ensure overall liveness: If some correct process multicasts a message m, then, all correct processes deliver m too. Reliable multicast

26. R-multicast B-multicast reliable unicast “USES” Reliable R-multicast algorithm Integrity Agreement if some correct process B-multicasts a message m, then, all correct processes R-deliver m too. If no correct process B-multicasts m, then no correct processes R-deliver m. Integrity, Validity

27. FIFO ordering – If a correct process issues multicast(g,m) and then multicast(g,m’) then every correct process that delivers m’ will have already delivered m – Sender ordered Causal ordering – If multicast(g,m)  multicast(g,m’) then any correct process that delivers m’ will have already delivered m – Happens before Total ordering – If a correct process delivers message m before m’ (independent of the senders), then any other correct process that delivers m’ will have already delivered m. Message ordering

28. Virtual synchrony 28

29. FIFO ordering 29 Without failures all we need is to run algorithm on FIFO channels (i.e., TCP) With failures we need to be careful about the order in which things are done

30. Causal ordering 30 Start from FIFO ordering It has been shown that from it we can achieve causal ordering by adding vector time

31. Total ordering 31 Centralized sequencer Token based implementation Token has a sequence number When p holds token it can send out new burst of multicasts

32. Asynchrony FIFO and causal ordering can be used asynchronously – Can always be delivered at the time the multicast is sent – We gain concurrency at sender side Helps only if remainder of group is idle or doing non conflicting tasks But: – Things pile up in output buffers – Much is lost in case of failures, – We consume much of sender-side buffering space Total ordering stutters – Ordering is not known before a round of message exchange has been completed 32

33. Next lecture RPC and marshalling 33