1. Distributed Systems Distributed Algorithms 1 Brendan Tangney tangney@tcd.ie Distributed Systems (Distributed Algorithms – A Problem Based Learning Approach)

2. Distributed Systems Distributed Algorithms 2 Topics  Some Problems  Some Solutions  Why Are Distributed Algorithms Difficult  The End to End Argument  Naming  File Systems  Historical Perspective on “Distributed Operating Systems”

3. Distributed Systems Distributed Algorithms 3 Spot The Difference

4. Distributed Systems Distributed Algorithms 4 Reaching Agreement (The Two Generals Problem)  The leaders of 2 armies need to agree upon the time to attack the castle. They will take it iff they attack together.  The communicate by sending messengers.  Messengers can get shot!  Design a protocol that will ensure they attack at the same time!

5. Distributed Systems Distributed Algorithms 5 The Solution

6. Distributed Systems Distributed Algorithms 6 Load Balancing Perform “load balancing” on an intranet of workstations. Solutions to be presented in class on Wednesday.

7. Distributed Systems Distributed Algorithms 7 Two Classic Distributed Algorithms

8. Distributed Systems Distributed Algorithms 8 LAN Media Access Protocols  In ring and tree topology local area networks there is more than one point of access to the media, i.e. there is no one point of control.  But allocation of network bandwidth to nodes must be fair.  How does the token ring protocol control access to the medium?  How does the Ethernet (CSMA/CD) algorithm do it?

9. Distributed Systems Distributed Algorithms 9  Insight: The Token Passing Mutual Exclusion Algorithm maps easily onto a ring topology with the physical wiring giving the ordering between nodes.  A special frame called a "token" is passed from station to station.  Possession of the token constitutes permission to transmit.  Sender sizes the token and sends a data packet to the desired destination address (DA). DA copies the packet and fills in Frame Status byte of frame to indicate ok/error.  Packet is removed by the sender and the token sent on.  Very simple access algorithm.  Distributed  Pessimistic. Token Ring Access Protocol

10. Distributed Systems Distributed Algorithms 10 Token Passing

11. Distributed Systems Distributed Algorithms 11  Errors can occur during operation which would cause the token algorithm to break down.  E.g. a continuously circulating packet can be caused by i) the demise of a node which sent the data packet, ii) noise flipping the T bit. Error Handling

12. Distributed Systems Distributed Algorithms 12 The Monitor  One station on the ring is designated to be the ACTIVE MONITOR.  The Monitor sets the M bit in each data frame as it goes by.  Stations clear the M bit when they transmit a frame.  If a frame arrives at the monitor with the M bit set it must be an error. Monitor removes the frame.  The monitor injects a new token into the ring whenever it detects the absence of one.  Centralised Algorithm.  Moderately Complex.  Not invoked too frequently.

13. Distributed Systems Distributed Algorithms 13  The Monitor is now a single point of failure.  Need a dynamic distributed algorithm for detecting and recovering from Monitor loss. Monitor Failure

14. Distributed Systems Distributed Algorithms 14 Monitor Election  Stations may detect a missing monitor via timeouts.  The monitor sends out periodic AMP frames.  On detecting the absence of the active monitor nodes elect a new active monitor.  The election algorithm is based on nodes sending Claim Token frames claiming to be the new active monitor but deferring to nodes with higher addresses. (It is not a fair algorithm).  Distributed Algorithm.  Not fair (highest node always wins).  There is also another protocol for nodes entering the ring to ensure that their own address is unique on the ring and to determine their upstream neighbours’s address (UNA).

15. Distributed Systems Distributed Algorithms 15 More Complexity  ELECTION ALGORITHM  The CT frame contains the node’s own address and that of its Upstream Neighbour (UNA).  On receipt of a CT frame a node checks the value of SA and UNA in the frame.  IF SA /UNA match its own THEN it takes over AM functions ELSE IF SA < itself THEN continue to claim the token ELSE give up  Station with the highest address that is competing to become the new active monitor will take over.  Active Monitor and UNA  When the new Monitor sends an AMP frame it sets the A bit to 0  The first station receiving it (the downstream neighbor) records the SA Sets A and C bits to 1 before repeating the frame The station knows it immediate upstream neighbor address  The A bit equal to 1 informs the other stations that the arriving frame is not from their immediate neighbor  After receiving an AMP frame from its upstream neighbor (Monitor) The station sends and SMP frame with A=0  Its downstream neighbor receives the frame Record the SA and sets A and C bits to 1  This procedure allows each station to learn its upstream neighbor ‘s address  Network Disruption  If a node breaks down, or the ring is broken, one station on the ring will detect garbage or silence on it's input.  It transmits a management frame known as a "Beacon". This is addressed to all nodes, and contains the address of its up- stream neighbour (UNA).  This enables fault detection.

16. Distributed Systems Distributed Algorithms 16 Complexity and Frequency of Use of Token Ring Sub-algorithms Complexity Frequency of use Media Access Algorithm Token Loss Monitor Loss Fair Not Fair

17. Distributed Systems Distributed Algorithms 17

18. Distributed Systems Distributed Algorithms 18 Carrier Sense, Multiple Access, Collision Detect  Origin: University of Hawaii.  Insights: Network is mostly idle. Optimistic approach. Allow simultaneous transmission on the network (should happen rarely) and include a protocol to tidy up when the assumption is wrong. Use of randomness!

19. Distributed Systems Distributed Algorithms 19 CSMA/CD

20. Distributed Systems Distributed Algorithms 20 When a station has data to transmit 1. If medium idle, transmit. 2. If busy, listen until quiet - then transmit. 3. While transmitting listen to the network. If garbage then a collision has occurred. Transmit a jamming signal (collision enforcement). 4. Wait a random amount of time (backoff) and restart at step 1. CSMA/CD

21. Distributed Systems Distributed Algorithms 21  The algorithm used for step 4 is as follow. Attempts := 0; OK := TX(Frame) While (NOT OK) AND (Attempts < Attempt Limit) K : = Min (Attempts, Backoff Limit) R : = Random (0, 2 k -1) delay : = R * SlotTime; Wait (delay) Attempts++; OK := TX(Frame) End  This algorithm yields a "statistical fairness" in the sharing of bandwidth.  Attempt Limit is 16 and Backoff Limit 10. Backoff

22. Distributed Systems Distributed Algorithms 22 Behaviour In The Face Of Failure?  Node failure ?  Passive  Active  Cable break ?  No centralised points of control => a very robust algorithm.

23. Distributed Systems Distributed Algorithms 23 Comparison CSMA/CD  Optimistic. Optimised for the frequent case of one node transmitting. Complex backoff protocol if incorrect.  Robust in the face of failure.  No overhead in collision free cases.  Use of randomness for distributed control Token Passing  Pessimistic. Assumes simultaneous transmission and prevents it occurring.  Simple access protocol. Complexity needed to handle failure.  Always incurs a modest overhead on transmission.  Use of token for centralised control.