1. Lecture 12 Synchronization

2. EECE 411: Design of Distributed Software Applications Roadmap for today Project logistics Posted yesterday P01 due Wednesday Nov. 3 rd Apply for planetlab accounts Discuss quiz questions Synchronization in distributed systems

3. EECE 411: Design of Distributed Software Applications Before we start survey results

4. EECE 411: Design of Distributed Software Applications Useful Discussions structure20 good thinking exercises; helps understand how knowledge is applied; good to discuss quizz-like questions Assignments 9 closely related to class materials, usefull hands-on work, good that marking is done on coding style as much as functionality but *too few*! Slides 8 Good summary of material, Useful for assignment, good overlap with previous week to tie in, I like the repetition, makes it more obvious what we have to learn Real-world examples 7 Availability of TA / instructor || coding session1 hmmmm' voting technique1

5. EECE 411: Design of Distributed Software Applications Concerns Project not yet up10 description / grading scheme / project expectations PlanetLab tutorial Textbook is not a good reference 3 Epidemic2 I would like to see this topic Replication 1 How do large things work 1 Distributed decision making1 Cloud computing 1 Event-driven programming1 Examples of pseudocode 1 Virtualization1 Security 1

6. EECE 411: Design of Distributed Software Applications Suggestions Sample quizes/clearer idea on quiz expectations8 More sample questions / more sample problems7 Questions with answers (summarize discussions in slides)6 More detailed explanation within the slides3 Make slides available earlier2 Discussion board 2 Fixed course structure 1 More structured relationships between topics1 More coding sessions1 More short assignments 1 Make sure that students without adequate coding experience can not take the class 1 Results of a design rather than covering a bit about each1 Instructor-assigned groups (rather than based on student preference)1 Good on quiz one: lots of questions 1 Provide grading scheme beforehand 1 Provide more of a big picture / Better organization of content / Roadmap1 Tighter deadlines around project so that there is no cramming …1 Assignments that include written parts 1 if we learn about gossip should we have a chance to implement?1 More quantitative discussion questions1 Go slower 1 Be more clear in describing concepts 1 Grading scheme: Best 2 out of 3 three quizes1

7. EECE 411: Design of Distributed Software Applications Roadmap for today Project logistics Posted yesterday P01 due Wednesday Nov. 3 rd Apply for planetlab accounts Discuss quiz questions Synchronization in distributed systems

8. EECE 411: Design of Distributed Software Applications Q5.) Consider a circular Distributed Hash Table (DHT) with identifiers in the range [0; 127]. Suppose there are eight participating nodes with identifiers 1, 13, 43, 51, 70, 83, 100 and 115. The DHT is configured so that the successor list has length 2. Also, the DHT is configured so that the finger table has size one: i.e., each peer maintains only one ‘shortcut’ (or ‘finger’) – this aims to reduce the search space in half. Questions: a). Suppose that the following (key,value) pairs should be stored in the DHT: (0,’mama’), (3,’tata’), (7,’zaza’), (15,’bibi’), (110, ‘zizi’) and (125, ‘cici’). Which peers will store which (key,value) pair? Present your answer as a table. b.) Assume a search launched at node 13 for key 0. Describe the search process. c.) Suppose that peer 13 learns that peer 43 has left the DHT. How does peer 13 update its successor state information? Which peer is now its first successor? Its second successor? Is there any change in the set of keys each peer is responsible for? c.) Suppose that a new peer with the identifier 5 wants to join the DHT and it initially only knows the IP address of the peer 53. What steps are taken for peer 6 to join the system? How does the system look like after peer 6 joins?

9. EECE 411: Design of Distributed Software Applications Key placement Circular ID Space N13 N1 N70 N51 N43 Node ID K110 K3, K7 K0, K125 K15 128 0 N83 N100 N115

10. EECE 411: Design of Distributed Software Applications Search launched at N13 for K0 Each node maintains Successor list (2) Shortcuts (1) N13 N1 N70 N51 N43 Node ID K110 K3, K7 K0, K125 K15 N83 N100 N115

11. EECE 411: Design of Distributed Software Applications Search launched at N13 for K0 Each node maintains Successor list (2) Shortcuts (1) N13 N1 N70 N51 N43 Node ID K110 K3, K7 K0, K125 K15 N83 N100 N115

12. EECE 411: Design of Distributed Software Applications Search launched at N13 for K0 Each node maintains Successor list (2) Shortcuts (1) N13 N1 N70 N51 N43 Node ID K110 K3, K7 K0, K125 K15 N83 N100 N115

13. EECE 411: Design of Distributed Software Applications Search launched at N13 for K0 Each node maintains Successor list (2) Shortcuts (1) N13 N1 N70 N51 N43 Node ID K110 K3, K7 K0, K125 K15 N83 N100 N115

14. EECE 411: Design of Distributed Software Applications Peer 13 learns that peer 43 is dead. How does peer 13 update its successor state information? Which peer is now its first/second successor? Is there any change in the set of keys each peer holds? N13 N1 N70 N51 N43 Crashed K110 K3, K7 K0, K125 K15 N83 N100 N115

15. EECE 411: Design of Distributed Software Applications A new peer (6) wants to join the DHT and it initially only knows the IP address of the peer 51. What steps are taken for peer 6 to join the system? How does the system look like after peer 6 joins? 1. Lookup for 6?  N13 2. Predecessor of N13?  N1 3. Announce yourself to N1 4. N1 updates successor 5. N1 notifies predecesor 6. N115 updates successor 7. N6 joins 8. Creates successor list 9. Splits keys with N13 10. N13 updates predecesor N13 N1 N70 N51 K110 K3, K7 K0, K125 K15 N83 N100 N115 N6 joins Two invariants to maintain for correctness Key to node assignment Successor lists

16. EECE 411: Design of Distributed Software Applications 6.) What are the criteria to choose between a system based on consistent hashing and one based on a distributed hash table. Where are the differences? Key to node assignment?No difference Lookup?Yes (logN hops for DHT vs. 1 hop consistent hashing) Information used for lookup?Yes (logN vs. N) Impact of failures?Yes (logN vs. N) Ability to scale? Yes

17. EECE 411: Design of Distributed Software Applications Q8) In a sensor network, the sensors monitor temperature. a.) Sketch an epidemic style protocol so that each of the sensors present obtains an estimate of the maximum and average temperature measured by all the sensors in the network. b.) Sketch an epidemic style protocol to estimate the number of sensors in the network. c.) For the protocol presented at point a.) above please provide the pseudocode that presents in detail the operation of a sensor.

18. EECE 411: Design of Distributed Software Applications Assumptions: Each sensor already has a list L of all participating sensors Each sensor takes a temperature measurement each T seconds and if different from previous one tries to distribute it Each participating sensor Si keeps a table [S1, SN] with the latest values received

19. EECE 411: Design of Distributed Software Applications Roadmap for today Project logistics Posted yesterday P01 due Wednesday Nov. 3 rd Apply for planetlab accounts Discuss quiz questions Synchronization in distributed systems

20. EECE 411: Design of Distributed Software Applications Summary so far … A distributed system is: a collection of independent computers that appears to its users as a single coherent system Components need to: Communicate Point to point: sockets, RPC/RMI Point to multipoint: multicast, epidemic Cooperate Naming to enable some resource sharing Naming systems for flat (unstructured) namespaces: consistent hashing, DHTs Naming systems for structured namespaces: EECE456 for DNS Synchronization

21. EECE 411: Design of Distributed Software Applications Synchronization to support coordination Examples Distributed make Printer sharing Monitoring of a real world system Agreement on message ordering Why is synchronization more complex than in a single-box system No global views, multiple clocks, failures

22. EECE 411: Design of Distributed Software Applications Roadmap Physical clocks Provide actual / real time ‘Logical clocks’ Where only ordering of events matters Leader election How do I choose a coordinator?

23. EECE 411: Design of Distributed Software Applications Physical clocks (I) Problem: How to achieve agreement on time in a distributed system? A possible solution: use Universal Coordinated Time (UTC): Based on the number of transitions per second of the cesium 133 atom (pretty accurate). At present, the real time is taken as the average of some 50 cesium-clocks around the world. Introduces a leap second from time to time to compensate for days getting longer. UTC is broadcast through short wave radio and satellite. Accuracy ± 1ms (but if weather conditions considered ±10ms)

24. EECE 411: Design of Distributed Software Applications Physical clocks - underlying model Suppose we have a distributed system with a UTC-receiver somewhere in it. Problem: we still have to distribute time to each machine. Internal mechanism at each node Each machine has a timer Timer causes an interrupt H times a second Interrupt handler adds 1 to a software clock Software clock keeps track of the number of ticks since agreed-upon time in the past. Notation Value of clock on machine p at real time t is C p (t)

25. EECE 411: Design of Distributed Software Applications Physical clocks – main problem: clock drift Notation: Value of clock on machine p at real time t is C p (t) Ideally: C p (t) == t and dC p (t) = dt Real world: clock drift, i.e., |C p (t) - t | > 0 Clock value (C p ) guaranteed to progress: 1 - ρ ≤ (dC/dt) ≤ 1 + ρ ρ -- maximum drift rate Goal: Never let clocks in any two nodes in the system differ by more than x time units  synchronize at least every x/(2ρ) seconds.

26. EECE 411: Design of Distributed Software Applications Building a complete system … Option I: Every machine asks a time server for the accurate time at least once every x/(2ρ) seconds (Network Time Protocol). Okay, but need to account for network delays, including interrupt handling and processing of messages. Client updates time to: T new =C UTC +(T 2 -T 1 )/2 Fundamental: You’ll have to take into account that setting the time back is never allowed  smooth adjustments. Option II: Let the time server scan all machines periodically, calculate an average, and inform each machine how it should adjust its time relative to its present time. Note: you don’t even need to propagate UTC time.

27. EECE 411: Design of Distributed Software Applications Building a complete system … Option I: Every machine asks a time server for the accurate time at least once every x/(2ρ) seconds (Network Time Protocol). Okay, but need to account for network delays, including interrupt handling and processing of messages. Client updates time to

28. EECE 411: Design of Distributed Software Applications Real world: Network Time Protocol (NTP) Stratum 0 NTP servers – receive time from external sources (cesium clocks, GPS, radio broadcasts) Stratum N+1 servers synchronize with stratum N servers and between themselves Self-configuring network User configured to contact local NTP server Survey (N. Minar’99) > 175K NTP servers 90% of the NTP servers have <100ms offset fro synchronization peer 99% are synchronized within 1s

29. EECE 411: Design of Distributed Software Applications Uses of (synchronized) physical clocks in the real world NTP Global Positioning Systems Using physical clocks to implement at-most- once semantics

30. EECE 411: Design of Distributed Software Applications Summary so far Synchronization solutions Physical time synchronization Often costly, imperfect But with real applications (NEXT TIME)