1. CSE 812

2. Outline What is a distributed system/program? Program Models Program transformation

3. What is a distributed system? A network of processes. The nodes are processes, and the edges are communication channels

4. Code for Distributed Systems Each node would need some code. –CSE 231? Communication between nodes –Network level issues –CSE 824 Protocols/Programs for allowing this system to function –This course –Focus on abstract protocols (in my presentations) –Focus on implementation issues (Student presentations)

5. Important issues Knowledge is local Clocks are not synchronized No shared address space Most of the time Topology and routing Scalability Fault tolerance

6. Some common subproblems Leader election Mutual exclusion Time synchronization Distributed snapshot Replica management

7. A classification Client-server modelPeer-to-peer model: Notion of neighbors Server Clients

8. Outline What is a distributed system/program? Program Models Program transformation

9. Defining Programs Goal of this discussion is to extend the concept of programs from programs such as that in C/C++/… to more abstract programs Consider the map of MSU shown on the next page. A robot needs to be programmed so that it can go from point A to point B –Identify a program for such robot

10. A B

11. A B

12. Issues Non determinism –At a certain state, the program is presented from a set of multiple options. –Assumption: the program may choose either of these options non- deterministically. Note that this does not imply any fairness unless assumed otherwise.

13. Abstraction Thinking of the program as a finite automata

16. Defining Safety Example: –Consider the arrows in previous picture Intuitively, safety identifies transitions that should not be executed by the program

17. Example 2 Peterson’s mutual exclusion algorithm –Two processes State can be n (non critical), t (trying), or c (critical) It is necessary to ensure that both processes are not in state c simultaneously If a process is in state t, then it must eventually go to state c When a process in state n (respectively, t, c) changes its state, it must change it to t (respectively, c, n) –Additional variable turn

18. Automata for Peterson’s Mutual Exclusion n1,n2 t1,n2 n1,t2 c1,n2 n1,c2t1,t2 turn=1 t1,t2 turn=2 c1,t2 t1,c2

19. Safety Examples –[c1, c2] not reached –([n1, n2], [c1, n2]) is not permitted

20. Example 3 Car climate control –Driver side temperature –Passenger side temperature –Controls for increasing and decreasing temperature –A button for `Sync’ –Minimum and maximum temperature

21. Automata for Car Climate Control

22. Use of Invariants Sync ==> d_temp = p_temp d_temp >= min p_temp >= min d_temp <= max p_temp <= max

23. Designing Programs Given an Invariant

24. Stepping back Overview of different program models –Message passing model –Shared memory model Different semantics

25. Why do we need models? There are many dimensions of variability in distributed systems. Examples: - interprocess communication mechanisms, - failure classes, - security mechanisms etc.

26. Why do we need models? Models are simple abstractions that help understand the variability -- abstractions that preserve the essential features, but hide the implementation details from observers who view the system at a higher level. For example C program is an abstraction of the assembly program that preserves certain properties

27. A message passing model System topology is a graph G = (V, E), where V = set of nodes (sequential processes) E = set of edges Four types of actions by a process: - Internal action-input action - Communication action-output action

28. A Reliable FIFO channel Axiom 1. Message m sent  message m received Axiom 2. Message propagation delay is arbitrary but finite. Axiom 3. m1 sent before m2  m1 received before m2. P Q

29. Life of a process in message passing model When a message m is rec’d 1.Evaluate a predicate with m and the local variables; 2. if predicate = true then -update internal variables; -send zero or more messages; else skip {do nothing} end if

30. Synchrony vs. Asynchrony Send & receive can be blocking or non- blocking Postal communication is asynchronous: Telephone communication is synchronous Synchronous communication or not? (1)Remote Procedure Call, (2)Email Synchronous clocks Physical clocks are synchronized Synchronous processes Lock-step synchrony Synchronous channels Bounded delay Synchronous message-order First-in first-out channels Synchronous communication Communication via handshaking

31. Shared memory model Address spaces of processes overlap M1 13 2 4 M2 Concurrent operations on a shared variable are serialized

32. Variations of shared memory models 0 3 2 1 State reading model Each process can read the states of its neighbors 0 3 21 Link register model Each process can read from and write to adjacent registers. The entire local state is not shared.

33. Weak vs. Strong Models One object (or operation) of a strong model = More than one objects (or operations) of a weaker model. Often, weaker models are synonymous with fewer restrictions. One can add layers (additional restrictions) to create a stronger model from weaker one. Examples HLL model is stronger than assembly language model. Asynchronous is weaker than synchronous. Bounded delay is stronger than unbounded delay (channel)

34. Model transformation Stronger models - simplify reasoning, but - needs extra work to implement Weaker models - are easier to implement. - Have a closer relationship with the real world “Can model X be implemented using model Y?” is an interesting question in computer science. Sample problems Non-FIFO to FIFO channel Message passing to shared memory Non-atomic broadcast to atomic broadcast

35. Non-FIFO to FIFO channel P Q buffer m1m4m3m2

36. Non-FIFO to FIFO channel {Sender process P}{Receiver process Q} var i : integer {initially 0}var k : integer {initially 0} buffer: buffer[0..∞] of msg {initially  k: buffer [k] = emptyrepeat send m[i],i to Q; {STORE} receive m[i],i from P; i := i+1 store m[i] into buffer[i]; forever {DELIVER} while buffer[k] ≠ empty do begin deliver content of buffer [k]; buffer [k] := empty  k := k+1; end forever

37. Observations (a) Needs Unbounded sequence numbers and (b) Unbounded number of buffer slots (Both are bad) Now solve the same problem on a model where (a) The propagation delay has a known upper bound of T. (b) The messages are sent out @r per unit time. (c) The messages are received at a rate faster than r. The buffer requirement drops to r.T. Synchrony pays. Question. How to solve the problem using bounded buffer space if the propagation delay is arbitrarily large?

38. Message-passing to Shared memory {Read X by process i}: read x[i] {Write X:= v by process i} - x[i] := v; -Atomically broadcast v to every other process j (j ≠ i); -After receiving broadcast, process j (j ≠ i) sets x[j] to v. Understand the significance of atomic operations. It is not trivial, but is very important in distributed systems This is incomplete. There are more pitfalls here.

39. Non-atomic to atomic broadcast Atomic broadcast = either everybody or nobody receives {process i is the sender} for j = 1 to N-1 (j ≠ i) send message m to neighbor[j] (Easy!) Now include crash failure as a part of our model. What if the sender crashes at the middle? Implement atomic broadcast in presence of crash.

40. Mobile-agent based communication Communicates via messengers instead of (or in addition to) messages. What is the lowest Price of an iPod in Michigan? Carries both program and data

41. June 20, 2007ProSe: Programming Tool for Sensor Networks; 41 Computational Model in Sensor Networks More generally in wireless networks –Communication pattern is local broadcast However, if multiple nodes broadcast simultaneously then a collision may occur

42. Write-All with Collision Model Thus, communication in sensor networks is local broadcast with collision In one action, –A sensor can (1) write its own state and (2) the state of all its neighbors at once –However, simultaneous execution of 2 or more sensors may cause the state of the common neighbors to remain unchanged Can we convert a program in shared memory to write-all with collision model? –Why?

43. Other classifications of models Reactive vs Transformational systems A reactive system never sleeps (like: a server) A transformational (or non-reactive systems) reaches a fixed point after which no further change occurs in the system Named vs Anonymous systems In named systems, process id is a part of the algorithm. In anonymous systems, it is not so. All are equal. (-) Symmetry breaking is often a challenge. (+) Easy to switch one process by another with no side effect.