1. CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS Spring 2014 Prof. Jennifer Welch CSCE 668 Set 1: Introduction 1

2. Distributed Systems CSCE 668Set 1: Introduction 2  Distributed systems have become ubiquitous:  share resources  communicate  increase performance speed fault tolerance  Characterized by  independent activities (concurrency)  loosely coupled parallelism (heterogeneity)  inherent uncertainty

3. Uncertainty in Distributed Systems CSCE 668Set 1: Introduction 3  Uncertainty comes from  differing processor speeds  varying communication delays  (partial) failures  multiple input streams and interactive behavior

4. Reasoning about Distributed Systems CSCE 668Set 1: Introduction 4  Uncertainty makes it hard to be confident that system is correct  To address this difficulty:  identify and abstract fundamental problems  state problems precisely  design algorithms to solve problems  prove correctness of algorithms  analyze complexity of algorithms (e.g., time, space, messages)  prove impossibility results and lower bounds

5. Potential Payoff of Theoretical Paradigm CSCE 668Set 1: Introduction 5  careful specifications clarify intent  increased confidence in correctness  if abstracted well then results are relevant in multiple situations  indicate inherent limitations  cf. NP-completeness

6. Application Areas CSCE 668Set 1: Introduction 6  These areas have provided classic problems in distributed/concurrent computing:  operating systems  (distributed) database systems  software fault-tolerance  communication networks  multiprocessor architectures  Newer application areas:  cloud computing  mobile computing, …

7. Course Overview: Part I (Fundamentals) CSCE 668Set 1: Introduction 7  Introduce two basic communication models:  message passing  shared memory  and two basic timing models:  synchronous  asynchronous

8. Course Overview: Basic Models CSCE 668Set 1: Introduction 8 Message passing Shared memory synchronous asynchronous Yes No Yes (Synchronous shared memory model is PRAM)

9. Course Overview: Part I CSCE 668Set 1: Introduction 9  Covers the canonical problems and issues:  graph algorithms (Ch 2)  leader election (Ch 3)  mutual exclusion (Ch 4)  fault-tolerant consensus (Ch 5)  causality and time (Ch 6)

10. Course Overview: Part II (Simulations) CSCE 668Set 1: Introduction 10  Here "simulations" means abstractions, or techniques for making it easier to program, by making one model appear to be an easier model. For example:  broadcast and multicast (Ch 8)  distributed shared memory (Ch 9)  stronger kinds of shared variables (Ch 10)  more synchrony (Chs 11, 13)  more benign faults (Ch 12)

11. Course Overview: Part II CSCE 668Set 1: Introduction 11  For each of the techniques:  describe algorithms for implementing it  analyze the cost of these algorithms  explore limitations  mention applications that use the techniques

12. Course Overview: Part III (Advanced Topics) CSCE 668Set 1: Introduction 12  Push further in some directions already introduced:  randomized algorithms (Ch 14)  stronger kinds of shared objects of arbitrary type (Ch 15)  what kinds of problems are solvable in asynchronous systems (Ch 16)  failure detectors (Ch 17)  self-stabilization

13. Relationship of Theory to Practice CSCE 668Set 1: Introduction 13  time-shared operating systems: issues relating to (virtual) concurrency of processes such as  mutual exclusion  deadlock also arise in distributed systems  MIMD multiprocessors:  no common clock => asynchronous model  common clock => synchronous model  loosely coupled networks, such as Internet, => asynchronous model

14. Relationship of Theory to Practice CSCE 668Set 1: Introduction 14  Failure models:  crash: faulty processor just stops. Idealization of reality.  Byzantine (arbitrary): conservative assumption, fits when failure model is unknown or malicious  self-stabilization: algorithm automatically recovers from transient corruption of state; appropriate for long-running applications

15. Message-Passing Model CSCE 668Set 1: Introduction 15  processors are p 0, p 1, …, p n-1 (nodes of graph)  bidirectional point-to-point channels (undirected edges of graph)  each processor labels its incident channels 1, 2, 3,…; might not know who is at other end

16. Message-Passing Model CSCE 668Set 1: Introduction 16 1 1 2 2 11 3 2 p3p3 p2p2 p0p0 p1p1

17. Modeling Processors and Channels CSCE 668Set 1: Introduction 17  Processor is a state machine including  local state of the processor  mechanisms for modeling channels  Channel directed from processor p i to processor p j is modeled in two pieces:  outbuf variable of p i and  inbuf variable of p j  Outbuf corresponds to physical channel, inbuf to incoming message queue

18. Modeling Processors and Channels CSCE 668Set 1: Introduction 18 inbuf[1] p 1 's local variables outbuf[1] inbuf[2] outbuf[2] p 2 's local variables Pink area (local vars + inbuf) is accessible state for a processor.

19. Configuration CSCE 668Set 1: Introduction 19  Vector of processor states (including outbufs, i.e., channels), one per processor, is a configuration of the system  Captures current snapshot of entire system: accessible processor states (local vars + incoming msg queues) as well as communication channels.

20. Deliver Event CSCE 668Set 1: Introduction 20  Moves a message from sender's outbuf to receiver's inbuf; message will be available next time receiver takes a step p1p1 p2p2 m 3 m 2 m 1 p1p1 p2p2

21. Computation Event CSCE 668Set 1: Introduction 21  Occurs at one processor  Start with old accessible state (local vars + incoming messages)  Apply transition function of processor's state machine; handles all incoming messages  End with new accessible state with empty inbufs, and new outgoing messages

22. Computation Event CSCE 668Set 1: Introduction 22 cd e old local state a new local state b pink indicates accessible state: local vars and incoming msgs white indicates outgoing msg buffers

23. Execution CSCE 668Set 1: Introduction 23  Format is config, event, config, event, config, …  in first config: each processor is in initial state and all inbufs are empty  for each consecutive (config, event, config), new config is same as old config except:  if delivery event: specified msg is transferred from sender's outbuf to receiver's inbuf  if computation event: specified processor's state (including outbufs) change according to transition function

24. Admissibility CSCE 668Set 1: Introduction 24  Definition of execution gives some basic "syntactic" conditions.  usually safety conditions (true in every finite prefix)  Sometimes we want to impose additional constraints  usually liveness conditions (eventually something happens)  Executions satisfying the additional constraints are admissible. These are the executions that must solve the problem of interest.  Definition of “admissible” can change from context to context, depending on details of what we are modeling

25. Asynchronous Executions CSCE 668Set 1: Introduction 25  An execution is admissible for the asynchronous model if  every message in an outbuf is eventually delivered  every processor takes an infinite number of steps  No constraints on when these events take place: arbitrary message delays and relative processor speeds are not ruled out  Models reliable system (no message is lost and no processor stops working)

26. Example: Flooding CSCE 668Set 1: Introduction 26  Describe a simple flooding algorithm as a collection of interacting state machines.  Each processor's local state consists of variable color, either red or green  Initially:  p 0 : color = green, all outbufs contain M  others: color = red, all outbufs empty  Transition: If M is in an inbuf and color = red, then change color to green and send M on all outbufs

27. Example: Flooding CSCE 668Set 1: Introduction 27 p1p1 p0p0 p2p2 MM p1p1 p0p0 p2p2 M M deliver event at p 1 from p 0 computation event by p 1 deliver event at p 2 from p 1 p1p1 p0p0 p2p2 M M MM p1p1 p0p0 p2p2 M M computation event by p 2

28. Example: Flooding (cont'd) CSCE 668Set 1: Introduction 28 deliver event at p 1 from p 2 computation event by p 1 deliver event at p 0 from p 1 etc. to deliver rest of msgs p1p1 p0p0 p2p2 M M M M p1p1 p0p0 p2p2 M M M M p1p1 p0p0 p2p2 M M M p1p1 p0p0 p2p2 M M M

29. Nondeterminism CSCE 668Set 1: Introduction 29  The previous execution is not the only admissible execution of the Flooding algorithm on that triangle.  There are several, depending on the order in which messages are delivered.  For instance, the message from p 0 could arrive at p 2 before the message from p 1 does.

30. Termination CSCE 668Set 1: Introduction 30  For technical reasons, admissible executions are defined as infinite.  But often algorithms terminate.  To model algorithm termination, identify terminated states of processors: states which, once entered, are never left  Execution has terminated when all processors are terminated and no messages are in transit (in inbufs or outbufs)

31. Termination of Flooding Algorithm  Define terminated processor states as those in which color = green. CSCE 668Set 1: Introduction 31

32. Message Complexity Measure CSCE 668Set 1: Introduction 32  Message complexity: maximum number of messages sent in any admissible execution  This is a worst-case measure.  Later we will mention average-case measures.

33. Message Complexity of Flooding Algorithm CSCE 668Set 1: Introduction 33  Message complexity: one message is sent over each edge in each direction. So number is 2m, where m = number of edges.

34. Time Complexity Measure CSCE 668Set 1: Introduction 34  How can we measure time in asynchronous executions?  Produce a timed execution from an execution by assigning non-decreasing real times to events such that time between sending and receiving any message is at most 1.  Essentially normalizes the greatest message delay in an execution to be one time unit; still allows arbitrary interleavings of events.  Time complexity: maximum time until termination in any timed admissible execution.

35. Time Complexity of Flooding Algorithm CSCE 668Set 1: Introduction 35  Recall that terminated processor states are those in which color = green.  Time complexity: diameter + 1 time units. (A node turns green once a "chain" of messages has reached it from p 0.)  Diameter of a graph is the maximum, over all nodes v and w in the graph of the shortest path from v to w

36. Synchronous Message Passing Systems CSCE 668Set 1: Introduction 36  An execution is admissible for the synchronous model if it is an infinite sequence of "rounds"  What is a "round"?  It is a sequence of deliver events that move all messages in transit into inbuf's, followed by a sequence of computation events, one for each processor.

37. Synchronous Message Passing Systems CSCE 668Set 1: Introduction 37  The new definition of admissible captures lockstep unison feature of synchronous model.  This definition also implies  every message sent is delivered  every processor takes an infinite number of steps.  Time is measured as number of rounds until termination.

38. Example of Synchronous Model CSCE 668Set 1: Introduction 38  Suppose flooding algorithm is executed in synchronous model on the triangle.  Round 1:  deliver M to p 1 from p 0  deliver M to p 2 from p 0  p 0 does nothing (as it has no incoming messages)  p 1 receives M, turns green and sends M to p 0 and p 1  p 2 receives M, turns green and sends M to p 0 and p 1

39. Example of Synchronous Model CSCE 668Set 1: Introduction 39  Round 2:  deliver M to p 0 from p 1  deliver M to p 0 from p 2  deliver M to p 1 from p 2  deliver M to p 2 from p 1  p 0 does nothing since its color variable is already green  p 1 does nothing since its color variable is already green  p 2 does nothing since its color variable is already green

40. Example of Synchronous Model CSCE 668Set 1: Introduction 40 p1p1 p0p0 p2p2 M MM M p1p1 p0p0 p2p2 MM p1p1 p0p0 p2p2 round 1 events round 2 events

41. Complexity of Synchronous Flooding Algorithm CSCE 668Set 1: Introduction 41  Just consider executions that are admissible w.r.t. synchronous model (i.e., that satisfy the definition of synchronous model)  Time complexity is diameter + 1  Message complexity is 2m  Same as for asynchronous case.  Not true for all algorithms though…