Model and complexity Many measures Space complexity Time complexity Message complexity Bit complexity Round complexity Consider broadcasting in an n-cube (n=3) source

Broadcasting using messages {Process 0} sends m to neighbors {Process i > 0} repeat receive m {m contains the value}; if m is received for the first time then x[i] := m.value; send x[i] to each neighbor j > I else discard m end if forever What is the (1) message complexity (2) space complexity per process? Each process j has a variable x[j] initially undefined

Broadcasting using shared memory {Process 0} x[0] := v {Process i > 0} repeat if  a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j] {this is a step} else skip end if forever What is the time complexity? (i.e. no. of steps are needed?) Can be arbitrarily large! Each process j has a variable x[j] initially undefined

Broadcasting using shared memory Now, use “large atomicity”, where in one step, a process j reads the states of ALL its neighbors of smaller id, and updates x[j] only when these are equal, and different from x[j]. What is the time complexity? How many steps are needed? The time complexity is now O(n2) Each process j has a variable x[j] initially undefined

Time complexity in rounds Rounds are truly defined for synchronous systems. An asynchronous round consists of a number of steps where every process (including the slowest one) takes at least one step. How many rounds will you need to complete the broadcast using the large atomicity model? Each process j has a variable x[j] initially undefined

Representing distributed algorithms Why do we need these? Don’t we already know a lot about programming? Well, you need to capture the notions of atomicity, non-determinism, fairness etc. These concepts are not built into languages like JAVA, C++ etc!

Syntax & semantics: guarded actions <guard G>  <action A> is equivalent to if G then A

Syntax & semantics: guarded actions Sequential actions S0; S1; S2; . . . ; Sn Alternative constructs if . . . . . . . . . . fi Repetitive constructs do . . . . . . . . . od The specification is useful for representing abstract algorithms, not executable codes.

Syntax & semantics Alternative construct if G1  S1  G2  S2 …  Gn Sn fi When no guard is true, skip (do nothing). When multiple guards are true, the choice of the action to be executed is completely arbitrary.

Syntax & semantics do G1 S1  G2  S2 .  Gn  Sn od Repetitive construct do G1 S1  G2  S2 .  Gn  Sn od Keep executing the actions until all guards are false and the program terminates. When multiple guards are true, the choice of the action is arbitrary.

Example: graph coloring j  neighbor(i): c(j) = c(i)  c(i) := 1-c(i) 1 Will the computation terminate? 1

An example program uncertain; define x : integer; initially x = 0 do x < 4  x := x + 1  x = 3  x := 0 od Question. Will the program terminate? (Here fairness becomes an important issue)

The adversary A distributed computation can be viewed as a game between the system and an adversary. The adversary comes up with feasible schedules to challenge the system, and a correct algorithm must have an adequate response to meet the challenge.

Non-determinism Token server Non-determinism is abundant in the real world. Determinism is a special case of non-determinism. (Program for a token server - it has a single token} repeat if (req1 and token) then give the token to client1 else if (req2 and token) then give the token to client2 else if (req3 and token) then give the token to client3 forever Now, assume that all three requests are sent simultaneously. Client 1 always gets the token! Token server 3 1 2

Atomicity (or granularity) Atomic = all or nothing Atomic actions = indivisible actions do red  x:= 0 {red action}  blue  x:=7 {blue action} od Regardless of how nondeterminism is handled, we would expect that the value of x will be an arbitrary sequence of 0's and 7's. Right or wrong? x The answer depends on atomicity

Atomicity (or granularity) Does hardware guarantee any form of atomicity? Yes! Transactions are atomic by definition (in spite of process failures). Also, critical section codes are atomic. We will assume that G  A is an atomic operation if x ≠ y  y:= x fi x y if x ≠ y  y:= x fi

Atomicity {Program for P} define b: boolean initially b = true do b  send msg m to Q ¬ empty(R,P) receive msg; b := false od Suppose it takes 15 seconds to send the message. After 5 seconds, P receives a message from R. Will it stop sending the remainder of the message? NO. R P Q b

Fairness Defines the choices or restrictions on the scheduling of actions. No such restriction implies an unfair scheduler. For fair schedulers, the following types of fairness have received attention: Unconditional fairness Weak fairness Strong fairness

Fairness Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od An unfair scheduler may never schedule the second (or the third actions). So, x may always be equal to zero. An unconditionally fair scheduler will eventually give every statement a chance to execute without checking their eligibility. (Example: process scheduler in a multiprogrammed OS.)

Weak fairness A scheduler is weakly fair, when it eventually executes every guarded action whose guard becomes true, and remains true thereafter A weakly fair scheduler will eventually execute the second action, but may never execute the third action. Why? Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od

Strong fairness A scheduler is strongly fair, when it eventually executes every guarded action whose guard is true infinitely often. The third statement will be executed under a strongly fair scheduler. Why? Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od