1. Model and complexity Many measures Space complexity Time complexity
Message complexity
Bit complexity
Round complexity
Consider broadcasting
in an n-cube (n=3)
source

2. Broadcasting using messages
{Process 0} sends m to neighbors
{Process i > 0}
repeat
receive m {m contains the value};
if m is received for the first time
then x[i] := m.value; send x[i] to each neighbor j > I
else discard m
end if
forever
What is the (1) message complexity
(2) space complexity per process?
Each process j
has a variable x[j]
initially undefined

3. Broadcasting using shared memory
{Process 0} x[0] := v
{Process i > 0}
repeat
if  a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j]
{this is a step}
else skip
end if
forever
What is the time complexity?
(i.e. no. of steps are needed?)
Can be arbitrarily large!
Each process j
has a variable x[j]
initially undefined

4. Broadcasting using shared memory
Now, use “large atomicity”, where
in one step, a process j reads
the states of ALL its neighbors of
smaller id, and updates x[j] only
when these are equal, and
different from x[j].
What is the time complexity?
How many steps are needed?
The time complexity is now O(n2)
Each process j
has a variable x[j]
initially undefined

5. Time complexity in rounds
Rounds are truly defined for
synchronous systems. An
asynchronous round consists
of a number of steps where
every process (including the
slowest one) takes at least
one step.
How many rounds will you need
to complete the broadcast using
the large atomicity model?
Each process j
has a variable x[j]
initially undefined

6. Representing distributed algorithms
Why do we need these?
Don’t we already know a
lot about programming?
Well, you need to capture the notions of atomicity, non-determinism, fairness etc. These concepts are not built into languages like JAVA, C++ etc!

7. Syntax & semantics: guarded actions
<guard G>  <action A>
is equivalent to
if G then A

8. Syntax & semantics: guarded actions
Sequential actions S0; S1; S2; ; Sn
Alternative constructs if fi
Repetitive constructs do od
The specification is useful for representing abstract algorithms, not executable codes.

9. Syntax & semantics Alternative construct if G1  S1  G2  S2 …
 Gn Sn fi
When no guard is true, skip (do nothing). When multiple guards are true, the choice of the action to be executed is completely arbitrary.

10. Syntax & semantics do G1 S1  G2  S2 .  Gn  Sn od
Repetitive construct
do G1 S1
 G2  S2 .
 Gn  Sn od
Keep executing the actions until all guards are false and the program terminates. When multiple guards are true, the choice of the action is arbitrary.

11. Example: graph coloring
j  neighbor(i): c(j) = c(i) 
c(i) := 1-c(i) 1
Will the computation terminate? 1

12. An example program uncertain; define x : integer; initially x = 0
do x < 4  x := x + 1
 x = 3  x := 0 od
Question. Will the program terminate?
(Here fairness becomes an important issue)

13. The adversary
A distributed computation can be viewed as a game between the system and an adversary. The adversary comes up with feasible schedules to challenge the system, and a correct algorithm must have an adequate response to meet the challenge.

14. Non-determinism Token server
Non-determinism is abundant in the real world.
Determinism is a special case of non-determinism.
(Program for a token server - it has a single token}
repeat
if (req1 and token) then give the token to client1
else if (req2 and token) then give the token to client2
else if (req3 and token) then give the token to client3
forever
Now, assume that all three requests are sent
simultaneously. Client 1 always gets the token!
Token server 3 1 2

15. Atomicity (or granularity)
Atomic = all or nothing
Atomic actions = indivisible actions
do red  x:= 0 {red action}
 blue  x:=7 {blue action} od
Regardless of how nondeterminism
is handled, we would expect that
the value of x will be an arbitrary
sequence of 0's and 7's.
Right or wrong? x
The answer depends on atomicity

16. Atomicity (or granularity)
Does hardware guarantee any form of atomicity? Yes!
Transactions are atomic by definition (in spite of process failures). Also, critical section codes are atomic.
We will assume that G  A is an atomic operation
if x ≠ y  y:= x fi x y
if x ≠ y  y:= x fi

17. Atomicity {Program for P} define b: boolean initially b = true
do b  send msg m to Q
¬ empty(R,P) receive msg;
b := false od
Suppose it takes 15 seconds to
send the message. After 5 seconds,
P receives a message from R. Will it
stop sending the remainder of the
message? NO. R P Q b

18. Fairness
Defines the choices or restrictions on the scheduling of actions. No such restriction implies an
unfair scheduler. For fair schedulers, the following types of fairness have received attention:
Unconditional fairness
Weak fairness
Strong fairness

19. Fairness Program test define x : integer {initial value unknown}
do true  x : = 0
 x = 0  x : = 1
 x = 1  x : = 2 od
An unfair scheduler may never schedule the second (or the third actions). So, x may always be equal to zero.
An unconditionally fair scheduler will eventually give every statement a chance to execute without checking their eligibility. (Example: process scheduler in a multiprogrammed OS.)

20. Weak fairness
A scheduler is weakly fair, when it eventually executes every guarded action whose guard becomes true, and remains true thereafter
A weakly fair scheduler will eventually execute the second action, but may never execute the third action. Why?
Program test
define x : integer
{initial value unknown}
do true  x : = 0
 x = 0  x : = 1
 x = 1  x : = 2 od

21. Strong fairness
A scheduler is strongly fair, when it eventually executes every guarded action whose guard is true infinitely often.
The third statement will be executed under a strongly fair scheduler. Why?
Program test
define x : integer
{initial value unknown}
do true  x : = 0
 x = 0  x : = 1
 x = 1  x : = 2 od