1. ITEC452 Distributed Computing Lecture 3 Models in Distributed Systems
Hwajung Lee
ITEC452 Distributed Computing Lecture 3 Models in Distributed Systems

2. Why do we need models?
Models are simple abstractions that help understand the variability -- abstractions that preserve the essential features, but hide the implementation details from observers who view the system at a higher level.
Results derived from an abstract model is applicable to a wide range of platforms.

3. Two Primary Models
Two primary models that capture the essence of interprocess communication:
Message-passing Model
Shared-memory Model

4. Message Passing Model (1)
Process actions in a distributed system can be represented in the following notation:
System topology is a graph G = (V, E), where
V = set of nodes (sequential processes)
E = set of edges (links or channels, bi/unidirectional)

5. Message Passing Model (2)
Four types of actions by a process:
Internal action
a process performs computations in its own address space resulting in the modification of one or more of its local variables.
Communication action
a process sends a message to another process or receives a message from another process
Input action
A process reads data from sources external to the system
Output action
A process sends data outside of the system (called “environment”)

6. Message Passing Model (3)
Possible Dimension of Variability in Distributed Systems
Type of Channels
Point-t0-point or not
Reliable vs. Unreliable channels
Type of Systems
Synchronous vs. Asynchronous systems

7. Message Passing Model (4) Type of Channels
Channels in a Message Passing Model
Message propagate along directed edges called channels
Communications are assumed to be point-to-point (a broadcasting is considered as a set of point-to-point communications)
Reliable vs. unreliable channels
Reliable channel : a loss of corruption of message is not considered
Unreliable channel: It will be considered in a later chapter

8. Message Passing Model (5) Assumption: A Reliable FIFO channel
Axiom 1. Message m sent  message m received
Axiom 2. Message propagation delay is arbitrary but finite.
Axiom 3. m1 sent before m2  m1 received before m2. P Q

9. Message Passing Model (6) Type of Systems: Synchrony vs. Asynchrony
Broad notion of Synchrony
Senders and receivers maintain synchronized clocks and operating with a rigid temporal relationship.
However, in practice, there are many aspects of synchrony, and transition from a fully asynchronous to a fully synchronous model is a gradual one.
Thus, we will see a few examples of the behaviors characterizing a synchronous systems.

10. Message Passing Model (7) Synchrony vs. Asynchrony System
Synchronous
clocks
Physical clocks are synchronized
Synchronous processes
Lock-step synchrony
Synchronous channels
Bounded delay
Synchronous message-order
First-in first-out channels
Synchronous communication
Communication via handshaking
Send & receive can be blocking or non-blocking
Postal communication is asynchronous:
Telephone communication is synchronous
Any restriction defines some form of synchrony …

11. Shared memory model (1) Address spaces of processes overlap M1 M2 1 3
Shared variable: some computing clusters support DSM (distributed Shared Memory)
Virtual address space
Do not share physical memory.
to relieve the users of the intricate details of message passing, and let them use the richer programming tools of parallel programming available for shared-memory systems. 2 4
Concurrent operations on a shared variable are serialized

12. Shared memory model (1) Variations of shared memory models1 2
State reading model
Each process can read
the states of its neighbors 3
Link register model
Each process can read from
and write to adjacent registers. The entire local state is not shared. 1 2
LINDA by David Gelernter at Yale University: a simple programming language using the shared variable model. 3

13. Modeling wireless networks
Communication via broadcast
Limited range
Dynamic topology

14. Modeling Mobile Agents (1)
a program code that migrates from one process to another
Can interact with the variables of programs running on the host machine, use its resources, and take autonomous routing decisions.
Strong mobility/weak mobility
Ex.: Dartmouth’s D’Agents, IBM’s Aglets

15. Modeling Mobile Agents (2)
Model: (I, P, and B)
I is the agent identifier and is unique for every agent.
P designates the agent program
B is the briefcase and represent the data variables to be used by the agent

16. Weak vs. Strong Models
Examples HLL model is stronger than assembly language model. Asynchronous is weaker than synchronous. Bounded delay is stronger than unbounded delay (channel)
One object (or operation) of a strong model = More than one objects (or operations) of a weaker model.
Often, weaker models are synonymous with fewer restrictions.
One can add layers (additional restrictions) to create a stronger model from weaker one.

17. Model transformation
“Can model X be implemented using model Y?” is an interesting question in computer science. Sample problems Non-FIFO to FIFO channel Message passing to shared memory Non-atomic broadcast to atomic broadcast
Stronger models - simplify reasoning, but - needs extra work to implement Weaker models - are easier to implement. - Have a closer relationship with the real world
Complicated
Simple
Remember that strong and weak are not absolute, but relative attributes.
The choice of a strong model:
simplifies algorithm design
simplifies correctness proofs
BUT
every application has to be implemented
It is simpler to implement a weaker model from a stronger one, but the implementation of a stronger model using a weaker one nay take considerable effort.

18. Non-FIFO to FIFO channel (1)m2 m3 m4 m1 P Q
buffer

19. Non-FIFO to FIFO channel (2)
{Sender process P} {Receiver process Q}
var i : integer {initially 0} var k : integer {initially 0}
buffer: buffer[0..∞] of msg
{initially  k: buffer [k] = empty
repeat repeat {STORE}
send m[i],i to Q; receive m[i],i from P;
i := i store m[i] into buffer[i];
forever {DELIVER} while buffer[k] ≠ empty do
begin
deliver content of buffer [k];
buffer [k] := empty; k := k+1;
end
forever

20. Observations (a) Needs Unbounded sequence numbers and
(b) Unbounded number of buffer slots
 Both are bad
Now solve the same problem on a model where
(a) The propagation delay has a known upper bound of T.
(b) The messages are sent per unit time.
(c) The messages are received at a rate faster than r.
 The buffer requirement drops to (r xT). Thus, Synchrony pays.
Question. How to solve the problem using bounded buffer space if the propagation delay is arbitrarily large?

21. Message-passing to Shared memory
{Read X by process i}: read x[i]
{Write X:= v by process i}
- x[i] := v;
Atomically broadcast v to
every other process j (j ≠ i);
After receiving broadcast,
process j (j ≠ i) sets x[j] to v.
Understand the significance of atomic operations. It is not trivial, but is very important in distributed systems
This is incomplete. There are
more pitfalls here.

22. Non-atomic to atomic broadcast
Atomic broadcast = either everybody or nobody receives
{process i is the sender}
for j = 1 to N-1 (j ≠ i) send message m to neighbor[j] (Easy!)
Now include crash failure as a part of our model.
What if the sender crashes at the middle?
Implement atomic broadcast in presence of crash.

23. Mobile-agent based communication
Communicates via messengers instead of (or in addition to) messages.
What is
the lowest
Price of an
iPod in Radford?
Carries both
program and data

24. Other classifications of models
Reactive vs Transformational systems
A reactive system never sleeps (like: a server or servers)
A transformational (or non-reactive systems) reaches a fixed point after which no further change occurs in the system (Examples?)
Named vs Anonymous systems
In named systems, process id is a part of the algorithm.
In anonymous systems, it is not so. All are equal.
(-) Symmetry breaking is often a challenge.
(+) Easy to switch one process by another with no side effect. Saves log N bits.

25. Model and complexity Many measures Space complexity Time complexity
Message complexity
Bit complexity
Round complexity
What do these mean?
Consider broadcasting
in an n-cube (n=3)
source

26. Broadcasting using messages
{Process 0} sends m to neighbors
{Process i > 0}
repeat
receive m {m contains the value};
if m is received for the first time
then x[i] := m.value; send x[i] to each neighbor j > I
else discard m
end if
forever
What is the
(1) message complexity
(2) space complexity per process?
Each process j
has a variable x[j]
initially undefined

27. Broadcasting using shared memory
{Process 0} x[0] := v
{Process i > 0}
repeat
if  a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j]
{this is a step}
else skip
end if
forever
What is the time complexity?
(i.e. how many steps are needed?)
Can be arbitrarily large! WHY?
Each process j
has a variable x[j]
initially undefined

28. Broadcasting using shared memory
Now, use “large atomicity”, where in one step, a process j reads the states of ALL its neighbors of smaller id, and updates x[j] only when these are equal, and different from x[j]. What is the time complexity? How many steps are needed? The time complexity is now O(n2)
Each process j
has a variable x[j]
initially undefined

29. Time complexity in rounds
Rounds are truly defined for synchronous systems. An asynchronous round consists of a number of steps where every process (including the slowest one) takes at least one step. How many rounds will you need to complete the broadcast using the large atomicity model?
Each process j
has a variable x[j]
initially undefined