1. Formal Models of Distributed Systems
Ali Ghodsi – UC Berkeley / KTH
alig(at)cs.berkeley.edu

2. Formal Modeling

3. Ali Ghodsi, alig(at)cs.berkeley.edu
Granularity of Models
Biggest challenge of modeling
Choosing the right level of abstraction!
Model must be powerful enough to construct
Impossibility proofs
A statement about all possible algorithms in a system
Our model should therefore be:
Precise: explain all relevant properties
Concise: explain a class of distributed systems compactly
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

4. Model of Distributed Systems
Based on model from Attiya & Welch

5. Model of Distributed Computing
What is a distributed system?
bunch of nodes/processes
sending messages over a network
to solve a common goal (algorithm)
How do we model this? p1 p2 p3 p4
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

6. Ali Ghodsi, alig(at)cs.berkeley.edu
Modeling a Node
A single node has a bunch of neighbors
Can send and receive messages
Can do local computations
Model node by state transition system (STS)
Like a finite state machine, except
Need not be finite
No input
Not necessarily finite state machine, but in reality nothing is infinite 
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

7. State Transition System (informal)
A state transition system consists of
A bunch of states
Rule for which state to go to from each state (transition function)
The set of starting states (initial states)
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

8. State transition system - example
Example algorithm: Using graphs:
X:=0;
while (X<2) do
X = X + 1;
endwhile
X:=1
Formally:
States {X0, X1, X2, X1’}
Transitions function {X0→X1, X1→X2, X2→X1’}
Initial states {X0} X1
X1’
start X0 X2
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

9. Ali Ghodsi, alig(at)cs.berkeley.edu
Modeling a Node
State machine of node i
Bunch of states Qi
Each state consists of
1 inbuffer set for each neighbor
1 outbuffer set for each neighbor
Other data relevant to algorithm
Initial states
inbuf[j] is empty for all j p2
outbuf[1]
inbuf[1]
outbuf[3]
inbuf[3]
outbuf[2]
inbuf[2] p1 p2 p3 p4
Outbuf represents the network, could have been called link or network
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

10. Ali Ghodsi, alig(at)cs.berkeley.edu
State of one node p2
outbuf[1]
inbuf[1]
outbuf[2]
inbuf[2]
outbuf[3]
inbuf[3]
Example states
x=0
outbuf[1]={}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
x=0
outbuf[1]={}
inbuf[1]={m0}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
x=1
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
x=1
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={m2}
outbuf[3]={}
inbuf[3]={}
x=1
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={m2,m3}
outbuf[3]={}
inbuf[3]={}
x=2
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={m4}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

11. Ali Ghodsi, alig(at)cs.berkeley.edu
Transition functions
All of the state, except outbufs, is called the accessible state of a node
Transition function f
takes accessible state and gives state, and
adds at most 1 new msg in each outbuf[i] of new state.
all inbuf[i] of new state must be empty
x=0
outbuf[1]={}
inbuf[1]={m0}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
x=1
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={} f
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

12. Transition functions formally
Model in Attiya & Welch is a little bit broken
This is a tedious fix…
State of a node (with k channels) is triple <I,O,s>
I is a vector of inbufs, <I[1],…,I[k]>
O is a vector of outbufs, <O[1],…,O[k]>
s is the local state
We require that for any two
f(<I1,O1,s1>)=<I2,O2,s2> and f(<I3,O3,s3>)=<I4,O4,s4>
I2=I4=<,…,>, i.e. all inbufs are empty, and
If I1=I3 and s1=s3 then
s2=s4, i.e. don’t “observe” channel (1),
O1[i]O2[i] and O3[i]O4[i], i.e. only append messages, and
O2[i]-O1[i] = O4[i]-O3[i], i.e. don’t “observe” channel (2)
Skip this slide if you like!
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

13. Single node perspective
This is how computers in a distributed system work:
1. Wait for message
2. When received message, do some local computation, send some messages
Goto 1.
Is this a correct model? [D]
Determinism?
I/O?
Atomicity?
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

14. Single Node to a Distributed System
A configuration is snapshot of state of all nodes
C = (q0,q1,…,qn-1) where qi is state of pi
An initial configuration is a configuration
where each qi is an initial state
Configuration p1 p2 p3
x=1
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={}
x=4
outbuf[1]={}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={m7}
inbuf[3]={}
x=11
outbuf[1]={m1}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
outbuf[3]={}
inbuf[3]={m3}
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

15. Single Node to a Distributed System
The system evolves through events
Computation event at node i, comp(i)
Delivery event of msg m from i to j, del(i,j,m)
Computation event comp(i)
Apply transition function f on node i’s state
Delivery event del(i,j,m)
Move message m from outbuf of pi to inbuf of pj
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

16. Ali Ghodsi, alig(at)cs.berkeley.edu
Execution
An execution is an infinite sequence of
config0, event1, config1, event2, config2…
config0 is an initial configuration
If eventk is comp(i)
configk-1 changes to configk by
applying pi’s transition function on i’s state in configk-1
If eventk is del(i,j,m)
moving m from i’s outbuf for link i↔j to j’s inbuf for link i-j
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

17. Single Node to a Distributed Systemp2
Example execution p1 1 1 p3 2 1
config0
config1
config2
x=1
outbuf[1]={m1}
inbuf[1]={}
x=1
outbuf[1]={m1}
inbuf[1]={}
x=1
outbuf[1]={m1}
inbuf[1]={} p1 p1 p1
event1
event2
x=4
outbuf[1]={m7}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
x=4
outbuf[1]={}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={}
x=4
outbuf[1]={}
inbuf[1]={}
outbuf[2]={}
inbuf[2]={} p2
del(2,3,m7) p2
comp(3) p2
x=11
outbuf[1]={}
inbuf[1]={}
x=11
outbuf[1]={}
inbuf[1]={m7}
x=12
outbuf[1]={m8}
inbuf[1]={} p3 p3 p3
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

18. Some definitions for later use…
Each comp(i) is associated with a transition
If f of process i maps state1 to state2:
the triple (state1,state2,i) is called a transition
Transition (s1,s2,j) is applicable in configuration c if
The accessible state of node j is s1 in c
A del(i,j,m) is applicable in configuration c if
m is in outbuf for link i↔j of node i in c
A transition should really also have the process id …
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

19. Single Node to a Distributed System
Example execution p1 p2 1
config0
config1
x=1
outbuf[1]={m1}
inbuf[1]={}
x=1
outbuf[1]={m1}
inbuf[1]={} p1 p1
event1
comp(2)
x=11
outbuf[1]={}
inbuf[1]={m7}
x=12
outbuf[1]={m8}
inbuf[1]={} p2 p2
associated with
transition
x=11
inbuf[1]={m7}
x=12
outbuf[1]={m8}
inbuf[1]={}
( , , 2)
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

20. Some definitions for later use… (2)
If transition e=(s1,s2,i) is applicable to conf c
Then app(e,c) gives new configuration after the event comp(i)
If e=del(i,j,m) is applicable to conf c
Then app(e,c) gives new configration after the event del(i,j,m)
Some notation abuse… overloading function app
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

21. Schedules (Asynchronous Model)
Our processes are deterministic
Given some message, update state, send some messages, and wait…
Non-determinism comes from asynchrony
Messages take arbitrary time to be delivered
Processes execute at different speeds
A schedule is the sequence of events
Message asynchrony determined by del(i,j,m)
Process speeds determined by comp(i)
All non-determinism embedded in schedule!
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

22. Ali Ghodsi, alig(at)cs.berkeley.edu
Schedules (2)
Given the initial configuration
The schedule determines the whole execution
Not all schedules allowed for an initial conf.
del(i,j,m) only allowed if m is in outbuf of i in previous configuration
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu

23. Admissible executions (aka fairness)
An execution is admissible if
each process has infinite number of comp(i), and
every message m sent is eventually del(i,j,m)
Why infinity?
Executions are infinite
When algorithm is finished, only make dummy transitions (same state)
11/20/2018
Ali Ghodsi, alig(at)cs.berkeley.edu