1. Totally Asynchronous Iterative Algorithms
EE692
Parallel and Distribution Computation | Prof. Song Chong
Ch. 6
Totally Asynchronous Iterative Algorithms

2. Totally Asynchronous Algorithmic Model

3. Totally Asynchronous Algorithmic Model

4. Two Examples of Asynchronous Distributed System
Example of Message Passing System

5. Two Examples of Asynchronous Distributed System
Shared Memory System
Assumption 1.1 (total Asynchronism) x1 x2 x3
t=1 2 3 4 5 6 7 8 9 10 R W
compute

6. General Convergence Theorem
Assumption 2.1

7. General Convergence Theorem
• A typical example where the Box Condition holds is when x(k) is a sphere in Rn w.r.t some weighted maximum norm.

8. Prop. 2.1 (Asynchronous Convergence theorem)
If Assumption 2.1 holds, and the initial solution estimate
belongs to the set X(0),
then every limit point of {x(t)} is a fixed point of f.
Pf.) Show by induction that for each ,
there exists a time such that:
(a)
(b) For all i and with ,
we have
where
( In other words, after some time, all estimates will be in X(k) and all estimates used in the iteration
will come from X(k). )

9. For k=0, since the iteration estimates are assumed to be in X(0),
the induction hypothesis holds.
Suppose that the hypothesis is true for a given k.
Now, show the existence of such a time
For each i=1, …, n, let be the first element of such that
Then, by the Synchronous Convergence Condition,
we have , and by the Box Condition, we have
Similarly, for every ,
Since does not change between elements of ,
Let Then,

10. Finally, since by the Assumption 1.1, ,
we can choose a time that is sufficiently large so that
Then,
This implies that
The induction is complete Q.E.D.
• A number of useful extensions of the Asynchronous Convergence Theorem are found in Exercise
• The challenge in applying the theorem is to identify a suitable sequence of sets {X(k)}.

12. Applications to Problem’s involving Maximum Norm : Contraction Mappings
Suppose that is a contraction mapping w.r.t. a weighted maximum norm , and suppose that for
i=1, …, n and Define the sets
where x* is the unique fixed point of f, and a<1 is the contraction modulus
 {x(k)} satisfies the synchronous convergence condition and the box condition of A.2.1.
pf.) H.W
A similar conclusion holds if f is a pseudo-contraction mapping w.r.t. a weighted maximum norm.

13. Linear Systems of Equations

14. Unconstrained Optimization