CONNECTED-COMPONENTS ALGORITHMS FOR MESH-CONNECTED PARALLEL COMPUTERS S. Kumar, S. M. Goddard and J. F. Prins Presenter: Ze Li CSCE 5013 Parallel Algorithms University of Arkansas

Outline Reviews Introductions Main Algorithms for connected components Mesh- based Algorithm for connected components Evaluations Conclusions

Reviews Connected component: A maximum connected sub-graph of G. Pram: Common MEM Pointer Jumping: Tree Hanging.

Conceptual Idea 21 22 4 11 45 16 24 8 12 14 6 19 10 7 2 28 1 15 13 35 Every node locally make decision to choose a smaller node and converge to

Conceptual Idea 21 22 4 11 45 16 24 8 12 14 6 19 10 7 2 28 1 15 13 35

Conceptual Idea 21 22 4 11 45 16 24 8 12 14 6 19 10 7 2 28 1 15 13 35 Local jump linking

Conceptual Idea 21 22 4 11 45 16 24 8 12 14 6 19 10 7 2 28 1 15 13 35 Local jump linking

Conceptual Idea 2 4 1 6 Local jump linking

Conceptual Idea 2 1 4 6 Every node locally make decision to choose a smaller node and converge to

Conceptual Idea 1 4

Conceptual Idea

Common Characters PRAM Similar ideas Point Jumping (shortcutting) Tree Hanging (tree hooking) Different Implementations algorithms: Wes: (1)shortcutting and two kind of hooking (work O(mlogn)) (2) shortcutting and one hooking ((work O(mlogn)) (3) random mateCluster ((work O(mlogn)) Baha: Circle tree. Time O((log2n)2) n2 or n2/logn processors Jeremy: Use flag to reduce the node involved in iteration O((log2n)2) n2/logn

Introduction and Motivation PRAM: Simplified structure, the assumption of the operation cost is unrealistic Arbitrary reading memory= Arbitrary routing in the networks. Possible collisions Advantages: Provide several algorithms that can be modified and applied to other topologies.

Mesh-based Connected Component Computing Start with a Pram based algorithm Modify it for the mesh to minimize the communication time For dense graph, used Modified algorithm based on the adjacency matrix representation. Time complexity O(log2n) Work load O(n2logn) For sparse graph, only distributed the edges present in the matrix across processors. Work load O(mlogn)

Main Algorithem A graph G(V,E). V is the vertices sets. E is edge the sets Start from Parent function P: V->V, such that P(v)=v for all v in V. P is modified until P(v) =P(w) iff v and w belong to the same component. In each step, a vertex is connected to the parent of its lowest numbered neighbor. A vertex is its own neighbor

Main Algorithm 4 3 7 1 6 5 2 9 8 First iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ P(i):=r=min{ P{j}|OldP(j)=i } /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) P(i):=P(min{P(j)|node j is adjacent to node i in G}) 6 7 5 2 4 3 9 oldP(i) Physical link P(i) Check if its physically closed node pointing to a smaller node. If Yes, point to that

Main Algorithm 4 3 7 1 6 5 2 9 8 First iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ P(i):=r=min{ P{j}|OldP(j)=i } /*Tree hanging */ Until P stops changing. P(i):= i; 5 8 6 2 4 3 7 1 9 OldP(i):=P(i) P(i):=P(min{P(j)|node j is adjacent to node i in G}) oldP(i) Physical link P(i)

Main Algorithm 4 3 7 1 6 5 2 9 8 First iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) P(i):=P(min{P(j)|node j is adjacent to node i in G}) 6 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i) The root node check if there are any leaves currently pointing to a smaller node

Main Algorithm 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

Main Algorithm 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

Main Algorithm 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i) Check if its physically closed node pointing to a smaller node. If Yes, point to that

Algorithm Examples 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

Algorithm Examples 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i) The root node check if there are any leaves currently pointing to a smaller node

Algorithm Examples 4 3 7 1 6 5 2 9 8 second iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

Algorithm Examples 4 3 7 1 6 5 2 9 8 Third iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

Algorithm Examples 4 3 7 1 6 5 2 9 8 Third iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i) Check if its physically closed node pointing to a smaller node. If Yes, point to that

Algorithm Examples 4 3 7 1 6 5 2 9 8 Third iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ Until P stops changing. P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 3 9 oldP(i) Physical link P(i)

OVER!!!! Algorithm Examples 4 3 7 1 6 5 2 9 8 Third iteration FOREACH node i IN G REPEAT /* One Pointer jumping step*/ /*Tree hanging */ P(i):= i; 1 8 OldP(i):=P(i) 6 P(i):=P(min{P(j)|node j is adjacent to node i in G}) 7 5 P(i):=r=min{ P{j}|OldP(j)=i } 2 4 P stops changing 3 OVER!!!! 9 oldP(i) Physical link P(i) The root node check if there are any leaves currently pointing to a smaller node

Correctness Claim 1: If P(i)=P(j), i and j belong to the same connected component. Proof: (by induction) Starts: P(i)=i. i and P(i) in same component. Assume: The claim is true after k iterations Derivation: At the k+1 iteration. P(i) is changed to P(r). Since r is adjacent to i, r is in the same component as P(r), therefore P(i), P(r) in the same component

Correctness Claim 2: If i and j belong to the same connected component. P(i)=P(j). Proof: (by contradiction). Suppose, i and j in the same component and P(i)!=P(j). Since there is a path from i to j. There must exist two consecutive nodes m and n on the path such that the edge (m,n) in the G, and P(m)!=P(n). Since (m,n) are neighbor node, the algorithm can not terminate.

Compexity Define a chain as follows: i and P(i) lie on a chain. Succ(i) =P(i) and Pred (P(i))=i. If P(i) and j lie on the chain C, so does i. The edges on this chain are directed and are called P-edges.

Theorem The algorithm makes no more than O(logn) iterations Proof If no node in the chain change its chance. Logn If node I change the chain to the k chain. In the k chain , it still take logn times to reach the root. Therefore O(logn) j i k

Implementation on a Mesh They define the following matrix M for a graph G with n vertices. The size of M is n*n 5 8 6 2 4 3 7 1 9 M(u,v)= (u,v) is an edge in G? min(u,v)|∞; n/n 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 1. Find the minimum P values of all neighbors 5 8 6 2 4 3 7 1 9 Suppose each vertex is maintained by the row n/n 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 1. Find the minimum P values of all neighbors FOREACH j IN [1..n] LET r=min {M(i,j) | i IN [1..n] } 1 8 j/i 1 2 3 4 5 6 7 8 9 6 7 5 2 4 3 9

The Basic Operation of the Algorithm 2. Do a Pointer jumping step. M(M(j, j), j) := M(r, j) 1 8 j/i 1 2 3 4 5 6 7 8 9 6 7 5 2 4 3 9

The Basic Operation of the Algorithm 3. Do Tree hanging M(j, j) := M(r, j) 1 8 j/i 1 2 3 4 5 6 7 8 9 6 7 5 2 4 3 9

The Basic Operation of the Algorithm 4. Spread the new names for each vertex in its row FOREACH i in [i..n] LET r=min{M(i,j) | j IN [1..n]} IN FOREACH j IN [1..n] M(i,j) :=r. 1 8 j/i 1 2 3 4 5 6 7 8 9 j/i 1 2 3 4 5 6 7 8 9 6 7 5 2 4 3 9

The Basic Operation of the Algorithm 1. Find the minimum P values of all neighbors FOREACH j IN [1..n] LET r=min {M(i,j) | i IN [1..n] } 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 2. Do a Pointer jumping step. M(M(j, j), j) := M(r, j) 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 3. Do Tree hanging M(j, j) := M(r, j) 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 4. Spread the new names for each vertex in its row FOREACH i in [i..n] LET r=min{M(i,j) | j IN [1..n]} IN FOREACH j IN [1..n] M(i,j) :=r. 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 1. Find the minimum P values of all neighbors FOREACH j IN [1..n] LET r=min {M(i,j) | i IN [1..n] } 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 2. Do a Pointer jumping step. M(M(j, j), j) := M(r, j) 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 3. Do Tree hanging M(j, j) := M(r, j) 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9

The Basic Operation of the Algorithm 4. Spread the new names for each vertex in its row FOREACH i in [i..n] LET r=min{M(i,j) | j IN [1..n]} IN FOREACH j IN [1..n] M(i,j) :=r. 5 8 6 2 4 3 7 1 9 j/i 1 2 3 4 5 6 7 8 9 stop

Two Representations Matrix Representation: M(i,j) is mapped to PE(i mod p, j mod p) Working complexity: n2logn. Each PE does O(n/p) work per iteration. There are O(n/p) elements per PE. The min operations take O(n/p+logp) steps Adjacency List Representation. Distributed edges to the PE Work complexity O(n2logn). Consider load balance O(n)

Adjacency List Representation

Improvements Load balancing: Distributed load evenly to achieve a complexity as O(mlogn). A fast graph pre-conditioner that will determine load balance and sparsity of the graph before it is distributed. Creating a new edge in underloaded PE. for each edge (u, v) in G, there exists either (u1, v) in G1 or (u2, v) in G2. In addition the edge (u1, u2) is created. Type of Graph: Identify the best possible algorithm for a given classes of graphs and given a graph.

Questions ?