Routing in a Parallel Computer

A network of processors is represented by graph G=(V,E), where |V| = N. Each processor has unique ID between 1 and N. Each edge e=(v1,v2) represents a communication link between processors v1 and v2.

Routing in a Parallel Computer All communication proceeds in a sequence of synchronous steps. Each link ca carry one packet in a step. During a step processor can send at most one message to each one of his neighbours.

Permutation Routing Initially each processor contains packet destined for some processor in the network. i:processor ID. v i :packet that processor i contains. d(i):destination processor of the packet v i. For 1 ≤ i ≤ N, d(i)’s form a permutation of {1,.2,…,N}.

Permutation Routing Route: A sequence of edges packet follow from its source to destination. Algorithm’s goal: Find a route for each packet. Note: A packet may occasionally have to wait at an intermediate node because the next edge on its route is "busy" transmitting another packet. We assume that each node contains one queue for each edge leaving the node.

Oblivious Algorithm The route followed by v i depends on d(i) alone, and not on d(j) for any j ≠ i. Oblivious routing algorithms are simple to implement. Theorem: For any deterministic oblivious permutation routing algorithm on a network of N nodes each of out-degree d, there is an instance of permutation routing requiring steps.

Boolean Hypercube Network N=2 n n: Hypercube dimensions. N: Number of nodes. Each node i represented as n-bits binary number: (i 0,i 1,…,i n-1 ) {0,1} n There is edge (i,j) iff i and j differs in exactly one bit. Example: (1001, 1101)

Boolean Hypercube Network n=3 N=

Boolean Hypercube Network Algorithm: s ending v i from node i to node σ(i): Scan the bits of σ(i) from left to right, and compare them with the address of the current location of v i. Send v i out of the current node along the edge corresponding to the left-most bit in which the current position and σ(i) differ. Example: Sending (1011) to (0000) in 4-dimensional hypercube. Route: (1011), (0011), (0001), (0000)

Randomized Oblivious Routing Algorithm Phase 1: Pick a random destination σ(i) from {1,...,n}. Packet v i travels to node σ(i). Phase 2: Packet v i travels from σ(i) to its destination d(i). In each phase, each packet uses the bit-fixing strategy to determine its route.

Randomized Oblivious Routing Algorithm Notes: σ(i) are chosen randomly. So they are not a permutation. Each node maintains a queue for each outgoing edge, with packets leaving in FIFO order (ties should be broken arbitrarily).

Randomized Oblivious Routing Algorithm How many steps elapse before packet v i reaches its destination? Let us first consider this question for Phase 1: Let p i be route for v i in Phase 1. The number of steps taken by v i : - The length of p i, which is at most n. - Number of steps it is queued.

Randomized Oblivious Routing Algorithm (Lemma 1) Lemma 1: View routes in Phase 1 as a directed path. For two different packets’ routes in Phase 1: once they separate, they do not rejoin.

Randomized Oblivious Routing Algorithm (Lemma 1) Proof: Each path may be seen as a sequence of binary numbers, representing nodes. Let’s consider the paths parts starting from the first common node. Mark: - v - the last common node. - v1 – the next node in the first path - v2 – the next node in the second path

Randomized Oblivious Routing Algorithm (Lemma 1) b1b1 c2c2 b2b2 c1c1 V: V1:V1: V2:V2: b 1 ≠b 2 ^ c 1 ≠c 2 The order of updating the bits is left-to-right.  So the bit b 1 in the first path and bit b 2 in the second path will never change.  And thus the paths will never pass the node with the same ID. b2b2 c1c1

Randomized Oblivious Routing Algorithm (Lemma 2) Lemma 2: Let the route of v i follow the sequence of edges p i = (e 1,e 2,...,e k ). Let S be the set of packets (other than v i ) whose routes pass through at least one of {e 1,e 2,...,e k }. Then: the delay incurred by v i is at most |S|.

Randomized Oblivious Routing Algorithm (Lemma 2) Proof: A packet in S is said to leave p i at that time step at which it traverses an edge of p i for the last time. For each packet v passing p i we define: lag(v) = t-j  v is ready to follow edge e j at time t. The lag of v i is initially zero, and the delay incurred by v i is its lag when it traverses e k. We will show that each step at which the lag of v i increases by one can be charged to a distinct member of S. So the time that takes v i to get to its intermediate target is: k+|S| ≤ k+N

Randomized Oblivious Routing Algorithm (Lemma 2) e1e1 e2e2 e3e3 e4e4 Example: Next step time is 4 pi = (e1,e2,e3,e4) v Lag(v) = t – j = 4 – 3 = 1

Randomized Oblivious Routing Algorithm (Lemma 2) Claim: If the lag of v i reaches l + 1, some packet in S leaves p i with lag l. Let t’ be the last time step at which any packet in S has lag l. Claim: Some packet of S leaves p i at t’.

Randomized Oblivious Routing Algorithm (Lemma 2) So for each such l there is a packet v such that lag(v)=l. When the lag of v i increases from l to l + 1, there must be at least one packet (from S) that wishes to traverse the same edge as v i at that time step,

Randomized Oblivious Routing Algorithm (Lemma 2) At time t’ there is a packet v ready to follow edge e j’ at t’ such that t’ – j’ = l. Some packet u in S follows e j ’ at t’ (maybe u=v). Claim: u leaves p i at t’. Proof: Let’s assume it’s not true.  So u will want to follow the e j’+1 at time t’+1.  lag(u) at time t’+1 is ((t’+1)-(j’+1)) = l.  In contradiction that t’ is the last time when any packet in S has lag l.

Randomized Oblivious Routing Algorithm (Lemma 2) When v i is delayed, one packet in S leaves p i. According to Lemma 1, once the paths are separated, they never rejoin.  After |S| delays no packet except v i travels thru p i.  v i may be delayed up to |S| steps.  It takes v i up to k + |S| steps to get to it’s intermediate target. Proof summary:

Randomized Oblivious Routing Algorithm Random variables H ij : H ij = 1, if p i and p j share at least one edge H ij = 0, otherwise The intermediate nodes for each packet are chosen independently. So H ij for some i are independent. By lemma 2, delay of package v i is at most:

Randomized Oblivious Routing Algorithm

Random variables T(e): The number of routes that pass through edge e. For any route p i =(e 1,e 2,…,e k ):

Randomized Oblivious Routing Algorithm Lemma:

Randomized Oblivious Routing Algorithm

The probability for package to wait in Phase 1 for more than 6n steps is less than 2 -6n. The probability for one of the packages to wait in Phase 1 for more than 6n steps is less than N * 2 -6n =2 n * 2 -6n = 2 -5n. Add the length of the longest possible path and we get that the probability to finish Phase 1 in more than 7n steps is less than 2 -5n.

Randomized Oblivious Routing Algorithm Phase 2: Phase 2 can be seen as reflection of Phase 1. Paths of Phase 2 can intersect with paths of Phase 1. In order to prevent this we wait 7n steps to be sure all packets got to their intermediate destination, before starting Phase 2.

Randomized Oblivious Routing Algorithm The probability for the process not to finish in 2 * 7n steps is less than 2 * 2 -5n = 2 -4n. With probability at least 1 - (1/N), every packet reaches its destination in 14n or fewer steps.

Summary The deterministic algorithm takes: steps. The randomized 14n steps. The probability algorithm is exponentially better than the deterministic one.