Static Interconnection Networks Miodrag Bolic

Linear Arrays and Rings Ring arranged to use short wires Linear Array Asymmetric network Degree d=2 Diameter D=N-1 Bisection bandwidth: b=1 Allows for using different sections of the channel by different sources concurrently. Ring d=2 D=N-1 for unidirectional ring or for bidirectional ring Difference between the linear array and the bus Ring: unidirectional and bidirectional

Ring Fully Connected Topology Chordal ring Needs N(N-1)/2 links to connect N processor nodes. Example N=16 -> 136 connections. N=1,024 -> 524,288 connections D=1 d=N-1 Chordal ring N=16, d=3 -> D=5

Multidimensional Meshes and Tori 3D Cube 2D Grid Mesh Popular topology, particularly for SIMD architectures since they match many data parallel applications (eg image processing, weather forecasting). Illiac IV, Goodyear MPP, CM-2, Intel Paragon Asymmetric d= 2k except at boundary nodes. k-dimensional mesh has N=nk nodes. Torus Mesh with looping connections at the boundaries to provide symmetry. K is the dimension Ring D=k*(n-1), B=n Torus: d=4, D= 2 * ring diameter, B=2n

Trees Diameter and ave distance logarithmic Fixed degree k-ary tree, height d = logk N address specified d-vector of radix k coordinates describing path down from root Fixed degree Route up to common ancestor and down Bisection BW? Binary tree Number of nodes 2^k-1 Leaves, roots

Trees (cont.) Fat tree Star The channel width increases as we go up Solves bottleneck problem toward the root Star Two level tree with d=N-1, D=2 Centralized supervisor node

Hypercubes Each PE is connected to (d = log N) other PEs d = log N Binary labels of neighbor PEs differ in only one bit A d-dimensional hypercube can be partitioned into two (d-1)-dimensional hypercubes The distance between Pi and Pj in a hypercube: the number of bit positions in which i and j differ (ie. the Hamming distance) Example: 10011  01001 = 11010 Distance between PE11 and PE9 is 3 : select any bit position, those PEs which have 0's at that bit position will make up one partition and others will form the second partition In an n-dimensional hypercube, n separate paths always exist between two nodes. This makes the hypercube highly fault tolerant Degree increases linearly with dimension, which limits scalability. 100 110 000 010 111 101 001 011 0-D 1-D 2-D 3-D 4-D 5-D *From Parallel Computer Architectures; A Hardware/Software approach, D. E. Culler

Hypercube routing functions Example Consider 4D hypercube (n=4) Source address s = 0110 and destination address d = 1101 Direction bits r = 0110 1101 = 1011 1. Route from 0110 to 0111 because r = 1011 2. Route from 0111 to 0101 because r = 1011 3. Skip dimension 3 because r = 1011 4. Route from 0101 to 1101 because r = 1011

k-ary n-cubes Rings, meshes, torii and hypercubes are special cases of a general topology called a k-ary n-cube Has n dimensions with k nodes along each dimension An n processor ring is a n-ary 1-cube An nxn mesh is a n-ary 2-cube (without end-around connections) An n-dimensional hypercube is a 2-ary n-cube N=kn Routing distance is minimized for topologies with higher dimension Cost is lowest for lower dimension. Scalability is also greatest and VLSI layout is easiest.

Cube-connected cycle d=3 D=2k-1+ Example N=8 We can use the 2CCC network

References Advanced Computer Architecture and Parallel Processing, by Hesham El-Rewini and Mostafa Abd-El-Barr, John Wiley and Sons, 2005. Advanced Computer Architecture Parallelism, Scalability, Programmability, by  K. Hwang, McGraw-Hill 1993.