High Performance Computing & Bioinformatics Part 2 Dr. Imad Mahgoub

Overview of Interconnection Networks SIMD Interconnection Networks Bidirectional Single Loop Structure Advantages Total # of links is small (N) The degree of a node is "2" Disadvantages Diameter is big (N/2)  average message delay and message traffic density is proportional to N (for N processor loop) Fault in any two non-adjacent nodes will disconnect the loop January 11, 2019 Dr. Mahgoub

Suitable Applications Parallel algorithms that contain assignment of the following type: x [i] := x [i-1] + x [i+1] -2* x [i] January 11, 2019 Dr. Mahgoub

Completely Connected Networks Advantages Minimum diameter (1) Disadvantages Cost is prop. to N2 => N < 5 (N  # of nodes) Degree of a node (N-1) is high (N=4) Completely Connected Network January 11, 2019 Dr. Mahgoub

Tree Structure Binary Tree Advantages Low degree of a node Good inter-node distance Line and connection costs is prop. to N (N  # of nodes) Disadvantages High message traffic density through single nodes (especially at the second level) January 11, 2019 Dr. Mahgoub

• Any fault at the root will completely dislocate the system B  # of branches L  # of levels Tree Structure B=2, L=3 January 11, 2019 Dr. Mahgoub

Hypercube Connection Structures In general a hypercube structure can be obtained if the number of nodes N equals WD; W and D being integers if W = 2 and D = n (n is an integer)  the hypercube structure reduces to the Boolean n-cube January 11, 2019 Dr. Mahgoub

The Boolean n -cube N=2ⁿ January 11, 2019 Dr. Mahgoub

Mesh Nodes arranged into a q-dimensional lattice Communication is allowed only between neighboring nodes  interior nodes communicate with 2q other processors Some variants of the mesh allow (toroidal) wrap-around connections between processors on the edge January 11, 2019 Dr. Mahgoub

Mesh Two dimensional mesh With wrap -around ? With toroidal wrap - around ? January 11, 2019 Dr. Mahgoub

Applications Efficient sorting and Matrix Multiplication Algorithms have been designed for meshes of processors. Suitable for solving systems of second-order partial differential equations. January 11, 2019 Dr. Mahgoub

Consists of n=2K nodes numbered Shuffle - Exchange Consists of n=2K nodes numbered 0, 1, ..., n-1, and two kinds of connections, called shuffle and exchange. Exchange Links pairs of nodes whose numbers differ in their least significant bit Perfect Shuffle i --> 2i mod (n-1) (node n - 1 is connected to itself) January 11, 2019 Dr. Mahgoub

<--------> : Shuffle ------------: Exchange Shuffle - Exchange Network with 8 Nodes January 11, 2019 Dr. Mahgoub

Cube-Connected Cycle Obtained by replacing each node of the k-dimensional cube by a ring or a cycle of k nodes  has kx2k nodes Rings numbered from 0 to 2k-1 Nodes on a ring numbered from 0 to k-1 If two rings have numbers differing by "2i," connect node "i" on these rings January 11, 2019 Dr. Mahgoub

3 – Cube Cube Connected Cycles January 11, 2019 Dr. Mahgoub

MIMD Networks Bus network Multiple-Bus Crossbar Multistage Single Bus Network January 11, 2019 Dr. Mahgoub

Advantages and Disadvantages of Single Shared Bus Networks Multiple-Bus Networks Emerged as a solution for the bus contention problem in the single bus schemes Allow easy incremental expansions of the number of processors and memory modules in the system The buses can be configured in a variety of ways to provide a range of trade-offs between bandwidth, connection cost and reliability January 11, 2019 Dr. Mahgoub

An MxNxB Multiple - Bus Multiprocessor January 11, 2019 Dr. Mahgoub

Crossbar Networks A crossbar can be viewed as a number of vertical and horizontal links interconnected by a switch in each intersection A 4 x 4 Crossbar Switch January 11, 2019 Dr. Mahgoub

Advantages and Disadvantages of Crossbar Networks Multistage Networks Less complex than the crossbar Contains a number of switching elements (like crossbar switches) which typically have the same number of inputs and outputs (say "k") The switching elements are organized in "logk N" stages with "N/k" switching elements in each stage where N # of processors and memory modules January 11, 2019 Dr. Mahgoub

• One unique path exists between every processor-memory pair A request must pass through all stages of switching elements to reach its final destination  The latency time equals O (logk N) January 11, 2019 Dr. Mahgoub

Omega Networks Each switch box (2x2) has four switch functions: Straight Through Interchange January 11, 2019 Dr. Mahgoub

Upper Broadcast Lower Broadcast January 11, 2019 Dr. Mahgoub

010---------><101> 8 x 8 Omega Network January 11, 2019 Dr. Mahgoub

Typically distributed Omega is a self routing network Routing of requests Typically distributed Omega is a self routing network Outputs of a switching element are numbered "0" and "1“ If the destination address is <do d1 ... dn-1> where N=2n, then the switching element in stage "i" sends the message to output "di" January 11, 2019 Dr. Mahgoub

Disadvantages Improvements Advantages January 11, 2019 Dr. Mahgoub