1. High Performance Computing & Bioinformatics Part 2 Dr. Imad Mahgoub

2. Overview of Interconnection Networks
SIMD Interconnection Networks Bidirectional Single Loop Structure
Advantages
Total # of links is small (N)
The degree of a node is "2"
Disadvantages
Diameter is big (N/2)
 average message delay and message traffic density is proportional to N (for N processor loop)
Fault in any two non-adjacent nodes will disconnect the loop
January 11, 2019
Dr. Mahgoub

3. Suitable Applications
Parallel algorithms that contain assignment of the following type:
x [i] := x [i-1] + x [i+1]
-2* x [i]
January 11, 2019
Dr. Mahgoub

4. Completely Connected Networks
Advantages
Minimum diameter (1)
Disadvantages
Cost is prop. to N2 => N < 5 (N  # of nodes)
Degree of a node (N-1) is high
(N=4) Completely Connected Network
January 11, 2019
Dr. Mahgoub

5. Tree Structure Binary Tree Advantages Low degree of a node
Good inter-node distance
Line and connection costs is prop. to N (N  # of nodes)
Disadvantages
High message traffic density through single nodes (especially at the second level)
January 11, 2019
Dr. Mahgoub

6. • Any fault at the root will completely dislocate the system B  # of branches L  # of levels
Tree Structure B=2, L=3
January 11, 2019
Dr. Mahgoub

7. Hypercube Connection Structures
In general a hypercube structure can be obtained if the number of nodes N equals WD;
W and D being integers
if W = 2 and D = n (n is an integer)
 the hypercube structure reduces to the Boolean n-cube
January 11, 2019
Dr. Mahgoub

8. The Boolean n -cube N=2ⁿ
January 11, 2019
Dr. Mahgoub

9. Mesh Nodes arranged into a q-dimensional lattice
Communication is allowed only between neighboring nodes  interior nodes communicate with 2q other processors
Some variants of the mesh allow (toroidal) wrap-around connections between processors on the edge
January 11, 2019
Dr. Mahgoub

10. Mesh Two dimensional mesh With wrap -around ?
With toroidal wrap - around ?
January 11, 2019
Dr. Mahgoub

11. Applications
Efficient sorting and Matrix Multiplication Algorithms have been designed for meshes of processors.
Suitable for solving systems of second-order partial differential equations.
January 11, 2019
Dr. Mahgoub

12. Consists of n=2K nodes numbered
Shuffle - Exchange
Consists of n=2K nodes numbered
0, 1, ..., n-1, and two kinds of connections, called shuffle and exchange.
Exchange
Links pairs of nodes whose numbers differ in their least significant bit
Perfect Shuffle
i --> 2i mod (n-1) (node n - 1 is connected to itself)
January 11, 2019
Dr. Mahgoub

13. <--------> : Shuffle ------------: Exchange
Shuffle - Exchange Network with 8 Nodes
January 11, 2019
Dr. Mahgoub

14. Cube-Connected Cycle
Obtained by replacing each node of the k-dimensional cube by a ring or a cycle of k nodes
 has kx2k nodes
Rings numbered from 0 to 2k-1
Nodes on a ring numbered from 0 to k-1
If two rings have numbers differing by "2i," connect node "i" on these rings
January 11, 2019
Dr. Mahgoub

15. 3 – Cube
Cube Connected Cycles
January 11, 2019
Dr. Mahgoub

16. MIMD Networks Bus network Multiple-Bus Crossbar Multistage
Single Bus Network
January 11, 2019
Dr. Mahgoub

17. Advantages and Disadvantages of Single Shared Bus Networks
Multiple-Bus Networks
Emerged as a solution for the bus contention problem in the single bus schemes
Allow easy incremental expansions of the number of processors and memory modules in the system
The buses can be configured in a variety of ways to provide a range of trade-offs between bandwidth, connection cost and reliability
January 11, 2019
Dr. Mahgoub

18. An MxNxB Multiple - Bus Multiprocessor
January 11, 2019
Dr. Mahgoub

19. Crossbar Networks A crossbar can be viewed as a number of vertical and
horizontal links
interconnected by
a switch in each
intersection
A 4 x 4 Crossbar Switch
January 11, 2019
Dr. Mahgoub

20. Advantages and Disadvantages of Crossbar Networks
Multistage Networks
Less complex than the crossbar
Contains a number of switching elements (like crossbar switches) which typically have the same number of inputs and outputs (say "k")
The switching elements are organized in "logk N" stages with "N/k" switching elements in each
stage where N # of processors and memory modules
January 11, 2019
Dr. Mahgoub

21. • One unique path exists between every processor-memory pair
A request must pass through all stages of switching elements to reach its final destination
 The latency time equals O (logk N)
January 11, 2019
Dr. Mahgoub

22. Omega Networks Each switch box (2x2) has four switch functions:
Straight Through
Interchange
January 11, 2019
Dr. Mahgoub

23. Upper Broadcast
Lower Broadcast
January 11, 2019
Dr. Mahgoub

24. 010---------><101>
8 x 8 Omega Network
January 11, 2019
Dr. Mahgoub

25. Typically distributed Omega is a self routing network
Routing of requests
Typically distributed
Omega is a self routing network
Outputs of a switching element are numbered "0" and "1“
If the destination address is
<do d1 ... dn-1> where N=2n, then the
switching element in stage "i" sends the message to output "di"
January 11, 2019
Dr. Mahgoub

26. Disadvantages
Improvements
Advantages
January 11, 2019
Dr. Mahgoub