Parallel Architectures Based on Parallel Computing, M. J. Quinn CIS 5930-04 – Spring 2006: Parallel Computing Parallel Architectures Based on Parallel Computing, M. J. Quinn Ashok Srinivasan www.cs.fsu.edu/~asriniva Florida State University

Outline Interconnection Networks Architectures Flynn’s Taxonomy Mesh Binary Tree Hypercube Architectures Processor Arrays Multiprocessors Centralized Multiprocessors Distributed Multiprocessors Multicomputers Flynn’s Taxonomy SISD SIMD MIMD

Mesh 2-D Mesh 2-D Mesh with wraparound (2-D torus) 3-D Mesh 3-D torus Diameter = 2 N0.5 Bisection width = N0.5 Edges/node = 4 2-D Mesh with wraparound (2-D torus) 3-D Mesh 3-D torus

Binary Tree Binary Tree Fat Tree Diameter = 2 log N Bisection width = 1 Edges/node = 3 Fat Tree Double the number of edges at each level up the tree

Hypercube Hypercube N = 2d Diameter = log N Bisection width = N/2 Edges/node = log N

Processor Arrays

Centralized Multiprocessors Also called SMP/UMA

Cache Coherence Problem Solve through snooping

Distributed Multiprocessors Also called NUMA

Cache Coherence - Directory Based Solution

Multicomputers Also called a distributed memory system

Flynn’s Taxonomy SISD SIMD MIMD Single Instruction, Single Data Traditional computer SIMD Single Instruction, Multiple Data MIMD Multiple Instruction, Multiple Data