1. 1 Introduction to Data Parallel Architectures Sima, Fountain and Kacsuk Chapter 10 CSE462

2.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 2 Basic Concept of Data Parallelism Memory Register 1Register 2 8 Bit ALU

3.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 3 Basic Concept of Data Parallelism 8 bit wide Memory cells 8 bit wide Registers 8 bit ALU made of single bit ALUs

4.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 4 Why is this useful? Every cell of a matrix Every pixel of an image Every record of an database Process:

5.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 5 Thinking Machines l Connection Machine l Up to 65,535 processors in CM-2 http://www.astrosurf.com/lombry/Images/connection-machine-cm5.jpg http://emeagwali.com/gifs/cm2.jpg

6.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 6 Connectivity l Want to support basic computations required at cell level –E.g. A[i,j] = (A[i-1,j] + A[i+1,j] + A[i,j-1] + A[i,j+1])/4 l To achieve this, cells can be connected in a variety of ways –Near neighbours –Tree –Graph –Pyramid –Hypercube –Multistage –Reconfigurable –Crossbar –Bus.

7.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 7 Nearest Neighbours l Mapping spatially coherence data onto SIMD systems –Spatially correlated like images l Common to connect to NSEW, but diagonal has also been implemented –Applied to massively parallel systems –Scalable –Simple to implement

8.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 8 Trees and Graphs l Problems expressed as graphs –E.g. database searching, model matching, expert systems, etc l No mathematically regular structure l Reconfigurability required l Binary an Quad trees common l Data bottlenecks going through roots of sub trees

9.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 9 The Pyramid l Combination of mesh and tree –Supports nearest neighbour plus (quad) tree communication –Local communications of mesh –Global communication of tree Consider example of moving data from one corner to another l Useful for data stored at multiple resolutions –Like images

10.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 10 Hypercubes l 2 N processors – each of which has N links. l Fault tolerant l Shorter pathways than mesh

11.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 11 Different Data Parallel Architectures l SIMD Program PE Array

12.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 12 Different Data Parallel Architectures l Systolic or Pipelined DataALU 1ALU 2 ALU 3ALU 4 PE Array

13.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 13 Different Data Parallel Architectures l Vectorizing Vector of Vectors Vector ALU Automatic Sequence

14.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 14 Different Data Parallel Architectures l Associative and Neural Comparator Category Object Database

15.  David Abramson, 2004 Material from Sima, Fountain and Kacsuk, Addison Wesley 1997 15 Principal Characteristics of data- parallel systems PropertySIMDSystolicPipelineVectorizingNeuralAssociative ProgrammabilityGoodFixed GoodPoorGood AvailabilityGoodPoor GoodPoor ScalabilityGoodFixed Good ApplicabilityWideNarrow WideNarrowWide