1. Parallel Computing Basic Concepts Computational Models Synchronous vs. Asynchronous The Flynn Taxonomy Shared versus Distributed Memory Interconnection networks Bus structures Static Dynamic Performance Issues Amdahl versus Gustafson-Barsis

2. Parallel Algorithms Languages for Parallel Computing PVM and MPI Memory Issues To be treated with memory in general Parallel Computing cont.

3. Basic Concepts Introduction Why? … the von Neumann Bottleneck Problems … the Grand Challenges The Gordon Bell Prize Common parallelism buses piplining Computational Models Synchronous vs Asychronous The Flynn Taxonomy SISD MISD SIMD MIMD DATA INSTRUCTIONS

4. Flynn Taxonomy SISD superscalar architectures VLIW Vector processors SIMD fine grained parallelism generally many processors MISD not used MIMD coarse grained parallelism few processors Super-Flynn SPMD single program … multiple data very common

5. Memory Shared versus Distributed tightly coupled versus loosely coupled multiprocessor versus multicomputer Interconnects Static versus Dynamic Hardwired versus switched Other Catagorizations

6. Shared Memory Advantages Price/performance … use “off the shelf” components Simplicity … reliability Scalability Disadvantages Device contention Local cache coherency Topologies bus bus contention processes 1-D multiple buses crossbar switch multiport memory 2-D multiple buses hierarchical bus architecture NUMA … non-uniform shared memory … KSR1

7. Dynamic Interconnects Crossbar Delta CLOS Static Interconnects Goals Fewest interconnects Fewest hops 1-D Linear 2-D Ring Grid / Mesh Toroid Fully Connected Tree, Quadtree Fat Tree Hypercube

8. Shared Memory Machines Sequent Symmetry … bus FX/8 … bus Cray Y-MP … multiport memory Distributed Memory Machines Intel Paragon … 2-D grid Wavetracer … 3-D grid MasPar MP-2 2-D torus and 3-stage CLOS nCUBE … hypercube iPSC/2 … hypercube CM - 2 … 2-D grid and 12-D hypercube CM-5 … Fat Tree Kendall Square Research KSR1 … rings Alliant Campus … crossbars (multi) Paradigm … bus(es) and interconnect