1. Multiprocessor Systems
A presentation by
Chris Hargreaves

2. Flynn’s Classification
SISD: Single instruction single data
SIMD: Single instruction multiple data
MISD: Multiple instruction single data
MIMD: Multiple instruction multiple data

3. Bisection Bandwidth
Split the system into a half (or next to half) and find the bottleneck.

4. System Topologies
The pattern of connections between the processors

5. Shared Bus Processors communicate over a shared bus
Most cases the processors communicate directly with their locally memory
More processors cause delays as traffic increases

6. Ring Direct connections between processors
Allows simultaneous links to be active
Data may have to travel through multiple processors before reaching its final destination

7. Shared Bus & Ring

8. Tree Topology Each processor has three connections Lowers the diameter
Only one unique path between any pair of processors

9. Mesh Topology
Each processor connects to the it’s left and right, top and bottom
Without wraparound connections has a diameter of 2 n^(1/2). With it has n^(1/2)

10. Tree & Mesh

11. Hypercube Topology
Each processor connects to all other processors whose binary value differs by only one bit
Ex: Processor 0 connects to 1, 2, 4 and 8
Low Diameter (lg n). Good.

12. Completely Connected Each processor has (n-1) connections
Diameter is a constant 1

13. Hypercube & Completely Connected

14. MIMD Architecture
Refers to connections with respect to system memory

15. SMP (Symmetric Multiprocessor)
UMA – Uniform Memory Access
NUMA - Nonuniform Memory Access
CC-NUMA – Cache Coherent NUMA
COMA – Cache Only Memory Access
NOW & COW - Multicomputer

16. UMA Equal access to all locations in shared memory
Each processor may also have its own cache memory
Interact with memory over a communication mechanism such as a shared bus

17. UMA

18. NUMA
Can access the memory closest
Non Uniform access times to memory

19. NUMA

20. CC-NUMA Same as NUMA except each processor includes cache memory
Problem occurs when two caches hold the same piece of data
Must have coherence to make sure caches contain the same information

21. COMA Solution to CC-NUMA Local Memory treated as cache
If data is loaded multiple times, one is marked as the “Master Copy” which is kept current
Few of these machines have every been built

22. Multicomputing NOW – Network of workstations
COW – Cluster of workstations
United Devices (

23. MMP Massively Parallel Processor
Process massive ammounts of data in parallel
Typically at the level of the ALU
Communicate using shared memory
Example: IBM Blue Gene Computer