1. Multi-Processing in High Performance Computer Architecture:

2. What is Multiprocessing?
Enables several programs to run concurrently
Coordinated processing of Programs by more than one processor
Use of 2 or more CPUs within a single computer system
Ability of a system to support more than one processor and to allocate tasks between them

3. Idealism (Target for Processor Performance):

4. Memory Hierarchies:

5. Memory in Modern Processor (L1 Cache?):

7. Why Not Uni-Processors:
Making a wider Processor can efficiently run parallel programs but not programs that have dependencies
In Uni-Processor, the instructions a = b + c; d = e + f can be run in parallel because non of the results depend on other calculations
The instructions b = e + f; a = b + c, might not run in parallel due to data dependencies. There are several more complex data dependencies.
Parallel part : Fast, One at a time(Stalls) : Slow

8. Why Multi-Processors:

9. Multi-Processor needs Parallel Programs(Years to Develop):

10. Centralized Shared Memory: (UMA)
Distance from memory to each core is approximately same(UMA)
Replicate Core and Caches to build a Multi Dimensional System(SMP)

11. Issues in Centralized Main Memory:

12. Distributed Memory:
Each Core has it’s own local memory and cache forming a single core computer system.
A network interface card, connected to a network of interconnections
Cache miss goes directly to the local processor’s memory and if needed it can only access the other processor’s memory through the network message passing.
To communicate, data is sent EXPLICITLY to the core. Programmer is forced to be aware of communication between cores and try to minimize it.
Think of it as a set of machines communicating over a network.

13. Distributed (Multicomputer) Memory System(NUMA):

14. Shared Memory vs Message Passing (Hardware vs Software):

15. Simple Implementation of Summing Array in Parallel on 4 cores (Message Passing):

16. A Simple Shared Memory Implementation:

17. Performance Metrics (Message Passing vs Shared Memory):

18. Multithreading as Shared Memory Hardware:

19. Analyzing Multithreading Performance:

20. Summary: Multithreaded Categories