2. 2. Multiprocessors Main Structures 2.1 Shared Memory x Distributed Memory Shared-Memory (Global-Memory) Multiprocessor:  All processors can access all memory locations.  All the necessary variables are shared by all processors. They may read or write any shared variable. Mem Proc Mem Proc Mem Proc Memory Access Interconnection Network Shared-Memory Multiprocessor Mem Proc Mem Proc Mem Proc Message Passing Interconnection Network Distributed-Memory Multiprocessor (Multicomputer) Distributed-Memory Multiprocessor (Multicomputer):  Each processor has its own local memory.  Each processor can read or write only its own local (cache) memory. Any synchronization has to be done using explicit message passing between processors.

3. 2.1 Shared Memory x Distributed Memory Shared memory multiprocessors always have local cache memories private to each processor. This reduces global memory conflicts. As the size of the cache memory increases, then the usefulness of global memory is reduced to the buffering of interprocess data (Message Passing Interconnect Network). Concurrent processes reside on various processors. When some processors fail, the remaining processors can continue the work, though at a lesser throughput, but ensuring high availability. Recovery and process redistribution after a failure is more difficult in distributed- memory multiprocessors (the recovery process requires that other processors read the faulty processor cache). 2. Multiprocessors Main Structures

4. 2.2 Fine Grain x Coarse Grain Grain: refers to the number of instructions executed in a processor before synchronizing or communicating some data with another processor. Fine-grain parallel processing involves synchronizing the processors after few instructions. Coarse-grain parallel processing involves synchronizing the processors after tens of thousands of instructions. Medium-grain parallel processing has characterized a grain size somewhere in the middle, for example, several hundred instructions between synchronizations. 2. Multiprocessors Main Structures

5. 2.3 Moderate Parallel x Massively Parallel Moderate Parallel Processing: architecture using 10 to 100 processors. Massively Parallel Processing: architecture using hundreds of processors. 2. Multiprocessors Main Structures

6. 2.4 SIMD x MIMD SIMD: Single Instruction Multiple Data Processor. - A central controller broadcasts the same instruction to different processors, each processor then executes the instruction on its data. - Some processors can be masked from the instruction by appropriately setting some mask registers at each processor. 2. Multiprocessors Main Structures

7. 2.4 SIMD x MIMD MIMD: Multiple Instruction Multiple Data Processor. - Processors execute different instructions on different data. - Shared-memory MIMD multiprocessors are programmed assuming that all the necessary variables are shared by all processors, and that they may read or write any shared variable. - Distributed-memory MIMD multiprocessors are programmed assuming that each processor can read or write only its own local memory. Any synchronization has to be done using explicit message passing between processors. 2. Multiprocessors Main Structures

8. 2.5 Topology of Interconnect Typical interconnection networks include: BUS CROSSBAR MULTISTAGE MESHES TREES HYPERCUBES 2. Multiprocessors Main Structures