1. CHAPTER SEVEN
PARALLEL PROCESSING
© Prepared By:
Razif Razali

2. Contents Multiple Processor Organizations Symmetric Multiprocessors
© Prepared By:
Razif Razali

3. Introduction Introduction
Traditionally, the computer has been viewed as a sequential machine
Most computer programming languages require the programmer to specify algorithms as sequences of instructions
Processors executes programs by executing machine instructions in a sequence and one at a time
Each instructions is executed in a sequence of operations (fetch instruction, fetch operation, perform operation, store results)
A traditional way to increase system performance is to use multiple processors that can execute in parallel to support a given workload.
The two most common multiple-processor organizations are Symmetric multiprocessors (SMPs) and clusters
More recently, nonuniform memory access (NUMA) have been introduced commercially
© Prepared By:
Razif Razali

4. Multiple Processor Organization.
Types of Parallel Processor Systems are :
Single instruction, single data stream - SISD
Single instruction, multiple data stream - SIMD
Multiple instruction, single data stream - MISD
Multiple instruction, multiple data stream- MIMD
© Prepared By:
Razif Razali

5. Single instruction, single data stream - SISD
Single processor executes a single instruction stream to operate on data stored in single memory.
Uni-processor fall into this category
Single instruction, single data stream - SISD
© Prepared By:
Razif Razali

6. Single instruction, multiple data stream – SIMD
A single machine instruction controls simultaneous execution of a number of processing elements on a lockstep basis.
Each processing element has associated data memory, so that each instruction is executed on different set of data by different processors.
Example : Vector and array processors
Single instruction, multiple data stream - SIMD

7. MISD & MIMD Multiple instruction, single data stream - MISD
Sequence of data
Transmitted to set of processors
Each processor executes different instruction sequence
Never been implemented
Multiple instructions, multiple data stream- MIMD
Set of processors
Simultaneously execute different instruction sequences
Different sets of data
SMPs, clusters and NUMA systems
MIMD is divided into two :
Shared Memory (tightly coupled) :
Distributed Memory (loosely coupled)
© Prepared By:
Razif Razali

8. Multiple instruction, multiple data stream - MIMD shared memory
© Prepared By:
Razif Razali

9. Multiple instruction, multiple data stream- MIMD distributed memory
© Prepared By:
Razif Razali

10. Taxonomy of Parallel Processor Architectures
© Prepared By:
Razif Razali

11. Symmetric Multiprocessor
A stand alone computer with the following characteristics :
There are two or more similar processors of comparable capability
These processors share the same main memory and I/O facilities and are interconnected by a bus or other internal connection scheme, such that memory access time is approximately the same for each processor
All processors share access to I/O devices
Either through same channels or different channels that provide paths to same devices
All processors can perform the same functions (hence the term symmetric)
The system is controlled by an integrated operating system that providing interaction between processors and their programs at the job, task, file and data element levels
© Prepared By:
Razif Razali

12. Symmetric Multiprocessor
Advantages
Performance
If some work can be done in parallel, then a system with multiple processors will yield greater performance than one with a single processor of the same type
Availability
Since all processors can perform the same functions, failure of a single processor does not stop the system
Incremental growth
User can enhance performance by adding additional processors
Scaling
Vendors can offer range of products based on number of processors
© Prepared By:
Razif Razali

13. Block Diagram of Tightly Coupled Multiprocessor
© Prepared By:
Razif Razali

14. Symmetric Multiprocessor
CONCLUSIONS
Types of Parallel Processor Systems are :
Single instruction, single data stream - SISD
Single instruction, multiple data stream - SIMD
Multiple instruction, single data stream - MISD
Multiple instruction, multiple data stream- MIMD
Symmetric Multiprocessor
© Prepared By:
Razif Razali

15. END OF THESE SUBJECT
© Prepared By:
Razif Razali