1. Parallel Processing Presented by: Wanki Ho CS147, Section 1

2. The single processor problem Question: Why is parallel processing necessary?

3.  Reasons for needing computational power  Longer time to completion  Higher complexity  More detail  Solutions must be found within a reasonable time by human standards

4.  Examples of computational problems  Weather prediction  Air traffic simulations  DNA mapping  Big challenges  Computer AIs

5.  Underlying problem: Need more memory and speed  The solution: Parallel processing  Connect a lot of computers together using an interconnection network so that they act together

6. Types of Parallel Processors  Various types depending on how the processors interact with memory  Flynn’s Taxonomy  SISD Single Instruction Single Data Stream  SIMD Single Instruction Multiple Data Stream  MISD Multiple Instruction Single Data Stream  MIMD Multiple Instruction Multiple Data Stream

7.  SISD  Von Neumann architecture. Your typical everyday computer  MISD  Wierd stuff. Noone uses.  SIMD  Computers with special instructions for handling large data in parallel (ex. Array processor in Intel’s MMX chip)

8.  MIMD  Many processors performing operations on many data  Multiprocessors  Networks of interconnected computers

9. Parallel Processing Problems  Multiple processors accessing the same memory.  What happens if one processor reads and another processor writes to the same block of memory?  How do two processors maintain separate caches concurrently?

10. Network Topologies  Bus Network  Ring  Tree  Mesh  Hypercube  Complete Connectivity

11. Ring  Every computer has two branches connecting it to another computer or outside the ring

12. Tree  Computers are connected to each other in a tree structure

13. Mesh  Each computer is connected to the computer below and above and to the left and right of it

14. Hypercube  Each computer is connected to 4 other computers  Consistent distance between two computers. Useful for weight balancing

15. Complete Connectivity  Each computer is connected to every other computer in the network  Optimal connectivity  Complexity grows geometrically

16.  The End