Parallel Processing Presented by: Wanki Ho CS147, Section 1

The single processor problem Question: Why is parallel processing necessary?

 Reasons for needing computational power  Longer time to completion  Higher complexity  More detail  Solutions must be found within a reasonable time by human standards

 Examples of computational problems  Weather prediction  Air traffic simulations  DNA mapping  Big challenges  Computer AIs

 Underlying problem: Need more memory and speed  The solution: Parallel processing  Connect a lot of computers together using an interconnection network so that they act together

Types of Parallel Processors  Various types depending on how the processors interact with memory  Flynn’s Taxonomy  SISD Single Instruction Single Data Stream  SIMD Single Instruction Multiple Data Stream  MISD Multiple Instruction Single Data Stream  MIMD Multiple Instruction Multiple Data Stream

 SISD  Von Neumann architecture. Your typical everyday computer  MISD  Wierd stuff. Noone uses.  SIMD  Computers with special instructions for handling large data in parallel (ex. Array processor in Intel’s MMX chip)

 MIMD  Many processors performing operations on many data  Multiprocessors  Networks of interconnected computers

Parallel Processing Problems  Multiple processors accessing the same memory.  What happens if one processor reads and another processor writes to the same block of memory?  How do two processors maintain separate caches concurrently?

Network Topologies  Bus Network  Ring  Tree  Mesh  Hypercube  Complete Connectivity

Ring  Every computer has two branches connecting it to another computer or outside the ring

Tree  Computers are connected to each other in a tree structure

Mesh  Each computer is connected to the computer below and above and to the left and right of it

Hypercube  Each computer is connected to 4 other computers  Consistent distance between two computers. Useful for weight balancing

Complete Connectivity  Each computer is connected to every other computer in the network  Optimal connectivity  Complexity grows geometrically

 The End