1. 1 Recap

2. 2 No. of Processors C.P.I 1 2.... Computational Power Improvement Multiprocessor Uniprocessor

3. 3 Summing m Numbers Example: summing m numbers On a sequential computer we have, sum = a[0]; for (i=1;i<m;i++) { sum = sum + a[i]; } Would expect the running time be be roughly proportional to m.

4. 4 Summing m Numbers in Parallel What if we have N processors, with each calculating the m/N numbers assigned to it? We must add these partial sums together to get the total sum.

5. 5 Summing Multiprocessors P11P11 P13P13 P12P12 P21P21 P23P23 P22P22 P31P31 P33P33 P32P32 j i Suppose we have N processors. The algorithm is as follows: 1.Each processor finds the local sum of its m/N numbers 2.Each processor passes its local sum to another processor in a coordinated way 3.The global sum is finally in processor P11.

6. 6 Shared Memory Multiprocessors P/C Cache NIC MB P/C Cache NIC MB Bus/Custom-Designed Network Shared Memory

7. 7 MPP (Massively Parallel Processing) Distributed Memory Multiprocessors P/C LM NIC MB P/C LM NIC MB Custom-Designed Network MB : Memory BusNIC : Network Interface Circuitry

8. 8 Cluster Commodity Network (Ethernet, ATM, Myrinet) MB P/C M NIC P/C M Bridge LD NIC IOB LD : Local DiskIOB : I/O Bus

9. 9 Grid P/C SM NIC LD Hub/LAN Internet IOC P/C SM NIC LD Hub/LAN IOC

10. 10 TOP500 architectures