1. .1 Intro to Multiprocessors [Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005]

2. .2 The Big Picture: Where are We Now? Processor Control Datapath Memory Input Output Input Output Memory Processor Control Datapath  Multiprocessor – multiple processors with a single shared address space  Cluster – multiple computers (each with their own address space) connected over a local area network (LAN) functioning as a single system

3. .3 Applications Needing “Supercomputing”  Energy (plasma physics (simulating fusion reactions), geophysical (petroleum) exploration)  DoE stockpile stewardship (to ensure the safety and reliability of the nation’s stockpile of nuclear weapons)  Earth and climate (climate and weather prediction, earthquake, tsunami prediction and mitigation of risks)  Transportation (improving vehicles’ airflow dynamics, fuel consumption, crashworthiness, noise reduction)  Bioinformatics and computational biology (genomics, protein folding, designer drugs)  Societal health and safety (pollution reduction, disaster planning, terrorist action detection)

4. .4 Encountering Amdahl’s Law  Speedup due to enhancement E is Speedup w/ E = ---------------------- Exec time w/o E Exec time w/ E  Suppose that enhancement E accelerates a fraction F (F 1) and the remainder of the task is unaffected ExTime w/ E = ExTime w/o E  Speedup w/ E =

5. .5 Encountering Amdahl’s Law  Speedup due to enhancement E is Speedup w/ E = ---------------------- Exec time w/o E Exec time w/ E  Suppose that enhancement E accelerates a fraction F (F 1) and the remainder of the task is unaffected ExTime w/ E = ExTime w/o E  ((1-F) + F/S) Speedup w/ E = 1 / ((1-F) + F/S)

6. .6 Examples: Amdahl’s Law  Consider an enhancement which runs 20 times faster but which is only usable 25% of the time. Speedup w/ E =  What if its usable only 15% of the time? Speedup w/ E =  Amdahl’s Law tells us that to achieve linear speedup with 100 processors, none of the original computation can be scalar!  To get a speedup of 99 from 100 processors, the percentage of the original program that could be scalar would have to be 0.01% or less Speedup w/ E =

7. .7 Examples: Amdahl’s Law  Consider an enhancement which runs 20 times faster but which is only usable 25% of the time. Speedup w/ E = 1/(.75 +.25/20) = 1.31  What if its usable only 15% of the time? Speedup w/ E = 1/(.85 +.15/20) = 1.17  Amdahl’s Law tells us that to achieve linear speedup with 100 processors, none of the original computation can be scalar!  To get a speedup of 99 from 100 processors, the percentage of the original program that could be scalar would have to be 0.01% or less Speedup w/ E = 1 / ((1-F) + F/S)

8. .8 Supercomputer Style Migration (Top500)  In the last 8 years uniprocessor and SIMDs disappeared while Clusters and Constellations grew from 3% to 80% Nov data http://www.top500.org/lists/2005/11/ Cluster – whole computers interconnected using their I/O bus Constellation – a cluster that uses an SMP multiprocessor as the building block

9. .9 Multiprocessor/Clusters Key Questions  Q1 – How do they share data?  Q2 – How do they coordinate?  Q3 – How scalable is the architecture? How many processors can be supported?

10. .10 Flynn’s Classification Scheme  Now obsolete except for...  SISD – single instruction, single data stream l aka uniprocessor - what we have been talking about all semester  SIMD – single instruction, multiple data streams l single control unit broadcasting operations to multiple datapaths  MISD – multiple instruction, single data l no such machine (although some people put vector machines in this category)  MIMD – multiple instructions, multiple data streams l aka multiprocessors (SMPs, MPPs, clusters, NOWs)

11. .11 SIMD Processors  Single control unit  Multiple datapaths (processing elements – PEs) running in parallel l Q1 – PEs are interconnected (usually via a mesh or torus) and exchange/share data as directed by the control unit l Q2 – Each PE performs the same operation on its own local data PE Control

12. .12 Example SIMD Machines MakerYear# PEs# b/ PE Max memory (MB) PE clock (MHz) System BW (MB/s) Illiac IVUIUC197264 1132,560 DAPICL19804,0961252,560 MPPGoodyear198216,384121020,480 CM-2Thinking Machines 198765,5361512716,384 MP-1216MasPar198916,384410242523,000

13. .13 Multiprocessor Basic Organizations  Processors connected by a single bus  Processors connected by a network # of Proc Communication model Message passing8 to 2048 Shared address NUMA8 to 256 UMA2 to 64 Physical connection Network8 to 256 Bus2 to 36

14. .14 Shared Address (Shared Memory) Multi’s  UMAs (uniform memory access) – aka SMP (symmetric multiprocessors) l all accesses to main memory take the same amount of time no matter which processor makes the request or which location is requested  NUMAs (nonuniform memory access) l some main memory accesses are faster than others depending on the processor making the request and which location is requested l can scale to larger sizes than UMAs so are potentially higher performance  Q1 – Single address space shared by all the processors  Q2 – Processors coordinate/communicate through shared variables in memory (via loads and stores) l Use of shared data must be coordinated via synchronization primitives (locks)

15. .15 N/UMA Remote Memory Access Times (RMAT) YearTypeMax Proc Interconnection Network RMAT (ns) Sun Starfire1996SMP64Address buses, data switch 500 Cray 3TE1996NUMA20482-way 3D torus300 HP V1998SMP328 x 8 crossbar1000 SGI Origin 30001999NUMA512Fat tree500 Compaq AlphaServer GS 1999SMP32Switched bus400 Sun V8802002SMP8Switched bus240 HP Superdome 9000 2003SMP64Switched bus275 NASA Columbia2004NUMA10240Fat tree???

16. .16 Single Bus (Shared Address UMA) Multi’s  Caches are used to reduce latency and to lower bus traffic  Must provide hardware to ensure that caches and memory are consistent (cache coherency)  Must provide a hardware mechanism to support process synchronization Processor Cache Single Bus Memory I/O

17. .17 Message Passing Multiprocessors  Each processor has its own private address space  Q1 – Processors share data by explicitly sending and receiving information (messages)  Q2 – Coordination is built into message passing primitives (send and receive)

18. .18 Summary  Flynn’s classification of processors – SISD, SIMD, MIMD l Q1 – How do processors share data? l Q2 – How do processors coordinate their activity? l Q3 – How scalable is the architecture (what is the maximum number of processors)?  Shared address multis – UMAs and NUMAs  Bus connected (shared address UMAs) multis l Cache coherency hardware to ensure data consistency l Synchronization primitives for synchronization l Bus traffic limits scalability of architecture (< ~ 36 processors)  Message passing multis

19. .19 Review: Where are We Now? Processor Control Datapath Memory Input Output Input Output Memory Processor Control Datapath  Multiprocessor – multiple processors with a single shared address space  Cluster – multiple computers (each with their own address space) connected over a local area network (LAN) functioning as a single system

20. .20 Multiprocessor Basics # of Proc Communication model Message passing8 to 2048 Shared address NUMA8 to 256 UMA2 to 64 Physical connection Network8 to 256 Bus2 to 36  Q1 – How do they share data?  Q2 – How do they coordinate?  Q3 – How scalable is the architecture? How many processors?

21. .21 Single Bus (Shared Address UMA) Multi’s  Caches are used to reduce latency and to lower bus traffic l Write-back caches used to keep bus traffic at a minimum  Must provide hardware to ensure that caches and memory are consistent (cache coherency)  Must provide a hardware mechanism to support process synchronization Proc1 Proc2Proc4 Caches Single Bus Memory I/O Proc3 Caches

22. .22 Multiprocessor Cache Coherency  Cache coherency protocols l Bus snooping – cache controllers monitor shared bus traffic with duplicate address tag hardware (so they don’t interfere with processor’s access to the cache) Proc1 Proc2 ProcN DCache Single Bus MemoryI/O Snoop

23. .23 Bus Snooping Protocols  Multiple copies are not a problem when reading  Processor must have exclusive access to write a word l What happens if two processors try to write to the same shared data word in the same clock cycle? The bus arbiter decides which processor gets the bus first (and this will be the processor with the first exclusive access). Then the second processor will get exclusive access. Thus, bus arbitration forces sequential behavior. l This sequential consistency is the most conservative of the memory consistency models. With it, the result of any execution is the same as if the accesses of each processor were kept in order and the accesses among different processors were interleaved.  All other processors sharing that data must be informed of writes

24. .24 Handling Writes Ensuring that all other processors sharing data are informed of writes can be handled two ways: 1. Write-update (write-broadcast) – writing processor broadcasts new data over the bus, all copies are updated l All writes go to the bus  higher bus traffic l Since new values appear in caches sooner, can reduce latency 2. Write-invalidate – writing processor issues invalidation signal on bus, cache snoops check to see if they have a copy of the data, if so they invalidate their cache block containing the word (this allows multiple readers but only one writer) l Uses the bus only on the first write  lower bus traffic, so better use of bus bandwidth

25. .25 A Write-Invalidate CC Protocol Shared (clean) Invalid Modified (dirty) write-back caching protocol in black read (miss) write (hit or miss) read (hit or miss) read (hit) or write (hit or miss) write (miss)

26. .26 A Write-Invalidate CC Protocol Shared (clean) Invalid Modified (dirty) write-back caching protocol in black read (miss) write (hit or miss) read (hit or miss) read (hit) or write (hit or miss) write (miss) send invalidate receives invalidate (write by another processor to this block) write-back due to read (miss) by another processor to this block send invalidate signals from the processor coherence additions in red signals from the bus coherence additions in blue

27. .27 Write-Invalidate CC Examples l I = invalid (many), S = shared (many), M = modified (only one) Proc 1 A S Main Mem A Proc 2 A I Proc 1 A S Main Mem A Proc 2 A I Proc 1 A M Main Mem A Proc 2 A I Proc 1 A M Main Mem A Proc 2 A I

28. .28 Write-Invalidate CC Examples l I = invalid (many), S = shared (many), M = modified (only one) Proc 1 A S Main Mem A Proc 2 A I 1. read miss for A 2. read request for A 3. snoop sees read request for A & lets MM supply A 4. gets A from MM & changes its state to S Proc 1 A S Main Mem A Proc 2 A I 1. write miss for A 2. writes A & changes its state to M Proc 1 A M Main Mem A Proc 2 A I 1. read miss for A3. snoop sees read request for A, writes- back A to MM 2. read request for A 4. gets A from MM & changes its state to S 3. P2 sends invalidate for A 4. change A state to I 5. change A state to S Proc 1 A M Main Mem A Proc 2 A I 1. write miss for A 2. writes A & changes its state to M 3. P2 sends invalidate for A 4. change A state to I

29. .29 Block Size Effects  Writes to one word in a multi-word block mean l either the full block is invalidated (write-invalidate) l or the full block is exchanged between processors (write-update) -Alternatively, could broadcast only the written word  Multi-word blocks can also result in false sharing: when two processors are writing to two different variables in the same cache block l With write-invalidate false sharing increases cache miss rates  Compilers can help reduce false sharing by allocating highly correlated data to the same cache block AB Proc1Proc2 4 word cache block

30. .30 Other Coherence Protocols  There are many variations on cache coherence protocols  Another write-invalidate protocol used in the Pentium 4 (and many other micro’s) is MESI with four states: l Modified – same l Exclusive – only one copy of the shared data is allowed to be cached; memory has an up-to-date copy -Since there is only one copy of the block, write hits don’t need to send invalidate signal l Shared – multiple copies of the shared data may be cached (i.e., data permitted to be cached with more than one processor); memory has an up-to-date copy l Invalid – same

31. .31 MESI Cache Coherency Protocol Processor write or read hit Modified (dirty) Invalid (not valid block) Shared (clean) Processor write miss Processor shared read miss Processor write [Send invalidate signal] [Write back block] Processor shared read Invalidate for this block Another processor has read/write miss for this block [Write back block] Exclusive (clean) Processor exclusive read Processor exclusive read miss Processor exclusive read Processor exclusive read miss [Write back block] Processor shared read

32. .32 Commercial Single Backplane Multiprocessors Processor# proc.MHzBW/ system Compaq PLPentium Pro4200540 IBM R40PowerPC81121800 AlphaServerAlpha 21164124402150 SGI Pow ChalMIPS R10000361951200 Sun 6000UltraSPARC301672600

33. .33 Summary  Key questions l Q1 - How do processors share data? l Q2 - How do processors coordinate their activity? l Q3 - How scalable is the architecture (what is the maximum number of processors)?  Bus connected (shared address UMA’s(SMP’s)) multi’s l Cache coherency hardware to ensure data consistency l Synchronization primitives for synchronization l Scalability of bus connected UMAs limited (< ~ 36 processors) because the three desirable bus characteristics -high bandwidth -low latency -long length are incompatible  Network connected NUMAs are more scalable

34. .34 MESI Protocol Dirty Block Copyback Invalidation Read with intent to modify Cache block fill

35. .35 MESI Protocol

36. .36 MESI Protocol P1 cache P2 cache P3 cache P4 cache Main Memory the same cache block in four processors

37. .37 MESI protocol

38. .38 MESI

39. .39 MESI Write Miss