1. Parallel Computers Today
Two Nvidia 8800 GPUs
> 1 TFLOPS
Oak Ridge / Cray Jaguar
> 1.75 PFLOPS
Intel 80-core chip
> 1 TFLOPS
TFLOPS = 1012 floating point ops/sec
PFLOPS = 1,000,000,000,000,000 / sec (1015)

2. Pleiades (56320-processor SGI Altix, 544 Teraflops, NASA Ames Research Center)
                  

3. Beowulf (18-processor cluster, lab machine, 2002)

4. AMD Opteron quad-core die

5. The nVidia G80 GPU 128 streaming floating point processors @1.5Ghz
1.5 Gb Shared RAM with 86Gb/s bandwidth
500 Gflop on one chip (single precision)

6. The Computer Architecture Challenge
Most high-performance computer designs allocate resources to optimize Gaussian elimination on large, dense matrices.
Originally, because linear algebra is the middleware of scientific computing.
Nowadays, mostly for bragging rights. U P A L = x

7. Top 500 List

8. Generic Parallel Machine Architecture
Storage
Hierarchy
Proc
Proc
Proc
Cache
Cache
Cache
L2 Cache
L2 Cache
L2 Cache
L3 Cache
L3 Cache
L3 Cache
potential
interconnects
Memory
Memory
Memory
Key architecture question: Where is the interconnect, and how fast?
Key algorithm question: Where is the data?

9. Multicore SMP Systems Intel Clovertown AMD Opteron Sun Niagara2
4MB
Shared L2
Core2
FSB
Fully Buffered DRAM
10.6GB/s
Chipset (4x64b controllers)
10.6 GB/s(write)
21.3 GB/s(read)
Intel Clovertown
AMD Opteron
1MB
victim
Opteron
Memory Controller / HT
DDR2 DRAM
10.6GB/s
4GB/s
(each direction)
Crossbar Switch
Fully Buffered DRAM
4MB Shared L2 (16 way)
42.7GB/s (read), 21.3 GB/s (write)
8K D$
MT UltraSparc
FPU
179 GB/s
(fill)
90 GB/s
(writethru)
Sun Niagara2
4x128b FBDIMM memory controllers
XDR DRAM
25.6GB/s
EIB (Ring Network)
<<20GB/s
each
direction
SPE
256K
PPE
512K L2
MFC
BIF
XDR
IBM Cell Blade
Note, 3 are dual socket SMPs, + the single N2
Note, N2 is 8 threads/core

10. More Detail on GPU Architecture

11. Michael Perrone (IBM): Proper Care and Feeding of Multicore Beasts
1-arch/feeding_the_beast_perrone.pdf

12. Cray XMT (highly multithreaded shared memory)