1. IBM RS/6000 SP POWER3 SMP Jari Jokinen Pekka Laurila

2. IBM RS/6000 Series Series of advanced IBM Unix computers. Available from entry level to high-end machines. Allows many different configurations. Current high-end configurations use POWER3 type processors. Shared memory MIMD (Multiple Instructions, Multiple Data) architecture.

3. POWER3 processor 64-bit PowerPC RISC-processor. Symmetric multi-processor. 64-bit addressability and double- word integer operations. Eight execution units. High bandwidth memory with four floating point operations per cycle.

4. POWER3 processor

5. 200MHz processor achieves: 623 MFLOPS 1.1Gbps memory bandwidth The technology allows processor speeds up to 500MHz. Allows complex switch-based configurations to be built using POWER processor.

6. Node architecture SP POWER3 SMP high node architecture. Up to 16 POWER3 processors per node with up to 64 GB of memory. Scalable architecture allowing system from one node up to 512 nodes to be built using same technology.

7. Node architecture Up to four processor cards. Each card has up to four processors. The Node Controller chips for data have 4GB/s bandwidth per processor and 16GB/s bandwidth to the Active Backplane Planar. Memory and I/O functions have 16GB/s bandwidth to Planar. Tree topology inside the node.

8. Node architecture

9. Connecting nodes SP Switch2 is used to connect nodes into a supercomputer. Hardware + software = Communication Subsystem (CSS) Communication path, monitoring of the switch hardware, controling the network, error detection and recovery action. Multistage switching technology

10. SP Switch2 32 links, 16 for nodes and 16 for other switches. For very big networks, switch boards have to be connected together. Also available is an 8 node switch board for those who do not need more than 8 nodes.

11. Switch board

14. Two switch boards connected

15. Topology 2 to 80 nodes can be connected together with maximum of 5 switch boards using star topology. Data passes through, at most, two switch boards.

16. Multiple boards connected

17. 80 to 256 nodes When more than 80 nodes are required, at least six switch boards must be used. Star topology does not allow this. Instead, additional boards must be used as intermediate switch boards.

18. Intermediate switch boards

19. 257 to 512 nodes If more than 257 nodes are required, two frames of switch boards have to be used. 32 NSBs times 16 ISBs equals 512 nodes.

20. Operating System IMB ’ s own AIX operating system is used in the machines based on this architecture. 64-bit Unix system. Each node has its own operating system.

21. NERSC IBM SP RS/6000 Uses SP POWER3 SMP architecture. 208 16-processor nodes where 184 are actual computing nodes, 16 for fileserver (GPFS) with 20 Terabytes of disk space, 3 login nodes, 1 network node and 4 service nodes. 375MHz POWER3 processors.

22. NERSC IBM SP RS/6000 Currently fifth on the top500 supercomputer list. Usage restricted to scientific projects funded by Department of Energy.

23. NERSC IBM SP RS/6000 Peak performance of 4.5TFLOPS Actual benchmarked performance: With MPI, latency between nodes is 17 microseconds, within nodes it is 9 microseconds. Bandwidth between nodes is 60-400MB/s (16-2 nodes), within nodes 250-500MB/s (16-2 processors).

24. NERSC IBM SP RS/6000 MPI programming environment, C/C++, FORTRAN Other compilers: OpenMP, Java for AIX, Assembler for AIX Tools: Parallel Operating Environment (POE) for AIX, LoadLeveler for job queueing, TotalView for debugging, HPMcount for monitoring hardware and Xprofiler for profiling serial and parallel applications. Many mathematical libraries.