1. High Performance Computing G Burton – ICG – Oct12 – v1.1 1

2. Agenda Commodity Clusters Compute Pool Interconnects and Networks Shared Storage Login Layer and Middleware

3. HPC alternatives

4. HPC Facts IBM Sequoia - Number 1 in top 500 with 1.572 million Cores. 20 petaFLOPS ( Floating Point Operations / Second ). Sciama 10 teraFLOPS. China has Number 5 plus 62 other systems in top 500, now ahead of Germany, UK, Japan and France. 75% of Top 500 use Intel processors.

5. Demystifying the Techno Babble

6. Demystifying the Techno Babble (2)

7. Commodity Clusters 7

8. Made from commodity (off-the-shelf) components (read PC’s). Consequently (relatively) cheap. Usually Linux based High availability storage (no single point of failure) Generic compute pool (cloned servers that can easily be replaced).

9. Cluster Concept

10. 10 Compute Pool - Just a bunch of PC’s

11. In the “good-ol-days” things were simple ………. 11

12. In the “good-ol-days” things were simple ………. 12

13. … these days much more packed into the same space … but basically the same! 13 These are the building blocks of HPC similar to Sciama

14. Total ICG Compute Pool > 1000 Cores 14

15. Coke or Pepsi – Chalk & Cheese Only 2 remaining commodity CPU makers are Intel and AMD. Latest AMD “Bulldozer” architecture competing with Intel “Sandy Bridge” Architecture. Both architectures are multi core ( Intel 48 cores) Architectures use same memory / video cards / hard drives etc CPU speed constraints down to on-chip transmission delays and heat dissipation ( 22nm ).

16. Intel-AMD – Bangs per Buck

17. Graphical Processing Units (GPU’s actually GPGPU’s – General purpose ) CPU’s still in charge Special programming language. CUDA and OpenCL Three players:- Intel AMD Nvidia Cpu – multiple cores Gpu – 100’s of cores

18. Interconnects and Networks 18

19. Interconnects and Networks Moving away from the Processor towards the Internet you get slower and slower due to Increased Latency and Reduced Bandwidth

20. Processor Interconnects For processor running at 3.2GHz – QPI Bus would be 25GBytes / Second ( Kindle version of “War & Peace” is 2GBytes)

21. Peripheral Component Interconnect Express (PCIe) PCI bus is interconnect to the outside world

22. External Networks Interconnects are Parallel – Bytes / second Networks are serial – bits / second (shown here in B/s for comparison – eg. DAS is 6Gb/s)

23. Sciama Network

24. Sciama Traffic

25. Connecting to Sciama

26. 26 Shared Storage

27. Raw Disks are Dumb Remember: PATA, IDE (Advanced Technology Attachment)

28. Intelligence is in the File System

29. HPC’s Require many disks

30. Use High Capacity Arrays

31. HPC’s require large chucks of storage

32. Many RAID options 1-6 / 10 /50

33. Directly Attached Storage (DAS)

34. Of limited use as cannot be shared.

35. Network Attached Storage (NAS)

36. NAS or Network Appliance

37. Network BW is often the bottleneck

38. NAS - Lustre File System Lustre is an example of a distributed file system. There are many more. Sometimes called a “Cluster” file system

39. NAS – Lustre Often used with an Infiniband fabric.

40. Storage Area Networks

41. Storage Area Network using iSCSI

42. Fibre Channel High Availability SAN

43. Sciama NAS Storage

44. Sciama Storage Hardware Storage is expensive. 250gbp / Tbytes No Backup

45. Highly Available Hardware

46. Two paths to most components

47. Additional Sciama Storage /mnt/astro(1-5) 10 GbE

48. Login Layer and Middleware 48

49. Why Login Servers Login servers will provide the gateway to the cluster. Users can remotely login into the servers using “ssh” or a Remote Desktop Client. A desktop client gives a full working desktop in the environment (can full screen)

50. Some users are at remote locations.. 50

51. Use of Remote Login Client 51

52. Executable and Jobscript setup in Login Layer

53. Jobs submitted to the queues > qsub –q Queue1 run_script.sh

54. Scheduler prioritises and deems a job ready to run. Job passed to resource manager

55. Resource manager (Torque) checks for available resources.

56. Job either runs in the compute pool or returns to queue