1. Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 1 High Performance Computing Helvi Hartmann FIAS Inverted CERN School of Computing, 23-24 February 2015

2. Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS What is High Performance Computing?

3. Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory

4. Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS Why do we need it?

5. Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Example: Weather Forecast

6. Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

7. Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

8. Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Big Data

9. Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Why not make a single fast computer? processing 10 16 operation/s instead of 10 9 operation/s

10. Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns memory 27cycles + 40ns = 148ns

11. Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns L1 Cache 4 cycles = 16ns L2 Cache 12 cycles = 48ns L3 Cache 27 cycles = 108ns memory = 148ns Caches —> located directly next to the CPU to reduce access time - small space available - expensive in power usage

12. Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Moore‘s Law Amount of transistor on a chip will double every second year —> „computerspeed doubles every second year“ Use of multiprocessors

13. Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Microarchitecture PipeliningMultithreadingPrefetching CachingOut-of-order execution Vectorization Only small gains as compared to if Moore‘s law remained valid

14. Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS Shared Memory

15. Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall

16. Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency

17. Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS

18. Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency Expensive!!!

19. Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS What is possible except improving the Microarchitecture?

20. Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS Cluster Computing

21. Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS Distributed Memory

22. Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS Shared Memory Cheap Standard processor very powerful Scalable High Latencies

23. Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS High Latencies? Detour: Network Topologies

24. Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS

25. Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS Does it work any easier?

26. Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS switches

27. Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS Parallelization

28. Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS Amdahl‘s Law sequential part limits speedup significantly

29. Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS Parallelization While data points increase by a factor 1000 communication grows by 10

30. Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS Parallelization

31. Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS History of parallel computing

32. Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing 1994 NASA: Goddard Space Flight Center calculation of 2D streaming properties 16 Linux PC’s Beowulf Cluster

33. Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS

34. Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing Linux free and open software stability and portability PVM free and open standard

35. Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing search for extraterrestrial intelligence analyse data of radioteleskopes for extraterrestrial signals 1999 5 million people shared computing time April 2005 60TFlops, fastest computer 70,7TFlops, second fastest 51,8TFlops SETI@home

36. Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS CBM - FLES CBM untriggered Experiment Datarate of 1TB/s 1000 input nodes

37. Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS Conclusion

38. Message Passing Computing 38 iCSC2015,Helvi Hartmann, FIAS 05.11.2014: http://creatia2013.wordpress.com/2013/03/12/dna-is-like-a-computer-program-but- far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/ http://www.lhc-facts.ch/index.php?page=datenverarbeitung http://www.computerhistory.org/fellowawards/hall/bios/Gene,Amdahl/ http://www.heise.de/newsticker/meldung/AMD-bringt-neuen-Notebook-Prozessor- Llano-1259681.html http://en.wikipedia.org/wiki/TBR1 http://www.adpic.de/lizenzfreie_bilder/Geld%20_und_%20Boerse/Euro/goldener_g itter_globus_253863.html