Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 1 High Performance Computing Helvi Hartmann FIAS Inverted CERN School of Computing, February 2015

Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS What is High Performance Computing?

Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory

Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS Why do we need it?

Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Example: Weather Forecast

Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Big Data

Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Why not make a single fast computer? processing operation/s instead of 10 9 operation/s

Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns memory 27cycles + 40ns = 148ns

Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns L1 Cache 4 cycles = 16ns L2 Cache 12 cycles = 48ns L3 Cache 27 cycles = 108ns memory = 148ns Caches —> located directly next to the CPU to reduce access time - small space available - expensive in power usage

Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Moore‘s Law Amount of transistor on a chip will double every second year —> „computerspeed doubles every second year“ Use of multiprocessors

Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Microarchitecture PipeliningMultithreadingPrefetching CachingOut-of-order execution Vectorization Only small gains as compared to if Moore‘s law remained valid

Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS Shared Memory

Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall

Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency

Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency Expensive!!!

Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS What is possible except improving the Microarchitecture?

Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS Cluster Computing

Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS Distributed Memory

Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS Shared Memory Cheap Standard processor very powerful Scalable High Latencies

Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS High Latencies? Detour: Network Topologies

Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS Does it work any easier?

Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS switches

Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS Parallelization

Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS Amdahl‘s Law sequential part limits speedup significantly

Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS Parallelization While data points increase by a factor 1000 communication grows by 10

Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS Parallelization

Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS History of parallel computing

Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing 1994 NASA: Goddard Space Flight Center calculation of 2D streaming properties 16 Linux PC’s Beowulf Cluster

Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing Linux free and open software stability and portability PVM free and open standard

Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing search for extraterrestrial intelligence analyse data of radioteleskopes for extraterrestrial signals million people shared computing time April TFlops, fastest computer 70,7TFlops, second fastest 51,8TFlops

Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS CBM - FLES CBM untriggered Experiment Datarate of 1TB/s 1000 input nodes

Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS Conclusion

Message Passing Computing 38 iCSC2015,Helvi Hartmann, FIAS : far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/ Llano html itter_globus_ html