Claude TADONKI Mines ParisTech – LAL / CNRS / INP 2 P 3 University of Oujda (Morocco) – October 7, 2011 High Performance Computing Challenges and Trends

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI The need of competitive HPC systems Key Factors Large-scale scientific & technical computing (numerical and non numerical) Large-scale data mining and statistics in experimental physics Image and signal processing Video and 3D animations Molecular biology and structural genomic Sorting and pattern matching Meteorology, Atmospheric studies, Medical research Scientific and technical simulation High-standard industrial activities, services, and research investigations Gaming And more … Massively parrallel computers & systems Dedicated architectures Specialized processors Processor frequency High level integration Memory space, latency, and bandwidth High speed interconnection Advances in parallel algorithm synthesis and programming language features Powerful compilers Increasing Need of HPC (range of applications and computing power)

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI Parallel computing easily justifies nowadays ( processor frequency evolution ) HPC interest covers a larger spectra of applications, hence a wider audience Performance expectations in some specific areas are beyond the capacity of standard computers Frequency has been multiplied by 10 since 1993 The number of transistors in Intel proc has been multiplied by … since 1971 Core voltage has been reduced by 10 (1,2V, the min is 0,7V) However, this evolution is closed to its asymptotic threshold !!! Alternatives and trends Multi-core processors TLP-based parallel machines GPU (assume a skillful use!) Hardcoded (embedded) solutions Reconfigurable HPC Grid/Meta Computing Smallpox Research Grid (Oxford University/IBM/United Devices) Oustanding achievment in large-scale molecular analysis. Observations

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI The K Computer RIKEN / Fujitsu ( JAPAN ) Number one the 37th TOP petaflops (Linpack) 93% 68,544 energy-efficient CPUs 672 computer racks Complete deployment in petaflops expected (2012) 800 computer racks (2012) Worldwide shared use

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI The Sequoia System LLNL / IBM ( USA ) BlueGene technology 500 teraFLOPS 1.6 million of cores 96 computer racks 98,304 compute nodes 1.6 petabytes of memory 10 times faster than today’s Most powerful system Sequoia in 1 hour = 6.7 billion people calculating 24h/24h during 320 years Sequoia in 1 hour = 6.7 billion people calculating 24h/24h during 320 years

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI About sustained performance Before we calculate, we need data Time-to-CPU could be long % flops Memory space decreases with level Shared use of memory slowdown Synchronization and data exchange Control flow (if, while, for, case, …) Even an optimal algorithm will run at a fraction of the peak performance !!!

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI Important considerations Energy consumption and dissipation Integration ( more powerful unit/system in a smaller chip/surface ) Total cost of the system and maintenance issues Programmability (consider the case of the IBM CELL) Accessibility (remote and shared use among several entities) Software and tools (system, programming, monitoring, profiling, …) Lifetime and evolution of the system (extensibility, devices change/upgrade, …) Computing nodes interconnection (topology and speed) vital on embedded systems significant heat and cause of failure

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI Fundamental aspects Design of efficient parallel algorithms (modelling & scheduling) Complexity & Performance analysis Algorithmic and programming paradigms Code generation and transformations (automatic parallelization) Compilation techniques

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI Programming models Distributed memory parallel programming (MPI) Shared memory parallel programming (OpenMP, Pthreads) Accelerator-based programming (GPU, FPGA, CELL, …) Vector programming (SSE, VMX, SPU intrinsics, …) Hybrid programming (MPI+OpenMP/Pthread, CPU+GPU, PPU+SPU, …)

University of Oujda (Morocco) – October 7, 2011 High Performance Computing: Challenges and Trends Claude TADONKI