1 Toward Petascale Programming and Computing, Challenges and Collaborations Serge G. Petiton PAAP workshop, RIKEN

2 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenges 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

3 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenges 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

4 Introduction (Several) Petascale computing will call for a new ecosystem. Scientific end-users would have to adapt or rewrite software, propose new methods,….as they already have to on the past. Nevertheless, the evolution may be more complex this time. Initiated last year by the French Embassy in Tokyo, the French HPC mission concludes that such challenges would generate more intensive Japanese-French collaborations in this scientific domain and propose to organise a first workshop in 2007 in this subject. Applications, Algorithms, Middleware, Languages, Methods, and Programming Paradigms, for example, have to be proposed for future supercomputer/platforms/cluster/other with respect to several potential architectures.

5 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenge 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

6 Petascale and beyond Approximately another factor 10 has to be obtained to reach Petascale sustained performances. Nevertheless, limitation of the programming paradigms, software and the present ecosystem would probably occur around the 10 Petaflop frontier. We would be face to several complex computer scientific challenges which would have to integrate historical computational scientific end user behaviours since decade (such as languages, sotfware,…). We have to begin to predict and to evaluate the future programming paradigms for end users and to begin to extrapolate what would be systems, languages, algorithmic, arithmetic for future Petascale computers.

7 Parallel Architectures for HPC From vector pipelined architecture to Multi-SMP computer with accelerators, Hybrid, Heterogeneous computers Cluster of clusters, multi-level architectures Inter node communications are often the problems for many applications Power consumptions lead the Moore’s law to a dead end, processor frequencies would not increase as usually in the future. Instead cores will proliferate. Elementary Functions and Floating point arithmetic need to be adapted to the size of the new applications and numerical method stabilities have to increase.

8 GRID for HPC From virtual supercomputers to P2P desktop grids Virtualization of the computing resources for the end users Computational science data mining and data analysis, tools for scientists Multi-parameter task farming programming Mixed-methods, coupled software Multi-physic applications Asynchronous methods, Hybrid methods Fault tolerance, load balancing, security, orchestration, scheduling strategies,….. Communications are very slows compare to the computing resources (what is the reference memory level?)

9 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user versus Petascale Computing 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

10 Parallel and Distributed Computing for Petascale Applications Parallel programming paradigms and distributing computing methodologies merge toward a global large scale programming challenge. From Grid-oriented algorithms to Many-core programming optimizations Fault-tolerance, load-balancing, scheduling strategies, resource orchestration would have to be optimize for Petascale computers It is over to wait 18 months to get double sequential performances, we have to parallelize applications also to obtain accelerations with the many-core processors. From the Grid-level to the Many-core-level, communications would be the difficult criteria to optimize.

11 Several Correlated Challenges Large scale, several dozens of hundreds of thousand of processors Nodes with intra-communication problems SIMD or pipelined (or others) accelerators Many-core processors Low power processor From Grid-level to Core-level communications Integration of parallel computing and distributed computing toward a large scale global computing challenge Network : from worldwide latency to optical network for intra- core communications Multi-arithmetic and accurate normalize elementary functions

12 Hierarchical Computing GRID of supercomputer Supercomputer : clusters (of clusters) of nodes (dozen) Nodes (dozen or hundred of thousand) Processors (dozen, or less) Cores (dozen, hundreds) Threads (dozens) VLIW SIMD K-adic operations…………… What about Systems, Compilers, Languages and Programming. What efficiency?

13 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenges 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

14 The end user will have to program such computers and to obtain accurate results Applications, Numerical Methods Middleware/Language Programming paradigms, Algorithmic Hierarchical programming using component technologies have to be introduced. High level workflow Languages must be developed, each component also need other adapted Languages. End user expertises may be synthetized and communicated thought all the levels

15 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenges 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

16 Methods Krylov based methods for linear sparse linear algebra : Lanczos, Arnoldi, GMRES Dense Linear algebra methods : Block Gauss-Jordan Basic matrix computation : A(Ax+x)+x,…… Hybrid methods (co-methods) : MERAM, GMRES- ARNOLDI-LS. And : QR method and factorization, LS, inverse iterations,…

17 Platforms, Middleware and framework Several parallel computers Clusters in Tsukuba A Japanese-French platform, with dozens of PC in France and clusters in Tsukuba GRID5000 IBM Cell Tsubame (new) OminRPC, Condor, XtremWeb And YML

18 Experimentations Performance evaluation with respect to several parameters, with scheduling strategy emulations and extrapolations Co-methods and hybrid methods. Iterative restarted methods. Out-of-core computing to optimize the computing/communication ratio. …/…. Parallel methods adapted for cluster computing with strategies to save power without losing too much efficiency

19 World-wide grid experiments Experimental platforms, numerical settings Computing and network resources University of Tsukuba –Homogeneous dedicated clusters (Dual Xeon ~3GHz,1 to 4 GB) University of Lille 1 –Heterogeneous NOWs (Celeron 1.4 GHz to P4 3.2 Ghz, 128MB to 1GB) –Shared with students Internet OmniRPC 4 Platforms 2 local platforms: 29 / 58 nodes, Lille 2 world-wide platforms –58 (29 Lille+ 29 Tsukuba dual-proc.) –116 (58 Lille, 58 Tsukuba dual-proc.)

20 Grid'5000 experiments Presentation, motivations Up to 9 sites distributed in France Dedicated PC with reservation policy Fast and dedicated Network –RENATER (1GBit/s to 10GBit/s) PC are homogeneous (few exceptions) Homogeneous environment (deployment strategy) For those experiments Orsay: up to 300 single-CPU nodes Lille: up to 60 single-CPU nodes Nice: up to 60 dual-CPU nodes Rennes: up to 70 dual-CPU nodes

21 Hybrid GMRES(m,v,r)/Arnoldi(q,w,u)-LS(k,l)

22 Les résultats sur Grid5000(II) hybridation pour nG=2,5,8,10 comparee avec GMRES pur (n=1700,nA=2,m(GMRES)=358,m(Arnoldi)=150,k=30,l=30)

23 Les résultats sur Grid5000(Ш) Implantation sur deux sites comparee avec implantation sur un site (n=1700,nG=4,nA=2,m(GMRES)=400,m(Arnoldi)=256,k=10,l=1)

24 Résultats et comparaison (I) P-Time GMRES méthode hybride (n=3060,nA=2,m(GMRES)=400,m(Arnoldi)=50,K=30,L=30) Nombre de processeurs Temps d’exécution

25 Workflow programming for HPC End users have to program such platform They have to use component technologies They want to develop software independently of the middleware The end-user expertises have to be exploited YML propose such framework We experiment with three middleware and evaluate the generated overhead.

26 YML Design

27 Exemple de graphe de composants/tâches Begin node End node Graph node Dependence par compute tache1(..); signal(e1); // compute tache2(..); migrate matrix(..); signal(e2); // wait(e1 and e2); Par compute tache3(..); signal(e3); // compute tache4(..); signal(e4); // compute tache5(..); filter-datacut(..); signal(e5); visualize mesh(…) ; end par // wait(e3 and e4 and e5); compute tache6(..); compute tache7(..); end par Generic component node

28 Outline 1.Introduction 2.Present Problems and Future Architectures 3.From Worldwide GRID to a unique core 4.The End-user and Petascale Computing Challenges 5.Past and Present Researches to evaluate the Petascale Programming 6.Challenges and Collaborations

29 Challenges Many are well-defined but open problems are still numerous, Several of them will be discussed during the workshop, Application behaviors on these future number crushers have to be studied, Middleware and Languages for each level of the hierarchical execution model have to be proposed, and their interoperability well-defined. Basic numerical and linear algebra methods have to be stabilized and/or adapted for larger scale applications, and new ones have to be proposed Last but not least, the programming paradigms for end-users are the key for the future of the computers, and the expertise of the end users have to be exploited.

30 Collaborations are the main goal of this workshop. ………… rendez-vous tomorrow for the “discussions for the next”