1. Tools and Utilities for parallel and serial codes in ENEA-GRID environment CRESCO Project: Salvatore Raia SubProject I.2 C.R. ENEA-Portici. 11/12/2007

2.  GRID, cluster and parallel Computing (Intro)  ENEA-GRID. Architecture and functionality  My Activity for CRESCO project and results on ENEA-GRID  Conclusion and objectives C.R. ENEA-Portici. 11/12/2007OUTLINE:

3. What is a cluster ? -Collection of resources (HW, SW) connected via public or private network - Each CPU runs a separated istance of operating system -Administration: local cluster 1 cluster 1supercomputer Supercomputer= computer with many processors connected via high-speed computer bus and that share the memory (SMP). It runs one Operating system

4. - Collection of interconnected clusters geographically distributed - administration: sometimes clusters belong to different department or company cluster N cluster 3 cluster 2 cluster 1 C.R. ENEA-Portici. 11/12/2007 How to get a Grid ? GRID 1 GRID 1 GRID = nodes made of clusters and each node may have Shared or Distributed memory architectures (Hybrid ) that share processes. ENEA-GRIDENEA-GRID has the same structure ENEA-GRID With 6 clusters: Bologna, Casaccia, Frascati, Portici, Trisaia, Brindisi

5. ENEA-GRID structure (HW) C.R. ENEA-Portici. 11/12/2007

6. GRID features  Pro:  Shared resources  Low costs (clock ?)  Open systems  Scalability  Con:  Several platforms  Load balancing  User Access C.R. ENEA-Portici. 11/12/2007 How is it managed on How is it managed on ENEA-GRID ? ENEA-GRID ? Frequency scaling (domain ?) Power consumption P=C×V×V×F

7. ENEA-GRID structure (SW) C.R. ENEA-Portici. 11/12/2007 ICA client ICA client Resources management Resources management File System File System Operating Systems Operating Systems

8. C.R. ENEA-Portici. 11/12/2007 User Interface USER ACCESS  ICA client  ssh o telnet  web Switch host Switch host Run Appl. Run Appl. Jobs status Jobs status

9. Problem with:  Multi platforms  Load balancing  User Access C.R. ENEA-Portici. 11/12/2007 How to cope with ? How to cope with ? My activity on ENEA-GRID (CRESCO pr.) Serial and Parallel (MPI) codes User interfaces User interfaces LSF utilities LSF utilities Software dev. Software dev.

10. C.R. ENEA-Portici. 11/12/2007 Tools for Serial and Parallel (MPI) codes  Serial codes Compilers Compilers  GNU  PGI  IBM  Parallel codes (MPI) MPI Implementations MPI Implementations  MPICH  LAM-MPI  POE Multi Platform Problems with execution too …tools …So we need a lots of binaries for each platform. Launcher: after compiling our source code in each platform, we have “binary1”…”binaryN” for host1,…hostN. It is a shell script (placed on AFS) that selects the righteous “binary” for the selected host

11. C.R. ENEA-Portici. 11/12/2007 Some MPI problems

12.  SERIAL  Program for Fortran 77/90,C and C++ serial compiling (look Java Interface) (look Java Interface)  Launcher for “NS2” application (use external libraries)  PARALLEL (MPI)  Launcher for running a test program (check command)  Launcher for HPL test on AIX and Linux C.R. ENEA-Portici. 11/12/2007 Results: tools serial and parallel (MPI) codes user1 installation user2 installation

13. C.R. ENEA-Portici. 11/12/2007 Analizing LSF utilities Serial and Parallel codes  Serial codes  Resources definition  “NS2” application  Serial LSF utilities  Job array (Multicase)  “lsgrun”  Parallel codes (MPI)  Parallel LSF utilities  “mpijob” (MPICH)  “poejob” (POE) LSF Resources No correlation Correlation

14. C.R. ENEA-Portici. 11/12/2007 Results: Integration with other application  Serial codes  Parallel codes (MPI) (My)Java Interface

15. C.R. ENEA-Portici. 11/12/2007 Conclusion and objectives  Launcher + LSF utilities + User interface allow to create a omogeneous environment  Objectives:  Optimization of programs to launch serial and parallel codes, including checking resources to run the application (e.g. library, other programs, etc)  Exploitation of LSF utilities in order to make easy running MPI programs (mpijob, poejob, etc) and load balancing  Improve error handling for user interfaces … …

16. Andrew File System C.R. ENEA-Portici. 11/12/2007

17. LSF-Load Sharing Facilities