Fourth EGEE Conference Pise, October 23-28, 2005 DEISA Perspectives Towards cooperative extreme computing in Europe Victor Alessandrini IDRIS - CNRS

Slides:



Advertisements
Similar presentations
Network Resource Broker for IPTV in Cloud Computing Lei Liang, Dan He University of Surrey, UK OGF 27, G2C Workshop 15 Oct 2009 Banff,
Advertisements

Forschungszentrum Jülich in der Helmholtz-Gesellschaft December 2006 A European Grid Middleware Achim Streit
1 From Grids to Service-Oriented Knowledge Utilities research challenges Thierry Priol.
Jorge Gasós Grid Technologies Unit European Commission The EU e Infrastructures Programme Workshop, Beijing, June 2005.
"The views expressed in this presentation are those of the author and do not necessarily reflect the views of the European Commission" Mário Campolargo.
1 Ideas About the Future of HPC in Europe “The views expressed in this presentation are those of the author and do not necessarily reflect the views of.
High-Performance Computing
Towards a Virtual European Supercomputing Infrastructure Vision & issues Sanzio Bassini
Dinker Batra CLUSTERING Categories of Clusters. Dinker Batra Introduction A computer cluster is a group of linked computers, working together closely.
FP6−2004−Infrastructures−6-SSA [ Empowering e Science across the Mediterranean ] Grids and their role towards development F. Ruggieri – INFN (EUMEDGRID.
Silicon Graphics, Inc. Poster Presented by: SGI Proprietary Technologies for Breakthrough Research Rosario Caltabiano North East Higher Education & Research.
1 In VINI Veritas: Realistic and Controlled Network Experimentation Jennifer Rexford with Andy Bavier, Nick Feamster, Mark Huang, and Larry Peterson
The LHC Computing Grid – February 2008 The Worldwide LHC Computing Grid Dr Ian Bird LCG Project Leader 15 th April 2009 Visit of Spanish Royal Academy.
Lisbon, August A. Streit DEISA Forschungszentrum Jülich in der Helmholtz-Gesellschaft Achim Streit
1 Ideas About the Future of HPC in Europe “The views expressed in this presentation are those of the author and do not necessarily reflect the views of.
CSC Grid Activities Arto Teräs HIP Research Seminar February 18th 2005.
1 1 st Virtual Forum on Global Research Communities Brussels, 12 th July 2007 Mário Campolargo European Commission - DG INFSO Head of Unit “GÉANT and e-Infrastructures”
Chapter 2 Computer Clusters Lecture 2.1 Overview.
A.V. Bogdanov Private cloud vs personal supercomputer.
Experiences with using UNICORE in Production Grid Infrastructures DEISA and D-Grid Michael Rambadt
EuroCAMP, Malaga, October 19, 2006 DEISA requirements for federations and AA Jules Wolfrat SARA
GGF16 Athens, February DEISA Perspectives Towards cooperative extreme computing in Europe Victor Alessandrini IDRIS - CNRS
1 European policies for e- Infrastructures Belarus-Poland NREN cross-border link inauguration event Minsk, 9 November 2010 Jean-Luc Dorel European Commission.
Computing for ILC experiment Computing Research Center, KEK Hiroyuki Matsunaga.
Seaborg Cerise Wuthrich CMPS Seaborg  Manufactured by IBM  Distributed Memory Parallel Supercomputer  Based on IBM’s SP RS/6000 Architecture.
INFSO-RI Enabling Grids for E-sciencE EGEODE VO « Expanding GEosciences On DEmand » Geocluster©: Generic Seismic Processing Platform.
The John von Neumann Institute for Computing (NIC): A survey of its computer facilities and its Europe-wide computational science activities Norbert Attig.
From GEANT to Grid empowered Research Infrastructures ANTONELLA KARLSON DG INFSO Research Infrastructures Grids Information Day 25 March 2003 From GEANT.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Grid projects in Europe Giuseppe Andronico.
Enabling Grids for E-sciencE ENEA and the EGEE project gLite and interoperability Andrea Santoro, Carlo Sciò Enea Frascati, 22 November.
Results of the HPC in Europe Taskforce (HET) e-IRG Workshop Kimmo Koski CSC – The Finnish IT Center for Science April 19 th, 2007.
Neil Geddes GridPP-10, June 2004 UK e-Science Grid Dr Neil Geddes CCLRC Head of e-Science Director of the UK Grid Operations Support Centre.
RI User Support in DEISA/PRACE EEF meeting 2 November 2010, Geneva Jules Wolfrat/Axel Berg SARA.
Forschungszentrum Jülich in der Helmholtz-Gesellschaft Experiences with using UNICORE in Production Grid Infrastructures DEISA and D-Grid Michael Rambadt.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE – paving the way for a sustainable infrastructure.
INFSO-RI Enabling Grids for E-sciencE Plan until the end of the project and beyond, sustainability plans Dieter Kranzlmüller Deputy.
Introduction to Grid Computing Ed Seidel Max Planck Institute for Gravitational Physics
Leibniz Supercomputing Centre Garching/Munich Matthias Brehm HPC Group June 16.
The Swiss Grid Initiative Context and Initiation Work by CSCS Peter Kunszt, CSCS.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
EU-IndiaGrid (RI ) is funded by the European Commission under the Research Infrastructure Programme WP5 Application Support Marco.
RI The DEISA Sustainability Model Wolfgang Gentzsch DEISA-2 and OGF rzg.mpg.de.
Research Infrastructures Information Day Brussels, March 25, 2003 Victor Alessandrini IDRIS - CNRS.
ESFRI & e-Infrastructure Collaborations, EGEE’09 Krzysztof Wrona September 21 st, 2009 European XFEL.
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
April 10, 2008, Garching Claudio Gheller CINECA The DEISA HPC Grid for Astrophysical Applications.
Identity Management in DEISA/PRACE Vincent RIBAILLIER, Federated Identity Workshop, CERN, June 9 th, 2011.
7. Grid Computing Systems and Resource Management
Comprehensive Scientific Support Of Large Scale Parallel Computation David Skinner, NERSC.
Roadmap to Next Generation Internet: Indian Initiatives Subbu C-DAC, India.
NORDUnet NORDUnet e-Infrastrucure: Grids and Hybrid Networks Lars Fischer CTO, NORDUnet Fall 2006 Internet2 Member Meeting, Chicago.
14, Chicago, IL, 2005 Science Gateways to DEISA Motivation, user requirements, and prototype example Thomas Soddemann, RZG, Germany.
Migrating Desktop Uniform Access to the Grid Marcin Płóciennik Poznan Supercomputing and Networking Center Poland EGEE’08 Conference, Istanbul, 24 Sep.
E-Infrastructure the FP7 prospects Mário Campolargo European Commission - DG INFSO Head of Unit Research Infrastructures TERENA Networking Conference 2006.
1 Kostas Glinos European Commission - DG INFSO Head of Unit, Géant and e-Infrastructures "The views expressed in this presentation are those of the author.
European Perspective on Distributed Computing Luis C. Busquets Pérez European Commission - DG CONNECT eInfrastructures 17 September 2013.
EGEE Workshop on Management of Rights in Production Grids Paris, June 19th, 2006 Victor Alessandrini IDRIS - CNRS DEISA : status, strategies, perspectives.
Page : 1 SC2004 Pittsburgh, November 12, 2004 DEISA : integrating HPC infrastructures in Europe DEISA : integrating HPC infrastructures in Europe Victor.
Monterey HPDC Workshop Experiences with MC-GPFS in DEISA Andreas Schott
DutchGrid KNMI KUN Delft Leiden VU ASTRON WCW Utrecht Telin Amsterdam Many organizations in the Netherlands are very active in Grid usage and development,
EGI-InSPIRE EGI-InSPIRE RI The European Grid Infrastructure Steven Newhouse Director, EGI.eu Project Director, EGI-InSPIRE 29/06/2016CoreGrid.
Scientific Computing at Fermilab Lothar Bauerdick, Deputy Head Scientific Computing Division 1 of 7 10k slot tape robots.
1 MSWG, Amsterdam, December 15, 2005 DEISA security Jules Wolfrat SARA.
Grids and SMEs: Experience and Perspectives Emanouil Atanassov, Todor Gurov, and Aneta Karaivanova Institute for Parallel Processing, Bulgarian Academy.
Bob Jones EGEE Technical Director
Clouds , Grids and Clusters
DEISA : integrating HPC infrastructures in Europe Prof
University of Technology
Presentation transcript:

Fourth EGEE Conference Pise, October 23-28, 2005 DEISA Perspectives Towards cooperative extreme computing in Europe Victor Alessandrini IDRIS - CNRS

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 2 DEISA objectives To enable Europe’s terascale science by the integration of Europe’s most powerful supercomputing systems. Enabling scientific discovery across a broad spectrum of science and technology is the only criterion for success DEISA is an European Supercomputing Service built on top of existing national services. This service is based on the deployment and operation of a persistent, production quality, distributed supercomputing environment with continental scope. The integration of national facilities and services, together with innovative operational models, is expected to add substantial value to existing infrastructures. Main focus is High Performance Computing (HPC).

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 3 The DEISA Supercomputing Environment IBM AIX Super-cluster –FZJ-Julich, 1214 processors, 6,8 teraflops peak –RZG – Garching, 748 processors, 3,8 teraflops peak –IDRIS, 1024 processors, 6.7 teraflops peak –CINECA, 512 processors, 2,6 teraflops peak –CSC, 512 processors, 2,6 teraflops peak –ECMWF, 2 systems of 2276 processors each, 33 teraflops peak BSC, IBM PowerPC Linux system (MareNostrum) 4864 processeurs, 40 teraflops peak SARA, SGI ALTIX Linux system, 1024 processors, 7 teraflops peak LRZ, Linux cluster (2.7 teraflops) moving to SGI ALTIX system (5120 processors and 33 teraflops peak in 2006, 70 teraflops peak in 2007) HLRS, NEC SX8 vector system, 646 processors, 12,7 teraflops peak.

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 4 AIX SUPER-CLUSTER, September 2005 Full production status of dedicated (reserved bandwidth) 1 Gb/s network GPFS : Full production at FZJ, RZG, IDRIS, CINECA; CSC and ECMWF to follow JOB MIGRATION: test status in all sites, production expected in November CSC ECMWF Services Services: High performance datagrid via GPFS Access to remote files use the full available network bandwidth Job migration across sites Used to load balance the global workflow when a huge partition is allocated to a DEISA project in one site Common Production Environment

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 5 HPC and Grid computing Grid computing is not always HPC. Message Passing latencies are boosted in WANs from a few microseconds to millisecond, because the speed of light is not big enough. Deploying tightly coupled parallel applications in large scale grids may not be compatible with high performance requirements Direct Grid computing works best for (almost) embarasingly parallel applications, or coupled softwere modules with limited real time communications. It is better to run large, tightly coupled parallel applications in a single platform. DEISA implements this resuirement by rerouting jobs and balancing the computational workload at a European scale. A co-scheduling service will enable deployment of weakly coupled parallel applications on several platforms

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 6 Heterogeneous Grid services roadmap Extension of GPFS to non-AIX Linus systems. GPFS will work also on the extended Grid. Extensions to SGI Altix is validated. MareNostrum can also be integrated in DEISA’s GPFS Workflow applications. Based on UNICORE plus further extensions coming from EU funded projects. Available today. Co-allocation. Needed to support Grid applications running on the heterogeneous environment. First generation co-allocation service to be implemented by Platform Computing Global data management. Implementing access to distributed data, fast data transfers across sites, hierarchical data management at a continental scale. First services expected in 2006 Science Gateways and Portals; Specific Internet interfaces to hade complex supercomputing environments from end users, and facilitate the access of new, non traditional users communities.

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 7 DEISA Service Activities roadmap DEISA (existing) SA1: Networking SA2: Global File Systems SA3: Middleware SA4: User Support SA5: Security eDEISA (starting operation, not yet EU funded) eSA2: Operations eSA4: Applications Enabling eSA5: Visualization and Portals

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 8 Dedicated network roadmap (secured) FZJ IDRIS RZG CINECA LRZ SARA FZJ IDRIS RZG LRZ CINECA CSC ECMWF SARA BSC HLRS 1 Gb/s 10 Gb/s AIX site Other OS. Today 2006 Today: Six sites connected with dedicated bandwidth at 1 Gb/s Provided by GEANT and NRENs 2006: Ten sites connected at 1 Gb/s Four sites connected at 10 Gb/s (proof of concept network platform)

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 9 Dedicated network roadmap (planned) 2007: DEISA platforms All DEISA computing platforms connected at 10 Gb/s to a central router in Germany. Provided by GN2 and NRENs Star topology ? Supercomputers N x 10 Gb/s Scalable topology, internal backbone, Two or three entry pointd for 10 Gb/s links coming from supercomputers.

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 10 Enabling science Initially, DEISA had an « early users » program: a number of Joint Research Activities integrated in the project from the start As some services in the infrastructure reached production quality, we moved towards « exceptional users » The DEISA Extreme Computing Initiative: identification, deployment and operation of a number of « flagsjip » applications in selected areas of science and technology. Applications are selected on the basis of scientific excellence, innovation potential and relevance criteria (the application must require the extended infrastructure services) European call for proposals: April 1st -> May 30, 2005 (to be repeated evey year) Evaluation Juin -> September projects are starting operation.

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 11 Adapting applications to the infrastructure: the ATASKF Creation, in April 2005, of the Applications Task Force (ATASKF), to support the Extreme Computing Initiative. The ATASKF carries out a prospective action with the European Scientific community. It provides guidance to find the best fit betweed the users requirements and the DEISA supercomputing environment. For accepted projects, the ATASKF takes all the actions needed to adapt and optimize the aplications for efficient operation in the DEISA environment Most demanded actions are: hyperscaling of parallel applications, data management and improved I/O, workflows. We had in Extreme Computing proposals. 29 projects were retained for operation in Full information on DEISA Web server ( after November 8,

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 12 Extreme Computing proposals Bioinformatics 4 Biophysics 3 Astrophysics 11 Fluid Dynamics 6 Materials Sciences 11 Cosmology 3 Climate, Environment 5 Quantum Chemistry 5 Plasma Physics 2 QCD, Quantum computing 3 Profiles of applications in operation in 2005 – 2006 Huge parallel applications running in single remote nodes (dominant) Data Intensive applications of different kinds. Workflows (about 10%)

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 13 Projects from Plasma Physics Extreme Gyrokinetic Turbulence Simulations (related to ITER project) The nonlinear particle-in-cell code TORB uses a Monte Carlo particle approach to simulate the time evolution of turbulent field structures in fusion plasmas (J. Nuehrenberg, IPP, Greifswald & L. Villard, CRPP, Lausanne) Within DEISA, TORB has been improved for extreme scalability at IBM system at ECMWF: On 2048 procs: Speedup = 1680 Parallel efficiency = 82% Sustained performance = 1.3 TF 64 nodes = 2048 processors

Fourth EGEE Conference Pise, October , 2005 V. Alessandrini, IDRIS-CNRS 14 Conclusions DEISA adopts Grid technologies to integrate national supercomputing infrastructures, and to provide an European Supercomputing Service. Service activities are supported by the coordinated action of the national center's staffs. DEISA operates as a virtual European supercomputing centre. The big challenge we are facing is enabling new, first class computational science. DEISA aims at deploying a persistent, basic European infrastructure for general purpose high performance computing. Interfaced with other grid-enabled complementary infrastructures, DEISA expects to contribute to a global European eInfrastructure for science and technology Integrating leading supercomputing platforms with Grid technologies may enable new research dimensions in Europe.