Journée Présentation de lANR In conjunction with Perpi2006 RenPar'17 / SympA'2006 / CFSE'5 / JC' octobre 2006

2 Teams LIP/INRIA: Projet GRAAL Anne Benoît Raphaël Bolze Yves Caniou Eddy Caron Pushpinder Kaur Chouhan Frédéric Desprez Jean-Sébastien Gay Cédric Tedeschi IRISA/INRIA: Projet PARIS Gabriel Antoniu Luc Bougé Hinde Bouziane Loïc Cudennec Mathieu Jan Sébastien Monnet Christian Perez Thierry Priol LaBRI/INRIA: Projet RUNTIME Olivier Aumage Alexandre Denis ENSEEIHT: IRIT Michel Daydé Marc Pantel Daniel Hagimont CERFACS Eric Maisonnave ENS-Lyon: CRAL Hélène Courtois Julien Devriendt Romain Teyssier

3 Middleware Components Deployment Communications Data management Scheduling The Concept Workflow

4 Components Advanced Component Model Components and data-sharing service Composition based on data-access Data port Use of JuxMem Components and master-worker paradigm Collection + request scheduling Use of Diet Components and workflow Whats mean dependency for the component model? Components and legacy code No code re-writing Mechanism to deal between application and scheduler B A Data data_ref worker master

5 DIET Architecture LA MA LA Server front end Master Agent Local Agent Client MA CORBA or JXTA Middleware

6 Peer Firewall Peer TCP/IP HTTP Peer ID Firewall Toolbox for the development of P2P applications Set of protocols One peer Unique ID Several communication protocols (TCP, HTTP, …) JuxMem: a Grid Data-Sharing Service A peer-to-peer architecture for a data-sharing service in memory Persistence and data coherency mechanism Transparent data localization Data management

7 Communication Brick Communication for multi-paradigm programming model Message passing Remote procedure calls Distributed/Shared memory Cluster view: High-speed network Hardware heterogeneity Myrinet, Quadrics, Infiniband, SCI Gigabit Ethernet Software heterogeneity GM, MX Elan, Elan4 Sisci Sockets Contribution Madeleine library Communications Network Programming environments Generic communication support Message passing Service invocation (RPC, RMI) Madeleine Application processes EthernetMyrinetSCIQuadrics… Distributed shared memory

8 Communications Grid Communication: PadicoTM Grid communications between site Wide communications Specific communications Connectivity: firewalls, none-routed network, etc. Performance: High latency, low bandwidth Security: protection, accounting Middleware and applications integration Middleware upgrading for Madeleine? Existing code? Contribution A high-performance communication framework for Grids: PadicoTM PARIS project ( ) and RUNTIME (since 2004)

9 Scheduling Brick: into DIET Plug-in Scheduler Existing plug-in scheduling facilities Application-specific definition of appropriate performance metrics An extensible measurement system Tuneable comparison/aggregation routines for scheduling Composite requirements enables various selection methods basic resource availability processor speed, memory database contention future requests CORI Collector: an easy interface to gathering performance and load information for a specific SeD Two modules (currently): CoRI-Easy and FAST Possible to extend (new modules): Ganglia, Nagios, R-GMA, Hawkeye, INCA, MDS, … Scheduling

10 Scheduling Brick: Workflow Workflow management using component model Workflow and DIET Simple and high level API for the client Workflow description based on XML Use of different scheduling algorithms (RR, HEFT, etc.) Ability for the client to use its own workflow scheduler Automatic rescheduling mechanisms Support multi-workflows scheduling DIET hierarchy extended with a special agent: MA DAG Two execution modes of the MA DAG Complete scheduling provided : task priorities and resources mapping Partial scheduling provided : only task priorities Workflow Exemple from Cosmological Application

11 Deployment Brick: ADAGE Automatic deployment tool for grid environment Only one command to deploy 3 kinds of input information Resource description application description control parameter Planning model (random, round-robin), … Plug-in for each application Description convector Configuration of application CCM, MPICH-P4, MPICH-G2, JXTA Plug-in: from 400 to 1200 C++ lines Deployment

12 Ocean-atmosphere Numerical Simulations Energy transport: Equator Pole World climate behavior Platform Supercomputer approach large simulation (1000 years) Grid approach parameterization design independent and simultaneous simulations Code coupling ARPEGE v4.5 (atmospheric modelisation) OPA v9 +LIM (ocean modelisation) OASIS v3 (code coupling) Application

13 Cosmological Simulation RAMSES Computes the evolution of dark matter particles starting from the early universe's structure GALICS Performs structure detection (halos of dark matter) Builds the evolution tree of the particles Generates galaxies Application Simulation 1st part, 1 submission from the client: generating low resolution IC RAMSES post-processing with GALICS, results are sent back to the user 2nd part, n submissions from the client: generating high resolution IC centered on the wanted part of the universe RAMSES post-processing with GALICS, results are sent back to the user

14 Application Sparse Direct Solvers Sparse direct solvers in a client-server environment (DIET) Provide remote access to the algorithms we develop (e.g. MUMPS) Easy to use from a light client Data persistency on the servers is crucial Application: an expertise site for sparse linear algebra: ACI GRID TLSE (coordinated by ENSEEIHT-IRIT, Toulouse) On a users specific problem, compare execution time / accuracy / memory usage / … of various solvers: public domain … as well as commercial, sequential … as well as parallel Find best parameter values / reordering heuristics on a given problem Also bibliography, matrix collections, … All elementary requests executed on the/a GRID through DIET Must be highly evolving (new solvers with new parameters, new scenarii)

15 Conclusion Programming model brick Component model Grid middleware brick GridRPC environment: DIET. Data management brick Data-sharing system: JuxMeM Communication bricks Intra-cluster: Madeleine Grid communication: PadicoTM Scheduling brick DIETs Plug-in scheduler Workflow bricks DIETs DAG management Component management Deployment brick ADAGE Applications brick Ocean-atmosphere Numerical Simulations Cosmological Simulation Sparse Direct Solvers

Questions?

17 Workpackages WP1: Applications Responsable: CRAL Équipes impliquées : toutes WP2: Modèles de programmation Responsable: PARIS Équipes impliquées: PARIS, GRAAL, IRIT-TLSE WP3: Étude et modélisation de lordonnancement et du déploiement des applications Responsable: GRAAL Équipes impliquées: PARIS, GRAAL, RUNTIME WP4: Étude et modélisation des communications Responsable: RUNTIME Équipes impliquées: RUNTIME, PARIS, GRAAL WP5: Validation à grande échelle sur la plate-forme expérimentale Grid5000 Responsable: IRIT-TLSE Équipes impliquées: toutes