Denis Caromel1 Institut Universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECCOMAS, July 2004 ProActive: Components for Numerical GRIDs 1. The GridS 2. Asynchronous Distributed Objects 3. Hierarchical Components 4. Deployment and Environments 5. Application example: Electromagnetism 6. Conclusion, next step: Real Peer-to-Peer (P2P)

Denis Caromel2 GRIDs

Denis Caromel3 Enterprise Grids Internet EJBServlets Apache Databases

Denis Caromel4 Scientific Grids Internet Clusters Parallel Machine Large Equipment

Denis Caromel5 Internet Grids Internet Job management for embarrassingly parallel application (e.g. SETI)

Denis Caromel6 The multiple GRIDs Scientific Grids Enterprise Grids Intranet and Internet Grids Strong convergence in process! At least at the infrastructure level, i.e. WS Define adequate,sound,programming models Challenges: Composition and coupling of codes

Denis Caromel7 Programming ProActive: an integrated solution for the GRID Composing Deploying W r a p p i n g

Denis Caromel8 Distributed Objects ProActive Programming

Denis Caromel9 Model: Remote Mobile Objects, Group Communications Asynchronous Communications with synchro: automatic Futures OO SPMD, Migration, Non Functional Exceptions (NFE) Environment: XML Deployment, dynamic class-loading, PKI security Various protocols: rsh,ssh,LSF,Globus,BPS,... Graphical Visualization and monitoring: IC2D ProActive: A Java API + Tools for the GRID Parallel, Distributed, Mobile, Activities, across the world ! SMPClustersLAN Desktop

Denis Caromel10 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Cpt. and GRID JVM

Denis Caromel11 OO SPMD A ag = newSPMDGroup (“A”, […], VirtualNode) // In each member myGroup.barrier (“2D”); // Global Barrier myGroup.barrier (“vertival”); // Any Barrier myGroup.waitFor (“north”,”south”,“east”,“west”); A Still, not based on raw messages, but Typed Method Calls ==> Components

Denis Caromel12 Parallel, Distributed, Hierarchical Components for the Grid Composing

Denis Caromel13 A CORBA Component My Business Component Component interface Facets Event sources Event sinks Attributes Receptacles OFFERED REQUIRED Courtesy of Philippe Merle, Lille, OpenCCM platform

Denis Caromel14 Building CCM Applications = Assembling CORBA Component Instances Provide + Use, but flat assembly

Denis Caromel15 Content Controller The Fractal model: Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT

Denis Caromel16 Content Controller Interface = access point

Denis Caromel17 Content Controller Hierarchical model : composites encapsulate primitives encapsulate Java code

Denis Caromel18 Content Controller Component Binding

Denis Caromel19 Binding = at Deployment or Dynamically Content Controller Component = runtime entity if needed

Denis Caromel20 3. Parallel and composite component 1. Primitive component 2. Composite component ProActive Components for the GRID An activity, a process, … potentially in its own JVM C D Composite: Hierarchical, and Distributed over machines Parallel: Composite + Broadcast (group)

Denis Caromel21 Wrapping Legacy MPI Components MPI Code C/Fortran: Messages on Tags sent/converted to Method Calls ProActive Java: Method Calls sent as Messages on Tags Virtual Nodes for Deployments

Denis Caromel22 Environment Deploying

Denis Caromel23 How to deploy on the Various Kind of Grids ? Internet EJBServlets Apache Databases Internet Clusters Parallel Machine Large Equipment Internet Job management for embarrassingly parallel application (e.g. SETI)

Denis Caromel24 Abstract Deployment Model Problem: Difficulties and lack of flexibility in deployment Avoid scripting for: configuration, getting hosts, connecting, etc. Capture Parallel and Distributed nature of codes A key principle: Virtual Node (VN) + XML deployment file Abstract Away from source code: Machines Creation Protocols Lookup and Registry Protocols Protocols and infrastructures: Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF,...

Denis Caromel25 C AB VNaVNb C AB VNc = VN(a,b) XML Deployment (Not in source) Separate or Co-allocation

Denis Caromel26 Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications

Denis Caromel27 ProActive Applications

Denis Caromel28 Jem3D

Denis Caromel29 JEM 3D : Java 3D Electromagnetism together with Said El Kasmi, Stéphane Lanteri (caiman) Maxwell 3D equation solver, Finite Volume Method (FVM) Pre-existing Fortran MPI version: EM3D (CAIMAN INRIA) Up to 294 machines at the same time (Intranet and cluster) Large data sets: 150x150x150 (100 million facets) temps d'exécution de la boucle principale (sur cluster) nombre de processeurs temps (secondes) 21*21*21 31*31*31 43*43*43 55*55*55 81*81*81 97*97*97 113*113* *121*121 taille du maillage

Denis Caromel30 Recent Benchmarks Seq. Java/Fortran: 2 Comparison: Jem3D over - ProActive/RMI Sun - ProActive/RMI Ibis Em3D in - Fortran/MPI On 16 machines: Fortran MPI: 13.8 ProActive/Ibis: 12 ProActive/RMI: 8.8 Grid experiment on 5 clusters (DAS 2): Speed up of 100 on 150 machines

Denis Caromel31 Perspective: Real P2P

Denis Caromel32 Architectures: Server to Peer-To-Peer (P2P) Internet EJBServlets Apache Databases

Denis Caromel33 Pure P2P: Definition Only PEERs, no above everything, top level, server(s) Every peer is, somehow, also a server No master … No slave ! System get organized dynamically, without static configuration Coherent, desired behavior, dynamically emerges

Denis Caromel34 P2P Examples (1)

Denis Caromel35 P2P can be difficult: need to be fault-tolerant, self healing

Denis Caromel36 Not a P2P system

Denis Caromel37 Neither a P2P system

Denis Caromel38 P2P Examples (2)

Denis Caromel39 P2P Examples (2bis)

Denis Caromel40 A P2P system at work Credit: from the movie Atlantis, Luc Besson

Denis Caromel41 Conclusion Available in Open Source ProActive in Application: 3D Electromagnetism on 300 machines at once Goal: Towards a few 1000s ! Summary: Programming: Distributed Objects, OO SPMD Composing: Hierarchical Components Deploying: ssh,Globus,LSF,PBS,…,Web Services

Denis Caromel42 Next ProActive events EuroPar 2004, Pisa, Italy, Monday 30th August, Full Day Tutorial ProActive Tutorial, and Hands-on session User Group & Grid Interoperability, Nice, ETSI, Oct Detailed presentation of the platform User presentations and feedbacks Contest & Grid PLUGTESTS Current Figures: Web Page Hits: ~ 2K-3K / months, Downoad: / month, Users: us, mx, br, cl, ch, it,...