Denis Caromel1 Institut Universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECCOMAS, July 2004 ProActive: Components for Numerical GRIDs 1. The GridS 2. Asynchronous Distributed Objects 3. Hierarchical Components 4. Deployment and Environments 5. Application example: Electromagnetism 6. Conclusion, next step: Real Peer-to-Peer (P2P)
Denis Caromel2 GRIDs
Denis Caromel3 Enterprise Grids Internet EJBServlets Apache Databases
Denis Caromel4 Scientific Grids Internet Clusters Parallel Machine Large Equipment
Denis Caromel5 Internet Grids Internet Job management for embarrassingly parallel application (e.g. SETI)
Denis Caromel6 The multiple GRIDs Scientific Grids Enterprise Grids Intranet and Internet Grids Strong convergence in process! At least at the infrastructure level, i.e. WS Define adequate,sound,programming models Challenges: Composition and coupling of codes
Denis Caromel7 Programming ProActive: an integrated solution for the GRID Composing Deploying W r a p p i n g
Denis Caromel8 Distributed Objects ProActive Programming
Denis Caromel9 Model: Remote Mobile Objects, Group Communications Asynchronous Communications with synchro: automatic Futures OO SPMD, Migration, Non Functional Exceptions (NFE) Environment: XML Deployment, dynamic class-loading, PKI security Various protocols: rsh,ssh,LSF,Globus,BPS,... Graphical Visualization and monitoring: IC2D ProActive: A Java API + Tools for the GRID Parallel, Distributed, Mobile, Activities, across the world ! SMPClustersLAN Desktop
Denis Caromel10 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Cpt. and GRID JVM
Denis Caromel11 OO SPMD A ag = newSPMDGroup (“A”, […], VirtualNode) // In each member myGroup.barrier (“2D”); // Global Barrier myGroup.barrier (“vertival”); // Any Barrier myGroup.waitFor (“north”,”south”,“east”,“west”); A Still, not based on raw messages, but Typed Method Calls ==> Components
Denis Caromel12 Parallel, Distributed, Hierarchical Components for the Grid Composing
Denis Caromel13 A CORBA Component My Business Component Component interface Facets Event sources Event sinks Attributes Receptacles OFFERED REQUIRED Courtesy of Philippe Merle, Lille, OpenCCM platform
Denis Caromel14 Building CCM Applications = Assembling CORBA Component Instances Provide + Use, but flat assembly
Denis Caromel15 Content Controller The Fractal model: Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT
Denis Caromel16 Content Controller Interface = access point
Denis Caromel17 Content Controller Hierarchical model : composites encapsulate primitives encapsulate Java code
Denis Caromel18 Content Controller Component Binding
Denis Caromel19 Binding = at Deployment or Dynamically Content Controller Component = runtime entity if needed
Denis Caromel20 3. Parallel and composite component 1. Primitive component 2. Composite component ProActive Components for the GRID An activity, a process, … potentially in its own JVM C D Composite: Hierarchical, and Distributed over machines Parallel: Composite + Broadcast (group)
Denis Caromel21 Wrapping Legacy MPI Components MPI Code C/Fortran: Messages on Tags sent/converted to Method Calls ProActive Java: Method Calls sent as Messages on Tags Virtual Nodes for Deployments
Denis Caromel22 Environment Deploying
Denis Caromel23 How to deploy on the Various Kind of Grids ? Internet EJBServlets Apache Databases Internet Clusters Parallel Machine Large Equipment Internet Job management for embarrassingly parallel application (e.g. SETI)
Denis Caromel24 Abstract Deployment Model Problem: Difficulties and lack of flexibility in deployment Avoid scripting for: configuration, getting hosts, connecting, etc. Capture Parallel and Distributed nature of codes A key principle: Virtual Node (VN) + XML deployment file Abstract Away from source code: Machines Creation Protocols Lookup and Registry Protocols Protocols and infrastructures: Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF,...
Denis Caromel25 C AB VNaVNb C AB VNc = VN(a,b) XML Deployment (Not in source) Separate or Co-allocation
Denis Caromel26 Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications
Denis Caromel27 ProActive Applications
Denis Caromel28 Jem3D
Denis Caromel29 JEM 3D : Java 3D Electromagnetism together with Said El Kasmi, Stéphane Lanteri (caiman) Maxwell 3D equation solver, Finite Volume Method (FVM) Pre-existing Fortran MPI version: EM3D (CAIMAN INRIA) Up to 294 machines at the same time (Intranet and cluster) Large data sets: 150x150x150 (100 million facets) temps d'exécution de la boucle principale (sur cluster) nombre de processeurs temps (secondes) 21*21*21 31*31*31 43*43*43 55*55*55 81*81*81 97*97*97 113*113* *121*121 taille du maillage
Denis Caromel30 Recent Benchmarks Seq. Java/Fortran: 2 Comparison: Jem3D over - ProActive/RMI Sun - ProActive/RMI Ibis Em3D in - Fortran/MPI On 16 machines: Fortran MPI: 13.8 ProActive/Ibis: 12 ProActive/RMI: 8.8 Grid experiment on 5 clusters (DAS 2): Speed up of 100 on 150 machines
Denis Caromel31 Perspective: Real P2P
Denis Caromel32 Architectures: Server to Peer-To-Peer (P2P) Internet EJBServlets Apache Databases
Denis Caromel33 Pure P2P: Definition Only PEERs, no above everything, top level, server(s) Every peer is, somehow, also a server No master … No slave ! System get organized dynamically, without static configuration Coherent, desired behavior, dynamically emerges
Denis Caromel34 P2P Examples (1)
Denis Caromel35 P2P can be difficult: need to be fault-tolerant, self healing
Denis Caromel36 Not a P2P system
Denis Caromel37 Neither a P2P system
Denis Caromel38 P2P Examples (2)
Denis Caromel39 P2P Examples (2bis)
Denis Caromel40 A P2P system at work Credit: from the movie Atlantis, Luc Besson
Denis Caromel41 Conclusion Available in Open Source ProActive in Application: 3D Electromagnetism on 300 machines at once Goal: Towards a few 1000s ! Summary: Programming: Distributed Objects, OO SPMD Composing: Hierarchical Components Deploying: ssh,Globus,LSF,PBS,…,Web Services
Denis Caromel42 Next ProActive events EuroPar 2004, Pisa, Italy, Monday 30th August, Full Day Tutorial ProActive Tutorial, and Hands-on session User Group & Grid Interoperability, Nice, ETSI, Oct Detailed presentation of the platform User presentations and feedbacks Contest & Grid PLUGTESTS Current Figures: Web Page Hits: ~ 2K-3K / months, Downoad: / month, Users: us, mx, br, cl, ch, it,...