2. Denis Caromel1 Institut Universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECCOMAS, July 2004 ProActive: Components for Numerical GRIDs 1. The GridS 2. Asynchronous Distributed Objects 3. Hierarchical Components 4. Deployment and Environments 5. Application example: Electromagnetism 6. Conclusion, next step: Real Peer-to-Peer (P2P)

3. Denis Caromel2 GRIDs

4. Denis Caromel3 Enterprise Grids Internet EJBServlets Apache Databases

5. Denis Caromel4 Scientific Grids Internet Clusters Parallel Machine Large Equipment

6. Denis Caromel5 Internet Grids Internet Job management for embarrassingly parallel application (e.g. SETI)

7. Denis Caromel6 The multiple GRIDs Scientific Grids Enterprise Grids Intranet and Internet Grids Strong convergence in process! At least at the infrastructure level, i.e. WS Define adequate,sound,programming models Challenges: Composition and coupling of codes

8. Denis Caromel7 Programming ProActive: an integrated solution for the GRID Composing Deploying W r a p p i n g

9. Denis Caromel8 Distributed Objects ProActive Programming

10. Denis Caromel9 Model: Remote Mobile Objects, Group Communications Asynchronous Communications with synchro: automatic Futures OO SPMD, Migration, Non Functional Exceptions (NFE) Environment: XML Deployment, dynamic class-loading, PKI security Various protocols: rsh,ssh,LSF,Globus,BPS,... Graphical Visualization and monitoring: IC2D ProActive: A Java API + Tools for the GRID Parallel, Distributed, Mobile, Activities, across the world ! SMPClustersLAN Desktop

11. Denis Caromel10 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Cpt. and GRID JVM

12. Denis Caromel11 OO SPMD A ag = newSPMDGroup (“A”, […], VirtualNode) // In each member myGroup.barrier (“2D”); // Global Barrier myGroup.barrier (“vertival”); // Any Barrier myGroup.waitFor (“north”,”south”,“east”,“west”); A Still, not based on raw messages, but Typed Method Calls ==> Components

13. Denis Caromel12 Parallel, Distributed, Hierarchical Components for the Grid Composing

14. Denis Caromel13 A CORBA Component My Business Component Component interface Facets Event sources Event sinks Attributes Receptacles OFFERED REQUIRED Courtesy of Philippe Merle, Lille, OpenCCM platform

15. Denis Caromel14 Building CCM Applications = Assembling CORBA Component Instances Provide + Use, but flat assembly

16. Denis Caromel15 Content Controller The Fractal model: Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT

17. Denis Caromel16 Content Controller Interface = access point

18. Denis Caromel17 Content Controller Hierarchical model : composites encapsulate primitives encapsulate Java code

19. Denis Caromel18 Content Controller Component Binding

20. Denis Caromel19 Binding = at Deployment or Dynamically Content Controller Component = runtime entity if needed

21. Denis Caromel20 3. Parallel and composite component 1. Primitive component 2. Composite component ProActive Components for the GRID An activity, a process, … potentially in its own JVM C D Composite: Hierarchical, and Distributed over machines Parallel: Composite + Broadcast (group)

22. Denis Caromel21 Wrapping Legacy MPI Components MPI Code C/Fortran: Messages on Tags sent/converted to Method Calls ProActive Java: Method Calls sent as Messages on Tags Virtual Nodes for Deployments

23. Denis Caromel22 Environment Deploying

24. Denis Caromel23 How to deploy on the Various Kind of Grids ? Internet EJBServlets Apache Databases Internet Clusters Parallel Machine Large Equipment Internet Job management for embarrassingly parallel application (e.g. SETI)

25. Denis Caromel24 Abstract Deployment Model Problem: Difficulties and lack of flexibility in deployment Avoid scripting for: configuration, getting hosts, connecting, etc. Capture Parallel and Distributed nature of codes A key principle: Virtual Node (VN) + XML deployment file Abstract Away from source code: Machines Creation Protocols Lookup and Registry Protocols Protocols and infrastructures: Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF,...

26. Denis Caromel25 C AB VNaVNb C AB VNc = VN(a,b) XML Deployment (Not in source) Separate or Co-allocation

27. Denis Caromel26 Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications

28. Denis Caromel27 ProActive Applications

29. Denis Caromel28 Jem3D

30. Denis Caromel29 JEM 3D : Java 3D Electromagnetism together with Said El Kasmi, Stéphane Lanteri (caiman) Maxwell 3D equation solver, Finite Volume Method (FVM) Pre-existing Fortran MPI version: EM3D (CAIMAN team @ INRIA) Up to 294 machines at the same time (Intranet and cluster) Large data sets: 150x150x150 (100 million facets) temps d'exécution de la boucle principale (sur cluster) 0 100 200 300 400 500 600 700 800 900 010203040506070 nombre de processeurs temps (secondes) 21*21*21 31*31*31 43*43*43 55*55*55 81*81*81 97*97*97 113*113*113 121*121*121 taille du maillage

31. Denis Caromel30 Recent Benchmarks Seq. Java/Fortran: 2 Comparison: Jem3D over - ProActive/RMI Sun - ProActive/RMI Ibis Em3D in - Fortran/MPI On 16 machines: Fortran MPI: 13.8 ProActive/Ibis: 12 ProActive/RMI: 8.8 Grid experiment on 5 clusters (DAS 2): Speed up of 100 on 150 machines

32. Denis Caromel31 Perspective: Real P2P

33. Denis Caromel32 Architectures: Server to Peer-To-Peer (P2P) Internet EJBServlets Apache Databases

34. Denis Caromel33 Pure P2P: Definition Only PEERs, no above everything, top level, server(s) Every peer is, somehow, also a server No master … No slave ! System get organized dynamically, without static configuration Coherent, desired behavior, dynamically emerges

35. Denis Caromel34 P2P Examples (1)

36. Denis Caromel35 P2P can be difficult: need to be fault-tolerant, self healing

37. Denis Caromel36 Not a P2P system

38. Denis Caromel37 Neither a P2P system

39. Denis Caromel38 P2P Examples (2)

40. Denis Caromel39 P2P Examples (2bis)

41. Denis Caromel40 A P2P system at work Credit: from the movie Atlantis, Luc Besson

42. Denis Caromel41 Conclusion Available in Open Source ProActive http://ProActive.ObjectWeb.org in Application: 3D Electromagnetism on 300 machines at once Goal: Towards a few 1000s ! Summary: Programming: Distributed Objects, OO SPMD Composing: Hierarchical Components Deploying: ssh,Globus,LSF,PBS,…,Web Services

43. Denis Caromel42 Next ProActive events EuroPar 2004, Pisa, Italy, Monday 30th August, Full Day Tutorial ProActive Tutorial, and Hands-on session User Group & Grid Interoperability, Nice, ETSI, Oct. 18-20 Detailed presentation of the platform User presentations and feedbacks Contest & Grid PLUGTESTS Current Figures: Web Page Hits: ~ 2K-3K / months, Downoad: 150-300 / month, Users: us, mx, br, cl, ch, it,...