2. Denis Caromel1 Denis Caromel, Romain Quilici www.inria.fr/oasis/ProActive OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis Albufeira, Portugal 14-19 June 2003 1. Distributed and Asynchronous Objects, Groups 2. Components, XML Deployment 3. Graphical Interface (IC2D) 4. Towards PSE ProActive: From Active Objects to Components, Towards PSE

3. Denis Caromel2 PSEs: High Level View PSE = User Interface + libraries + knowledge base + integration (Rice, Boisvert 1996) Needs for scalability, software reuse, cost effective, flexibility, extensibility, …(non exhaustive) Must cope with heterogeneous languages, environment, resources, …(non exhaustive) ProActive, with its programming and component model, interactive environment could be one of the brick towards PSEs

4. Denis Caromel3 A uniform framework: An Active Object pattern A formal model behind: Prop. Determinism, insensitivity to deploy. Main features: Remotely accessible Objects ( RMI, JINI, --> UDDI, -->LDAP ) Asynchronous Communications with synchro: automatic Futures Group Communications, Migration (mobile computations) XML Deployment Descriptors Interfaced with various protocols: rsh,ssh,LSF,Globus, --> SOAP Graphical Visualization and monitoring: IC2D In the www. ObjectWeb.org Consortium (Open Source middleware) since April 2002 (LGPL license) ProActive: A Java API + Tools for Parallel, Distributed Computing

5. Denis Caromel4 ProActive : model Active objects : coarse-grained structuring entities (subsystems) Each active object: - possibly owns many passive objects - has exactly one thread. No shared passive objects -- Parameters are passed by deep-copy Asynchronous Communication between active objects Future objects and wait-by-necessity. Full control to serve incoming requests (reification)

6. Denis Caromel5 An object created with A a = new A (obj, 7); can be turned into an active and remote object: Instantiation-based: A a = (A)ProActive.newActive(«A», params, node); The most general case. To get Class-based: a static method as a factory To get a non-FIFO behavior (Class-based) : class pA extends A implements RunActive { … } Object-based: A a = new A (obj, 7);... a = (A)ProActive.turnActive (a, node); ProActive : Creating active objects

7. Denis Caromel6 Two key features: Polymorphism between standard and active objects Type compatibility for classes (and not only interfaces) Needed and done for the future objects also Dynamic mechanism (dynamically achieved if needed) Wait-by-necessity: inter-object synchronization Systematic, implicit and transparent futures Ease the programming of synchronizations, and the reuse of routines "A" "pA" a p_a foo (A a) { a.g (...); v = a.f (...);... v.bar (...); } ProActive: Reuse and seamless

8. Denis Caromel7 ProActive: Reuse and seamless Two key features: Polymorphism between standard and active objects Type compatibility for classes (and not only interfaces) Needed and done for the future objects also Dynamic mechanism (dynamically achieved if needed) Wait-by-necessity: inter-object synchronization Systematic, implicit and transparent futures (“value to come”) Ease the programming of synchronizations, and the reuse of routines "A" "pA" a p_a foo (A a) { a.g (...); v = a.f (...);... v.bar (...); } O.foo(a) : a.g() and a.f() are « local » O.foo(p_a): a.g() and a.f()are «remote + Async.» O

9. Denis Caromel8 ProActive Groups: Collective Communications Typed and polymorphic Groups of active and remote objects Dynamic generation of group of results Language centric, Dot notation Manipulate groups of Active Objects, in a simple and typed manner: Be able to express high-level collective communications (like in MPI): broadcast, scatter, gather, all to all A ag=(A)ProActiveGroup.newActiveGroup(«A»,{{p1},...},{Nodes,..}); V v = ag.foo(param); v.bar();

10. Denis Caromel9 Construction of a Result Group Typed Group Java or Active Object A ag = newActiveGroup (…) V v = ag.foo(param); v.bar(); V A

11. Denis Caromel10 ProActive : Mobility of active objects Migration is initiated by the active object itself through a primitive: migrateTo Can be initiated from outside through any public method The active object migrates with: all pending requests all its passive objects all its future objects Automatic and transparent forwarding of: requests (remote references remain valid) replies (its previous queries will be fulfilled)

12. Denis Caromel11 Characteristics and optimizations Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

13. Denis Caromel12 Characteristics and optimizations Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

14. Denis Caromel13 Characteristics and optimizations direct Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

15. Denis Caromel14 Characteristics and optimizations direct Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

16. Denis Caromel15 Characteristics and optimizations direct forwarder Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

17. Denis Caromel16 Characteristics and optimizations direct forwarder Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

18. Denis Caromel17 Characteristics and optimizations direct forwarder Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder)

19. Denis Caromel18 Characteristics and optimizations Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) Safe migration (no agent in the air!) Local references if possible when arriving within a VM Tensionning (removal of forwarder) direct forwarder

20. Denis Caromel19 Software Components A component in a given infrastructure is: a software module, with a standardized description of what it needs and provides, to be manipulated by tools for Composition and Deployment

21. Denis Caromel20 A CORBA Component My Business Component Component interface Facets Event sources Event sinks Attributes Receptacles OFFERED REQUIRED Courtesy of Philippe Merle, Lille, OpenCCM platform

22. Denis Caromel21 Building CCM Applications = Assembling CORBA Component Instances Provide+Use, but flat assembly

23. Denis Caromel22 Composite Components Composition of components to build new component: Composite (vs. Primitif) Cp. Inside Cp. Composite components: - Internal interfaces - Internal Bindings - Imbrication - IN/OUT operations - Controllers - Life cycle Mng. --> Reconfiguration D C A B C B A

24. Denis Caromel23 Conclusion on the basics: Component Orientedness Level 1: Instantiate - Deploy - Configure Simple Pattern Meta-information (file, XML, etc.)JavaBeans, EJB Level 2: Assembly (flat) Server and client interfacesCCM Level 3: Hierarchic CompositeFractal Level 4: Reconfiguration Binding, Inclusion, Location On going work … Interactions / Communications: Functional Calls: service, event, stream Non-Functional: instantiate, deploy, start/stop, inner/outer, re-bind

25. Denis Caromel24 B B C D Towards GRID Components Collections are essential: --> Group Communications --> Collective Interfaces Parallel component vs. Distributed: A given component instance can be distributed over machines Reconfigurations: bindings, in/out B B B’ D C B A A

26. Denis Caromel25 ProActive Components - Principles for Distributed Components - Abstract Deployment model - Composing Virtual Nodes - Descriptors: Primitive and Composite

27. Denis Caromel26 ProActive Component Definition A component is: Formed from one (or several) Active Object Executing on one (or several) JVM Provides a set of server ports (Java Interfaces) Uses a set of client ports (Java Attributes) Point-to-point or Group communication between components Hierarchical: Primitive component: define with Java code and a descriptor Composite component: composition of primitive components Descriptor: XML definition of primitive and composite Virtual node captures the deployment capacities and needs Virtual Node is a very important abstraction for GRID components

28. Denis Caromel27 Components vs. Activity and JVMs Activity JVM Component A B1 C B2 B3 Cp. are rather orthogonal to activities and JVMs: contain activities, span across several JVMs Here, co-allocation of components, within a composite one, with a collective port, using group com. Components are a way to globally manipulate distributed, and running activities

29. Denis Caromel28 Abstract Deployment Model Objectives Problem: Difficulties and lack of flexibility in deployment Avoid scripting for: configuration, getting nodes, connecting, etc. A key principle: Abstract Away from source code: Machines Creation Protocols Lookup and Registry Protocols Context: Grid Distributed Objects, Java Not legacy-code driven, but adaptable to it

30. Denis Caromel29 Descriptors: based on Virtual Nodes Virtual Node (VN): Identified as a string name Used in program source Configured (mapped) in an XML descriptor file --> Nodes Operations specified in descriptors: Mapping of VN to JVMs (leads to Node in a JVM on Host) Register or Lookup VNs Create or Acquire JVMs Program Source Descriptor (RunTime) |----------------------------------| |-------------------------------------------| Activities (AO) --> VN VN --> JVMs --> Hosts Runtime structured entities: 1 VN --> n Nodes in n JVMs

31. Denis Caromel30 Descriptors: Mapping Virtual Nodes Component Dependencies: Provides: … Uses:... VirtualNodes: Dispatcher RendererSet Mapping: Dispatcher --> DispatcherJVM RendererSet --> JVMset JVMs: DispatcherJVM = Current // (the current JVM) JVMset=//ClusterSophia.inria.fr/...

32. Denis Caromel31 Descriptors: Virtual Nodes in Programs Descriptor pad = ProActive.getDescriptor ("file:.ProActiveDescriptor.xml"); VirtualNode vn = pad.activateMapping ("Dispatcher"); // Triggers the JVMs Node node = vn.getNode();... C3D c3d = ProActive.newActive("C3D", param, node); log (... "created at: " + node.name() + node.JVM() + node.host() );

33. Denis Caromel32 Descriptors: Virtual Nodes in Programs Descriptor pad = ProActive.getDescriptor ("file:.ProActiveDescriptor.xml"); VirtualNode vn = pad.activateMapping ("Dispatcher"); // Triggers the JVMs Node node = vn.getNode();... C3D c3d = ProActive.newActive("C3D", param, node); log (... "created at: " + node.name() + node.JVM() + node.host() ); // Cyclic mapping: set of nodes VirtualNode vn = pad.activateMapping ("RendererSet"); while ( … vn.getNbNodes … ) { Node node = vn.getNode(); Renderer re = ProActive.newActive(”Renderer", param, node);

34. Denis Caromel33 Composing Virtual Nodes When composing at runtime A and B to form C. VNa, VNb --> 2 VNs : Distributed mapping Activity JVM Component A B VNa VNb C

35. Denis Caromel34 Composing Virtual Nodes VNa, VNb --> VNa+b : Co-allocation Composition of VNs can control distribution of composite Co-allocation in a composite Activity JVM Component C A B VNa VNb VNa+b

36. Denis Caromel35 Descriptor Example: Composite Component <interface-type name="controlComposite2" signature="test.component.car.Motor" /> <interface-type name="controlComposite1” signature="test.component.car.Motor" /> <primitive-component ….. Not to be written nor read by humans !! TOOLS

37. Denis Caromel36 ProActive Components: Characteristics Components characteristics: Software module = Java Classes and Interfaces + threads: forming Active Objects Standardized description = In source: Virtual Nodes, newActive API.ProActiveDescriptor : an XML file per component Tools: Composition = working on: IC2D--Compose Deployment = Java VM, and IC2D (Deploy + Monitor)

38. Denis Caromel37 IC2D: Interactive Control and Debugging of Distribution

39. Denis Caromel38 IC2D: Basic features Graphical Visualization: Hosts, Java Virtual Machines, Nodes, Active Objects Topology: reference and communications Status of active objects (executing, waiting, etc.) Migration of activities

40. Denis Caromel39 IC2D: Basic features Graphical Visualization: Hosts, Java Virtual Machines, Nodes, Active Objects Topology: reference and communications Status of active objects (executing, waiting, etc.) Migration of activities Textual Visualization: Ordered list of messages, Logical Clock Status: waiting for a request or for a data Causal dependencies between messages Related events (corresponding send and receive, etc.)

41. Denis Caromel40 IC2D: Basic features Graphical Visualization: Hosts, Java Virtual Machines, Nodes, Active Objects Topology: reference and communications Status of active objects (executing, waiting, etc.) Migration of activities Textual Visualization: Ordered list of messages, Logical Clock Status: waiting for a request or for a data Causal dependencies between messages Related events (corresponding send and receive, etc.) Control and Monitoring: Drag and Drop migration of executing tasks Creation of additional JVMs and nodes

42. Denis Caromel41 Next steps Interactively compose components with the component view Maintain component view at execution Formal Semantics of mixing: Functional, with Non Functional calls (start/stop, rebind, in/out, …)

43. Denis Caromel42 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction)

44. Denis Caromel43 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction) ProActive Component Model: Composing instead of Programming PSEs will rely on a component model to feature plug-and-play paradigm, reusability, composition, flexibility, components combination, modularity, …

45. Denis Caromel44 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction) ProActive Component Model: Composing instead of Programming PSEs will rely on a component model to feature plug-and-play paradigm, reusability, composition, flexibility, components combination, modularity, … IC2D as User Graphical Interface to control, monitor, create local or remote objects

46. Denis Caromel45 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction) ProActive Component Model: Composing instead of Programming PSEs will rely on a component model to feature plug-and-play paradigm, reusability, composition, flexibility, components combination, modularity, … IC2D as User Graphical Interface to control, monitor, create local or remote objects ProActive objects are created and accessible remotely: RMI, JINI, UDDI(Web Services)

47. Denis Caromel46 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction) ProActive Component Model: Composing instead of Programming PSEs will rely on a component model to feature plug-and-play paradigm, reusability, composition, flexibility, components combination, modularity, … IC2D as User Graphical Interface to control, monitor, create local or remote objects ProActive objects are created and accessible remotely: RMI, JINI, UDDI(Web Services) Hierarchical and Dynamic Groups for performance improvements, Grid and large scale applications

48. Denis Caromel47 Towards PSEs: Today! ProActive Middleware already addresses some issues for PSEs development ProActive is Grid enabled (Globus, LSF, …), and provides machine-independent deployment model (VirtualNodes abstraction) ProActive Component Model: Composing instead of Programming PSEs will rely on a component model to feature plug-and-play paradigm, reusability, composition, flexibility, components combination, modularity, … IC2D as User Graphical Interface to control, monitor, create local or remote objects ProActive objects are created and accessible remotely: RMI, JINI, UDDI(Web Services) Hierarchical and Dynamic Groups for performance improvements, Grid and large scale applications Migration capabilities

49. Denis Caromel48 Towards PSEs: Tomorrow! ProActive future improvements will fit some PSEs requirements Complete Component model IC2D components view will allow full control on components composition, integration, activities,… - IC2D-ComposeIC2D-Compose - IC2D-Deploy/MonitorIC2D-Deploy/Monitor IC2D pluggings to build dedicated PSEs Wrapping as ProActive Components of legacy codes(compose and deploy) Components acquisition through Web Services(UDDI) and published as solvers (compose only)

50. Denis Caromel49 Conclusion A huge work still to be done(non exhaustive !!!) – ProActive is not a domain-dedicated PSE – Higher level and specifics PSEs still to be built – Application intelligence to be added – TestBed – Databases access – Still need to code primitive components – Functional interface useful, but Behaviour description needed for effective composition –…

51. Denis Caromel50 Conclusion A huge work still to be done(non exhaustive !!!) – ProActive is not a domain-dedicated PSE – Higher level and specifics PSEs still to be built – Application intelligence to be added – TestBed – Databases access – Still need to code primitive components – Functional interface useful, but Behaviour description needed for effective composition –… Other technologies (Portlets, … ) to be needed and to interact with ProActive BUT ProActive could play a key role for software reuse and integration to build PSEs

52. Denis Caromel51 DEMO: Components with the IC2D monitor A simple Motors and Wheels demo case Parallel: –several instances of components with collective interface Composite: –3 levels of imbrication Level 4 component orientedness: –life cycle management, rebinding, in and out

53. Denis Caromel52 WP1WP2WP3 M1 M2 Motors and Wheels demo case WP4 WP5 W1 W2 composite2 composite1 WP6 parallel2 parallel1

54. Denis Caromel53 Motors and Wheels demo case composite1 composite2 WP1WP2WP3 M1 M2 WP4 WP5 W1 W2 WP6 parallel2 parallel1

55. Denis Caromel54 Motors and Wheels demo case composite2 M1 M2 W1 W2 WP1 WP2 WP3 parallel2 composite1

56. Denis Caromel55 Component Manipulation Selecting: component and deployment descriptors DEPLOY Managing: life cycle, rebinding, in and out

57. Denis Caromel56 Component Interface with IC2D

58. Denis Caromel57 Conclusion -- Perspectives Not all models are equivalent: Component Orientedness Level 1: Configuration 2: Assembly 3: Hierarchic 4:Reconfiguration Specificity for GRID Components: Parallel (HPC), Distributed, Collective Op., Deployment, … Reconfiguration Can programming models be independent of (Grid) Components ? Do not target the same objectives But can components … compose, … reconfigure without a clear model ? Reconfiguration is the next big issue: Life cycle management, but with direct communications as much as possible For the sake of reliability and fault tolerance ---> GRID –Error, Exception handling across components –Checkpointing: independent, coordinated, memory channel,... Other pending issues: Peer-to-peer (even more volatile … reconfiguration is a must), Security,...

59. Denis Caromel58 Adaptive GRID The need for adaptive middleware is now acknowledged, with dynamic strategies at various points in containers, proxies, etc. Can we afford adaptive GRID ? with dynamic strategies at various points (communications, checkpointing, reconfiguration, …) for various conditions (LAN, WAN, network, P2P,...) HPC vs. HPC High Performance Components vs. High Productivity Components

60. Denis Caromel59 ProActive : API for Mobile Agents Mobile agents (active objects) that communicate Basic primitive: migrateTo public static void migrateTo (String u) // string to specify the node (VM) public static void migrateTo (Object o) // joinning another active object public static void migrateTo (Node n) // ProActive node (VM) public static void migrateTo (JiniNode n) // ProActive node (VM)

61. Denis Caromel60 ProActive : API for Mobile Agents Mobile agents (active objects) that communicate // A simple agent class SimpleAgent implements Active, Serializable { public SimpleAgent () {} public void moveTo (String t){ // Move upon request ProActive.migrateTo (t) } public String whereAreYou (){ // Repplies to queries return (“I am at ” + InetAddress.getLocalHost ()); } public Live(Body myBody){ while (… not end of iterator …){ res = myFriend.whatDidYouFind () // Query other agents … } myBody.fifoPolicy(); // Serves request, potentially moveTo }

62. Denis Caromel61 ProActive : API for Mobile Agents Mobile agents that communicate Primitive to automatically execute action upon migration public static void onArrival (String r) // Automatically executes the routine r upon arrival // in a new VM after migration public static void onDeparture (String r) // Automatically executes the routine r upon migration // to a new VM, guaranted safe arrival public static void beforeDeparture (String r) // Automatically executes the routine r before trying a migration // to a new VM

63. Denis Caromel62 ProActive : API for Mobile Agents Itinerary abstraction Itinerary : VMs to visit specification of an itinerary as a list of (site, method) automatic migration from one to another dynamic itinerary management (start, pause, resume, stop, modification, …) API: myItinerary.add (“machine1’’, “routineX”);... itinerarySetCurrent, itineraryTravel, itineraryStop, itineraryResume, … Still communicating, serving requests: itineraryMigrationFirst (); // Do all migration first, then services, Default behavior itineraryRequestFirst (); // Serving the pending requests upon arrival before migrating again

64. Denis Caromel63 Collective Operations : Example class A {… V foo(P p){...} } class B extends A {...} A a1= PA.newAct( A,); A a2= PA.newAct( A,); B b1= PA.newAct( B,); // Build a group of « A » A ag = (A)ProActiveGroup.newGroup(« A », {a1,a2,b1}) V v = ag.foo(param); // foo(param) invoked // on each member // A group v of result of type V is created v.bar(); // starts bar() on each member of the result group upon arrival ProActiveGroup.waitForAll (v); //bloking -> all v.bar();//Group call V vi = ProActiveGroup.getOne(v); //bloking -> on vi.bar(); //a single call A a3= PA.newAct( A,); // => modif. of group ag : Group ga = ProActiveGroup. getGroup(ag); ga.add(a3); //ag is updated

65. Denis Caromel64 1.6 IC2D Interactive Control & Debug for Distribution Features: Graphical visualization Textual visualization Monitoring and Control

66. Denis Caromel65

67. Denis Caromel66

68. Denis Caromel67

69. Denis Caromel68 IC2D: Interactive Control and Debugging of Distribution Main Features: - Hosts, JVM, - Nodes - Active Objects - Topology - Migration - Logical Clock

70. Denis Caromel69 Motors and Wheels demo case composite1 composite2 WP1WP2WP3 M1 M2 WP4 WP5 W1 W2 WP6 parallel2 parallel1

71. Denis Caromel70 Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore at SC’02 Width of links proportional to the number of com- munications

72. Denis Caromel71 IC2D: Related Events Events: Textual and ordered list of events for each Active Object Logical clock: related events, ==> Gives a Partial Order

73. Denis Caromel72 IC2D: Dynamic change of Deployment New JVMs Creation, Acquisition of new JVMs, and Nodes Protocols: rsh, ssh Globus, LSF

74. Denis Caromel73 IC2D: Dynamic change of Deployment Drag-n-Drop Migration Drag-n-Drop tasks around the world

75. Denis Caromel74 IC2D: Cluster Visualization Visualization of 2 clusters (1Gbits links) Featuring the current communications (proportional)

76. Denis Caromel75 Wrappers on legacy code Virtual Node host1 jvm host7 host6 host5 host4 host3 host2 ProActive Node Wrapper MPI JNI MPI Primitive Component Parallel Data or Control

77. Denis Caromel76 Towards GRID Components Plus specificity : High performance Communication : Important Bandwidth Very High Latency Deployment complexity: --> Abstractions Various remote execution tools (rsh, ssh, Globus, Web Services) Various registry and lookup protocols (LDAP, RMI, WS, etc.) Large variations in nodes being used (1 to 5000, … 200 000) Debugging, Monitoring, and Reconfiguring Across the world ?? High-Performance a specificity ? Not sure: an EJB component handling 1 000 000 of requests already needs High-Performance! Networks grow faster than ProcsTechniques for hiding it Parallel and Distributed: --> Group Communications

78. Denis Caromel77 Programming vs. Composing The underlying model of parallel and distributed computing being used is FUNDAMENTAL. How to build components that actually compose: semantics, correctness, efficiency, predictability of performance,... without a clearly defined programming model ? For 50 years, Computer Science have been looking for abstractions that compose: functions, modules, classes, objects, … The semantics of a composite is solely and well defined from the semantics of inner components. The quest is not over !

79. Denis Caromel78 Towards Distributed Components Typed Group Java Object A ag = newActiveGroup (…) V v = ag.foo(param); v.bar(); V A Active Objects Example of component instance Compose Deploy Monitor- Control at the component level Components