1. Emil Salageanu ProActive Parallel Suite ActiveEon March 2008 ActiveEon Hands On Programming

2. Emil Salageanu Hands On Programming Implementation of a Monitor and Compute Agent 1.create an Active Object (the Monitoring Agent) 2.manage the lifecycle of an Active Object (record info on requests, service time, …) 3.deploy the Agent on a remote host 4.use inter-Active Object communications. Deal with synchronizations 5.migrate your Agent from one host to another to gather information 6.create, manage and use a group of Agents 7.publish the web Agent as web service 8.use your agent to perform some computation (check prime numbers) 9.use the Master Worker framework to solve the Prime Number problem 2 ActiveEon Exercises

3. Emil Salageanu Hands On Programming  The Project Exercises  contains the skeleton sources for the exercises  a different package for each exercise  each exercise consists in extending classes implemented in a previous exercise and adding new functionalities  each package contains a Main class corresponding to the exercise  Don’t worry, the solutions are in project Solutions…  Just copy and past the package if needed 3 ActiveEon Eclipse Workspace

4. Emil Salageanu ACTIVE OBJECTS Hands-on programming ActiveEon 4

5. Emil Salageanu Active Object Structure 5 ActiveEon Active objects : coarse-grained structuring entities (subsystems) Each active object: - possibly owns many passive objects - has exactly one thread. Remote Method Invocation (Object RPC) No shared passive objects -- Parameters are passed by deep-copy Asynchronous Communication between active objects Full control to serve incoming requests (reification)

6. Emil Salageanu Active objects 6 ActiveEon 6 A Proxy Java Object A ag = newActive (“A”, […], Node) V v1 = ag.foo (param); V v2 = ag.bar (param);... v1.bar(); //Wait-By-Necessity V Wait-By-Necessity provides Dataflow Synchronization JVM A Active Object Future Object Request Req. Queue Thread v1 v2 ag WBN!

7. Emil Salageanu Creating active objects 7 ActiveEon 7  Instantiation-based: A a = (A)ProActive.newActive( « A », params, node);  Class-based: class pA extends A implements RunActive { … }  Object-based: A a = new A (obj, 7);... a = (A)ProActive.turnActive (a, node);

8. Emil Salageanu Creating active objects  no direct access to field variables (use getter/setter methods)  provide a no-argument and preferably an empty constructor  provide remote functionalities as public methods with non-final and serializable type ActiveEon 8

9. Emil Salageanu Polymorphism 9 ActiveEon 9 "A" “A" a ra Public class O {... public void foo (A a) { a.g (...); v = a.f (...);... v.bar (...); }... } 1) o.foo(a) : a.g() and a.f() are « local » 2) o.foo(ra) a.g() and a.f() are «remote + Async.» O

10. Emil Salageanu Terminating Active Objects 10 ActiveEon 10 PAActiveObject.terminateActiveObject(ao, immediately)  ‘immediately’ specifies if the termination request should be put in the request queue of ao or executed preemptively (called an immediate service).

11. Emil Salageanu Exercise1  TODO  create an ActiveObject (CMAgent)  call a method on the object (getCurrentstate())  terminate your active object  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.simple  ProActive API  org.objectweb.proactive.api.PAActiveObject  org.objectweb.proactive.core.node.NodeException  org.objectweb.proactive.ActiveObjectCreationException 11 ActiveEon Getting Started with The Simple Computation And Monitoring Agent

12. Emil Salageanu ACTIVE OBJECTS LIFECYCLE Hands-on programming ActiveEon 12

13. Emil Salageanu Active Object Lifecycle 13 ActiveEon 13 class BoundedBuffer extends FixedBuffer implements InitActive, RunActive, EndActive { public void initActivity(Body body) { // executed only once when object is created }; public void runActivity(Body body){ // define object behaviour // request serving policies // additional treatment between request services } public void endActivity(Body body) { // executed before the termination of the Object }

14. Emil Salageanu Active Objects 14 ActiveEon Intra-object synchronization 14 class BoundedBuffer extends FixedBuffer implements InitActive, RunActive, EndActive { runActivity (Body myBody) { Service service = new Service(body); while (...) { if (this.isFull()) service.serveOldest("get"); else if (this.isEmpty()) service.serveOldest ("put"); else service.serveOldest (); //FIFO service.waitForRequest (); } }} Explicit control Library of service routines: Non-blocking services,... serveOldest (); serveOldest (f); Blocking services, timed, etc. serveOldestBl (); serveOldestTm (ms); Waiting primitives waitARequest(); etc.

15. Emil Salageanu Exercise 2  TODO  implement initActivity(), runActivity() and endActivity() methods in order to keep track of last request duration, number of requests served and total running time of your Monitoring Agent  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.initialized  ProActive API  org.objectweb.proactive.Body  org.objectweb.proactive.api.PALifeCycle  org.objectweb.proactive.InitActive,...EndActive,...RunActive  org.objectweb.proactive.Service  org.objectweb.proactive.core.util.wrapper.LongWrapper 15 ActiveEon Active Objects Lifecycle: Using InitActive and EndActive

16. Emil Salageanu PROACTIVE DEPLOYMENT Hands on programming ActiveEon 16

17. Emil Salageanu Virtualization of resources 17 ActiveEon Application VN1 VN2 GCM XML Deployment Descriptor node Host JVM node JVM Host node JVM

18. Emil Salageanu Virtualization of resources 18 ActiveEon Application VN1 VN2 node Host JVM node JVM Host node JVM

19. Emil Salageanu XML Deployment files  Virtual Node (VN): Identified as a string name Used in program source Configured (mapped) in the XML descriptor file --> Nodes  Operations specified in descriptors: Mapping of VN to JVMs (leads to Node in a JVM on Host) Register or Lookup VNs Create or Acquire JVMs Security ActiveEon 19  Runtime structured entities : 1 VN → n nodes in m JVMs on k hosts Program SourceDescriptor (RunTime) Activities(AO) →VN→NodesVN→JVMs→Hosts

20. Emil Salageanu ProActive Deployment  Virtual Nodes Definition  Define the Virtual Nodes provided by the deployment descriptors and the services that applies to these virtual nodes  Deployment  Mapping [VN→JVM] or Lookup VN  Map the Virtual Nodes to the JVMS: [1 VN – 1 JVM] or [1 VN – n JVMs] or [n VNs – m JVMs]  Lookup an already deployed Virtual Node  Register Virtual Node  JVMs  Define a number of nodes for a JVM ([1 JVM – 1 Node] or [1 JVM – n Nodes])  Define the process for creation or lookup for each JVM  Infrastructure  JVM Creation/Lookup processes  Services (P2P, FT, Timit) 20 ActiveEon Structure of Deployment Descriptor Files

21. Emil Salageanu ProActive Deployment  In the Virtual Node definition are defined:  The Virtual Node name  The services that applies to this VN  The file transfer needed for this VN  Node attributes: property (unique/multiple), timeout, waitForTimeout, minNodeNumber 21 ActiveEon Virtual Nodes Definition <virtualNode name="Workers" property="multiple" ftServiceId="appli" />

22. Emil Salageanu ProActive Deployment Mapping of VNs on JVMs → Nodes  Mapping [1 VN – 1JVM] 22 ActiveEon Deployment: Mapping …

23. Emil Salageanu ProActive Deployment  Mapping [1 VN – n JVM] 23 ActiveEon Deployment: Mapping …

24. Emil Salageanu ProActive Deployment  Mapping [n VN – m JVM] 24 ActiveEon Deployment: Mapping

25. Emil Salageanu ProActive Deployment  Register a Virtual Node to make it accessible by another applications 25 ActiveEon Deployment: Virtual Node Registration or …

26. Emil Salageanu ProActive Deployment  Acquire a VirtualNode already deployed by another application 26 ActiveEon Deployment: Virtual Node Acquisition....................

27. Emil Salageanu ProActive Deployment  Specify a process for the creation of the JVM 27 ActiveEon Deployment: JVMs creation............

28. Emil Salageanu ProActive Deployment  Acquire already created JVMs  Use of the RMIRegistryLookup service or the P2PService 28 ActiveEon Deployment: JVMs acquisition............

29. Emil Salageanu ProActive Deployment  Define the processes used in the ‘deployment’ section  Create and acquire JVMs  Define  classpath environment variable  the java path  the policy file path  the log4j properties file path  the ProActive properties file path  parameters to the JVM to be created 29 ActiveEon Infrastructure

30. Emil Salageanu ProActive Deployment  The class attribute defines the class to instantiate in order to create the process 30 ActiveEon Infrastructure: Local JVMs.......

31. Emil Salageanu ProActive Deployment  Several Protocols implemented:  rsh, ssh, rlogin, lsf, pbs, oar, prun, globus, unicore, arc (nordugrid), glite  Hierarchy of processes:  each remote process must have a pointer either on another remote process or on a jvmProcess to create a jvm 31 ActiveEon Infrastructure: Remote JVMs

32. Emil Salageanu ProActive Deployment  Use ssh public/private keys 32 ActiveEon Infrastructure: Remote JVMs (SSH Process)

33. Emil Salageanu ProActive Deployment  Build a list of JVMs processes using protocols like RSH, SSH, RLOGIN 33 ActiveEon Infrastructure: Remote JVMs (Process List)......

34. Emil Salageanu ProActive Deployment  LSF Process  Create Nodes(JVMs) on a cluster  PBS Process  Create jobs on cluster managed by PBS, PBSPro or Torque  Sun Grid Engine Process  This protocol is used to create jobs on cluster managed by Sun Grid Engine  GLOBUS Process  Etc... 34 ActiveEon Infrastructure: Job Scheduler Support

35. Emil Salageanu ProActive Deployment  Use a service to acquire an already running JVM  RMIRegistryLookup  P2PService 35 ActiveEon Infrastructure: Acquire JVMs rmi://localhost:3000

36. Emil Salageanu Using Virtual Nodes in the Application 36 ActiveEon 1. create object representation of the deployment file: ProActiveDescriptor pad=PADeployment.getProactiveDescriptor(“file:…”) 2. activate all virtual nodes: pad.activateMappings(); 3. obtain the virtual nodes VirtualNode[] vns= pad.getVirtualNodes(); VirtualNode virtualNode=vns[0]; 4. obtain the nodes: Node[] nodes=virtualNode.getNodes(); Or: Node node = virtualNode.get(index); 5. create an active object on the node: A a = PAActiveObject.newActive(A.class.getName(),[…],node) log("created at: ”+node.name()+node.JVM()+node.host());

37. Emil Salageanu Exercise 3  TODO  Write code for obtaining a virtual node from your deployment descriptor and deploy your agent on a Node of the Virtual Node  Modify the deployment descriptor in order to deploy your agent on some other machine without modifying the source code  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.deployed  ProActive API  org.objectweb.proactive.api.PADeployment  org.objectweb.proactive.core. descriptor.data.ProActiveDescriptor  org.objectweb.proactive.core.descriptor.data.VirtualNode  org.objectweb.proactive.core.node.Node  org.objectweb.proactive.core.node.NodeException 37 ActiveEon Application Deployment

38. Emil Salageanu SYNCHRONIZATION AND POLYMORPHISM Hands on programming ActiveEon 38

39. Emil Salageanu Call on an Active Object 39 ActiveEon  Internally, a method call on an active object consists in 2 steps A query : name of the method, parameters… A Reply : the result of the method call  A query returns a Future object which is a placeholder for the result  The callee will update the Future when the result is available  The caller can continue its execution event if the Future has not been updated 39 foo () { Result r = a.process(); //do other things... r.toString(); } Result process() { //perform long //calculation return result; } will block if not available

40. Emil Salageanu Call between Objects 40 ActiveEon Parameter Passing: Active Objects 40 ba x Copy at serializ ation Object passed by Deep Copy - Active Object by Reference b.foo(x, c) c c Reference Passing

41. Emil Salageanu Wait-By-Necessity: Automatic Continuation 41 ActiveEon 41 ba Futures are Global Single-Assignment Variables V= b.bar () c c c.gee (V) v v b

42. Emil Salageanu Synchronizations 42 ActiveEon  Explicit Synchronization: - ProActive. isAwaited (v); // Test if available -. waitFor (v); // Wait until availab.  Vectors of Futures: -. waitForAll (Vector); // Wait All -. waitForAny (Vector); // Get First 42 A ag = newActive (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity  Implicit Synchronization: Wait-by-necessity

43. Emil Salageanu Polymorphism 43 ActiveEon 43 "A" “A" a ra Public class O {... public void foo (A a) { a.g (...); v = a.f (...);... v.bar (...); }... } 1) o.foo(a) : a.g() and a.f() are « local » 2) o.foo(ra) a.g() and a.f() are «remote + Async.» O

44. Emil Salageanu Active Objects 44 ActiveEon The meaning of “this” 44 public class A { B b = … public void foo() { b.gee(this); OR: b.gee(PAActiveObject.getStubOnThis()) } With public class B { public void gee(A a) { //do something with a } ?

45. Emil Salageanu Standard system at Runtime 45 ActiveEon Dynamic Topology, Asynchrony, WbN,... but no sharing Denis Caromel 45

46. Emil Salageanu Exercise 4  TODO  Implement inter-Agent communications in order to gather the states on several machines  Deal with implicit and explicit synchronization  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.sync  ProActive API  Use the ProActive API you have encountered in the previous exercises 46 ActiveEon Synchronization and Polymorphism: inter-Agent communications

47. Emil Salageanu MIGRATION OF ACTIVE OBJECTS Hands on programming ActiveEon 47

48. Emil Salageanu ActiveEon 48 Migration of active objects Proxy Body Object  Migration is initiated by the active object through a request  The active object migrates with  its passive objects  the queue of pending requests  its future objects 2 Techniques : Forwarders or Centralized server Calling Object ForwarderForwarder

49. Emil Salageanu Migration API 49 ActiveEon  Migrate to a given node: PAMobileAgent.migrateTo(Node nodeToMigrateTo);  Migrate to the node with the URL given as parameter: PAMobileAgent.migrateTo(String NodeUrl);  Migrate to the location of the ActiveObject given as parameter: PAMobileAgent.migrateTo(Object activeObject);

50. Emil Salageanu Migration API 50 ActiveEon On departure / On arrival public class A implements InitActive { public void initActivity(Body body) { MigrationStrategyManagerImpl myStrategyManager = new MigrationStrategyManagerImpl((Migratable) body); myStrategyManager.onArrival(“ afterMigration "); myStrategyManager.onDeparture(“ beforeMigration "); } public void beforeMigration (){ //actions to be executed before the migration } public void afterMigration () { //actions to be executed after migration (on the new location) }

51. Emil Salageanu Exercise 5  TODO  Migrate the Agent from one machine to another and report the state of each JMV on the machines.  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.migration  ProActive API  org.objectweb.proactive.api.PAMobileAgent 51 ActiveEon Monitoring Several Computers Using Migration

52. Emil Salageanu TYPED GROUPS Hands on programming ActiveEon 52

53. Emil Salageanu ProActive Groups 53 ActiveEon 53 Typed and polymorphic Groups of local and remote objects Dynamic generation of group of results Language centric, Dot notation  Manipulate groups of Active Objects, in a simple and typed manner: A ag=(A)ProActiveGroup.newGroup(«A»,{{p1},...},{Nodes,..}); V v = ag.foo(param); v.bar();

54. Emil Salageanu Collective Operations : Example 54 ActiveEon 54 // Build a Group of OBJECTS « A » (Standard and/or Active) A agS = (A)ProActiveGroup.newGroup(“A”,....) V v = ag.foo(param); // foo(param) invoked // on each member // A group v of result of type V is created // Build an Active Group of OBJECTS « A » //(Standard and/or Active) A ag = (A) newActiveGroup(“A”,{…}, Nodes) V v=ag.foo(param);//invoked on each member

55. Emil Salageanu Two Representations Scheme 55 ActiveEon 55 Group of objects ‘Group’ Typed group ‘A’ getGroupByType static method of class ProActive getGroup method of class Group Management of the group Functional use of the group

56. Emil Salageanu Two Representation Scheme 56 ActiveEon  groupAFunctional – has type ‘A’ : functional representation A groupAFunctional = (A)ProActiveGroup.newGroup(“A”,{…})  groupAManagement – has type ‘Group’: management representation Group groupAManagement = ProActiveGroup.getGroup(groupAFunctional); groupAManagement.add(new A()); groupAManagement.add(new B()); //B inherits from A A groupAFunctional_copy = (A)groupAManagement.getGroupeByType();  Variables groupAFunctional, groupAManagement, groupAFunctionalCopy reference the same group object 56

57. Emil Salageanu Example 57 ActiveEon pubic Class A { public V foo(…) {…} } // Build a group of ACTIVE OBJECTS « A » A ag = (A)ProActiveGroup.newGroup(“A”,{…}, Nodes) V v=ag.foo(param);//invoked on each member // Get the management representation of group v Group vGroup = ProActiveGroup.getGroup(v ); vGropup.addMember(……) ….

58. Emil Salageanu Implementation 58 ActiveEon Stub Class automatic generated Stub inherits from the class of object Stub connects a proxy special proxy for group result is stub+proxy 58 groupA.foo(); Stub A Proxy for group groupA Stub B Proxy for group groupB B groupB =

59. Emil Salageanu Broadcast and Scatter 59 ActiveEon 59 JVM ag cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg) ; ag.bar(cg); // scatter cg c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 s c1 c2 c3 s  Broadcast is the default behavior  Use a group as parameter, Scattered depends on rankings

60. Emil Salageanu Handling Group Failures 60 ActiveEon 60 JVM ag vg V vg = ag.foo (param) ; Group groupV = PAG.getGroup(vg); el = groupV.getExceptionList();... vg.gee(); failure Except. List

61. Emil Salageanu Active and Hierarchical Groups  Active Group: a group being a (Remote) Active Object  A group becomes remotely accessible  Multiple coherent view of a single Group from several JVMs  Hierarchical group  A group that contains either standard objects, active objects or groups  Dynamically built  Achieves scalability ActiveEon 61

62. Emil Salageanu Hierarchical Groups 62 ActiveEon  Add a typed group into a typed group  vs. Adding an element in a group  Scalability of groups  Can match the network topology 62 A A G1 G2

63. Emil Salageanu Grid deployment 63 ActiveEon 63 grid frontal cluster

64. Emil Salageanu Exercise 6  TODO  Create a group object containing Monitoring Agents  Use the functional representation of the group to perform monitoring actions  Use the management representation of your group to add and remove agents  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.groups  ProActive API  org.objectweb.proactive.api.PAGroup  org.objectweb.proactive.core.group.Group 64 ActiveEon Use of ProActive Groups

65. Emil Salageanu WEB SERVICES Hands on programming ActiveEon 65

66. Emil Salageanu Exercise 7  TODO  Expose the Monitoring Agent as a Web Service  Use a Web Service Client to perform a call on the Agent through the Web Service  Exercise source package  org.objectweb.proactive.examples.userguide.cmagent.webservice  ProActive API  org.objectweb.proactive.extensions.webservices.WebServices 66 ActiveEon Expose the Monitoring agent as a Web Service

67. Emil Salageanu DISTRIBUTED PRIMES Hands on programming ActiveEon 67

68. Emil Salageanu Exercise 8  TODO  Use your Agent to perform computation: check if a candidate number has a divisor in a given range  You will have to transform your sequential application in a parallel/distributed one  Exercise source packages  org.objectweb.proactive.examples.userguide.primes.sequential  org.objectweb.proactive.examples.userguide.primes.distributed  ProActive API  org.objectweb.proactive.api.PAFuture  org.objectweb.proactive.core.util.wrapper.BooleanWrapper 68 ActiveEon Distributed Prime Numbers

69. Emil Salageanu MASTER-WORKER Hands on programming ActiveEon 69

70. Emil Salageanu ActiveEon 70 How it works ?

71. Emil Salageanu ActiveEon 71 Creating/Terminating the Master  Local creation : public ProActiveMaster();  Remote creation : public ProActiveMaster(Node remoteNodeToUse); public ProActiveMaster(URL descriptorURL, String masterVNName);  Termination : public void terminate(boolean freeResources);

72. Emil Salageanu ActiveEon 72 Adding Resources public void addResources(URL descriptorURL, String virtualNodeName); public void addResources(VirtualNode virtualNode); public void addResources(Collection nodes); public void addResources(URL descriptorURL);

73. Emil Salageanu ActiveEon 73 Task Definition / Submission  Task definition : public interface Task extends Serializable { public R run(WorkerMemory memory) throws Exception; }  Task submission : public void solve(List tasks) throws TaskAlreadySubmittedException;

74. Emil Salageanu ActiveEon 74 Retrieving the Results  Result reception order : public void setResultReceptionOrder(OrderingMode mode);  Blocking calls : public List waitAllResults() throws TaskException; public List waitKResults(int k) throws TaskException; public R waitOneResult() throws TaskException;  Non-Blocking calls : public boolean isEmpty(); public int countAvailableResults();

75. Emil Salageanu ActiveEon 75 Results reception order CompletionOrder Mode SubmissionOrder Mode

76. Emil Salageanu Exercise 9  TODO  Use the same algorithm as the one used at the previous exercise and the Master Worker API to solve the Prime Number problem  Exercise source packages  org.objectweb.proactive.examples.userguide.primes.distributedmw  ProActive API  org.objectweb.proactive.extensions.masterworker.ProActiveMaster  org.objectweb.proactive.extensions.masterworker.interfaces.Task 76 ActiveEon Check Prime Numbers using the Master Worker Framework