Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) From Theory to Practice in Distributed Component Systems Denis Caromel, + GridCOMP + OASIS members GridCOMP Scientific Coordinator, INRIA-Nice Univ.-CNRS, OASIS Team Contents: 1.GridCOMP: Practice 2.ProActive: Practice 3.ASP: Theory 4.GCM Component: Practice 5.VerCors Model Checking: Theory Practice

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) 1. Practice:

Grid programming with components: an advanced COMPonent platform for an effective invisible grid Objectives + Structuring of GridCOMP Project “GRID PROGRAMMING WITH COMPONENTS:  Parallel, Distributed, Concurrent AN ADVANCED COMPONENT PLATFORM  a lot of money is needed FOR AN EFFECTIVE INVISIBLE GRID”  Mang. Of Non Functional aspects  Grid for the D.

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 4  GCM: Grid Component Model  GCM Being defined in the NoE CoreGRID (42 institutions)  Open Source ObjectWeb ProActive implements a preliminary version of GCM  Service Oriented: NESSI relation  GridCOMP takes:  GCM as a first specification,  ProActive as a starting point, and Open Source reference implementation. The vision: GCM to be the GRID GSM

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 5 GCM Partners

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain Innovation: Composite Components Multicast and GatherCast MultiCast GatherCast

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) 2. Practice:

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 8 Objectives Parallelism + Concurrency Multi-Core to Distributed

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 9 ProActive’s Framework in a nutshell Open Source Active Objects

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 10 A ProActive : Active objects Proxy Java Object A ag = newActive (“A”, […], VirtualNode) V v1 = ag.foo (param); V v2 = ag.bar (param);... v1.bar(); //Wait-By-Necessity V Wait-By-Necessity is a Dataflow Synchronization JVM A Active Object Future Object Request Req. Queue Thread v1 v2 ag WBN!

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 11 First-Class Futures Update

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 12 Wait-By-Necessity: First Class Futures ba Futures are Global Single-Assignment Variables V= b.bar () c c c.gee (V) v v b

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 13 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Cpt. and GRID JVM

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 14 Broadcast and Scatter JVM ag cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg) ; ag.bar(cg); // scatter cg c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 s c1 c2 c3 s Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 15 Static Dispatch Group JVM ag cg c1 c2 c3 c4 c5 c6 c7 c8c0 c9c1 c2 c3 c4 c5 c6 c7 c8c0 c9 c1 c2 c3 c4 c5 c6 c7 c8c0 c9 Slowest Fastest empty queue ag.bar(cg);

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 16 Dynamic Dispatch Group JVM ag cg c1 c2 c3 c4 c5 c6 c7 c8c0 c9c1 c2 c3 c4 c5 c6 c7 c8c0 c9 c1 c2 c3 c4 c5 c6 c7 c8c0 c9 Slowest Fastest ag.bar(cg);

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) 3.Theory: ASP Asynchronous Sequential Processes

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 18 A Theory of Distributed Objects D. Caromel, L. Henrio, Springer 2005, Monograph A Calculus: ASP: Asynchronous Sequential Processes  Based on Sigma-Calculus (Abadi-Cardelli)  Formal Proofs of determinism: in an asynchronous model  Releases a few important implementation constraints

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 19 Local Creating an Activity Sending a Request Sending a Reply Service ASP

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 20 Denis Caromel 20 Standard system at Runtime: Dynamic Topology, Asynchrony, Wait-by-Necessity,... but no sharing Proofs of Determinism

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 21 Diagram of Properties

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 22 ASP theory: Summary and Results  ASP  Confluence and Determinacy  Proved Properties:  Future updates can occur at any time, in any order  Asynchronous FIFO point-to-point is enough for Requests  Execution characterized by the order of request senders  Applications:  Determinacy of programs based on a dynamic property (DON)  Determinacy of programs communicating over Trees  Deterministic Components, …

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) 4. Practice: GCM Component

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 24 Objects to Distributed Components (1) Typed Group Java or Active Object ComponentIdentity Cpt = newActiveComponent (params); A a = Cpt ….getFcInterface ("interfaceName"); V v = a.foo(param); V A Example of component instance JVM Truly Distributed Components

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 25 Content Controller Using The Fractal model: Hierarchical Component Orange & INRIA (E. Bruneton, T. Coupaye, J.B. Stefani)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 26 Content Controller The Fractal model: Hierarchical Components Common component model of the ObjectWeb consortium

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 27 Content Controller Interfaces = Provided and Required Provided, Server Interfaces Required, Client Interfaces

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 28 Content Controller Hierarchical model : Composites encapsulate Primitives, Primitives encapsulate Code

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 29 Content Controller Binding = in an external file (XML ADL), Not in programs

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 30 Content Controller Binding = in an external file (XML ADL), Not in programs

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 31 Migration Capability of composites Migrate sets of components, including composites

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 32 Migration Capability of composites Migrate sets of components, including composites

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 33 Collective Interfaces

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 34 Collective Interfaces  Simplify the design and configuration of component systems  Expose the collective nature of interfaces  Cardinality attribute  Multicast, Gathercast, gather-multicast  The framework handles collective behaviour  at the level of the interface  Based on ObjectWeb Fractal API :  Dedicated controller  Interface typing  Verifications

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 35 Multicast interfaces  Transform a single invocation into a list of invocations  Multiple invocations  Parallelism  Asynchronism  Dispatch  Data redistribution (invocation parameters)  Parameterizable distribution function  Broadcast, scattering  Dynamic redistribution (dynamic dispatch)  Result = list of results

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 36

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 37 Multicast interfaces  Results as lists of results  Invocation parameters may also be distributed from lists

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 38 Gathercast interfaces  Transform a list of invocations into a single invocation  Synchronization of incoming invocations  ~ “join” invocations  Timeout / Drop policy  Data gathering  Aggregation of parameters into lists  Result: redistribution of results  Redistribution function

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 39 Environment

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 40 On-going work : GUI for Components

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 41 ProActive / GCM Environment

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 42 Current GCM experiments in ProActive  JEM3D: 3D electromagnetic application: a single Cp on 300+ CPUs on Grid  Vibro-Acoustic application with EADS (legacy MPI coupling)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) 4. Theory: VerCors Behavioral Specification and Model Checking Eric Madelaine, OASIS

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 44 Formal verification and model checking Principles of the VERCORS platform Behaviour of Primitive Components Specificatio n of the Architecture Parameterized Model Tree of Finite LTSs Parameterized LTS Synchronisation networks + controllers Model Checkin g Finite Abstraction of parameter domains

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 45 Behaviour of Primitives  Functional behaviour  Given by the user  Static source code analysis (with ProActive primitives)  Currently supported languages: FC2 and LOTOS  Usage  Parameterized LTS encoding the behaviour specification

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 46 Vercors Status and Relation to GCM-ProActive  Current state of tools  Functional behaviour: Ready and available (ADL2N)  NF controllers and ProActive's semantics  To be released in ADL2N v1  Current state of model  Functional and Non-Functional distributed components  Extensions still needed:  Exception handling is mandatory  Collective interfaces  A few other features of ProActive  Future Work  Modeling and Specification Language and GUI for ProActive community: TTools+

Grid programming with components: an advanced COMPonent platform for an effective invisible grid TTool+ for ProActive  Design model of hierarchical components for ProActive: High Level Design Tool mapped on a formal semantics, Easy to understand, easy to use  TTool+ : An extension of TTool: Developed by Ludovic Apvrille, ENST, LabSoC)

Producer-Consumer System in TTool+ : a. First Level Component Design

b. Adding Subcomponents

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) c. Binding components through Interfaces

d. Adding a new client (consumer)

e. Defining behavior using State Machine Diagrams

Grid programming with components: an advanced COMPonent platform for an effective invisible grid TTool+ for ProActive  Alpha Version Provides:  User design of distributed components with asynchronous calls  Interactions between distributed components: build behavior models (  use model checkers)  Future work:  High Level Design of ProActive components: automatic generation of controllers of component management  GCM: Generation of multiple instances of components and managing Group Communications  Generation of ADL files, ProActive Template Code

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) CONCLUSIONS

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 55 A few Key Aspects  Distributed Objects : Active Objects  Asynchronous Method Calls First-Class Futures Groups  Calculus: ASP  Confluence (very General and Dynamic)  Determinism only based on Request-Sender Reception Order  Dist. Component Specification: GCM  Hierarchical and Runtime (Fractal)  Distributed (VN, …), Multicast, Gathercast  Middleware: ProActive  Programming, Composing, Deploying + GUI  Model Checking: VerCors  Hierarchical, Parameterized,  Practical (Multi. Source for Information, Checking vs. Telling)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 56 Conclusion: Why does it scale?  Thanks to a few key features:  Asynchrony: Connection-less  Messages rather than long-living interactions RMI+JMS unified

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 57 Conclusion: Why does it Compose?  Thanks to a few key features:  Because it Scales: asynchrony !  Thanks to First Class Futures  Because it is Typed: RMI with interfaces ! No unstructured Call Backs and Ports

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Perspectives

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 59 Perspectives  Putting everything together:  ASP, ProActive/GCM, Vercors  Still a lot of work ! Collaborations Welcome!  Behavioral specification of component composition (ongoing)  Specify and Study Non-Functional Aspects  More Specifically: Life-Cycle, Reconfiguration in distributed environments

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 60 Example of a Perspective: Components in the membrane:  GCM to express reconfiguration strategy  AO to implement the Cp (Groups, Futures)  Model Checking to prove the strategy using ASP properties

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 61 Perspectives  Putting everything together:  ASP, ProActive/GCM, Vercors  Still a lot of work ! Collaborations Welcome!  Behavioral specification of component composition (ongoing)  Specify and Study Non-Functional Aspects  More Specifically: Life-Cycle, Reconfiguration in distributed environments GridCOMP.ercim.org ProActive.ObjectWeb.org inria.fr/oasis/Vercors VerCors