Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Denis Caromel GridCOMP Scientific Coordinator Beijing, October 2007 Middleware Comparison / GCM

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) ProActive / GCM Overview

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 3 ProActive Parallel Suite (1)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 4 ProActive Parallel Suite (2)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Integrated development environments (IDE)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 6 Rich GUI for Grid / Parallel Programming

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 7 Component GUI under Dev. at Westminster Univ.

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Programming Models

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 9 ProActive Parallel Suite (3)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 10 Programming models Basic programming models supported by the middleware GUIXML Format Procedural API Objects API WorkflowYes SkeletonYes ComponentsYes Master worker (slave)Yes Branch & BoundYes SPMD : MPI, object SPMD Yes Active object (actors)Yes

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) GCM + Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain Innovation: Composite Components Multicast and GatherCast MultiCast GatherCast

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Data Management

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 13 Data management: File transfer  Currently ProActive provides support for the following type of transfers:  To a remote node (Push)  From a remote node (Pull)  It could be used at:  Deployment time (XML)  Retrieval time (XML)  Execution time (Objects API)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Infrastructure: Resource management & SLA

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 15 Scheduler: User Interface

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 16 Scheduler: Resource Manager Interface

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 17 Infrastructure management  Virtualisation of multi-core, multi-processors, servers, clusters: Yes  ProActive/GCM makes it possible to:  Virtualize the infrastructure using host and VM capacity  Map the application onto the resulting virtual infrastructure  Meta scheduler, Broker: Yes  ProActive Scheduler and Infrastructure Manager, two graphical application, can talk with primitive schedulers.  Scheduler: Yes  ProActive Scheduler can deploy directly a list of jobs over various machines.  Peer to peer support for auto-organization : Yes  Desktop P2P can be used to build desktop grid. Windows and Unix for heterogeneous grid. No need to change code of ProActive/GCM applications.  Hardware resources handled by the middleware:  computers: Yes  Networks, storage devices, scientific instruments: No  Security:  AIC: Authentification, Integrity, confidentiality : Yes  both middleware and application communications are handled by mechanism  Role management: Partially  Full support for construction & administration of VO: No

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 18 Orthogonal questions  Globus based or not  No  Approach to SOA  Yes, both component and Active Object can be exported as Web Service. An application can itself registered into a registry and lookup other services for SOA integration.  Dynamic reconfiguration of Components  Main languages: (C++, Java, C#, Fortran...)  Java as principal language, and support wrapping of native legacy applications  Interoperability with other Grid middleware: Yes !!  SSH, RSH, OARSH, ARC NorduGrid, CGSP China Grid, EGEE gLite, Fura/InnerGrid, GLITE, GLOBUS, GridBus, IBM Load Leveler, LSF, MPI, OAR, PBS, PRUN, SGE, Sun Grid Engine, UNICORE

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) Applications

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 20 Current GCM experiments in ProActive  JEM3D: 3D electromagnetic application: a single Cp on 300+ CPUs on Grid  Vibro-Acoustic application with EADS (legacy MPI coupling)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 21 Applications: successfully executed  GridCOMP use cases  Biometric Identification System  Days Sales Outstanding  The mean time that clients delay to pay an invoice  EDR processing  Extended Data Record, extract, transform and Load in database large amount of data  WingDesign  Computes the aerodynamic wing performance for a given configuration

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 22 Other ProActive/GCM applications  Scilab Parallel execution of Scilab Scripts on clusters and desktop grids.  Mathlab  BIDistribution of databases for Business Intelligence  FlowShop A distributed approach for solving the FlowShop problem.  Nbody The nbody problem (SPMD)  NQueen An example of the N Queen problem distributed  JEM3D An electromagnetism application  C3DA distributed and collaborative 3D renderer.  Monte carloStock pricing (EU and USA options)  ProPotts MonteCarlo simulation for NLP  OPSSA On-line power systems security analysis  Salishan A resource sharing and simulation problem  SAT solverDistributed SAT solvers in Java based on the SAT4J library  ProFractal Parallel fractal image generation  GeB Grid-enabled BLAST  Cruise Control A real-time java controller to maintain a cruise speed  NASNASA benchmark suite (5 cores/5, 1 application/7)  e-science, e-business, e-engineering

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 23 Distribution model  Open Source  Yes, the source code is under GPL version 2 licence.  Close Source  No  Commercial support  Yes, the ActiveEon company provides commercial support for the ProActive/GCM middleware.

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 24 ActiveEon

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 25 Summary-Perspective: Full-fletched Upper Ware Active Objects SPMD

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 26

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 27

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 28

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 29 Form of GCM Interoperability Deployment  Just an XML Schema: Specifies the deployment of the application Virtual Nodes onto the infrastructure (machine, OS, protocols, schedulers, etc.)  Example: EGEE gLite schema:

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 30 XML Deployment Descriptors

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 31 Components and SOA  From the review:  Definition:  Components:  Services and SCA:  In GridCOMP:  Autonomicity, Reconfiguration  Lookup, Dynamic Bindings, etc.

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 32 ProActive/GCM Component Definition  A component:  Is formed from one (or several) Active Object  Is executed on one (or several) JVM  Provides a set of server ports: Java Interfaces  Uses a set of client ports: Java Attributes  Uses Point-to-point or Group communication between components  Hierarchical:  Primitive component: define with Java code and a descriptor  Composite component: calls in composites  Descriptor: composition of primitive + composite  Parallel component: multicast of  XML definition of primitive and composite (ADL)  Virtual nodes capture the deployment capacities and needs  Virtual Node is a very important abstraction for GRID components

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 33 GCM Architecture

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 34 GCM Technical Structure 1. Component Specification as an XML schema or DTD 2. Run-Time API defined in several languages C, Java 3. Description and Information for Deployment (XML DD, Virtual Nodes, File Transfer, …) 3. Packaging described as an XML schema

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 35 Main Technical Activities --- Objectives  WP 2, Task 2.1: Primitive Component Programming  WP 2, Task 2.2: Legacy Code Wrapping, Interoperability  WP 2, Task 2.3: Composition, Composite, and Deployment For memory:  Autonomic features (WP 3)  IDE for GCM, Composition GUI (WP 4)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 36 Collective Interfaces

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 37 Collective Interfaces  Simplify the design and configuration of component systems  Expose the collective nature of interfaces  Cardinality attribute  Multicast, Gathercast, gather-multicast  The framework handles collective behaviour at the level of the interface  Based on Fractal API :  Dedicated controller  Interface typing  Verifications

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 38 Multicast interfaces Transform a single invocation into a list of invocations  Multiple invocations  Parallelism  Asynchronism  Dispatch  Data redistribution (invocation parameters)  Parameterisable: Distribution Function  Broadcast, scattering  Dynamic redistribution: Dynamic dispatch  Result = list of results

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 39 Multicast interfaces

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 40 Multicast interfaces  Results as lists of results  Invocation parameters may also be distributed from lists

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 41 Gathercast interfaces Transform: a list of invocations   a single invocation  Synchronization of incoming invocations  ~ “join” invocations  Timeout / Drop policy  Bidirectional Bindings (callers  callee)  Data gathering Aggregation of parameters into lists  Result: Redistribution of results Redistribution function

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 42 Industrial exploitation : ActiveEon start-up  Support and Services around ProActive+GCM  Use standard GCM deployment for all developments  Use standard GCM programming model to develop components, and build component applications  Collaboration with GridSystems

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 43 ActiveEon

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 44 ActiveEon

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 45 ActiveEon

Grid programming with components: an advanced COMPonent platform for an effective invisible grid © GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442) More to come on Friday: ETSI TC Grid Meeting

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 47 Example

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 48 Apologies for being tough with refusal of August 30 th Evaluation Date Samuel, August 30 th 2007, 19H07, 3.7 Kg