ServiceSs, a new programming model for the Cloud Daniele Lezzi, Rosa M. Badia, Jorge Ejarque, Raul Sirvent, Enric Tejedor Grid Computing and Clusters Group Barcelona Supercomputing Center OGF-Europe COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 Outline StarSs programming model COMP Superscalar framework COMP Superscalar towards SOA and Clouds ServiceSs 2

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9); }... Sequential Application T1 0 T2 0 T3 0 T4 0 T5 0 T1 1 T2 1 T3 1 T4 1 T5 1 T1 2 … Resource 1 Resource 2 Resource 3 Resource N Task graph creation based on data precedence Task selection + parameters direction (input, output, inout)‏ Scheduling, data transfer, task execution Synchronization, results transfer Parallel Resources (cluster, grid) Star Superscalar Programming Model 3

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 StarSs programming model GRIDSs, COMPSs Tailored for Grids or clusters Data dependence analysis based on files C/C++, Java SMPSs Tailored for SMPs or homogeneous multicores Altix, JS21 nodes, Power5, Intel-Core2 C or Fortran CellSs Tailored for Cell/B.E. processor C or Fortran GPUSs (Next session talk by Rosa M. Badia) NestedSs Hybrid approach that combines SMPSs and CellSs

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs Componentisation of the GRID superscalar runtime Each component in charge of a functionality 5 Base technologies: Java as programming language ProActive: Reference implementation of the GCM model Used to build the components JavaGAT API that provides uniform access to different kinds of Grid middleware Used for job submission and file transfer

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 initialize(f1); for (int i = 0; i < 2; i++) { genRandom(f2); add(f1, f2); } print(f2); Java application COMPSs Programming model – Application + interface public interface = "Linux") void = Type.FILE, direction = Direction.OUT) String f } Task constraints Parameter metadata Implementation Java interface 6

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 Custom Java Class Loader Java app code COMPSs runtime Annotated interface Javassist inserts calls to Custom Loader uses input C/C++ app code COMPSs runtime Interface inserts calls to Stubs Generator input JNI

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs and XtreemOS COMPSs has been ported in the XtreemOS framework by means of the Simple API for Grid Applications (SAGA). COMPSs benefits from SAGA by delegating to it the job execution including resource allocation and data transfers.

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 Integration in a Service-Oriented and Cloud infrastructure Main idea: moving the COMPSs runtime from the client side to a server SOA platform Characteristics of this environment: Execution of application tasks offered as services N applications can be served simultaneously Several COMPSs can be deployed, to serve the tasks from one or more applications Resource provisioning brought by a Cloud 9

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs and EMOTIVE Cloud

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs and EMOTIVE Cloud The Virtualization Resource Management and Monitoring (VRMM) layer wraps each virtualized node and monitors its state Within the VRMM layer, the Virtualization Manager (VtM) provides an application-specific virtual machine (VM) for each application; the application is given full control of its execution environment without introducing risk to the underlying system or other applications The VMs are created on demand, according to application requirements, and are consolidated in the provider’s physical resources to optimise their use A Resource Monitor (RM) monitors task and resource status, including historical information SERA (Semantically-Enhanced Resource Allocator): Resources allocation using semantics and agents

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 ServiceSs envisioned architecture 12 COMPSs API Java App WS Container Runtime Manager COMPSs runtime instance 1 Cloud Scheduler COMPSs runtime instance N Worker VM 1 Worker VM 1 Worker VM 1 Worker VM M COMPSs Application Side Java App Java App Cloud WS Container User Side App WS Container WS Container Worker VM 1 External WS Worker FP7 OPTIMIS Project (Optimized Infrastructure Services)

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs and EMOTIVE Cloud – Step 1 VM1 VMn VM2 1.Existing pool of EMOTIVE VMs 2.COMPSs executes tasks on these VMS

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 COMPSs and EMOTIVE Cloud – Step 1 VM1 VMn VM2 1.The Task Scheduler asks SERA for a pool of VMs 2.COMPSs executes tasks on these VMS 3.COMPSs requests the creation of more or “bigger” VMs (memory, CPU, etc) VMn+1

COS on Software Development Tools for Distributed Computing OGF28, Munich 15 March 2010 Next steps 1.Extension of the COMPSs API to support the new features 2.Extension of the Runtime Manager 3.Tools for deployment of the core elements and composite applications. COMPSs API Java App WS Container Runtime Manager COMPSs runtime instance 1 Cloud Scheduler COMPSs runtime instance N Worker VM 1 Worker VM 1 Worker VM 1 Worker VM M COMPSs Application Side Java App Java App Cloud WS Container User Side App WS Container WS Container Worker VM 1 External WS Worker