StratusLab is co-funded by the European Community’s Seventh Framework Programme (Capacities) Grant Agreement INFSO-RI Demonstration StratusLab First Periodic Review Brussels, Belgium 4 July 2011

2 Introduction Manual Installation  Script-assisted installation of complete StratusLab cloud. Tutorial  Selection of VM image via the Marketplace.  Abbreviated tutorial showing VM lifecycle.

3 Introduction Grid Site Installation  Services described via OVF are deployed automatically by Claudia.  Contextualization includes the provision of service certificates. Autoscaling of Grid Service  Show that number of worker nodes grows and shrinks with queue length.

4 Infrastructures “Private” Machines  Couple of machines with Fedora 14 preinstalled.  Used for demonstration of installation.  Used for tutorial. Reference Infrastructure  Used for tutorial in case of problems with previous installation. “Pre-installed” Machines  Set of machines pre-installed with StratusLab v1.0.  Demonstration of grid site installation and elasticity.

5 Installation & Tutorial

6 Installation Demo Services Involved

7 Installation Demo Deployment Front-endNode GRNET Marketplace Appliance Repository TCD

8 Grid Site Deployment & Scalability

9 Introduction Objective  To deploy and configure a grid site automatically  To scale a grid site already deployment Main work  Virtualization of the grid services (images in the appliance repository)  Definition of a grid site in a standard format  Introduction of Claudia for deploying, controlling and configuring the grid site  Usage of Claudia for scalability  Usage of OpenNebula as infrastructure manager WPs involved  WP6 provide new functionalities developed  WP4 provides component integrations in StratusLab distribution  WP5 provides the appliance repository, testbed and requirements  WP2 provides documentation

10 Grid Site Management The grid site is considered a whole (as a set of VMs, networks, certificates, software to configure …). Service manager (Claudia) controls the grid site lifecycle as a whole (it does not manage single VMs): deploy grid site, undeploy grid site Grid site configuration is done by contextualization  StratusLab contextualization  OVF contextualization for grid software configuration

11 Large-scale Grid Infrastructures

12 Compute Element Worker Node Internet glite_public … 1 to n 1 CPU 2GB MB of memory Storage Element 4 CPU 2048 MB of memory 4 CPU 2048 MB of memory APEL 1 CPU 512 MB of memory Grid site specified in a OVF

13 OVF Extracts <File ovf:id="storageelement" ovf:href=" sl-5.5-x86_64-grid.se/4.0/sl-5.5-x86_64-grid.se-4.0.qcow.gz" rsrvr:driver="qcow2"/> Network information Public Network The gLite 3.2 packages for CREAM Computing Element gLite virtual CPU 3 4

14 App Repo App Repo Site Client Crede ntial Server Crede ntial Server Service Manager Monitoring Web service (Tcloud) Web service (X) Site 1. The grid site user describes the Service using OVF 2. The User generates all the images (VMs) involved in the service and stores them into the App Repository. 3. Using the TCloud API, the client requests the service deployment. This request includes the OVF descriptor. 4. The SM revises the OVF descriptor in search of useful information for the service lifecycle management (e.g. scalability rules, startup order, needed KPIs,etc.). It also translate the OVF descriptor to ONE templates 6. Using the Tcloud API, the SM stars the final phase of the deployment : network generation and the VMs startup, etc. Site Node … Front End Front End Web service (Tcloud) OpenNebula 7. OpenNebula manages the final phase of the deployment process SEWNSECE SE ONE Templates OVF Descriptor WN APEL

15 Scalability The ability of a particular system to fit a problem as the scope of that problem increases (number of elements or objects, growing volumes of work and/or being susceptible to enlargement)  Horizontal scaling by adding more of the same software or hardware resources.  We increase the number of WNs if the number of jobs is increasing The service manager (Claudia) allows grid users to deploy and configure the grid site and provide scalability in the number of WNs KPI = jobqueueutilization = number of jobs running / total CPU slots

16 App Repo App Repo Crede ntial Server Crede ntial Server Service Manager Monitoring Web service (Tcloud) Web service (X) Site 2. The SM evaluates the elasticity rules to apply scalability if needed (e.g. queuedjobs > threshold  A new WN instance is needed). Node … Front End Front End Web service (Tcloud) OpenNebula 1. The monitoring systems collects the needed KPIs samples CEWNCEWN ONE Template 3. Using the Tcloud API, the SM requests the deployment of a new WN of the grid site WN 4. OpenNebula manages the deployment of the new WN of the service

17 Scalability in OVF KPIs Section jobqueue

18 Summary

19 Summary Manual Installation  Simplifies StratusLab installation for system administrators. Tutorial  Showed Marketplace use and standard VM lifecycle. Grid Site Installation  Automated installation of grid site services via OVF description. Autoscaling of Grid Service  Automated management of grid services, responding to peak loads.

20 Questions?

Copyright © 2011, Members of the StratusLab collaboration: Centre National de la Recherche Scientifique, Universidad Complutense de Madrid, Greek Research and Technology Network S.A., SixSq Sàrl, Telefónica Investigación y Desarrollo SA, and The Provost Fellows and Scholars of the College of the Holy and Undivided Trinity of Queen Elizabeth Near Dublin. This work is licensed under the Creative Commons Attribution 3.0 Unported License