WNoDeS – a Grid/Cloud Integration Framework Elisabetta Ronchieri (INFN-CNAF) for the WNoDeS Project

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 2E. Ronchieri, EGI TF, September 2011, Lyon, France

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 3E. Ronchieri, EGI TF, September 2011, Lyon, France

Requirements we started with Large computing centers used for a wide range of applications by multiple user communities – To determine proper and timely configuration of compute resources without sacrificing efficiency, flexibility and security User Communities – To allow accessing resources by using a number of different interfaces, authenticating via several methods, exploiting existing local and distributed infrastructures 4E. Ronchieri, EGI TF, September 2011, Lyon, France

In short To define a multi-layer framework to optimize the usage of computing center resources for multiple communities … starting from the INFN CNAF center (located in Bologna, Italy) that currently hosts about 10,000 computing cores, and 9 PB of disk space and 10 PB of tape space each day hosts about 50,000 jobs supports tens of international experiments of astro-particle physics (e.g., AMS2, Argo, Auger, Fermi/Glast, Magic, Pamela, Virgo) with about 15% of its computing resources supports the Italian Tier-1 for CERN-based LHC experiments of high energy physics (i.e, ATLAS, CMS, LHCb, Alice) and Tier-0/1 for several others adopts Platform LSF as batch system and IBM GPFS as shared File System 5E. Ronchieri, EGI TF, September 2011, Lyon, France

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 6E. Ronchieri, EGI TF, September 2011, Lyon, France

The Worker Nodes on Demand Service, a.k.a. WNoDeS, in one slide Is a software framework created by INFN to integrate Grid and Cloud provisioning through virtualization – All resources (Grid, Cloud, or else) are taken from a common pool Is scalable and reliable – it is in production at several Italian centers, including the INFN Tier-1 (CNAF, Bologna) since November 2009 – Currently WNoDeS is managing about 2000 on‐demand Virtual Machines (VMs) there Is totally transparent for Grid users and for users of traditional Computing Centers batch systems Supports a native Cloud interface – OCCI (Open Cloud Computing Interface) compliant – A Cloud Web portal Integrates authentication, policy and accounting It is not required to convert an entire farm to WNoDeS: it can coexist and share resources with a traditional computing cluster not using virtual machines 7E. Ronchieri, EGI TF, September 2011, Lyon, France

Key Characteristics Use of Linux KVM to virtualize resources on-demand that are available and customized for – direct job submissions by local users [1-2] – Grid job submissions (with direct support for the EMI CREAM-CE and WMS components) [1-2] – instantiation of Cloud resources [6] – instantiation of Virtual Interactive Pools (VIP) [3] Use of LRMS (a “batch system software”) to handle VM scheduling – There is no need to develop special (and possibly unscalable, inefficient) resource brokering systems – The LRMS is totally invisible to users for e.g. Cloud instantiations Not Use of “Cloud over Grid” (or “Grid over Cloud”)’s concept to handle resources – WNoDeS simply uses all resources and dynamically presents them to users as users want to see and access them 8E. Ronchieri, EGI TF, September 2011, Lyon, France

Key Advantages Use of a common pool of resources – There is no need to dedicate resources to “user interfaces”, “Grid computing”, “Cloud computing”, “local users” – All types of resources are taken from a common pool, resulting in overall better utilization of resources Integrated support of old and new use cases (local access, Grid computing, Cloud computing) Re-use of ten years of worldwide development, expertise and resources brought about by Grid Computing applied to the key areas of Authentication, Authorization, Accounting, Information Systems, Brokering – This will make it possible for example to inter-connect Clouds without starting from scratch in these areas again Flexibility and scalability – At the core of WNoDeS there is a standard batch system used for resource provisioning and policing a mature, stable piece of software found in any sizeable resource center – There is no need to rewrite this key and complex part 9E. Ronchieri, EGI TF, September 2011, Lyon, France

Requirements Collected on the base of the needs of INFN users and resource providers to virtualize systems and services 10 Published in the document ‘WNoDES: Requirements for Virtualized Services’ [7] – It is a continuous work in progress due to the dynamic nature of requirements and the constant evolution of WNoDeS E. Ronchieri, EGI TF, September 2011, Lyon, France

Req. #1: Jobs submitted by users of a traditional computing center Users log on to a frontend system of a resource center and submit jobs to a computing farm 11 WNoDeS can associate jobs belonging to a user (or to a set of users) to VMs specifically created for them and customized to match the job requirements [1,2] – This is completely transparent for users – There is no need to change anything from the users’ point of view – The jobs will be executed on dynamically-created VMs Non WNoDeS WNs WN Standard job WN WNoDeS enabled WNs Virtual Machin e E. Ronchieri, EGI TF, September 2011, Lyon, France

Req. #2: Jobs submitted by users of a Grid-based distributed infrastructure WNoDeS handles these jobs as an extension of the Req. #1 simply taking into account special Grid requirements if present [1,2] There are two possibilities here: – All jobs beloning to certain Virtual Organizations (VOs) can be directed to pre‐packaged VMs This is completely transparent for users – Grid users can insert a special request into their JDL script for example to specify which VM they want their jobs to run on This is done by using standard EMI (European Middleware Initiative) gLite job management tools 12E. Ronchieri, EGI TF, September 2011, Lyon, France

Req. #3: Self-allocation of systems by users of a traditional computing center Local users need direct and local access (or interactive access) to the batch system or the distributed infrastructure without requiring root access. WNoDeS provides a mode called Virtual Interactive Pool (VIP) [3] – Systems are provisioned so that users can log on with their local account – Users can specify characteristics of VM such as machine type, number of CPUs, amout of RAM, local filesystems to be mounted – This is a kind of cloud computing applied to a traditional computing center designed to efficiently offer new services, without incurring the overhead to dedicate resources for this purpose – A caching mechanism is available to reduce provisioning time 13E. Ronchieri, EGI TF, September 2011, Lyon, France

Req. #4: Self-allocation of compute resources by distributed users (Cloud Computing) Distribute users like to autonomously self- allocate compute resources defined by several parameters WNoDeS can deliver Cloud services via [6] – A standard API called the Open Cloud Computing Interface (OCCI), developed by the Open Grid Forum (OGF) Rarely employed directly by users – A Cloud Web portal A user-friendly way to self-allocate resources 14E. Ronchieri, EGI TF, September 2011, Lyon, France

Req. #5: Resource access by distributed users through multiple authentication methods Users need to be authenticated whatever is the users’ authentication method WNoDeS can use any authentication method as long as it can be mapped into a known and unique resource ownership [2] – Ownership can be shared across different resource interfaces such as Grid and Cloud – Non-Grid users will be authenticated as well using Kerberos or Shibboleth by the IGI Web Portal [6] 15E. Ronchieri, EGI TF, September 2011, Lyon, France

Status WNoDeS 1 is in production at the INFN Tier-1 and at other INFN sites in Italy – Focusing on providing virtualization services for local and Grid accesses WNoDeS 2 is planned for release in November 2011 – Introducing (among other things) the Cloud portal, the VIP interface and support for multiple batch systems WNoDeS 2 installation and configuration also works with Torque and Maui as Open Source Batch System (tested at INFN Bari) – Using properties for queue, jobs and WN WNoDeS 2 configuration also works with Lustre as Open Source Shared File System to serve data and distribute Virtual Node images (tested at INFN Bari) – Observed good performance and scalability 16E. Ronchieri, EGI TF, September 2011, Lyon, France

Developments Program focuses in particular on: – Dynamic network virtualization – VM image abstraction – Efficient access to large storage systems Gateway NFS/GPFS – Inter-cloud connectivity 17E. Ronchieri, EGI TF, September 2011, Lyon, France

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 18E. Ronchieri, EGI TF, September 2011, Lyon, France

WNoDeS Cloud Web Interface 19E. Ronchieri, EGI TF, September 2011, Lyon, France

Process Flow for the WNoDeS Web-Based Cloud Management 20E. Ronchieri, EGI TF, September 2011, Lyon, France

Mixed Mode We are investigating the possibility to use the same physical WN to execute both real jobs and VM jobs – This will increase the flexibility of the farm and allow better optimization of the resource usage There is no need to “reserve” WN to WNoDeS 21E. Ronchieri, EGI TF, September 2011, Lyon, France

Mixed Mode Old WNoDeS New WNoDeS Batch Server Manager WN Virtual Machine Standard job WNoDeS jobs & Standard jobs WN WNoDeS enabled WNs Virtual Machine Non WNoDeS WNs Batch Server Manager WNoDeS jobs Standard jobs WN Standard job 22E. Ronchieri, EGI TF, September 2011, Lyon, France

Software WNoDeS has been ported into ETICS during summer 23 – To handle daily builds via local remote build in order to get repository RPMs – To automate its deployment via WNoDeS metapackages – To adopt ETICS quality checks – To automate internal test procedures E. Ronchieri, EGI TF, September 2011, Lyon, France

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 24E. Ronchieri, EGI TF, September 2011, Lyon, France

WNoDeS and IGI IGI, the Italian Grid Infrastructure, plans to offer an integrated computing infrastructure providing – Grid computing services – Cloud computing services – Inter-connection of Cloud computing resource centres for the Italian and research community The WNoDeS framework is the production solution for integrating Grid and Cloud infrastructure offered by IGI 25E. Ronchieri, EGI TF, September 2011, Lyon, France

Overview Problem Statement WNoDeS as solution New features and why WNoDeS and IGI Details 26E. Ronchieri, EGI TF, September 2011, Lyon, France

Information For further information and questions – WNoDeS mailing list, The WNoDeS Web site – Acknoledgements for this talk – D.Salomoni (INFN CNAF), A. Italiano (INFN CNAF), G. Dalla Torre (INFN CNAF), D. Andreotti (INFN CNAF), G. Donvito (INFN Bari), V. Spinoso (INFN Bari) 27E. Ronchieri, EGI TF, September 2011, Lyon, France

References [1] A. Italiano, et al, WNoDeS, a Tool for Integrated Grid/Cloud Aaccess and Computing Farm Virtualization, CHEP 2010, Taipei [2] D. Salomoni, et al, An Authentication Gateway for Integrated Grid and Cloud Access, CHEP 2010, Taipei [3] C. Grandi, et al, Virtual Pools for Interactive Analysis and Software Development through an Integrated Cloud Environment, CHEP 2010, Taipei [4] D. Salomoni, A. K. Calabrese Melcarne, A. Chierici, G. Dalla Torre, A. Italiano, “Performance improvements in a large scale virtualization system”, PoS(ISGC 2011 & OGF 31)049 [5] D. Salomoni, D. Andreotti, L. Cestari, G. Potena, P. Solagna, “A Web-based portal to access and manage WNoDeS Virtualized Cloud resources”, PoS(ISGC 2011 & OGF 31)054 [6] M. Bencivenni, et al, “A portal for an easy access to the IGI grid infrastructure”, poster 9 at EGI TF, September 2011, Lyon, France [7] D. Salomoni, E. Ronchieri, “WNoDeS: Requirements for Virtualized Services”, 28E. Ronchieri, EGI TF, September 2011, Lyon, France

Thanks! 29E. Ronchieri, EGI TF, September 2011, Lyon, France

Backup Slides 30E. Ronchieri, EGI TF, September 2011, Lyon, France

Measurements Using Lustre shared file-system to distribute Virtual Node images determines: – Good performance Tested already at 3-5GByte/s on a farm with 700TB of storage – Scalability Already tested with thousands of client and hundreds of file servers 31E. Ronchieri, EGI TF, September 2011, Lyon, France