Blueprint of Persistent Infrastructure as a Service for NA61 Experiment “Enabling long term data preservation using virtualization technology” CernVM - SoFTware Development for Experiments Group @CERN Predrag Buncic & Mihajlo Mudrinic NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland

Requirements for the new IaaS NA61 Collaboration Meeting Remainder :: Historical Background and Definition of Goals and Objectives The NA61 Collaboration together with CernVM (R&D in PH/SFT Group) initiated project: “Enabling long term data preservation using virtualization technology” (The project got 6 months (1/2 FT PJAS) financial support from CERN Directorate) The project focus is on providing support for the NA61 Experiment and facilitate smooth transition to the new NA61 software framework for simulation, reconstruction and data analysis. It will also help the NA61 Collaboration to embrace and take advantage of new “Cloud Computing” paradigm. What we want to deliver and why? Build so called Infrastructure as a Service (IaaS) which will provide job work-load management system and experimental data access using virtualization technologies. Requirements for the new IaaS The NA61 software framework should be independent of underlying operating system and hardware! To make flexible and scalable testbed for the software development and assure long- term stability of the software environment for data acquisition and data analysis! Reproducibility of the reconstruction/simulation results at the beginning of data taking period should be available over the next two decades! NA61 Collaboration Meeting 15th February 2011, Warsaw , Poland Page 2.

NA61 Collaboration Meeting CernVM Cloud Infrastructure as a Service CernVM Cloud is a production environment which is scalable and independent on a hosting computer farm hardware infrastructure. As a building block of the CernVM Cloud production environment we are using CernVM image. The CernVM image provides portable and easy to configure user environment in form of a Virtual Machine. The CernVM image supports automatic update of new software releases via web protocol HTTP and CernVM-FS file system. The CernVM image has release number and it can be recreated on exactly the same specification in future independent on version of hypervisor. CernVM Cloud will also provide network isolation of VM instances preventing possible security risk provoked by running legacy system software. CernVM Software Repository NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 3.

NA61 Collaboration Meeting Components of CernVM Cloud OpenNebula is a virtual infrastructure engine that enables the dynamic deployment and re-allocation of virtual machines in a pool of physical resources. It manages virtual network, computing resources, storage capacity, information handling and accounting. CernVM for NA61. KVM - Linux native hypervisor. Condor - Condor is a specialized workload management system for compute intensive jobs. It provides a job querying mechanism, scheduling policy, resource monitoring and resource execution management. Xrootd server - High performance file server, used by LHC experiments and distributed with ROOT. Xrootd redirector – Able to aggregate up to 64 servers, can cluster up to 64k servers. Xrootd supports access to MSS systems using SRM extension. Standard HTTP proxy (Squid). CernVM-FS repository for software distribution. xcfs - POSIX File System for Xrootd with Castor backend. GANGA - User interface for job submission. Supports various front-ends to batch systems and Grid. Ganglia - Monitoring and accounting. CernVM Storage Server Batch Worker Hypervisor 11 1 9 4 3 2 Storage Proxy 5 HTTP Proxy 7 NAT Gateway Batch Master 3 6 8 10 MSS S/W TCP/IP API NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 4.

NA61 Collaboration Meeting Components of CernVM Cloud: OpenNebula Feature Function Internal Interface Unix-like CLI for fully management of VM life-cycle and physical boxes XML-RPC API and libvirt virtualization API Scheduler Requirement/rank matchmaker allowing the definition of workload and resource-aware allocation policies Virtualization Management Xen, KVM, and VMware Image Management General mechanisms to transfer and clone VM images Network Management Definition of isolated virtual networks to interconnect VMs Service Management and Contextualization Support for multi-tier services consisting of groups of inter-connected VMs, and their auto-configuration at boot time Security Management of users by the infrastructure administrator Fault Tolerance Persistent database backend to store host and VM information Scalability Tested in the management of medium scale infrastructures with hundreds of servers and VMs (no scalability issues has been reported) Flexibility and Extensibility Open, flexible and extensible architecture, interfaces and components, allowing its integration with any product or tool NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 5.

NA61 Collaboration Meeting Components of CernVM Cloud: OpenNebula Part of services supported by existing CernVM Cloud OpenNebula + CernVM for NA61 + Ganglia NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 6.

NA61 Collaboration Meeting Components of CernVM Cloud: Condor work-load management system HostingNode2.cern.ch CernVMBN1 CernVMBN2 Condor Node Condor Node Hypervisor Job Execution HostingNode1.cern.ch HostingNode3.cern.ch Condor Master Collector CernVMBN3 CernVMBN4 Negotiator Job Submission Scheduler Condor Node Condor Node CernVM-HN UserNode.cern.ch Hypervisor Manage Virtual Infrastructure NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 7.

NA61 Collaboration Meeting Components of CernVM Cloud: Ganga User Frontend for Job Submission Ganga is an user frontend for job definition and management. To submit a job the user create Python program/script which makes it easy to define and run user-level production or data analysis. User should provide ssh-rsa public key to the CernVM Cloud administrator to get authorization to run job on the CernVM Cloud. Ganga also provides graphic user interface and plugin classes for different types of application, backend, dataset, splitter and merger and is widely used by LHC Experiments. NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 8.

NA61 Collaboration Meeting Components of CernVM Cloud: Ganga Graphical User Interface NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 9.

NA61 Collaboration Meeting CernVM Cloud The Project Timeline 2010 2011 O N D J F M A M J J A S Legend Deliverable Date Status M1 CernVM version 2.2.0 31/10/2010 DONE T1 NA61 release certification in CernVM 2.2.0 ???? T2 Testing components, designing system 30/11/2010 T3 Installing Cluster 15/12/2010 M2 Cluster installed T4 Developing and testing job submission system, Ganga interface, monitoring 31/01/2011 T5 Developing and testing xrootd, xcfs 28/02/2011 IN PROGRESS T6 Testing complete system 31/03/2011 M3 First usable version 30/10/2011 M1 CernVM 2.2.0 T1 T2 T3 M2 T4 T5 T6 M3 NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 10.

NA61 Collaboration Meeting IN PROGRESS CernVM Cloud: Xrootd data access system Application Xroot Client Linux Xroot Server Redirector Access to Xrootd hosted data xrdcp native copy command via root framework direct using file name root://… manualy using TXNetFile() object POSIX preload library bild your own library for the Collaboration. GridFTP The LHC grid data access (FUSE/POSIX Library). BeStMan (SRM add-on) srmpc for srm-to-srm copy native set A CernVM B Grid HYPERVISOR HARDWARE CASTOR NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 11.

NA61 Collaboration Meeting Summary and outlook Work on the prototype of CernVM Cloud as a production environment for the NA61 experiment is going according to the schedule. CernVM Cloud Infrastructure as a Service will remove NA61 software dependency on underlying OS and hardware. The project is introducing cloud computing paradigm combined with the most current software technologies and services used by LHC Experiments. CernVM Cloud will deliver scalable infrastructure integrated with the current NA61 software framework and it will be at the same time testing infrastructure for the new software development. At the beginning of March 2011 we are expecting to have all components of CernVM Cloud in place (first prototype). During March 2011 we will start testing data access using Xrootd and CPU intensive jobs. NA61 Collaboration Meeting 15th February 2011, Warsaw, Poland Page 12.