1. Blueprint of Persistent Infrastructure as a Service
for NA61 Experiment
“Enabling long term data preservation using virtualization technology”
CernVM - SoFTware Development for Experiments Group @CERN
Predrag Buncic & Mihajlo Mudrinic
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland

2. Requirements for the new IaaS NA61 Collaboration Meeting
Remainder :: Historical Background and Definition of Goals and Objectives
The NA61 Collaboration together with CernVM (R&D in PH/SFT Group) initiated project:
“Enabling long term data preservation using virtualization technology”
(The project got 6 months (1/2 FT PJAS) financial support from CERN Directorate)
The project focus is on providing support for the NA61 Experiment and facilitate smooth transition to the new NA61 software framework for simulation, reconstruction and data analysis. It will also help the NA61 Collaboration to embrace and take advantage of new
“Cloud Computing” paradigm.
What we want to deliver and why?
Build so called Infrastructure as a Service (IaaS) which will provide job work-load management system and experimental data access using virtualization technologies.
Requirements for the new IaaS
The NA61 software framework should be independent of underlying operating system and hardware!
To make flexible and scalable testbed for the software development and assure long- term stability of the software environment for data acquisition and data analysis!
Reproducibility of the reconstruction/simulation results at the beginning of data taking period should be available over the next two decades!
NA61 Collaboration Meeting
15th February 2011, Warsaw , Poland
Page 2.

3. NA61 Collaboration Meeting
CernVM Cloud Infrastructure as a Service
CernVM Cloud is a production environment which is scalable and independent on a hosting computer farm hardware infrastructure.
As a building block of the CernVM Cloud production environment we are using CernVM image.
The CernVM image provides portable and easy to configure user environment in form of a Virtual Machine.
The CernVM image supports automatic update of new software releases via web protocol HTTP and CernVM-FS file system.
The CernVM image has release number and it can be recreated on exactly the same specification in future independent on version of hypervisor.
CernVM Cloud will also provide network isolation of VM instances preventing possible security risk provoked by running legacy system software.
CernVM
Software Repository
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 3.

4. NA61 Collaboration Meeting
Components of CernVM Cloud
OpenNebula is a virtual infrastructure engine that enables the dynamic deployment and re-allocation of virtual machines in a pool of physical resources. It manages virtual network, computing resources, storage capacity, information handling and accounting.
CernVM for NA61.
KVM - Linux native hypervisor.
Condor - Condor is a specialized workload management system for compute intensive jobs. It provides a job querying mechanism, scheduling policy, resource monitoring and resource execution management.
Xrootd server - High performance file server, used by LHC experiments and distributed with ROOT.
Xrootd redirector – Able to aggregate up to 64 servers, can cluster up to 64k servers.
Xrootd supports access to MSS systems using SRM extension.
Standard HTTP proxy (Squid).
CernVM-FS repository for software distribution.
xcfs - POSIX File System for Xrootd with Castor backend.
GANGA - User interface for job submission. Supports various front-ends to batch systems and Grid.
Ganglia - Monitoring and accounting.
CernVM
Storage
Server
Batch
Worker
Hypervisor 11 1 9 4 3 2
Storage
Proxy 5
HTTP
Proxy 7
NAT
Gateway
Batch
Master 3 6 8 10
MSS
S/W
TCP/IP
API
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 4.

5. NA61 Collaboration Meeting
Components of CernVM Cloud: OpenNebula
Feature
Function
Internal Interface
Unix-like CLI for fully management of VM life-cycle and physical boxes
XML-RPC API and libvirt virtualization API
Scheduler
Requirement/rank matchmaker allowing the definition of workload and resource-aware allocation policies
Virtualization Management
Xen, KVM, and VMware
Image Management
General mechanisms to transfer and clone VM images
Network Management
Definition of isolated virtual networks to interconnect VMs
Service Management and Contextualization
Support for multi-tier services consisting of groups of inter-connected VMs, and their auto-configuration at boot time
Security
Management of users by the infrastructure administrator
Fault Tolerance
Persistent database backend to store host and VM information
Scalability
Tested in the management of medium scale infrastructures with hundreds of servers and VMs (no scalability issues has been reported)
Flexibility and Extensibility
Open, flexible and extensible architecture, interfaces and components, allowing its integration with any product or tool
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 5.

6. NA61 Collaboration Meeting
Components of CernVM Cloud: OpenNebula
Part of services supported by existing
CernVM Cloud
OpenNebula + CernVM for NA61 + Ganglia
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 6.

7. NA61 Collaboration Meeting
Components of CernVM Cloud: Condor work-load management system
HostingNode2.cern.ch
CernVMBN1
CernVMBN2
Condor Node
Condor Node
Hypervisor
Job Execution
HostingNode1.cern.ch
HostingNode3.cern.ch
Condor Master
Collector
CernVMBN3
CernVMBN4
Negotiator
Job Submission
Scheduler
Condor Node
Condor Node
CernVM-HN
UserNode.cern.ch
Hypervisor
Manage Virtual Infrastructure
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 7.

8. NA61 Collaboration Meeting
Components of CernVM Cloud: Ganga User Frontend for Job Submission
Ganga is an user frontend for job definition and management.
To submit a job the user create Python program/script which makes it easy to define and run user-level production or data analysis.
User should provide ssh-rsa public key to the CernVM Cloud administrator to get authorization to run job on the CernVM Cloud.
Ganga also provides graphic user interface and plugin classes for different types of application, backend, dataset, splitter and merger and is widely used by LHC Experiments.
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 8.

9. NA61 Collaboration Meeting
Components of CernVM Cloud: Ganga Graphical User Interface
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 9.

10. NA61 Collaboration Meeting
CernVM Cloud The Project Timeline
2010
2011 O N D J F M A M J J A S
Legend
Deliverable
Date
Status M1
CernVM version 2.2.0
31/10/2010
DONE T1
NA61 release certification in CernVM 2.2.0
???? T2
Testing components, designing system
30/11/2010 T3
Installing Cluster
15/12/2010 M2
Cluster installed T4
Developing and testing job submission system, Ganga interface, monitoring
31/01/2011 T5
Developing and testing xrootd, xcfs
28/02/2011
IN PROGRESS T6
Testing complete system
31/03/2011 M3
First usable version
30/10/2011 M1
CernVM 2.2.0 T1 T2 T3 M2 T4 T5 T6 M3
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 10.

11. NA61 Collaboration Meeting
IN PROGRESS
CernVM Cloud: Xrootd data access system
Application
Xroot Client
Linux
Xroot Server
Redirector
Access to Xrootd hosted data
xrdcp
native copy command
via root framework
direct using file name root://…
manualy using TXNetFile() object
POSIX preload library
bild your own library for the Collaboration.
GridFTP
The LHC grid data access (FUSE/POSIX Library).
BeStMan (SRM add-on)
srmpc for srm-to-srm copy
native set A
CernVM B
Grid
HYPERVISOR
HARDWARE
CASTOR
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 11.

12. NA61 Collaboration Meeting
Summary and outlook
Work on the prototype of CernVM Cloud as a production environment for the NA61 experiment is going according to the schedule.
CernVM Cloud Infrastructure as a Service will remove NA61 software dependency on underlying OS and hardware.
The project is introducing cloud computing paradigm combined with the most current software technologies and services used by LHC Experiments.
CernVM Cloud will deliver scalable infrastructure integrated with the current NA61 software framework and it will be at the same time testing infrastructure for the new software development.
At the beginning of March 2011 we are expecting to have all components of CernVM Cloud in place (first prototype).
During March 2011 we will start testing data access using Xrootd and CPU intensive jobs.
NA61 Collaboration Meeting
15th February 2011, Warsaw, Poland
Page 12.