Predrag Buncic (CERN/PH-SFT) CernVM - a virtual software appliance for LHC applications C. Aguado-Sanchez 1), P. Buncic 1), L. Franco 1), A. Harutyunyan 2), P. Mato 1), Y. Yao 3) 1) CERN, Geneva, 2) Yerevan Physics Institute, Yerevan, 3) LBNL, Berkeley

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ CernVM Project Talk Outline  CernVM Project Building blocks User Interface and API Scalability and performance  Future plans & directions  Release status  Conclusions  Links

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ CernVM Project Portable Analysis Environment using Virtualization Technology (WP9)  Approved in 2007 (2+2 years) as R&D activity in CERN/PH Department  Started January 2008  Sister project to Multicore R&D Project goals:  Provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid independent of physical software and hardware platform (Linux, Windows, MacOS)  Decouple application lifecycle from evolution of system infrastructure  Reduce effort to install, maintain and keep up to date the experiment software  Lower the cost of software development by reducing the number of compiler-platform combinations

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ A complete Data Analysis environment available for each experiment  Code check-out, edition, compilation, local small test, debugging, …  Grid submission, data access…  Event displays, interactive data analysis, … No software installation required Suspend/resume capability

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Key Building Blocks rBulder from rPath (  A tool to build VM images for various virtualization platforms rPath Linux 1  Slim Linux OS binary compatible with RH/SLC4 rAA - rPath Linux Appliance Agent  Web user interface  XMLRPC API Can be fully customized and extended by means of plugins (  401) CVMFS - CernVM file system  Read only file system optimized for software distribution Aggressive caching  Operational in offline mode For as long as you stay within the cache Build types  Installable CD/DVD  Stub Image  Raw Filesystem Image  Netboot Image  Compressed Tar File  Demo CD/DVD (Live CD/DVD)  Raw Hard Disk Image  Vmware ® Virtual Appliance  Vmware ® ESX Server Virtual Appliance  Microsoft ® VHD Virtual Apliance  Xen Enterprise Virtual Appliance  Virtual Iron Virtual Appliance  Parallels Virtual Appliance  Amazon Machine Image  Update CD/DVD  Appliance Installable ISO

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ class Root(CPackageRecipe): name='root' version=' ' buildRequires = ['libpng:devel', 'libpng:devellib','krb5:devel', 'libstdc++:devel’,'libxml2:devel', 'openssl:devel','python:devel', 'xorg-x11:devel', 'zlib:devel', 'perl:devel', 'perl:runtime'] def setup(r): r.addArchive('ftp://root.cern.ch/root/%(name)s_v%(version)s.source.tar.gz’) r.Environment('ROOTSYS',%(builddir)s') r.ManualConfigure('--prefix=/opt/root ') r.Make() r.MakeInstall() Conary Package Manager

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ CernVM Variations group-cernvm (core packages) group-cernvm (core packages) group-cernvm-devel (development tools) group-cernvm-devel (development tools) group-slc4 (SLC4 compatibility libs) group-slc4 (SLC4 compatibility libs) group- (groups and extra packages required by experiment) group- (groups and extra packages required by experiment) 100 MB compat-db4 compat-openssl compat-libstdc++slc3 compat-libxml2 compat-readline compat-tcl compat-tk group- -desktop (lightweight X environment) group- -desktop (lightweight X environment) group-cernvm-desktop X11 group-cernvm-desktop X11

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ import xmlrpclib import os server = xmlrpclib.ServerProxy(url) r = server.cernvm.Config.configGridUIVersion(” ") As easy as 1,2,3 1. Login to Web interface 2. Create user account 3. Select experiment, appliance flavor and preferences

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ CVMFS CernVM File System (CVMFS) is derived from Parrot ( and its GROW-FS plugin code base and adapted to run as a FUSE kernel module adding extra features like:  possibility to use multiple file catalogues on the server side  transparent file compression under given size threshold  dynamical expansion of environment variables embedded in symbolic links

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Scalable Infrastructure

Publishing Releases 1. Each experiment is given a VM to install and test their software using own installation tools 2. Publishing is an atomic operation

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Primary (master) software repository Secondary software repository Secondary software repository Reverse Proxy Load Balancing Reverse Proxy Load Balancing Reverse Proxy CernVM (thin client) Site Reverse Proxy Load Balancing Reverse Proxy Regional Reverse Proxy The aim is to reduce latency which is the most important issue for distributed network file systems 3/9/0912 Current deployment model

Benchmark setup ROOT (version 5.22, 100Mb) The benchmark runs in Xen VM and includes compilation and execution of ROOT stressHepix suite  The system has to find and fetch binaries, libraries and headers from the network file system but, once loaded, the libraries remain in a local cache We compare the performance of CVMFS to AFS by artificially introducing network latency and limiting bandwith (using tc tool)

CVMFS vs AFS These plots show the time penalty  t=t AFS,CVMFS - t local  resulting from having the application binaries, search paths, and include paths reside on a network file system,  Additional performance loss due to virtualization (not shown here) is consistently <5% CVMFS shows consistently better performance than AFS in case of ‘cold cache’  ‘Worm cache’ performance is in case of AFS and CVMFS equal to local file access

Proxy Server Proxy Server CernVM HTTP server HTTP server Proxy Server HTTP server HTTP server Proxy Server HTTP server HTTP server Proxy Server Removing single point of failure (1) Slave servers could be deployed on strategic locations to reduce latency and provide redundancy

HTTP server HTTP server Proxy Server CernVM Content Distributio n Network WAN Use existing Content Delivery Networks to remove single point of failure  Amazon CloudFront (  Coral CDN ( LAN Use P2P like mechanism for discovery of nearby CernVMs and cache sharing between them. No need to manually setup proxy servers (but they could still be used where exist) Removing single point of failure (2)

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Bridging Grids & Clouds BOINC  Open-source software for volunteer computing and grid computing  CernVM CoPilot development  Based on BOINC, experience and CernVM image  Image size is of outmost importance to motivate volonteers  Can be easily adapted to Pilot Job frameworks (AliEn,Dirac, Panda) … or Condor Worker, or proofd..  Aims to demonstrate running of ATLAS simulation using BOINC infrastructure and PanDa BOINC E PanDA Pilot

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Adapter CernVM CoPilot 0. Send host JDL (free disk space, free memory, available packages) 1. Append framework specific information and request a job 3. Send input files and commands for execution (packages are already there) 5. Register output files 4. When the job is done send back the output files (and the result of validation) 2. Send user job JDL from Task Queue AliEn/DIRAC/PanDA

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Simplifying Grid deployment Nimbus (former Globus Workspace Service)  Nimbus is a set of open source tools that together provide an "Infrastructure-as-a-Service" (IaaS) cloud computing solution  Google Summer School (hosted at ANL) project to deploy a one-click, auto-configuring virtual Grid overlay for Alice/AliEn Successfully created virtual AliEn site for ALICE with one command See CHEP’09 contribution by Artem Harutyunyan (poster #214)

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Release now available Available now for download from  Can be run on  Linux (KVM, Xen,VMware Player, VirtualBox)  Windows(WMware Player, VirtualBox)  Mac (Fusion, Parallels, VirtualBox) Release Notes Appliance can be configured and used with ALICE, LHCb, ATLAS (and CMS) software frameworks  ALICE (AliEn 2.15,2.16, AliRoot v4-14-Rev-04, v4-15-Rev-03, v4-16-Rev-01)  ATLAS ( , , , , , )  CMS (CMSSW_2_2_5, CMSSW_2_2_6 )  LHCB (GAUDI_v19r8, GAUDI_v19r9, GAUDI_v20r0, GAUDI_v20r2, GAUDI_v20r3, GAUDI_v10r11, DAVINCI_v20r3, DAVINCI_v19r12, DAVINCI_v20r3,DAVINCI_v22r1)

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Summary If you work for one of the LHC experiments and you have one or a few problems listed below then CernVM might be what you are looking for:  Your experiment software is not compatible with your hardware or s/w platform running on your laptop/desktop  You do not want to spend time to manually install and keep software up to date  Your want to profit from the latest developments in CPU technology and use your multi/many core CPU to its maximal potential without modifying your application  You want to run your software on voluntary resources beyond the current Grid CernVM is a ‘thin’ virtual machines capable of running software of all four LHC experiments  Based on rPath Linux, binary compatible with SL4  Extensible user interface (plugins)  Package groups can be tailored to individual user group Experiment software is injected in VM by means of a file system (CVMFS) specially optimized for efficient software distribution  This allows us to develop, deploy and maintain only ONE image  Easy for end user but also great potential for deployment on cloud & grid  Using HTTP protocol allows efficient caching using standard technology Version available at

CHEP 2009 CernVM – A Virtial Machine for LHC Experiments Prague, 26/03/ Mailing lists  (open list for announcements and discussion) (end-user support for the CernVM project) Savannah Portal  Please submit bugs and feature requests to Savannah at CernVM Home Page:  rBuilder & Download Page:  ATLAS Wiki  CernVM Links…