Predrag Buncic (CERN/PH-SFT) Virtualization – the road ahead

CERN, 18/06/ Clouds over Grid Over past couple of years new buzzwords appeared threatening to cast the shadow over Grid Amazon Elastic Computing Cloud –Breakthrough in industry approach to computing –Everything is for rent (CPU, Storage, Network, Accounting) Blue Cloud (IBM) is comming Software as a Service (SaaS) Google App Engine Virtual Software Appliances and JeOS In all this cases, the virtualisation is emerging as a key enabling technology Trend supported by CPU manufacturers  Multiple cores  Hardware virtualization (Intel VT, AMD-V)

CERN, 18/06/ One problem we could tackle using this technology LHC Experiment(s)  Millions of lines of code  Complicated software installation/update/configuration procedure, different from experiment to experiment  Only tiny portion of it is really used at runtime in most of the cases  Often incompatible or lagging behind OS version on desktop/laptop Multi core CPUs with hardware support for virtualization  Making laptop/desktop ever more powerful and underutilised Using virtualization and extra cores to get extra comfort  Zero effort to install, maintain and keep up to date the experiment software  Reduce the cost of software development by reducing the number of compiler-platform combinations  Decouple application lifecycle from evolution of system infrastructure

CERN, 18/06/ Choosing the right way..

CERN, 18/06/ How do we want to do that? Build a “thin” Virtual Software Appliance for use by LHC experiments This appliance should  provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid  be independent of physical software and hardware platform (Linux, Windows, MacOS) This should minimize the number of platforms (compiler-OS combinations) on which experiment software needs to be supported and tested thus reducing the overall cost of LHC software maintenance. Funded as R&D work package in the framework of White Paper  All this to be done in collaboration with LHC experiments, OpenLab reusing existing solutions where possible  Resources 7.5 FTE (staff) FTE (fellow) over 4 years

CERN, 18/06/ Conary Recipe Based on rPath Linux

CERN, 18/06/ Groups & Builds Build types  Installable CD/DVD  Stub Image  Raw Filesystem Image  Netboot Image  Compressed Tar File  Demo CD/DVD (Live CD/DVD)  Raw Hard Disk Image  Vmware ® Virtual Appliance  Vmware ® ESX Server Virtual Appliance  Microsoft ® VHD Virtual Apliance  Xen Enterprise Virtual Appliance  Virtual Iron Virtual Appliance  Parallels Virtual Appliance  Amazon Machine Image  Update CD/DVD  Appliance Installable ISO

CERN, 18/06/ “Thin” Software Appliance JeOS (based on rPath Linux) rAArAA KERNEL fuse module FILESYSTEMFILESYSTEM rAA plugin Extra Liibs & Apps Cache HTTPDHTTPD Software Repository 10 GB1 GB0.1 GB LAN/WAN (HTTP)

CERN, 18/06/ Beta Release 0.6 Available for download from  Can be run on  Linux (Xen 3.1,VMware Player or Server)  Windows(WMware Player or Server) free  Mac (Fusion, Parallels) both require a license Release Notes  HowTo  Limited testing to verify that appliance can be configured and used with Alice, LHCb, ATLAS (and CMS) software frameworks

CERN, 18/06/

CERN, 18/06/

CERN, 18/06/

CERN, 18/06/ NOTE - initial settings are: User Name: admin Password: password

CERN, 18/06/ Please change Web admin password!

CERN, 18/06/ Select experiment for which VM will be configured

CERN, 18/06/ Status page

CERN, 18/06/ Select your top level group and migrate appliance configuration to get the latest version

CERN, 18/06/ Create local user account. First account created using this interface will be allowed to to executed sudo comand

CERN, 18/06/ Hands on… pbuncic$ ssh -X password: Last login: Fri Jul 11 10:15: from ~]$ mkdir.alien ~]$ scp -r ~]$ alienv list AliRoot/v4-13-Rev-02 GEANT3/v1-9 ROOT/v

CERN, 18/06/ ~]$ alienv use AliRoot/v4-13-Rev-02 … Great! Your compiler setup is ok! ****************************************************** Attention: You don't have a valid grid proxy - doing grid-proxy-init ****************************************************** Your identity: /DC=ch/DC=cern/OU=…. Enter GRID pass phrase for this identity: Creating proxy Done Your identity: pbuncic Creating token Done Your token is valid until: Sat Jul 12 10:18:

CERN, 18/06/ bash-3.00$ which aliroot /opt/alice/packages/AliRoot/v4-13-Rev-02/bin/tgt_linux/aliroot bash-3.00$ aliroot -b ******************************************* * W E L C O M E to R O O T * * Version 5.19/04 14 May 2008 * * You are welcome to visit our Web site * * * ******************************************* ROOT 5.19/04 Jun , 02:19:19 on linux) CINT/ROOT C/C++ Interpreter version , Jan 08, 2008 Type ? for help. Commands must be C++ statements. Enclose multiple statements between { }. root [0]

CERN, 18/06/ bash-3.00$ cernvm-cache -i CONFIGURATION VARIABLES CERNVM_CONFIG_URL= CERNVM_ORGANISATION=ALICE CERNVM_PLATFORM=i686-pc-linux-gnu CURRENT LOCATION Public Hostname: pb-d cern.ch Domain: cern.ch Time zone: Europe/Zurich Location: Geneva WEB FILE SYSTEM Server: cernvm-webfs.cern.ch Protocol: grow Cashe directory: /tmp/parrot.500 Cache size: 191 MB Cache: ~200 MB

CERN, 18/06/ Other interesting projects BOINC  Open-source software for volunteer computing and grid computing  CernVM is now extended to include BOINC client  Summer student project in OpenLab based on experience and CernVM image to run ATLAS simulation using BOINC infrastructure  Globus Virtual Workspace Service  An open source infrastructure for the deployment and management of virtual machines  Google Summer School (hosted at ANL) project to deploy a one- click, auto-configuring virtual Grid overlay for Alice/AliEn 

CERN, 18/06/ Roadmap Release 0.6 (30/06/08)  Bug fixes  Adding missing packages and features based on feedback from experiments  Addressing SLC4 binary compatibility issues (new package group structure) Release 0.7 (04/08/08)  Write access to software repository Allowing experiments to publish their releases  Support for x86_64 platform Release 0.8 (01/09/08)  New (simplified and improved) file system Release 0.9 (29/09/08)  Bug fixes Release 1.0 (03/11/08)

CERN, 18/06/ Mailing lists  (Open list to discuss about design, user experience and related issues with the CernVM project)  (End-user support for the CernVM project) Savannah Portal  Please submit bugs and feature requests to Savannah at Web site(s)   Building the community…