1. 1 BOINC Case Study: LHC@home Ben Segal / CERN b.segal@cern.ch with: Predrag Buncic / CERN Daniel Lombrana Gonzalez / Univ. Extremadura David Weir / Imperial College ASIA@home Workshop, Academica Sinica Taiwan, April 16, 2009

2. 2 CERN Mont Blanc, 4810 m Geneva airport LHC tunnel CERN

3. 3 The LHC

4. 4 LHC@home Calculates stability of proton orbits in CERN’s new LHC accelerator System is nonlinear and unstable so numerically very sensitive. Hard to get identical results on all platforms About 40 000 users, 70 000 PC’s… over 1500 CPU years of processing Objectives: extra CPU power and raising public awareness of CERN and the LHC - both successfully achieved. Started as an outreach project for CERN 50 th Anniversary 2004; used for Year of Physics (Einstein Year) 2005

5. 5 SixTrack program SixTrack is a Fortran program by F. Schmidt, based on DESY program SixTrack simulates 60 particles for 100k-1M LHC orbits Can include measured magnet parameters, beam-beam interactions LHC@home revealed reproducibility issues, solved by E. McIntosh Phase space images of a particle for a stable orbit (left) and unstable chaotic orbit (right).

6. 6 >3000 CPU-years >60k volunteers

7. 7 Server managed by QMUL Cool screensaverMessage boards Credit for processing

8. 8 LHC@home future plans Sixtrack for LHC upgrade studies (W. Herr, CERN) Sixtrack for other accelerators (Yun Luo et al., Brookhaven National Lab) Garfield: for the detailed simulation of gases in detectors (R. Veenhof, CERN) Rivet, Jetweb: Monte Carlo event generator, validator and web archiver (J. Butterworth. UCL) ATLFAST and ATHENA: event simulation for ATLAS using virtualization (B. Segal, CERN) Volunteer thinking for LHC data analysis (discussions with Jim Virdee, John Ellis)

9. 9

10. 10

11. 11

12. 12 BOINC & Virtualization Work began at CERN in 2006 (w. Daniel Lombrana Gonzales + others) – Chose VMware Player over (k)QEMU, etc. – Proved feasibility and built a working prototype – Continued project at Univ. Extremadura Work continued at CERN in 2007 (w. David Weir) – Succeeded to build a VM for full ATLAS Athena environment – Reduced size of VM image by mounting runtime libraries over AFS

13. 13 BOINC & Virtualization Work accomplished at CERN in 2008 (w. Pedrag Buncic) – Established collaboration with the CernVM Project – General interface to all CERN physics software – Size of VM images now optimized and cached locally – Able to build VMs for full ATLAS, LHCb, ALICE and CMS environments and tested a VM with full Athena environment – Included an interface to physics groups’ production chain (tested with ATLAS PanDA job management)

14. 14 BOINC & Virtualization Work remaining at CERN in 2009 (in progress): – Improve Wrapper’s handling of VMware VM’s Cleanly Start, Stop, Suspend, Restart guest VM’s from host OS (using VIX API available for VMware Server) Monitor host OS resources of guest VM’s (for BOINC credit) OS’s to include Linux, Windows, possibly MacOS – Extend to Sun VirtualBox VM’s with their API – Possibly extend to kQEMU also

15. 15 Predrag Buncic (CERN/PH-SFT) CernVM - Virtual Machine for LHC Experiments

16. 16 CernVM Background Over the past couple of years, the industry is redefining the meaning of some familiar computing terms – Shift from glorious ideas of a large public infrastructure and common middleware towards end-to-end custom solutions and private corporate grids New buzzwords Amazon Elastic Computing Cloud (EC2) – Breakthrough in industry approach to computing – Everything is for rent (CPU, Storage, Network, Accounting) Blue Cloud (IBM) is coming Software as a Service (SaaS) Google App Engine Virtual Software Appliances and JeOS In all these cases, virtualization is emerging as a key enabling technology, and is supported by computer manufacturers – Multiple cores – Hardware virtualization (Intel VT, AMD-V)

17. 17 Motivation Software @ LHC Experiment(s) – Millions of lines of code – Complicated software installation/update/configuration procedure, different from experiment to experiment – Only a tiny portion of it is really used at runtime in most cases – Often incompatible or lagging behind OS versions on desktop/laptop Multi core CPUs with hardware support for virtualization – Making laptop/desktop ever more powerful and underutilised Using virtualization and extra cores to get extra comfort – Zero effort to install, maintain and keep up to date the experiment software – Reduce the cost of software development by reducing the number of compiler-platform combinations – Decouple application lifecycle from evolution of system infrastructure

18. 18 How do we want to do this? Build a “thin” Virtual Software Appliance for use by the LHC experiments This appliance should – provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid – be independent of physical software and hardware platforms (Linux, Windows, MacOS) This should minimize the number of platforms (compiler-OS combinations) on which experiment software needs to be supported and tested, thus reducing the overall cost of LHC software maintenance. All this is to be done – in collaboration with the LHC experiments and OpenLab – By reusing existing solutions where possible

19. 19 Key Building Blocks rBuilder from rPath (www.rpath.org) – A tool to build VM images for various virtualization platforms rPath Linux 1 – Slim Linux OS binary compatible with Red Hat / SLC4 rAA - rPath Linux Appliance Agent – Web user interface – XMLRPC API Can be fully customized and extended by means of plugins (  401) CVMFS - CernVM file system – Read-only file system optimized for software distribution Aggressive caching – Operational in offline mode For as long as you stay within the cache Build types – Installable CD/DVD – Stub Image – Raw File System Image – Netboot Image – Compressed Tar File – Demo CD/DVD (Live CD/DVD) – Raw Hard Disk Image – VMware ® Virtual Appliance – VMware ® ESX Server Virtual Appliance – Microsoft ® VHD Virtual Appliance – Xen Enterprise Virtual Appliance – Virtual Iron Virtual Appliance – Parallels Virtual Appliance – Amazon Machine Image – Update CD/DVD – Appliance Installable ISO – Sun Virtual Box Image

20. 20 “Thin” Software Appliance JeOS (based on rPath Linux) rAArAA KERNEL fuse module FILESYSTEMFILESYSTEM rAA plugin Extra Libs & Apps Cache HTTPDHTTPD Software Repository 10 GB1 GB0.1 GB LAN/WAN (HTTP)

21. 21 CernVM File System App On same host: /opt/lcg -> /chirp/localhost/opt/lcg open(“/opt/lcg”) On File Server /opt/lcg -> /grow/host/opt/lcg Cache Kernel NFSLFSFUSE CernVM Fuse !Cache

22. 22 Bridging Grids & Clouds BOINC – Open-source software for Volunteer Computing and Grid computing – CernVM is being extended to support BOINC client – http://boinc.berkeley.edu/ http://boinc.berkeley.edu/ CernVM CoPilot development – Based on BOINC, LHC@home experience and CernVM image – Image size is of utmost importance to motivate volunteers – Can be easily adapted to Pilot Job frameworks (AliEn,Dirac, PanDA) … or Condor Worker, or proofd.. – Aims to demonstrate running of ATLAS simulation using BOINC infrastructure and PanDA BOINC LHC@hom e PanDA Pilot

23. 23 Adapter CernVM CoPilot 0. Send host JDL (free disk space, free memory, available packages) 1. Append framework- specific information and request a job 3. Send input files and commands for execution (packages are already there) 5. Register output files 4. When the job is done send back the output files (and the result of validation) 2. Send user job JDL from Task Queue AliEn/DIRAC/PanDA

24. 24 Mailing lists – cernvm-talk@cern.ch (Open list to discuss about design, user experience and related issues with the CernVM project) – cernvm.support@cern.ch (End-user support for the CernVM project) Savannah Portal – Please submit bugs and feature requests to Savannah at http://savannah.cern.ch/projects/cernvm Reference Web site(s): – http://cernvm.cern.ch http://cernvm.cern.ch – http://rbuilder.cern.ch http://rbuilder.cern.ch Building the CernVM community…