Virtualization and Clouds ATLAS position Simone Campana (CERN-IT/ES) on behalf of ATLAS Computing

Clouds and Virtualization ATLAS R&D project on Cloud and Virtualization evaluate the available cloud technologies in relation to the use-cases presented by ATLAS data management, processing and analysis design a model for transparently integrating cloud computing resources with the ADC software and service stack implement the ATLAS cloud computing model into DDM, PanDA and related tools and services. Dedicated activities on CVMFS and Multicores (see later) R&D coordinators daniel.colin.vanderster@cern.ch fernando.harald.barreiro.megino@cern.ch kaushik@uta.edu rodney.walker@physik.uni-muenchen.de

ATLAS Distributed Computing use cases to be explored Clouds R&D use cases ATLAS Distributed Computing use cases to be explored Monte Carlo simulation on the cloud with stage-out to traditional grid storage or long-term storage on the cloud Data reprocessing in the cloud (with a strong caveat related to cost Distributed analysis on the cloud using data which is accessed remotely to the grid sites, or analysis of data which is located in the cloud Resource capacity bursting which is managed centrally (e.g. to handle urgent reprocessing tasks) or regionally (e.g. to handle urgent local analysis requests)

Clouds R&D plans and milestones Initial Steps Get access to existing academic clouds Work ongoing at Magellan@Argonne and Magellan@LBNL Many other flavors at various locations are being considered. StratusLab@EU, OpenNebula@CERN, Eucalyptus@Edinburgh, Nimbus@CA … Some work going on also on commercial clouds BNL investigating on how to expand capacity into Amazon to absorb peaks Cloud-like environment already in production for ATLAS at CNAF WNoDeS integrated with Panda for simulation jobs. Understand the protocols and working model Both workload and data management Define the scenario for adaptation of existing ATLAS services to clouds Panda, DDM, AGIS, … GOAL: Prototype by end of 2011 for Simulation, Reprocessing and Analysis Kickstart workshop: https://indico.cern.ch/conferenceDisplay.py?confId=136751 talks from ATLAS, CMS, BaBar, CERN IT -- open attendance ----- Meeting Notes (5/10/11 15:41) -----

Clouds and Virtualization The R&D is also responsible in ATLAS to define a model, together with WLCG, for defining and using Virtual Machines at sites. Needs to be defined: Who builds the VM and what gets inside How the VM is instantiated at the site and by who Requirements on the VM (cores, memory, disk cache,…) (Many other bits to be sorted out, once you think about it) Would be good to have one WLCG proposal and discuss it with the VOs

(Virtualization) and Multicores Usage of multicores requires a dedicated discussion And in ATLAS there is a dedicated TF Only “soft” correlation with virtualization Some tests on 8 cores VMs Pushed back till AthenaMP is in production “on the bare metal” (P. Calafiura) Status AthenaMP is ready with caveats to run in a multicore environment Adaptations to the ATLAS framework (Panda) at good stage Successful tests in dedicated queue at CERN_8CORE (“all node”) and other sites ATLAS might be ready to utilize this in production very soon https://indico.cern.ch/getFile.py/access?contribId=7&sessionId=4&resId=0&materialId=slides&confId=119169 Based on outcome, ATLAS might decide to push for the “all node” approach Too early to discuss this now ATLAS Contacts douglas@cern.ch, lblcalaf@gmail.com

CVMFS ATLAS would like to use CVMFS for: Distribution (on and off Grid) of the official ATLAS Software (athena) Distribution of the Computing Client tools (DDM, AGIS, AMI, …) and their dependencies if needed Distribution of the Software Nightlies Distribution of the Condition Data FLATFILES

CVMFS status ATLAS is ready to use CVMFS for 1. and 2. Installation system has been adapted Just configuration changes for Panda queues Repository deployed and supported today by PH-SFT ATLAS is ready to test CVMFS for 3. Repository running on ATLAS VOBOX at CERN ATLAS can test access to Condition Data via CVMFS (4.) “today” Repository contains currently relevant condition files Work being done for automation in adding new conditions Now needs to be evaluated in comparison with “standard” mechanism (“HOTFILES” in the SE) Performance, operational impact, impact on the infrastructure

CVMFS deployment ATLAS has very positive experience with CVMFS Both in testing phase and production ATLAS would like to ask for a broad deployment of CVMFS starting from now. Rollout plan? Ultimately ATLAS understand that the CVMFS service will be run by CERN IT. Timescale? ATLAS contacts douglas.benjamin@cern.ch, desilva@triumf.ca (overall coordination) undrus@bnl.gov (nightlies) Alessandro.DeSalvo@roma1.infn.it (Grid SW installation) Mikhail.Borodin@cern.ch (Condition Data)