Catalin Condurache STFC RAL Tier-1 GridPP OPS meeting, 10 March 2015

Short history September 2012 – non-LHC Stratum-0 at RAL gridpp.ac.uk CVMFS domain initially for UK VOs, then extended August 2013 – CVMFS TF kick-off meeting to establish a CVMFS infrastructure that allows EGI VOs to use it as a standard method of distribution of their software at grid sites

Recent developments Domain gridpp.ac.uk less likely to continue requests to create, host and publish repositories for various EGI VOs (not only UK) Following CVMFS WG meeting (March 2014) it was agreed that egi.eu would be the new CVMFS domain for new requests Work started to migrate gridpp.ac.uk -> egi.eu existing gridpp.ac.uk repositories duplicated as egi.eu

– current status 15 ‘established’ repositories (only 5 at kick-off) hosted and published at RAL – ~500 GB – egi.eu 6 ‘emerging’ repos – incl. UK reg. VOs (gridpp.ac.uk) List of VOs supported (egi.eu and gridpp.ac.uk) -biomed, cernatschool.org, glast.org, hone, hyperk.org, km3net.org, mice, na62.vo.gridpp.ac.uk, pheno, phys.vo.ibergrid.eu, snoplus.snolab.ca, t2k.org, enmr.eu List of VOs and research groups supported (egi.eu only) - auger, comet.j-parc.jp, ligo, supernemo List of VOs supported (gridpp.ac.uk only) - vo.londongrid.ac.uk, vo.scotgrid.ac.uk, vo.southgrid.ac.uk, vo.northgrid.ac.uk

RAL infrastructure Stratum-0 – VM, 20GB RAM, 1.2TB HDD CVMFS uploader – VM, 8GB RAM, 1TB HDD Stratum-1 – HA 2-node cluster, metal boxes, 32GB, 12TB cvmfs-egi it replicates egi.eu, opensciencegrid.org, desy.de, nikhef.nl plan to integrate with WLCG Stratum-1 (cernvmfs) Squid machines are shared for Frontier and LHC/non-LHC CVMFS access

Extended EGI infrastructure Stratum-0 at DESY and NIKHEF (not as egi.eu) 31 repos replicated at RAL (non-LHC, non-OSG) Other Stratum-1 replicas for egi.eu at NIKHEF, ASGC, TRIUMF Big help with cvmfs-keys v1.5 egi.eu configured by default more help with cvmfs v no longer CERN-centric configuration new puppet module available gridpp.ac.uk to be phased out outside UK soon will remain in use for geographically UK VOs only

EGI + OSG CVMFS infrastructure Stratum-0 NIKHEF nikhef.nl Stratum-1 NIKHEF Proxy Hierarchy Stratum-1 CERN Proxy Hierarchy Stratum-0 DESY desy.de Stratum-1 RAL Stratum-0 RAL egi.eu Proxy Hierarchy Stratum-1 DESY Proxy Hierarchy Stratum-1 ASGC Stratum-1 TRIUMF Stratum-0 OSG opensciencegrid.org

Software installation mechanism at RAL CernVM Users Workshop, CERN, March 2015 /home/augersgm /home/biomedsgm. /home/t2ksgm GSI Interface CVMFS Uploader Stratum-0 GSIssh/scp DN credentials VOMS Role credentials VO Software Grid Manager /cvmfs/auger.egi.eu /cvmfs/biomed.egi.eu. /cvmfs/t2k.egi.eu

HOWTO new repo at RAL Stratum-0 Request at For registered VOs access on CVMFS uploader based on DNs and/or VOMS role For research groups membership of UK regional VOs recommended if grid access is required repository created within gridpp.ac.uk space repository could be moved later within egi.eu (once VO registered)

Proposal for rsync of repos at RAL Stratum-0 Currently CVMFS uploader contains the master copies of all repositories – used space slowly increasing Could VOs keep their own master copies locally? Then CVMFS uploader (or Stratum-0) would just check and rsync if necessary

Catalin’s requests to all CVMFS sites Please check that you are at cvmfs v Please, please have cvmfs-keys v1.5-1 installed Please update to the latest cvmfs-puppet module v0.3.3 Be ready to update to cvmfs v2.1.20

CernVM-FS Users Workshop, 5-6 March, CERN

Status and roadmap of CernVM - CernVM virtual appliance – complete and portable environment for developing and running HEP data processing tasks - Use cases - IaaS Clouds: AWS community image, CERN OpenStack - Development environment: Xfce graphical UI - Volunteer computing: Test4Theory - Long-term analysis preservation: ALEPH software - Outreach and education: CERN OpenData portal - Future plans - Lightweight virtualization: containers in CernVM (Docker and lxc)

Status and roadmap of CernVM Future plans - Lightweight virtualization: containers in CernVM (Docker and lxc) - Security Updates - Contextualization - SL7 support

Status and roadmap of CernVM-FS - CernVM-FS – critical WLCG service, increasing number of non-LHC repos -Repository server migration to 2.1.x -Consolidation configuration of CernVM-FS - Disentangle CernVM-FS from CERN-specific configuration - Simplify CernVM-FS client configuration - Allow for 3rd party configuration packages - Facilitate support for non-HEP VOs

Status and roadmap of CernVM-FS New configuration methods in CernVM-FS Introduction of cvmfs-config-packages - Ability to use configuration repositories - Automatic location aware ordering of Stratum1 servers - Push replication - Stratum-0 will actively announce updates to Stratum-1s - Will significantly lower the update dissemination latency to CVMFS clients

Feedback from the LHC experiments Feedback from the users community - OSG and EGI – similar interface for users for maintaining the repositories - Asia - requests for ACL - notes from GridPP user engagement programme – TomW CVMFS as a high speed filesystem for auxiliary data Volunteer computing projects at CERN

Technological trends - Docker containers in distributed applications Sebastien Goasguen (Citrix) - Opportunistic computing for CMS at large scale Douglas Thain (Univ Notre Dame) - Hybrid cloud environments and networking on AWS Giulio Soro (AWS) - Big data in the Cloud – processing and performance Anthony Voellm (Google)