GridPP Storage perspectives Brian Davies pre-GDB Data Management , CERN 13 September 2016

Areas of interest of GridPP Storage ATLAS analysis for new schema. Volumes Network needs/slot capacities T2 Evolution What changes happen to those SEs which remain as well as those which remain? Structure? Testing particular areas of interest (Non)-LHC VO support LFC,FTS(dev/ops),WMS, perfSONAR/TCP tuning (cloud/VO support) T1 activity to put Grid middleware on CEPH

UK ATLAS T2 evolvement use case Site Maximal Slot Count Normal Slot Count Pledged Storage (TB) Actual (TB) Additional slots Factor increase Glasgow 4500 4000 1305 2583 2100 1.53 QMUL 4300 3000 1386 2585 1.70 Lancaster 3100 2500 1229 1700 1.68 Manchester 3900 1080 1750 1400 1.47 Σ Main T2s 15800 12500 5000 9018   RHUL 3400 2000 963 1418 ECDF 600 400 540 891 Oxford 1300 800 729 823 RALPP 1200 360 757 L'pool 1000 486 735 Sheffield 760 387 429 B'Ham 200 189 335 Cambridge 250 207 321 Sussex 100 45 50 Durham 2200 44 Brunel 1150 33 UCL 60 10 81 19 Imperial Σ Small T2s 15050 7370 3987 5874 RAL T1 11400 8000 5300 1 Site need to accommodate additional capacities Large increase in WAN traffic 90% LAN traffic is WN 6/15GB/s of WN traffic at T1 is for ATLAS Jobs Disk servers/ head nodes to cope with additional connections CMS use ~2.5MB/s per job Storage pledge meet What to do with 5874 TB of “free” space

T2 Evolution Context VOs want to use fewer storage endpoints Fewer sites want to support storage Reduction in funded manpower Sites need to run with fewer people. (And need to transition with fewer people, too, as decline has already started) Sites also need to work well with non-LHC assumptions, techniques. https://twiki.cern.ch/twiki/bin/view/LCG/WLCGSiteSurveyT2s

T2 Storage Characteristics Churn (cache like?) but low reuse rate Large Size variation - 3PB down to <500TB Access methods SRM, Xroot, GridFTP, HTTP… X509+VOMS authentication Total Capacity Unlikely that total demand for T2 storage will fall. What do we do with existing storage? Allow to "slowly degrade" at T2Cs? "Physically consolidate" at T2Ds?

Reduce Expenditure by: Reliability Requirements Features Maintenance Manpower

Expenditure Method Types: Reduce Reliability Requirements Features Maintenance Manpower Egalitarian Barebones Network Caching/Read only

Some existing solutions..

Data CVMFS Caching / Read only Read only data provision, optimised for small, slowly changing datasets. CVMFS + (Xrootd backhaul) + [future cache tiers] Deployed for LIGO US by OSG - tested in production. Some exploration at RAL for non-LHC VOs Not suitable for rapidly changing, large datasets. Easy transition (for all sites with CVMFS up to date…) Caching / Read only

T2C Caching WLCG Data Caching Group. Two solutions: Xrootd - mature, not widely used?, compatible with xrootd federation "DPM" - In development, protocol agnostic, but DPM specific. Easy transition (for sites with DPM or Xrootd services) Caching / Read only

ARC Cache ARC "prefetch" cache for jobs Working happily at Durham for some months now. Easy transition (for sites with ARC CEs) Caching / Read only

Cache questions None of the mentioned solutions solve job output issues. Centralised object store for log files? Is caching efficient? Small scale studies suggest 90% of data is read maybe twice, at best. Caching as an proxy for acceptance of low reliability? How is this different from redirection, in practice? Caching / Read only

Pure network model UCL CPU paired to remote SE at QMUL. Original localSE model from 10+ years ago. Can this apply to all sites/Regions Tests by Alastair Dewhurst against Oxford T2 for ATLAS for more sophisticated (federated) solutions. VOs already federating so use their frameworks Currently resolving HammerCloud's assumptions re data locality. Obviously, only test ATLAS workloads… Network

Simpler storage Current T2 storage is specialist - big RAID-6 arrays. Work by M.Ebert on ZFS, software 'RAID': faster, more flexible, more extensible than HW RAID small efficiency gains via compression (~4%) Small step (server local changes). (Also heads off RAID6 scaling limit @ 8TB disks) Egalitarian

SRM retirement? Barebones Egalitarian but spacetokens still an issue. Slow movement on this, due to existing dependencies. Agreement "in principle" that not necessary for T2 workflows but spacetokens still an issue. Still no good, general-purpose alternative for protecting/guaranteeing VO space. Barebones Egalitarian

Single protocol? Barebones Egalitarian HTTP(S)/WebDAV? HTTP Deployment taskforce signed off on features XROOT? [or xrootd] "unusual", but xrootd also supports HTTP(S) GridFTP? Advantage(?) of "Globus Connect" compatibility Barebones Egalitarian

SE "retirement" Without SRM, what is the need for a traditional SE? Revert to Classic SEs Xrootd, HTTP, GridFTP endpoints in front of: ceph/hdfs/other distributed storage [~posix] Object storage via [xrootd/S3/SWIFT/…] Transition is hard. (Unless we can dump all of the data at a site…) Unless churn is used to our advantage ATLAS delete 2PB/month from UK T2s (is this normal?) Barebones Egalitarian

Support for non-LHC VOs LFC, FTS, WMS, DIRAC usage for non-LHC VOs non negligible Support for products… Sharing technology/monitoring/hardware useful New communities require alternative technologies Licensing issues

Tier 1 storage evolution CERN switched from Castor to EOS for disk only storage several years ago. RAL have been working on a replacement storage service. Why not choose DPM/dCache? Grid storage not popular with non-LHC users. They want things like S3. Other groups at RAL are also using Ceph so expertise readily available. Ceph is an object store: Fits LHC use case well (external file catalogues, don’t update files) Want to take advantage of this simplicity Need to add support for XrootD and GridFTP

GridFTP and XrootD for Echo CERN have developed Ceph plugin for xrootd Being used for the Castor tape buffers. Significant work now focusing on performance improvements. RAL are developing GridFTP plugin for Ceph With help from CERN and Brian Bockelman. Most functionality in place, starting looking at performance. Multiple stream assembly planned to improve GridFTP performance via FTS.

Echo Status No SRM – is that a problem? The Castor SRM will deal with all the tape stuff. Currently just support ATLAS and CMS who claim to not need one!! Other VOs to follow. Accounting will be provide via .json file Echo is now registered as a ‘T3D’ for ATLAS. Starting to add SE and run functional tests Andrew Lahiff is validating CMS jobs. Waiting for GridFTP improvements before adding to PhEDEx Aim to have production quality SE by April 2017.

Future areas of interest AAI/AARC EUDAT/SAGE/ESiWACE project collaboration/co-ordination Further non-HEP community support

Summary GridPP supporting/following various storage topics. Balance LHC/nonLHC requirements T2 evolution on track T1 (r)evolution progressing But Tape staying the Same Collaboration between LHC/nonLHC beneficial to both parties

Questions Thanks to A.Dewhurst and S.Skipsey for additional/majority material for this talk. Thanks to gridpp-storage/GridPP community( particulary J.Jensen and M.Ebert ) gridpp-storage@jiscmail.ac.uk

Backups

DYNaFED

Network

Network