StoRM Some basics and a comparison with DPM Wahid Bhimji University of Edinburgh GridPP Storage Workshop 31-Mar-101Wahid Bhimji – StoRM

This talk: Considering moving to StoRM ??? – Comparison with DPM – Benefits / Problems – Status 31-Mar-10Wahid Bhimji – StoRM2

Whats StoRM? Lightweight SRM server Works on top of POSIX filesystems Relies on that filesystem to replicate data etc. Used widely in Italy and a few place outside 31-Mar-10Wahid Bhimji – StoRM3

DPM - reminder 31-Mar-10Wahid Bhimji – StoRM4 DPM Head Node DPM Head Node Pool Node Pool Node Head Node (dpm-mysql) - SRMV2 Interface - DPM server: Request tracking (MySQL) - DPM Nameserver (DPNS) (MySQL) User Worker Node Worker Node DISK “Pool” node (dpm-disk) -Directly mounted disk - RFIO– local transfers -GridFTP– WAN transfers … Pool Node Pool Node DISK Pool Node Pool Node DISK gridftp rfio …

StoRM: Can be one machine. StoRM: Can be one machine. Storm Structure 31-Mar-10Wahid Bhimji – StoRM5 Storm Frontend Storm Frontend Storm Backend Storm Backend Frontend: SRM Interface Request Tracking (MySQL) Backend: Namespace (by file path) Filesytem Drivers Disk Servers User Worker Node file:// GridFTP server(s)

Differences to DPM Namespace derived from filename Local transfers through file:// POSIX access Uses ACLs for permissions Lot less (no) user tools – So no equivalent of dpm-* dpns-* tools – But filesystem operations are just du * rm * ls * – For spacetoken allocations etc. edit file: /opt/storm/backend/etc/namespace.xml 31-Mar-10Wahid Bhimji – StoRM6

Namespace srm://se2.glite.ecdf.ed.ac.uk:8444/srm/managerv2? =/atlas/atlasdatadisk/bill/bob DPM this would be DPNS path and to get to physical file you would need to use DB -On Storm there is no DB. The physical file is at: SPACE_ROOT/bill/bob SPACE_ROOT is defined for each VO/ spacetoken: Might be just /lustre/atlas/atlasdatadisk 31-Mar-10Wahid Bhimji – StoRM7

Versions Storm 1.5 released recently 1.4 no longer “supported” Currently for SL4 (32bit – prob. works on 64) SL5 – “after easter” Installations at QMUL and Edinburgh – both have issues at the moment – probably resolvable. INFN-T1 are using 1.5 most other sites Mar-10Wahid Bhimji – StoRM8

Instalation Obtain repos glite-generic.repo ; lcg-ca.repo; ig.repo; jpackage.repo “Standard” yum install / YAIM configure : yum install lcg-CA ig_SE_storm_backend ig_SE_storm_frontend /opt/glite/yaim/bin/ig_yaim -c -s siteinfo/site-info.def –n ig_SE_storm_backend -n ig_SE_storm_frontend Setup filesystem (Lustre/GPFS etc.; acls needed) You also need users etc (as for DPM + other nodes) 31-Mar-10Wahid Bhimji – StoRM9

Weighing it up StormDPM SimplicityScalability File:// performance SupportSRM features 31-Mar-10Wahid Bhimji – StoRM10

Benefits Easiest current way to make use of Lustre / GPFS or other cluster filesystem: file:// POSIX access – better local access performance that rfio (afternoon discussion) Scalability / resilience (INFN-T1 uses Storm) Simplicity: Navigable directories/ familiar posix commands (but you also have to maintain Lustre / GPFS if you use those) 31-Mar-10Wahid Bhimji – StoRM11

Key reasons not to move Limited UK experience, smaller user community than DPM /dCache – mostly Italian sites Some implementation of SRM features lacking (in 1.4 – not clear if fixed in storm 1.5) – Access permissions – Information provider – Space reservation? ;-) : 31-Mar-10Wahid Bhimji – StoRM12

StoRM space reservations? ;-) 31-Mar-10Wahid Bhimji – StoRM13

Conclusions StoRM is not (so) scary. – It has some quirks/limits but so does DPM – You can use the same hardware – SRM performance looks to be similar to DPM – Growing maturity and user community It is currently the easiest way to use cluster filesystems to give scalability and the performance of local file:// access Discuss (after Chris’s talk…) 31-Mar-10Wahid Bhimji – StoRM14