SRM v2.2 / v3 meeting report SRM v2.2 meeting Aug. 29

Slides:



Advertisements
Similar presentations
30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage.
Advertisements

J Jensen CCLRC RAL Data Management AUZN (mostly about SRM though) GGF 16, Athens J Jensen.
Jens G Jensen CCLRC/RAL hepsysman 2005Storage Middleware SRM 2.1 issues hepsysman Oxford 5 Dec 2005.
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Storage Issues: the experiments’ perspective Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 9 September 2008.
CASTOR SRM v1.1 experience Presentation at SRM meeting 01/09/2004, Berkeley Olof Bärring, CERN-IT.
Data Grid Web Services Chip Watson Jie Chen, Ying Chen, Bryan Hess, Walt Akers.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 experience Sophie Lemaitre WLCG Workshop.
Maarten Litmaath (CERN), EGEE User Forum, CERN, 2006/03/02 (v3) Use of the SRM interface Use case What is the SRM? –Who develops it? –Is it a standard?
CERN, 29 August 2006 Status Report Riccardo Zappi INFN-CNAF, Bologna.
Status report on SRM v2.2 implementations: results of first stress tests 2 th July 2007 Flavia Donno CERN, IT/GD.
Status of SRM 2.2 implementations and deployment 29 th January 2007 Flavia Donno, Maarten Litmaath IT/GD, CERN.
SRM 2.2: tests and site deployment 30 th January 2007 Flavia Donno, Maarten Litmaath IT/GD, CERN.
SRM 2.2: status of the implementations and GSSD 6 th March 2007 Flavia Donno, Maarten Litmaath INFN and IT/GD, CERN.
A. Sim, CRD, L B N L 1 OSG Applications Workshop 6/1/2005 OSG SRM/DRM Readiness and Plan Alex Sim / Jorge Rodriguez Scientific Data Management Group Computational.
Data Management The GSM-WG Perspective. Background SRM is the Storage Resource Manager A Control protocol for Mass Storage Systems Standard protocol:
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
Δ Storage Middleware GridPP10 What’s new since GridPP9? CERN, June 2004.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/02/08 VOMS deployment Extent of VOMS usage in LCG-2 –Node types gLite 3.0 Issues Conclusions.
SRM workshop – September’05 1 SRM: Expt Reqts Nick Brook Revisit LCG baseline services working group Priorities & timescales Use case (from LHCb)
Placeholder ES 1 CERN IT Experiment Support group Authentication and Authorization (AAI) issues concerning Storage Systems and Data Access Pre-GDB,
Report on Installed Resource Capacity Flavia Donno CERN/IT-GS WLCG GDB, CERN 10 December 2008.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
WLCG Grid Deployment Board, CERN 11 June 2008 Storage Update Flavia Donno CERN/IT.
SRM & SE Jens G Jensen WP5 ATF, December Collaborators Rutherford Appleton (ATLAS datastore) CERN (CASTOR) Fermilab Jefferson Lab Lawrence Berkeley.
1 LHCb File Transfer framework N. Brook, Ph. Charpentier, A.Tsaregorodtsev LCG Storage Management Workshop, 6 April 2005, CERN.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/03/08 An update on SRM Mumbai & subsequent discussion summary –
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/06/07 SRM v2.2 working group update Results of the May workshop at FNAL
CERN SRM Development Benjamin Coutourier Shaun de Witt CHEP06 - Mumbai.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Padova, 5 October StoRM Service view Riccardo Zappi INFN-CNAF Bologna.
Storage Classes report GDB Oct Artem Trunov
Report from GSSD Storage Workshop Flavia Donno CERN WLCG GDB 4 July 2007.
Storage Interfaces and Access pre-GDB Wahid Bhimji University of Edinburgh On behalf of all those who participated.
Ákos FROHNER – DataGrid Security n° 1 Security Group TODO
WLCG Grid Deployment Board CERN, 14 May 2008 Storage Update Flavia Donno CERN/IT.
SRM-2 Road Map and CASTOR Certification Shaun de Witt 3/3/08.
Handling of T1D0 in CCRC’08 Tier-0 data handling Tier-1 data handling Experiment data handling Reprocessing Recalling files from tape Tier-0 data handling,
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Storage Element Model and Proposal for Glue 1.3 Flavia Donno,
SRM-iRODS Interface Development WeiLong UENG Academia Sinica Grid Computing 1.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
1 SRM v2.2 Discussion of key concepts, methods and behaviour F. Donno CERN 11 February 2008.
The Grid Storage System Deployment Working Group 6 th February 2007 Flavia Donno IT/GD, CERN.
Andrea Manzi CERN EGI Conference on Challenges and Solutions for Big Data Processing on cloud 24/09/2014 Storage Management Overview 1 24/09/2014.
SRM 2.2: experiment requirements, status and deployment plans 6 th March 2007 Flavia Donno, INFN and IT/GD, CERN.
Bologna, March 30, 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna.
CERN IT Department CH-1211 Genève 23 Switzerland t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November.
Implementation of GLUE 2.0 support in the EMI Data Area Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User.
SRM v2.2: service availability testing and monitoring SRM v2.2 deployment Workshop - Edinburgh, UK November 2007 Flavia Donno IT/GD, CERN.
a brief summary for users
StoRM: a SRM solution for disk based storage systems
Vincenzo Spinoso EGI.eu/INFN
Status of the SRM 2.2 MoU extension
Flavia Donno, Jamie Shiers
Flavia Donno CERN GSSD Storage Workshop 3 July 2007
GDB 8th March 2006 Flavia Donno IT/GD, CERN
GGF OGSA-WG, Data Use Cases Peter Kunszt Middleware Activity, Data Management Cluster EGEE is a project funded by the European.
Storage Interfaces and Access: Introduction
John Gordon EDG Conference Barcelona, May 2003
SRM v2.2 planning Critical features for WLCG
SRM Developers' Response to Enhancement Requests
SRM2 Migration Strategy
Proposal for obtaining installed capacity
Ákos Frohner EGEE'08 September 2008
The INFN Tier-1 Storage Implementation
Data Management cluster summary
GIN-Data : SRM Island Inter-Op Testing
INFNGRID Workshop – Bari, Italy, October 2004
Presentation transcript:

SRM v2.2 / v3 meeting report SRM v2.2 meeting Aug. 29 http://agenda.cern.ch/fullAgenda.php?ida=a063257 Status reports and discussions SRM v3 meeting Aug. 30 – Sep. 1 http://agenda.cern.ch/fullAgenda.php?ida=a062357 Many discussions also very relevant to v2.2 (v2.3) Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

CASTOR status No significant development remaining for minimal WLCG model C++ client may not catch bugs found by LBNL Java client Java Axis is stricter than gSOAP Stability of CASTOR core code at RAL should be OK now Disk server at CERN should now be available Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

dCache status Directory functions implemented srmLs output format to be decided Space reservation alpha version ready authZ based on VO group and role when gPlazma + PRIMA + GUMS are used Framework allows for LCMAPS plugin space access semantics to be decided Data transfer functions implemented srmCopy can copy to/from v1.1 and v2.2 srmBringOnline in October Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

DPM status All methods for minimal WLCG model implemented a few options are not yet fully treated Some unresolved issues Space ownership vs. primary FQAN content Space ACLs srmLs output format Paths or SURLs File lifetimes DB schema should allow for v3/v2.3 enhancements Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

Berkeley SRM status Almost all methods for minimal WLCG model implemented Currently supports 2 of the 3 storage classes: CUSTODIAL + NEARLINE == Tape1Disk0 REPLICA + ONLINE == Tape0Disk1 Will be deployed for Earth System Grid and STAR experiment Will be made available through VDT Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

StoRM status Collaboration of INFN and ICTP (EGRID) http://www.egrid.it/sw/storm/ Disk-based SRM on top of high-performance parallel file systems e.g. GPFS (supports space reservation) Allows SRM file systems with ACLs to be directly accessed from WNs Apply just-in-time ACLs All methods for minimal WLCG model expected around end September Discovered some ambiguities in v2.2 specification/MoU Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

FTS status Use space token or description, if supplied, on Put and Copy srmLs only one single files bringOnline for pre-staging early next year ACL propagation (also for leading directory creation) to be decided Best effort support for v1.1 WLCG v2.2 support to be in release end of October Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

GFAL/lcg-utils status Space token to be provided in explicit call to set the context not as extra arguments to the POSIX-like interface ACL propagation to be decided WLCG v2.2 support to be available early October Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

Storage Classes WG issues Tape1Disk1 back-end implementation not completely evident the other two are standard allowed transitions to/from the other two? want to avoid making physical copies explicit stageout Tape0  Tape1 awkward to implement: disallow? Space reservation refers to the pool from which the data is used data typically accessed from LAN only WAN transfer pool takes up some HW, but is not exposed Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

Berkeley SRM-Tester status Test client independent of any server implementation Basic functionality tests Advanced tests provoke failed requests check the server's subsequent behavior Basic and advanced interoperability tests with srmCopy Results published on web pages http://sdm.lbl.gov/srm-tester/ Stress tests implementation dependent on extra funding Work part of GGF-GIN project Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

Space reservation issues Ownership/ACL semantics Based on VOMS FQANs Proposals, no decisions yet v3/v2.3: allow subspaces for dynamic reservation v3/v2.3: one can add a file to a space not writable by the file owner Compare with hard link in home directory to someone else’s file Prevent purgeFromSpace anomaly: last writable instance cannot be removed with purgeFromSpace Owner can always use srmRm instead v2.2: a file can only be in one space! Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

File lifetimes v2.2 v3/v2.3 v3 core does not have spaces SURL, bringOnline (pin), TURL v3/v2.3 SURL, space, bringOnline (pin), TURL v3 core does not have spaces All basic functionality must also be available without space tokens Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

xrootd, quotas, httpg xrootd Quotas httpg vs. https BaBar have important use case for file update  immutable files file can change as long as not in catalog want prepareToPut with delay option to allow for updates etc. tested with modified Berkeley SRM Quotas still desired for v3/v2.3 complementary to spaces httpg vs. https httpg always delegates proxy, while only needed for srmCopy separate delegation service reduces security overhead https is standard can use alternative implementations, less code to maintain Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

GLUE schema changes New schema for SE to reflect properties of SRM v2.2 and higher Storage Area  space Access Policy info per SA per VO Foresees quotas Enhances Access Protocol info Removes obsolete/unused notions Submitted to GLUE schema working group Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2

Plan of work Settle open issues in phone conferences + mailing lists ACL semantics Files and directories Spaces srmLs output format File lifetimes Avoid non-trivial changes to WSDL/spec for the time being Collaborate with Storage Classes WG Continuous interoperability testing Berkeley SRM-Tester Jiri Mencak’s test suite  Flavia Donno looking into it Monitor FTS/GFAL/lcg-utils progress RAL integration week cancelled Nobody found it useful at this time New meeting to be decided Maarten Litmaath (CERN), WLCG Management Board meeting, 2006/09/12 – v2