1 SRM v2.2 Discussion of key concepts, methods and behaviour F. Donno CERN 11 February 2008.

Slides:



Advertisements
Similar presentations
Jens G Jensen CCLRC/RAL hepsysman 2005Storage Middleware SRM 2.1 issues hepsysman Oxford 5 Dec 2005.
Advertisements

INFSO-RI Enabling Grids for E-sciencE Agreement Service for Resource Reservation and Allocation: Overview Tiziana Ferrari, Elisabetta.
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Storage Issues: the experiments’ perspective Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 9 September 2008.
1 of 7 This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation.
Data Grid Web Services Chip Watson Jie Chen, Ying Chen, Bryan Hess, Walt Akers.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 experience Sophie Lemaitre WLCG Workshop.
A. Sim, CRD, L B N L GIN-Data : SRM Island Inter-Op Testing With SRM-TESTER Alex Sim, Vijaya Natarajan Computational Research Division Lawrence Berkeley.
DPM CCRC - 1 Research and developments DPM status and plans Jean-Philippe Baud.
CERN, 29 August 2006 Status Report Riccardo Zappi INFN-CNAF, Bologna.
Data management in grid. Comparative analysis of storage systems in WLCG.
Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.
Status report on SRM v2.2 implementations: results of first stress tests 2 th July 2007 Flavia Donno CERN, IT/GD.
Status of SRM 2.2 implementations and deployment 29 th January 2007 Flavia Donno, Maarten Litmaath IT/GD, CERN.
SRM 2.2: tests and site deployment 30 th January 2007 Flavia Donno, Maarten Litmaath IT/GD, CERN.
SRM 2.2: status of the implementations and GSSD 6 th March 2007 Flavia Donno, Maarten Litmaath INFN and IT/GD, CERN.
CCRC’08 Weekly Update Plus Brief Comments on WLCG Collaboration Workshop Jamie Shiers ~~~ WLCG Management Board, 29 th April 2008.
Data Management The GSM-WG Perspective. Background SRM is the Storage Resource Manager A Control protocol for Mass Storage Systems Standard protocol:
Concept: Well-managed provisioning of storage space on OSG sites owned by large communities, for usage by other science communities in OSG. Examples –Providers:
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
SRM 2.2 Issues Well, er, and 2.3 too Jens Jensen (STFC RAL/GridNet2) On behalf of GSM-WG OGF22, Cambridge, MA.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Andrew C. Smith – Storage Resource Managers – 10/05/05 Functionality and Integration Storage Resource Managers.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
SRM workshop – September’05 1 SRM: Expt Reqts Nick Brook Revisit LCG baseline services working group Priorities & timescales Use case (from LHCb)
Report on Installed Resource Capacity Flavia Donno CERN/IT-GS WLCG GDB, CERN 10 December 2008.
1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.
SRM Monitoring 12 th April 2007 Mirco Ciriello INFN-Pisa.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
WLCG Grid Deployment Board, CERN 11 June 2008 Storage Update Flavia Donno CERN/IT.
SRM & SE Jens G Jensen WP5 ATF, December Collaborators Rutherford Appleton (ATLAS datastore) CERN (CASTOR) Fermilab Jefferson Lab Lawrence Berkeley.
1 WWW. 2 World Wide Web Major application protocol used on the Internet Simple interface Two concepts –Point –Click.
1 LHCb File Transfer framework N. Brook, Ph. Charpentier, A.Tsaregorodtsev LCG Storage Management Workshop, 6 April 2005, CERN.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/06/07 SRM v2.2 working group update Results of the May workshop at FNAL
CERN SRM Development Benjamin Coutourier Shaun de Witt CHEP06 - Mumbai.
Busy Storage Services Flavia Donno CERN/IT-GS WLCG Management Board, CERN 10 March 2009.
6 Sep Storage Classes implementations Artem Trunov IN2P3, France
David Adams ATLAS ATLAS distributed data management David Adams BNL February 22, 2005 Database working group ATLAS software workshop.
The CMS Top 5 Issues/Concerns wrt. WLCG services WLCG-MB April 3, 2007 Matthias Kasemann CERN/DESY.
Introduction to Active Directory
EGI-Engage Data Services and Solutions Part 1: Data in the Grid Vincenzo Spinoso EGI.eu/INFN Data Services.
Storage Classes report GDB Oct Artem Trunov
Report from GSSD Storage Workshop Flavia Donno CERN WLCG GDB 4 July 2007.
WLCG Grid Deployment Board CERN, 14 May 2008 Storage Update Flavia Donno CERN/IT.
SRM-2 Road Map and CASTOR Certification Shaun de Witt 3/3/08.
Handling of T1D0 in CCRC’08 Tier-0 data handling Tier-1 data handling Experiment data handling Reprocessing Recalling files from tape Tier-0 data handling,
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Disk Space publication Simone Campana Fernando Barreiro Wahid.
SRM v2.2 Production Deployment SRM v2.2 production deployment at CERN now underway. – One ‘endpoint’ per LHC experiment, plus a public one (as for CASTOR2).
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Storage Element Model and Proposal for Glue 1.3 Flavia Donno,
SRM-iRODS Interface Development WeiLong UENG Academia Sinica Grid Computing 1.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
The Grid Storage System Deployment Working Group 6 th February 2007 Flavia Donno IT/GD, CERN.
SRM 2.2: experiment requirements, status and deployment plans 6 th March 2007 Flavia Donno, INFN and IT/GD, CERN.
Arie Shoshani – Dec 2002 Arie Shoshani meeting held at CERN December, 2002 Comments and Observations SRM + GLUE.
Bologna, March 30, 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SE Security Rémi Mollon, Ákos Frohner EGEE'08,
Status of the SRM 2.2 MoU extension
Flavia Donno, Jamie Shiers
Flavia Donno CERN GSSD Storage Workshop 3 July 2007
SRM v2.2 / v3 meeting report SRM v2.2 meeting Aug. 29
SRM v2.2 planning Critical features for WLCG
SRM Developers' Response to Enhancement Requests
SRM2 Migration Strategy
Introduction to reading and writing files in Grid
Proposal for obtaining installed capacity
Ákos Frohner EGEE'08 September 2008
GLUE 2 Support in gLite Data Management
The INFN Tier-1 Storage Implementation
Architecture of the gLite Data Management System
Presentation transcript:

1 SRM v2.2 Discussion of key concepts, methods and behaviour F. Donno CERN 11 February 2008

2 Priorities to be agreed: Protecting spaces from generic users usage It concerns dCache, CASTOR Use of tokens for PrepareToGet/BringOnline/srmCopy (input) It concerns dCache, DPM, StoRM Implementations fully VOMS-aware It concerns dCache, CASTOR Correct implementation of GetSpaceMetaData (correct size to be returned at least for T1D1) It concerns dCache, CASTOR Selecting tape sets by means of tokens, directory paths, ?? It concerns dCache, CASTOR, StoRM List of issues Priority to be established

3 Establish short, medium and long terms goals Short = ?? - date and goals to be defined Medium = ?? – goals to be defined Long = ?? – date and goals to be defined Proposed short term goals (exact list and priorities to be agreed): Selecting tape sets by means of tokens, directory paths, ?? It concerns dCache, CASTOR, StoRM Making the token truly orthogonal to the path It concerns StoRM Protecting spaces from generic users usage It concerns dCache, CASTOR Implementations fully VOMS-aware It concerns dCache, CASTOR Proposed Short term goals

4 Definition of SRM v2.2 space Space properties (Permission and Size) Files and spaces Copies and spaces Discussion of key concepts/behaviour

5 srmReserveSpace[/srmUpdateSpace] srmGetSpaceMetaData srmPrepareToGet/srmBringOnline srmCopy srmChangeSpaceForFiles srmPurgeFromSpace Discussion of key methods

6 Definition of SRM v2.2 space An SRM space is a logical view of a physical space allocation that is reserved for read/write operations on files. Space properties Retention Policy Information (Retention Policy and Access Latency) Owner Group (NEW) Owner/Group Permission (Create, Release, Update, Read-from-Space, Write-to- Space, Remove-from-Space) (NEW) – VOMS FQAN to express permissions Connection Type (WAN, LAN) Supported File Access/Transfer Protocols Space Token [Description] (optional) Status Total Size Guaranteed Size Unused Size Assigned Lifetime Left Lifetime Supported Access Patterns (TRANSFER, PROCESSING) (NEW) Client Networks The SRM v2.2 space

7 Permissions are specified in terms of VOMS FQAN. In WLCG spaces are statically reserved, although support for truly dynamic reservation can be provided by some implementations. When a file is removed from a space, the space occupied by the file is returned back to the original space allocation. The SRM v2.2 space

8 Definition: A file is a set of data with the following properties defined on pag. 19 of the SRM v2.2 specification document. A file is furthermore characterized by a Site URL and by the space the file was first created into. This space is referred to as the file master space that determines the characteristics of the file. A file can have several copies in several spaces. Definition: A copy of a file is physical instance of the file in a given space. It is characterized by the RequestID of the srm request that has generated the copy and by the properties defined on pag. 19 of the spec. The first copy of a file is referred to as the file master copy. Files and spaces