Bologna, March 30, 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna

Bologna, March 30, 2006 Outline 1. SRM and StoRM 2. StoRM and Space Reservation 3. Space Reservation and GPFS 4. Conclusion

Bologna, March 30, 2006 What StoRM is

Bologna, March 30, 2006 INFN / D-Grid meeting Page 4 SRM and StoRM StoRM is a disk based Storage Resource Manager which: –implements SRM specification version WS-I compliant version, named “2.1.1_modified”. –is designed to support guaranteed space reservation. –supports direct access (native posix I/O calls). Other access protocols remain available (e.g., rfio). –takes advantage of high performance Cluster File System with ACL support, such as GPFS. Other posix file systems are supported (e.g., ext3) –Authentication and Authorization are based on VOMS certificates.

Bologna, March 30, 2006 INFN / D-Grid meeting Page 5 StoRM release Latest release description: –main subset of SRM functionalities supported (details in the next slide) –VOMS proxy certificates are used Release date: –First release  Nov 2005 –Second release  Jan 2006 –Latest release  in deploy (in April) – for news

Bologna, March 30, 2006 INFN / D-Grid meeting Page 6 At T1 each node will belongs to GPFS cluster. StoRM works better in this scenario allowing direct access from WN to storage. At T1 we are using a GPFS cluster composed by roughly 40 TB as testbed. StoRM and T1

Bologna, March 30, 2006 INFN / D-Grid meeting Page 7 Front end (FE) has responsibilities of : –expose a web service interface –manage connection with authorized clients –store asynchronous request into data base –retrieve asynchronous request status –co-operate with backend directly for synchronous call –co-operate with external authorization service to enforce security policy on service –manage user authentication Data Base : –Store SRM request and status –Store application data Back end (BE) has responsibilities of : –execute all synchronous (active) action –get asynchronous request from data base –execute all asynchronous action –bind with underlying file system –enforce authorization policy on files –manage SRM file and space metadata StoRM architecture

Bologna, March 30, 2006 INFN / D-Grid meeting Page 8 Data Transfer: –srmPrepareToPut() –srmPrepareToGet() –srmCopy() –srmStatusOf () –srmPutDone() Space Management: –srmReserveSpace() –srmGetMetaDataSpace() Directory functions : –srmLs() with recursive option –srmRm() –srmRmDir() –srmMkDir() Clients : –Command line clients for every operation supported. Functionalities in StoRM

Bologna, March 30, 2006 Space Reservation in StoRM

Bologna, March 30, 2006 INFN / D-Grid meeting Page 10 “srmReserveSpace” in SRM srmReserveSpace –In: TUserIDauthorizationID, TSpaceType typeOfSpace, String userSpaceTokenDescription, TSizeInBytes sizeOfTotalSpaceDesired, TSizeInBytes sizeOfGuaranteedSpaceDesired, TLifeTimeInSeconds lifetimeOfSpaceToReserve, TStorageSystemInfo storageSystemInfo –Out: TSpaceType typeOfReservedSpace, TSizeInBytes sizeOfTotalReservedSpace, TSizeInBytes sizeOfGuaranteedReservedSpace, TLifeTimeInSeconds lifetimeOfReservedSpace, TSpaceToken referenceHandleOfReservedSpace, TReturnStatus returnStatus

Bologna, March 30, 2006 INFN / D-Grid meeting Page 11 “srmReserveSpace” in StoRM SRM_SUCCESS if: –user is authorized to perform space reservation request –AND there is enough space to allocate –AND space size requested is not greater than maximum permitted Output Values –In case of null input parameter, default values based on user identity are used for: Space size Space type Lifetime –referenceHandleOfReservedSpace: (i.e., a space Token) is generated in unique fashion by StoRM

Bologna, March 30, 2006 INFN / D-Grid meeting Page 12 Space utilization in StoRM (1) srmPrepareToPut (... TPutFileRequest[] arrayOfFileRequest,... ) typedef struct { TSURLInfo toSURLInfo, // local to SRM TLifeTimeInSeconds lifetime, // pin time TFileStorageType fileStorageType, TSpaceToken spaceToken, TSizeInBytes knownSizeOfThisFile } TPutFileRequest

Bologna, March 30, 2006 INFN / D-Grid meeting Page 13 For every srmPrepareToPut call with space handler specified (spaceToken): –an endpoint to a file (File) with desired amount of pre- allocated blocks is returned. –Reserved space (StorageSpace) will be updated. Space utilization in StoRM (2) Storage Space Storage Space File spaceToken

Bologna, March 30, 2006 INFN / D-Grid meeting Page 14 Space Utilization and security aspects 1.User perform srmPrepareToPut 2.StoRM verifies if the principal holds a valid proxy certificate and delegates the external policy decision point to validate the request. 3.StoRM then queries the Authorization Sources to verify if the user can perform the specified operation on the SURL 4.StoRM queries LCMAPS to obtain local user account corresponding to the grid identity of the requestor 5.Physical file name derives by SURL and user attributes (Virtual organization name space) 6.The file system wrapper enforces permissions by setting a new ACL on the physical file. 7.The user job can be executed into the worker node 8.The application can perform a standard POSIX call to access the file into/from the storage system.

Bologna, March 30, 2006 Space Allocation in GPFS

Bologna, March 30, 2006 INFN / D-Grid meeting Page 16 Space Allocation in GPFS GPFS exposes a programming interface. gpfs_prealloc( ) subroutine for storage space allocation –The gpfs_prealloc( ) function is used to preallocate disk storage for a file that has already been opened, prior to writing data to the file. –It assigns a number of empty block to I-Node pointed by a parameter. –The preallocation of disk space for a file provides an efficient method for allocating storage without having to write any data. –This can result in faster I/O compared to a file which gains disk space incrementally as it grows. –Existing data in the file will not be modified. Reading any of the preallocated blocks will return zeroes.

Bologna, March 30, 2006 INFN / D-Grid meeting Page 17 StoRM is a SRM v.2.1 solution for disk based storage with Cluster File System. StoRM permits native direct access to storage. StoRM permits guaranteed space reservation. StoRM uses VOMS proxy certificate for Authentication, and interacts with Authorization Service as G-PBox. Demonstration of StoRM integration with the WS- Agreement Service was done. Conclusion

Bologna, March 30, 2006 INFN / D-Grid meeting Page 18 Project lead by Antonia Ghiselli. Alberto Forti (CNAF), from 01 April Luca Magnoni (CNAF), 100% Riccardo Zappi (CNAF), 100% Ezio Corso (ICTP), 100% Antonio Messina (ICTP), 50% Riccardo Murri (ICTP), 50% StoRM Development Team

Bologna, March 30, 2006 INFN / D-Grid meeting Page 19 Questions and Suggestions ?!