1. Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 http://www.eu-datagrid.org/ The Storage Element

2. John Gordon - SE Workshop - 24/09/01° 2 An SE in Testbed1 u A disk server u running GDMP v2.0 u running information providers u running data servers (RFIO, GridFTP) u optionally NFS mount user data on local CE u optionally stage replicated data to MSS

3. John Gordon - SE Workshop - 24/09/01° 3 Long Term Goals for SE u This is not the long-term vision. u By Testbed2 we should have: n replaced current GDMP by WP2 ReplicaManager n a functioning integrated SE Grid Service u This talk is about that Integrated SE Service

4. John Gordon - SE Workshop - 24/09/01° 4 The SE Provides: u Grid interface to permanent storage s Grid security s Grid protocols: n Data transfer n Control n Information u Additional functionality s SE provides functionality not supported by underlying storage system s e.g. access control, logging, monitoring

5. John Gordon - SE Workshop - 24/09/01° 5 Interaction with other Work Packages

6. John Gordon - SE Workshop - 24/09/01° 6 Grid ClientsOther clients SE Mass Storage Systems

7. John Gordon - SE Workshop - 24/09/01° 7 Interface 1 Interface 3 Interface 2 Message Queue Session Manager System LogHouse Keeping MetaData MSS Interface MSS Interface MSS1MSS2 Top layer Core Bottom layer Clients Storage Element SE Architecture

8. John Gordon - SE Workshop - 24/09/01° 8 Clients u Resource Broker & Job Submission Service (WP1) u Replica Manager (WP2) n replicates files between StorageElements u Information services (WP3) u User Applications n running on a ComputingElement n command line utilities

9. John Gordon - SE Workshop - 24/09/01° 9 Control Interface Control interface allows: u File creation and deletion u Reserving files for reading (“pinning”) u Reserving space for writing u Changing file permissions and lifetime u Delegations u PPDG are developing an SRM API for some of these aspects which we plan to implement.

10. John Gordon - SE Workshop - 24/09/01° 10 We have written a query program edg_se_query that performs this query Information Interface SE acts as provider for Grid Information Service (MDS) u SE metadata: protocols supported, free space, policies u File metadata: creation date, ownership, access time, pinning status, size, estimated access/transfer time u API provided so clients are not dependent on underlying Information Service n So can support replacement for MDS

11. John Gordon - SE Workshop - 24/09/01° 11 Data Transfer Interface Will support:- u GridFTP n Standard FTP protocol extended to provide efficient transfer u RFIO n Used at CERN as interface to CASTOR u NFS n Not secure by default, SE must provide secure version Files are always stored in the storage system, but to clients it appears as if the file is stored in the SE.

12. John Gordon - SE Workshop - 24/09/01° 12 ClientSE Replica Manager/Catalog Storage 5’ 2 3 4 1 (1) The Client asks a catalogue to provide the location of a file (2) The catalogue responds with the name of an SE (3) The client asks the SE for the file (4) The SE asks the storage system to provide the file (5,6) The storage system sends the file to the client through the SE or (5’) directly 5 6

13. John Gordon - SE Workshop - 24/09/01° 13 Supported Storage Systems u UNIX disk systems u HPSS (High Performance Storage System) u CASTOR (through RFIO) u GridFTP servers u DMF u Enstore u SRM?... u ????????

14. John Gordon - SE Workshop - 24/09/01° 14 Security u Will integrate with EDG Security Model n authentication n SE access control n File access control n Grid ACL?

15. John Gordon - SE Workshop - 24/09/01° 15 Client APIs We will provide u Existing APIs: n Compatibility libraries (e.g. RFIO libraries) n Support for existing network protocols (e.g. GridFTP) u SE-specific APIs: n Libraries specifically written to access the SE n Command line utilities built using the above

16. John Gordon - SE Workshop - 24/09/01° 16 Interface Queue Request Manager Pipe Manager Named Pipe 1 23 4 5 7 8 Pipe Store Network Request processing in the SE Handler Tape Disk 6 MSS Interfaces Core and MSS Layer

17. John Gordon - SE Workshop - 24/09/01° 17 Summary u In Testbed1 WP5 will continue to enhance capability of RFIO, GridFTP, and Information Providers u By Testbed 2 we plan to have an Integrated SE grid service n which will provide extra functionality to ordinary data servers. u Reference: WP5 Architecture and Design document http://edms.cern.ch/document/336679 http://edms.cern.ch/document/