1. Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT Standard Protocols in DPM Ricardo Rocha

2. Grid Technology DPM Main Goals Provide lightweight “grid enabled” storage Manage space on distributed disk servers Manage a hierarchical namespace Expose interfaces for 3 use cases: –Space management (socket, SRM1.1, SRM2.2) –Remote data access (gridFTP, HTTP/HTTPS) –POSIX like access (rfio, xroot)

3. Grid Technology DPM Further Details Written mostly in C C/C++ API, Python API, Perl API, CLI Statistics from GStat –https://gstat-wlcg.cern.ch/gstat/stats/https://gstat-wlcg.cern.ch/gstat/stats/ –Over 200 grid sites use DPM Largest deployment: 1.5PB More Information –https://svnweb.cern.ch/trac/lcgdm/wiki/Dpmhttps://svnweb.cern.ch/trac/lcgdm/wiki/Dpm

4. Grid Technology Why Standards

5. Grid Technology Standards help with… Library dependency issues Requirement of user interfaces (UIs) –Entry points to the grid –Maintained by experts Very hard to use “standard” distributions –Even transition from SLC4 to 5 is problematic Validation takes a long time

6. Grid Technology Standards improve… Accessibility –Not limiting access to OS X version Y with library Z Validation –Using common validation and test tools Stability –Evolution discussed in a wide group Ease of implementation –Sharing of experiences, common code base No vendor lock-in …

7. Grid Technology Use Case: Space Management SRM

8. Grid Technology Storage Resource Manager (SRM) An interface to manage storage resources –Supported by all grid storage backends DPM supports all SRM versions in use –1.1 / 2.2 Stable, in production for several years Current focus: provide pure SSL access –Ongoing effort in the context of EMI (together with dCache, BeStMan, STORM, …) –Replacing httpg with simple https –Requires alternative for credential delegation

9. Grid Technology Use Case: Remote Data Access HTTP / WebDAV

10. Grid Technology HTTP / WebDAV DPM supports file access via HTTP –Currently only as a transfer protocol –Implemented as an apache module or cgi Firewall friendly Extensive client availability Work in progress: –Adding support for WebDAV (RFC 2518): locking, namespace management, metadata on files Currently not so interesting within HEP, but very popular among other communities –Improve the HTTP implementation (benefit from upcoming nameserver code refactoring) Major focus once NFS 4.1 is stable

11. Grid Technology Use Case: POSIX Data Access NFS 4.1

12. Grid Technology NFS 4.1 Very good match for our needs Strong authentication / authorization –KRB5 (ongoing work for X509), Basic permissions & ACLs Performance –Via bulk operations (few round trips), Sessions, Delegations –Even more relevant for WAN access Parallel / Clustered data access –Parallel NFS (pNFS), multiple layout types (file, object, block) Multi server namespace –Spawning multiple domains, Client redirection, Alternative replica selections Free Clients (Linux, Solaris, Windows) –Supported by major industry vendors, Free caching Free system administration and monitoring tools

13. Grid Technology DPM NFS 4.1 Details (client and server): https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/NFS41 https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/NFS41 Most work is on the Head Node (Metadata Server) Possibility to reuse existing open source code Looked into two alternatives –sPNFS: a kernel based implementation Working prototype, but kernel programming is hard –User space daemon (Ganesha) Mainly developed by CEA in France, in production for several years First DPM enabled release deployed internally, in validation phase For the moment we focus on the second option (but did not completely drop the first)

14. Grid Technology DPM NFS 4.1 DPM HEAD NODE DATA SERVER DATA SERVER CONTROL PROTOCOL (undefined) NFS4.1/pNFS ProtocolStorage Access Protocol (File Layout) DATA SERVER DATA SERVER … … Callbacks CLIENT Standard Distribution Kernel

15. Grid Technology DPM NFS 4.1: Status & Roadmap Prototype ready and deployed internally –Includes strong security –Automated build and testing environment –Finishing functionality validation Next step is to extend testing (starting now) –Increase the number of sites involved –Include a DPM instance in the CERN NFS 4.1 performance testing effort Wide deployment first half 2011 Further work: –Multi server namespace : global grid namespace and client redirection

16. Grid Technology DPM NFS 4.1: Multi Server Namespace Namespace spawning multiple domains Servers redirect clients when data is not local –Redirection is the key word here Can also be used to provide clients with alternative locations (transparent failover on server overload, unavailability) CLIENT NFS SITE A NFS SITE B 1 2 open (/grid/siteB/myFile) NFS4ERR_MOVED ( fs_locations )

17. Grid Technology DPM NFS 4.1: Extra goodies We can benefit from using a standard Existing validation / testing suites –Example: PyNFS Existing monitoring tools –Example: collectl Existing debugging tools –Example: wireshark / tshark #./testserver.py $DPMNFS_HOST:/grid/dpm/cern.ch -v dir # collectl -snf

18. Grid Technology DPM NFS 4.1: Extra goodies We can benefit from using a standard Existing validation / testing suites –Example: PyNFS Existing monitoring tools –Example: collectl Existing debugging tools –Example: wireshark / tshark #./testserver.py $DPMNFS_HOST:/grid/dpm/cern.ch -v dir # collectl -snf # tshark -R "nfs" -V Network File System, Ops(4): PUTFH LOOKUP GETFH GETATTR [Program Version: 4] [V4 Procedure: COMPOUND (1)] Tag: length: 0 contents: minorversion: 0 Operations (count: 4) Opcode: PUTFH (22) filehandle length: 112 [hash: 0x311d234b] decode type as: unknown filehandle: 000000004D00000000003030000000000300000004000000... Opcode: LOOKUP (15) Filename: cern.ch length: 7 contents: cern.ch …

19. Grid Technology Conclusion With HTTP/WebDAV and NFS4.1, DPM provides standard based solutions for all its use cases Benefits exist for both clients and system administrators (and even developers) DPM will continue its work on improving the status of grid data storage and access