1. File System Access (XRootd) Andrew Hanushevsky Stanford Linear Accelerator Center 13-Jan-03

2. Jauuary 13, 20032: XRootd Goals High Performance File-Based Access Scalable Extensible Mass Storage integration Flexible Security Dynamically Configurable Rootd Compatable

3. Jauuary 13, 20033: XRootd Achieving High Performance Multi-threaded server Ability to run multiple servers on one host MRU polling Last active connections serviced first Sticky sockets Socket/thread bound for “active” connections Using high-performance oofs layer Unlocked I/O, sync reduction, file descriptor partitioning

4. Jauuary 13, 20034: XRootd Achieving Scalability Socket multiplexing One connection per client/host Requires stream id protocol extensions Dynamic Load Balancing Requires client redirection protocol extensions Using scalable oofs layer File reuse, idle closeouts, cached file system

5. Jauuary 13, 20035: XRootd Achieving Extensibility Opaque Information Protocol Protocol extension allows client to relay outboard information to server Messaging interface Cache group Request priority Other things we haven’t thought of yet…

6. Jauuary 13, 20036: XRootd Achieving MSS Integration Using oofs layer Already integrates with almost any MSS No brainer

7. Jauuary 13, 20037: XRootd Achieving Flexible Security Negotiated Security Protocol Extensions Allows client/server to agree on protocol E.g., Kerberos, GSI, AFS Kerberos, etc. Using oofs layer Multi-protocol authentication support Scalable authorization

8. Jauuary 13, 20038: XRootd Achieving Configurability Servers must come up/down without harm Part of dynamic load balancing Requires client retry extension

9. Jauuary 13, 20039: XRootd Achieving Compatability Client sensitive to server version number Rootd servers default to base TNetFile class XRootd servers use derived TNetFile class Server requires client to turn on extensions Base TNetFile class restricted to rootd ops Derived TNetFile enables extensions Multi-streaming, Redirection, Request Deferral, Generic Security, Opaque Information

10. Jauuary 13, 200310: XRootd xrootd Architecture Database Protocol Layer Filesystem Logical Layer Filesystem Physical Layer Filesystem Implementation

11. Jauuary 13, 200311: XRootd Some Details Compatible Protocol Allows transparent use of rootd & xrootd Multi-Streaming Allows multiple files per connection Cache File System Dynamic Load Balancing Client directed to “appropriate” server Configurability & Recoverability

12. Jauuary 13, 200312: XRootd Compatible Protocol TNetFile XTNetFile Application xrootd rootd rootd compability

13. Jauuary 13, 200313: XRootd Multi-Streaming Each open() assigned a stream number Four character id that starts with 0-9 All requests/responses identify relevant stream Non-open file requests use arbitrary id Any four characters not starting with 0-9 E.g., stat(), mv(), unlink(), etc. Parallel Multi-Streaming allowed Client must explicitly enable it

14. Jauuary 13, 200314: XRootd Cache File System /cache1/databases:mydbfile /databases/mydbfile /cache2 /cache3 symlink Index Area Optional data cache Default data area Data Area Any number Any Size Chosen based on free space in LRU order Multiple Independent Filesystems Naming convention allows for audit and index recovery

15. Jauuary 13, 200315: XRootd Dynamic Load Balancing Dynamic Selection

16. Jauuary 13, 200316: XRootd DLB Implementation xrootd dlbd xrootd dlbd xrootd dlbd xrootd dlbd Client subscribe (any number) open wait open again try host:port who has the file? I do

17. Jauuary 13, 200317: XRootd Configurability & Recoverability Go hand in hand Configuration changes may be disruptive Must be able to recover from dynamic configuration changes Dynamic Load Balancing essential part

18. Jauuary 13, 200318: XRootd Recoverability Whenever client looses r/o connection Back to distinguished xrootd(s) for reselection Whenever client looses r/w connection Limited wait/retry loop on the same server We will be working to improve this next year! All handled in the XTNetFile class Disruptions merely delay the client

19. Jauuary 13, 200319: XRootd Configurability Since we have recoverability… Servers may come and go New servers can be added at any time Servers may be brought down for maintenance Files can be moved around in real-time Client will recover and adjust to the new configuration

20. Jauuary 13, 200320: XRootd Conclusion Goals for rootd file access can be met Most of this work relies on leveraging work done for the AMS Compatability allows us to lobby the community for xrootd to become the default “rootd”