1. Scalla Developer Summary xrootd /cmsd Andrew Hanushevsky SLAC National Accelerator Laboratory CERN Workshop 10-November-08 http://xrootd.slac.stanford.edu

2. 10-November-082: http://xrootd.slac.stanford.edu Supported Platforms FreeBSD New addition (best-effort support only) Linux i386_linux24, i386_linux26, x86_64_linux_26 MacOS ppc_darwin_70, ppc_darwin_80, x86_darwin_90 Solaris sun4x_58, sun4x_59, sun4x_510, sunx86_510 Windows XP (client only)

3. 10-November-083: http://xrootd.slac.stanford.edu Supported Compilers g++ Up to version 4.3.2 Used for FreeBSD, Linux, and MacOS icc Used for Itanium architectures SGI Linux Sun CC Always used for Solaris compilations

4. 10-November-084: http://xrootd.slac.stanford.edu Supported Build Environments In-House configure.classic Works on all platforms without problems Missing some features Make install Autotools make Supported by Derek Feichtinger Has some issues on certain platforms E.G., Unable to build XrdPosixPreload.so 32/64 bit compilation/linking on 64/32 bit platforms

5. 10-November-085: http://xrootd.slac.stanford.edu General Source Layout I XProtocol/ cmsd and xrootd protocol data structures Xrd/ Protocol driver for cmsd and xrootd (networking and scheduling) XrdAcc/ Default access control (i.e., authorization) XrdBwm/ Bandwidth manager plug-in XrdCS2/ Castor plug-in (obsolete) XrdClient/ Client related code (e.g., TXNetFile, xrdcp, etc) XrdCms/ Cluster Management Services protocol plug-in XrdCns/ Cluster Name Space daemon and other components

6. 10-November-086: http://xrootd.slac.stanford.edu General Source Layout II XrdCrypto/ Cryptography support classes for security plug-ins XrdMon/ Monitor data collection agent XrdNet/ All networking related classes (used by everyone) XrdOdc/ Open Distributed Cluster support (deprecated) XrdOfs/ Open File System plug-in XrdOlb/ Open Load Balancing protocol implementation (deprecated)

7. 10-November-087: http://xrootd.slac.stanford.edu General Source Layout III XrdOss/ Open Storage System plug-in XrdOuc/ Object Utility Classes (used by everyone) XrdPosix/ Posix compatibility libraries Includes preload library XrdPss/ Proxy plug-in Used as a storage system plug-in

8. 10-November-088: http://xrootd.slac.stanford.edu General Source Layout IV XrdSec/ Authentication protocol driver plug-in XrdSecgsi/ Gsi-based authentication plug-ins XrdSeckrb4/ Kerberos IV authentication plug-in XrdSeckrb5/ Kerberos V authentication plug-in XrdSecpwd/ Password authentication plug-in XrdSecsss/ Simple Shared Secret authentication plug-in XrdSecunix/ Unix (i.e., NFS-like) authentication plug-in

9. 10-November-089: http://xrootd.slac.stanford.edu General Source Layout V XrdSfs/ File System Interface definition Includes the default implementation XrdSut/ Security utility classes XrdSys/ OS-dependent classes (used by everyone) XrdTokenAuthzOfs/ ALICE ofs plug-in implementing ALICE security XrdVersion.hh Holder to identify the version everywhere XrdXr/ Proxy plug-in (obsolete, to be deleted) XrdXrootd/ Xrootd protocol plug-in

10. 10-November-0810: http://xrootd.slac.stanford.edu So, What’s A Plug-In? Run-Time Loadable Code A single class based on an abstract interface Plus an extern ‘C’ object instantiator Always packaged as a shared library Configuration file specifies location of library Automatically loaded Objects created as needed Extends xrootd or cmsd functionality Avoids massive rebuilds and code branching The base system uses pre-defined statically linked plug-ins

11. 10-November-0811: http://xrootd.slac.stanford.edu lfn2pfn prefix encoding Storage System (oss, drm/srm, etc) authentication (gsi, krb5, etc) Clustering (cmsd) authorization (name based) Storage System (Flash Based System) File System (ofs, sfs, alice, etc) Protocol (1 of n) (xrootd, cmsd) Scalla Plugin Architecture Protocol Driver (Xrd)

12. 10-November-0812: http://xrootd.slac.stanford.edu The Plug-Ins Dynamic plug-in always packaged as a shared library Authentication plug-in Always as libXrdSec.so xxxx Others as libXrdSecxxxx.so Authorization plug-in Currently, no plug-ins exist (default statically linked-in) File System plug-in Available: libXrdOfs.so Storage System plug-in Available: libXrdProxy.so (provides proxy access) Protocol plug-in PROOF team uses a plug-in to implement PROOF protocol External Management Interface plug-in (XMI) Used by Castor to interface cmsd to Castor name space

13. 10-November-0813: http://xrootd.slac.stanford.edu Closer Look At Security Plug-Ins xrootd has no security FALSE! By default, security is not enabled This simplifies setup for most sites You must configure security to get it Not difficult but yet another thoughtful step Once enabled, xrootd is as secure as you want So, the myth is now busted!

14. 10-November-0814: http://xrootd.slac.stanford.edu What Is Security? Two phases to security Authentication Done in the xrootd/cmsd protocol layer Multiple simultaneous authentication modes supported Authorization Done in the ofs filesystem layer Basic user/group/netgroup capability list mode available But, authorization is a plug-in Can substitute any other mode you’d like

15. 10-November-0815: http://xrootd.slac.stanford.edu xrootd Authentication Multi-protocol design Server provides available protocols Client chooses one of the possibilities Each protocol name comes with configuration data Allows client to self-configure for the protocol Each protocol implemented as a plug-in Easy to add new protocols

16. 10-November-0816: http://xrootd.slac.stanford.edu Currently Available Protocols gsi Grid certificate based authentication Extension gsiGMAPLDAP maps DN to username via LDAP krb4 Standard Kerberos IV krb5 Standard Kerberos V pwd Hidden password authentication sss Simple shared secret unix Basic Unix NFS-like [non-]security

17. 10-November-0817: http://xrootd.slac.stanford.edu Authentication Protocol Plug-Ins Each protocol is a shared library plug-in xxxx libXrdSecxxxx.so xxxx krb4 The xxxx is the protocol name (e.g., libXrdSeckrb4.so) The plug-ins are managed by libXrdSec.so Authentication protocol client/server driver Finds and loads appropriate shared libraries Uses abstract security interface for all interactions Credential generation & authentication

18. 10-November-0818: http://xrootd.slac.stanford.edu Abstract Class XrdSec class XrdSecProtocol { public: virtual int Authenticate( XrdSecCredentials *cred, // In XrdSecParameters **parms, // Out XrdSecClientName &client, // Out XrdOucErrInfo *einfo=0 ) // Out virtual XrdSecCredentials *getCredentials( XrdSecParameters *parms=0, // In XrdOucErrInfo *einfo=0 ) // Out virtual const char *getParms( int &psize, // Out const char *host ) // In }; C++ framework to manage protocols as plug-ins Generic protocol (XrdSecProtocol) Protocol implementations inherit from XrdSecProtocol server client Courtesy of Gerri Ganis

19. 10-November-0819: http://xrootd.slac.stanford.edu libXrdSec.so libXrdSec.so provides the Protocol Manager - Server: instantiated at start-up from configuration file: - load protocol plug-ins that server can / wants to run - binds (subsets of) the list to hosts or host patterns - controls authentication mode by host - Client: build-up list loading protocols the first time needed - library loaded only if authentication is required Plug-in implementations provide a public instantiator to create an instance of the protocol Simple negotiation: list of allowed protocols sent to the client, who chooses the one to try first Courtesy of Gerri Ganis

20. 10-November-0820: http://xrootd.slac.stanford.edu XrdSec Implementation Is Generic Depends only on network and utility modules XrdNet, XrdOuc, and XrdSys Can be easily used in a non-XROOTD context Working example of standalone client and server programs using XrdSec available at http://ganis.home.cern.ch/ganis/ROOT/SECURITY/testXrd.tgz Courtesy of Gerri Ganis

21. 10-November-0821: http://xrootd.slac.stanford.edu Authentication Architecture login Send credentials Send client-specific security configuration libXrdSec.so GetCredentials (Select Protocol) Get Security Configuration Multiple exchanges allowed libXrdSecgsi.so libXrdSeckrb4.so libXrdSeckrb5.so libXrdSecpwd.so Dynamically selected by client Server specifies availability Libraries managed by libXrdSec.so Config File 1 2 3n authenticate Ask for more get credentials 0

22. 10-November-0822: http://xrootd.slac.stanford.edu Authenticated Identity Passed to file system layer to be used for authorization char prot[8]; // Protocol used char *name; // Entity's name char *host; // Entity's host name char *vorg; // Entity's virtual organization char *role; // Entity's role char *grps; // Entity’s groups char *endorsements; // Protocol specific endorsements char *tident; // Trace identifier (do not modify) void *cert // Pointer to certificate (future) int clen; // Length of certificate (future) XrdSecEntity.hh

23. 10-November-0823: http://xrootd.slac.stanford.edu Why Do It This Way? Can implement almost any model needed Without changing any server/client code at all Simplifies security audit procedures Can quickly evolve as requirements change And support different modes for different moods SLAC Atlas uses sss authentication Fermi (a.k.a. GLAST) uses unix authentication But generally many experiments don’t want anything Only for reading but not if they are writing to xrootd

24. 10-November-0824: http://xrootd.slac.stanford.edu But Wait! Where is SSL? SSL applies to the transport layer The security framework applies to the protocol layer Recall, any protocol can optionally use this framework A transport protocol does not easily give you any options And SSH? You really mean using ssh keys, don’t you? No one wrote a plug-in for that yet

25. 10-November-0825: http://xrootd.slac.stanford.edu Let’s Recap xrootd/cmsd are all about plug-ins Expands applicability within the design focus Allows you to piggy-back new functionality For instance, PROOF Are all points pluggable? Just the obvious ones There is always room for improvement here E.G., MSS plug-in now part of the oss plug-in So, let’s see talk about writing plug-ins

26. 10-November-0826: http://xrootd.slac.stanford.edu Writing Plug-Ins I Read the documentation in the plug-in “hh” XrdAccAuthorize.hh Authorization abstract interface XrdCmsXmi.hh XMI abstract interface XrdOss.hh Storage System Abstract Interface XrdProtocol.hh Protocol abstract interface XrdSecInterface.hh Authentication abstract interface XrdSfsInterface.hh File system abstract interface Look at an existing plug-in of the same type

27. 10-November-0827: http://xrootd.slac.stanford.edu Writing Plug-Ins II General things All plug-ins must be thread-safe Avoid high-latency actions This generally causes pile-up/melt-down Usually because of thread starvation and timeouts Some interfaces allow background processing High latency is not an issue then Use existing classes in the repository libCrypto.a, libCryptoLite.a, libXrdNet.a, libXrdOuc.a, and libXrdSys.a

28. 10-November-0828: http://xrootd.slac.stanford.edu Writing Plug-Ins III For a plug-in to be included in the CVS repository Must follow naming conventions Adhere to the abstract interface Interface changes are rare and always backward compatible Be stand-alone Cannot rely on frameworks and external add-ons This includes STL A sore point but it’s saved us countless hours of debugging Have a good Makefile and Makefile.am Compile and run on all supported platforms Usually have detailed documentation See existing references (e.g., Security) http://xrootd.slac.stanford.edu/doc/sec_config/sec_config.htm Otherwise, we can simply reference your plug-in web page

29. 10-November-0829: http://xrootd.slac.stanford.edu Packaging Official releases are available via web site http://xrootd.slac.stanford.edu/ We are working on getting a better more generic URL Currently, Wilko Kroeger cuts official releases This does not preclude special integrated releases ALICE Castor PROOF Root These usually suffer a little drift from the official release We are working on making the CVS available r/o The fastest way is an AFS accessible directory /afs/slac.stanford.edu/public/software/scalla CVS head checked out for viewing and a gtar file of the same Planning for a web interface to the repository

30. 10-November-0830: http://xrootd.slac.stanford.edu Licensing & Contributions Currently, Scalla is under a BSD License http://www.opensource.org/licenses/bsd-license.php May change to Apache 2.0 license http://www.opensource.org/licenses/apache2.0.php We may need to restrict how names are used cmsd, Scalla, and xrootd This is to prevent confusion relating to derivative works Contributions happily accepted Must conform to licensing requirements Handled in the Linux tradition

31. 10-November-0831: http://xrootd.slac.stanford.edu SLAC Support Memo of understanding… The SLAC science program is heavily dependent on xrootd. I can therefore assure you that xrootd will be maintained by SLAC for at least five years. As you are aware, xrootd is an open source product and will remain freely available. I believe that xrootd is brings valuable and currently unique capabilities in scalable high-volume data analysis. It is part of SLAC's mission to encourage wide use of developments like xrootd where they can benefit national and international science programs. SLAC staff supporting xrootd will be encouraged to examine, as time permits, problems and suggestions submitted by users who are not connected with the SLAC program. Non-trivial work in response to such submissions would require that work be also beneficial to SLAC's use of xrootd. I particularly encourage a collaborative approach to maintaining and developing products like xrootd. This approach promotes wide use, and creates a situation where effort spent on issues raised by a collaborator would be considered valuable to SLAC by default. Richard Mount Richard Mount

32. 10-November-0832: http://xrootd.slac.stanford.edu Active Developers Current Active Software Developers Andreas Peters (Andreas.Joachim.Peters cern.ch)Andreas.Joachim.Peters cern.ch Castor/xrootd Andrew Hanushevsky (abh stanford.edu or Andrew.Bohdan.Hanushevsky cern.ch)abh stanford.eduAndrew.Bohdan.Hanushevsky cern.ch Server Bertrand Bellenot (Bertrand.Bellenot cern.ch)Bertrand.Bellenot cern.ch FreeBSD issues, Windows Derek Feichtinger (Derek.Feichtinger cern.ch)Derek.Feichtinger cern.ch Autotools Fabrizio Furano (Fabrizio.Furano cern.ch)Fabrizio.Furano cern.ch Client Gerardo Ganis (Gerardo.Ganis cern.ch)Gerardo.Ganis cern.ch Classic make, Cross-Platform issues, Security and PROOF Tofigh Azemoon (azemoon slac.stanford.edu)azemoon slac.stanford.edu Monitoring Wilko Kroeger (wilko slac.stanford.edu)wilko slac.stanford.edu MPS scripts, packaging, and release issues

33. 10-November-0833: http://xrootd.slac.stanford.edu Getting Support Currently available venues Official web site (always check there first) http://xrootd.slac.stanford.edu/ General problem mailing list (xrootd-l@slac.stanford.edu)xrootd-l@slac.stanford.edu Must be subscribed (see http://xrootd.slac.stanford.edu/xrootdlist.html)http://xrootd.slac.stanford.edu/xrootdlist.html CERN Web Site http://project-arda-dev.web.cern.ch/project-arda-dev/xrootd/site/index.html Request Tracker (RT) problem system at SLAC This is still experimental E-Mail the right developer Actual bugs, contributions, and enhancement requests User support An evolving issue for experiments and groups E.G., OSG provides 1 st level support for VDT Developers generally cannot provide direct user support

34. 10-November-0834: http://xrootd.slac.stanford.edu Acknowledgements Software Contributors CERN: Derek Feichtinger, Fabrizio Furano, Andreas Peters Fermi: Tony Johnson (Java) Root: Gerri Ganis, Bertrand Bellenot SLAC: Jacek Becla, Tofigh Azemoon, Wilko Kroeger Operational Collaborators BNL, INFN, IN2P3 Partial Funding US Department of Energy Contract DE-AC02-76SF00515 with Stanford University