Xrootd Proxy Service Andrew Hanushevsky Heinz Stockinger Stanford Linear Accelerator Center SAG September-04

20-Sept-042: xrootd The BaBar Experiment Use big-bang energies to create B meson particles Look at collision decay products Answer the question “where did all the anti-matter go?” 500 physicists collaborating from >70 sites in 10 countries USA, Canada, China, France, Germany, Italy, Norway, Russia, UK, Taiwan The experiment produces large quantities of data 300 TBytes/year for 10 years Most data stored as objects using Root persistency framework Some data stored in Objectivity/DB database Expected to double every year as detector luminosity increases Heavy computational load 5, GHZ CPU’s spread over 35 sites world-wide Work is distributed across the collaboration

20-Sept-043: xrootd BaBar is the Forerunner LHC at CERN The Large Hadron Collider Due to start in 2007 Will generate several order of magnitude more data Will require even more compute cycles Example: ATLAS Probe the Higgs boson energy range Explore the more exotic reaches of physics

20-Sept-044: xrootd The Data Access Need Scalable high performance access to data Must scale to 100’s if not 1000’s of data servers Most data is read-only Data is written only once Versioned Secondary access to distributed data As a backup strategy

20-Sept-045: xrootd Solution Fundamentals Extensible base server architecture Allows for high performance implementation Rich but efficient server protocol Combines file serving with P2P elements Allows client hints for improved performance Administrative security Implies a structured peer-to-peer framework

20-Sept-046: xrootd The Implementation High Performance File-Based Access Fluidly scalable Works well in single server environments Scales beyond 32,000 cooperative data servers Naively extensible Requirement for this level of scaling Servers can be added at any time without disruption Fully fault-tolerant Servers can be removed at any time without disruption Flexible Security Allowing use of almost any protocol

20-Sept-047: xrootd Entities & Relationships data xrootd olbdxrootd olbd Data Clients Redirectors Data Servers M S ctl olbd Control Network Managers & Servers (resource info, file location) xrootd Data Network (redirectors steer clients to data Data servers provide data)

20-Sept-048: xrootd Example: SLAC Configuration client machines kan01kan02kan03kan04 kanxx bbr-rdr03bbr-rdr04 bbr-rdr-a data servers redirectors

20-Sept-049: xrootd Data Growth & More Fault Tolerance BaBar Data Is Replicated Backup Strategy Processing Strategy Some data only available at one site Use grid techniques to make data accessible But, when thing go wrong would like access The proxy solution

20-Sept-0410: xrootd The 10,000 Foot View IN2P3 fr FZK de RAL uk INFN it SLAC us

20-Sept-0411: xrootd The Reality Sites has a fear of hosting… Distributed Denial of Service Attacks Massive illegal file sharing Only certain hosts allowed to get outside Rarely batch worker machines The ones that need remote data most The Firewall Issue

20-Sept-0412: xrootd A Closer Look SLAC IN2P3 RAL xrootd’s RAL proxy IN2P3 proxy Firewalls require Proxy servers

20-Sept-0413: xrootd Proxy Service Attempts to address competing goals Security Deal with firewalls Scalability Administrative Configuration Performance Ad hoc forwarding for near-zero wait time Intelligent caching in local domain

20-Sept-0414: xrootd Proxy Implementation Uses capabilities of olbd and xrootd Simply an extension of local load balancing Implemented as a special file system type Interfaces in the Logical File System layer (ofs) Functions in the Physical File System layer (oss) Primary developer is Heinz Stockinger

20-Sept-0415: xrootd Proxy Interactions client machines red01data02data03proxy01 local olb data01data02data03data04 proxy olb local olb RAL SLAC

20-Sept-0416: xrootd Why This Arrangement? Minimizes cross-domain knowledge Necessary for scalability in all areas Security Configuration Fault tolerance & recovery

20-Sept-0417: xrootd Scalable Proxy Security SLAC PROXY OLBDRAL PROXY OLBD Authenticate & develop session key 2 2 Distribute session key to authenticated subscribers 3 3 Data servers can log into each other using session key Data Servers

20-Sept-0418: xrootd Proxy Performance Introduces minimal latency overhead Virtually undetectably from US/Europe Negligible on faster links 2% slower on fast US/US links 10% slower on LAN Can be further improved Parallel streams Better window size calculation Asynchronous I/O

20-Sept-0419: xrootd Proxy Study Conclusion Proxy Service easily integrates into xrootd Largely due to peer-to-peer architecture Provides enhanced service at minimal cost Allows access to addition data sources Increases fault tolerance Covers up for grid transfer mistakes Scalable in all aspects Security, number of servers, administration

20-Sept-0420: xrootd Overall Conclusion xrootd provides high performance file access Improves over afs, ams, nfs, etc. Unique performance, usability, scalability, security, compatibility, and recoverability characteristics Should scale to tens of thousand clients Will be distributed as part of CERN’s root package Open software, supported by SLAC (server), INFN-Padova (client) CERN (security, packaging)