New Xrootd Monitoring Stream Federated Data Workshop September 13, 2012 Andrew Hanushevsky, SLAC http://xrootd.org

Current Monitoring State xrootd provides a wide range of data Periodic summary data (xml based) Real time detail data (binary based) Login identification Filename mapping information File access information Seek based I/O File close summary (bytes r/w only) Major efforts ongoing to digest data Atlas: Julia Andreeva, Danila Oleynik, Artem Petrosyan, and Ilija Vukotic CMS: Matevz Tadel

The Monitoring Issue Only detail stream provides sufficient data Used to derive intermediate information Real-Time per-file transfers Though difficult to get it reasonably correct Sigma’s for I/O block size Raw data far exceeds derived data size Desire to send just enough data Less likely to overwhelm collector Solution is to create a new “f” stream

The New Real-Time “f” Stream Binary stream for only file-based information Intermediate detail between summary and detail Provides accurate Real-Time per-file transfers Computed Sigma’s for I/O block size Configured via the xrootd.monitor directive Option: fstats interval [fn] [io] [ops] [sigma] fn include filename in open record io provide per-file I/O statistics each interval ops include operation counts in close record Sigma calculate sigma values

The “f” Stream Open Record Inserted into stream when file opened struct XrdXrootdMonFileHdr // 8 { char recType; // close | io | open | ops | time char recFlag; // Record type-specific flags (e.g. open mode, fn present, etc) short recSize; // Size of this record in bytes int fileID; // dictid of file for all rectypes except "time“ }; struct XrdXrootdMonFileOpn XrdXrootdMonFileHdr Hdr; // 8 long long fsz; // 8 file size at time of open // tfn[]; // n "<tid>\n<lfn>\0" of size(Hdr.RecSize - 16)

The “f” Stream I/O Record One record per open file (each interval) struct XrdXrootdMonStatIO { long long read; // Bytes read from file so far using read() long long readv; // Bytes read from file so far using readv() long long write; // Bytes written to file so far }; struct XrdXrootdMonFileIO XrdXrootdMonFileHdr Hdr; // 8 XrdXrootdMonStatIO Bytes; // 24

The “f” Stream close Record I One record per open file (each interval) Operation count information (only if ops or sigma selected) struct XrdXrootdMonStatOps { int read; // Number of read() calls int readv; // Number of readv() calls int write; // Number of write() calls short rsMin; // Smallest readv() segment count short rsMax; // Largest readv() segment count long long rsegs; // Number of readv() segments int rdMin; // Smallest read() request size int rdMax; // Largest read() request size int rvMin; // Smallest readv() segment size int rvMax; // Largest readv() segment size int wrMin; // Smallest write() request size int wrMax; // Largest write() request size };

The “f” Stream close Record II One record per closed file Operation sigma information (only if sigma selected) struct XrdXrootdMonStatSD { int read; // Sigma for read size int readv; // Sigma for readv size Int rsegs; // Sigma for seg size int write; // Sigma for write size wrMax; // Largest write() request size }; struct XrdXrootdMonFileOps XrdXrootdMonFileHdr Hdr; // 8 XrdXrootdMonStatOps Ops; // 48 XrdXrootdMonStatSD Sigma; // 16