LCG Accounting John Gordon Grid Deployment Board 13 th January 2004

Overview  Aim is to gather accounting information at the job level from each site and store in a central database so that arbitrary queries can be done against it.  Four parts  Schema  Gathering  Sending  Querying

Schema  Defined our own  Then discovered GGF Usage Schema 

Accounting Schema  LocalGroup - the group that the job ran as on the system  LocalJobID - the ID of the job on the system on which it ran **  LCGJobID - the job ID assigned by the RB (null for jobs submitted locally or via globus) **  ExecutingSite - the site at which the job ran **  ExecutingCE - the queue from which the job ran  SubmittingRB - the RB used to submit the job  LocalUserID - the ID of the user on the system on which the job ran **  LCGUserDN - the DN of the user within LCG **  LCGUserVO - the virtual organisation of this user for this job **  LocalStartTime - the local wallclock date/time at which the job commenced execution  LocalStopTime - the local wallclock date/time at which the job ceased execution  UTCStartTime - the wallclock date/time at which the job commenced execution in UTC (GMT) **  UTCStopTime - the wallclock date/time at which the job ceased execution in UTC **  ElapsedTime - StopTime-StartTime (calculated seconds)  BaseCpuTime - the actual CPU time recorded by the system on which the job ran (seconds)  BaseCpuPower - the power of the system on which the job ran in SpecInts (needs more precise definition)  SpecIntSecs - the computing power consumed by the job in SpecInt-Seconds (calculated from previous two fields) **  DiskSpace - the maximum aggregated amount of local disk space used by the job (Mb)  DiskIO - the aggregated volume of disk data read/written by the job (Mb)

Gathering  PBS doesn’t produce job accounts, only logs events  Four solutions  RAL processes event logs and populkates db  NIKHEF does similar  pbsacct in SourceForge  Job info available in pbs prolog. Do it direct.  Still considering  Assume unix group = vo name  Several solutions to mapping unix username to DN  Not yet chosen  Initial version probably VO only

 LSF will be easier as it has job accounts  Site ultimately responsible.  They can filter out certain info (eg non-LHC VOs)  Or remap groups to Vos  Or replace the whole thing by calls to their own internal database.

Sending  Send schema info over grid to central database  Considered  Spitfire  Develop our own web service  R-GMA R-GMA Archiver is a web-service which takes a row of information and inserts it in a persistent database  Chose R-GMA  Stream producer and general archiver  Retain records in local Mon to provide resilience against network failure.  Sites could also run archivers to keep a copy of their data local.

Querying  Simple SQL query to start  Canned queries with plotting  General queries later

Progress  Evaluated different components separately  Now integrating  Our Schema  PBS prolog  Calling R-GMA stream producer  With general archive consuming  Prototype by 23 rd.