LCG User Level Accounting John Gordon CCLRC-RAL LCG Grid Deployment Board October 2006

LCG John Gordon – GDB 04 October 2006 History  Dave Kelsey presented a draft outline on a User-Level Accounting Data Policy at GDB in Rome, April 2006  I presented subsequent policy proposals in June  Asked for VO feedback

LCG John Gordon – GDB 04 October 2006 Reminder  Dave proposed that the following issues needed to be covered in such a policy:-  User Consent  In the Grid AUP the user consents to accounts being stored.  What is stored and where?  log files and accounting records derived from them  user identification (DN, FQAN)  Access rights - Who can read?  authorised people including the user  Access for what purposes?  accounting only, keep confidential

LCG John Gordon – GDB 04 October 2006 Five Actors  VO Resource Manager  VO Member  User  GOC developers  Site Admin

LCG John Gordon – GDB 04 October 2006 VO Resource Manager  The VO Resource Manager is a new role we are proposing. They are the only person who has access to all data in the accounting database belonging to her VO including the DN.  Not the person who authorises membership of the VO  It is the responsibility of the VO Resource Manager to use the DN related information appropriately and make sure this information does not proliferate beyond the circles where it is needed.  Should see:  aggregated user data F(site, time)  No of jobs, cputime  Individual job details (for a period of time)

LCG John Gordon – GDB 04 October 2006 VO Member  The general member of the VO  Should see:  Aggregated CPU usage per Group/Role  No information belonging to other identifiable user  Anonymised aggregates  Eg cpu use by top ten users

LCG John Gordon – GDB 04 October 2006 User  The person who ‘owned’ the jobs  Should see:  aggregated information about their jobs.  Job level records (for a period of time).

LCG John Gordon – GDB 04 October 2006 GOC developers  Have necessary access to data as part of their job  Should see:  Everything  But governed by similar acceptable use policy as VO Resource Manager

LCG John Gordon – GDB 04 October 2006 Site Admin  At each site  Should see:  nothing  They do not need access to grid accounting to do their job (an assertion)  the admin staff have access to their local batch records under their existing legal and ethical guidelines.  Can trace jobs to investigate problems

LCG John Gordon – GDB 04 October 2006 Is Everybody Happy?

LCG John Gordon – GDB 04 October 2006 Implementation progress  APEL update in gLite RC1  Main features:  Support for gLiteCE using a BLAH accounting log parser  1024 RSA encryption  Captures GlobalJobId and UserFQAN from gLiteCE  Support for publishing data via DGAS HLR  Improved reliability of publisher for large datasets

LCG John Gordon – GDB 04 October 2006 DN information  The encryption module is in the updated publisher  Anyone using the publisher can send encrypted data to the GOC. This includes data captured by the DGAS HLR.  This publisher is not yet widely distributed across EGEE (only a few test sites, RHUL, QMUL, GRIF).  Rosario Piro has successfully implemented the publisher and published test records via the DGAS HLR (DGAS2APEL)  The publisher has two publish options: either "don't publish userDn", or "publish userDN".  Its not possible to publish an unencrypted UserDn. The default is "don't publish userDn".  Any site using this version of the publisher can republish ALL their data to the GOC and send ALL the UserDN data.  Once the data are published into RGMA, it must be processed. Processing is "extract from RGMA, decrypt, write to a private database, build summary by User"  This work is mostly completed.  Visualisation of user data will be authorised through GridSite by DN and/or FQAN

LCG John Gordon – GDB 04 October 2006 Next Steps  Once gLite released  Get a number of sites to install and publish DN information  Tier1s? GridPP? INFN?  Accounting on role/group requires use of VOMS proxies (encourage)  Finish display of user level information  Perhaps just simple database query to start with  Have a policy document that sites and people can sign (or accept)

LCG John Gordon – GDB 04 October 2006 Issues  Storing job level information centrally may not scale forever.  Push back to sites and only store summaries centrally  DGAS already does this.  Does Pilot jobs and GLEXEC break user-level accounting?

LCG John Gordon – GDB 04 October 2006 Other Accounting Issues/Status  Both APEL and DGAS should work in gLiteCE  Storage  New GIPs for DPM and Castor are in beta test  Alternative dCache GIP being tested in UK  Added SRMtype to schema  Worldwide  Working with Condor to improve accounting information from history file  OGF RUS to receive summary information under test  Preferred solution for receiving information from other grids