1. John Gordon EMI TF and EGI CF March 2012 Accounting Workshop

2. EMI INFSO-RI-261611 Storage Compute SSM and existing cpu Outline

3. EMI INFSO-RI-261611 dCache StoRM DPM Castor BestMan Gratia Gstat Cloud Storage (Unicore) (ARC) The middleware in scope for EGI and WLCG Storage Solutions

4. EMI INFSO-RI-261611 To produce a design All have achieved this Now need to correlate that the designs are compatible in the fields published and the semantics used. And the timescales for implementation and testing. EMI Milestone

5. EMI INFSO-RI-261611 WLCG Use Case https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChallenges/ WLCG_GlueSchemaUsage-1.8.pdf https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChallenges/ WLCG_GlueSchemaUsage-1.8.pdf Required fields/values StorageSystem = Storage Element FQDN StorageShare = StorageArea – Needs checking for consistency SE, SA, Storage Token The SubjectIdentity property MUST be present in the record – It MUST contain Group = VO name – It MAY contain: UserIdentity = X.509 UserDN of the owner of the data. GroupAttribute = VOMS FQAN EGI Profile for StAR

6. EMI INFSO-RI-261611 Allocated Space – equivalent to wallclock time for cpu. Shows resource blocked by VO. – 14728 50000 – Discussed and agreed in OGF since document published SITE – EGI and NGI deal with SITEs. Defined in GOCDB and used in many other tools inc BDII for resource discovery. – I thought this had been included after previous discussions. Surprised to see not. – Very Important to EGI StAR Issues

7. EMI INFSO-RI-261611 APEL Test service in place. A days work to add a new AMQ topic and database schema and tweak the record loader to the storage record format and schema APEL currently uses its own data format Fieldname: value Where the fieldnames match the UR fields Intention to accept XML document but not yet in place. APEL Specifics

8. EMI INFSO-RI-261611 CAR – latest APEL job schema has been changed to align fields with CAR names. Issues with semantics. APEL currently uses epoch timestamps, and seconds durations CAR has adopted ISO standard date and time forms. CREAM – doesn’t publish directly. Fieldname changes will be adopted in next emi-apel-client release. (not EMI-2.0) ArcCE - ???? Unicore – already adopted CAR so may have semantic issues mentioned above. CPU

9. EMI INFSO-RI-261611 Parallel Jobs MPI users pushing for consistent accounting of parallel jobs This is relevant to whole-node use and how one treats cpu/wallclock. Record nodecount and ncpus(cores) for batch jobs In new APEL schema for SSM publishers – Need new apel-client for others. Don’t understand how to present the data. – Portal deals with summaries. – What is a parallel job? UR doesn’t record this. – On many sites, parallel jobs will get swamped in all the serial jobs so average speedup and nodes/job etc will not be meaningful.

10. EMI INFSO-RI-261611 Summary Records – Gratia (OSG) – SGAS – EDGI – Gridsafe (Globus) JOB Records DGAS Unicore QCG (Mapper) CERN NL-T1 France The systems/infrastructures/sites testing SSM SSM Users

11. EMI INFSO-RI-261611 Summary Records SSM Users

12. EMI INFSO-RI-261611 Final Production testing today Issues raised by Tania this morning – Worldwide tree in portal – Verification – summary republish? Not instantaneous – SAM tests at site level. Gratia (OSG)

13. EMI INFSO-RI-261611 NorduGrid developer in Sweden left. Post unfilled Finland also run SGAS and have volunteered to implement SSM publishing Hope NorduGrid will adopt. SGAS

14. EMI INFSO-RI-261611 100k+ compute nodes Looking at publishing summaries EDGI

15. EMI INFSO-RI-261611 Unit tested Globus to Gridsafe and Gridsafe to APEL End to end tests next. Gridsafe (Globus)

16. EMI INFSO-RI-261611 DGAS Have written a requirements document Some issues already addressed.

17. EMI INFSO-RI-261611 Unicore Have released a version of CE with APEL publishing Test status unknown

18. EMI INFSO-RI-261611 QCG Compute MAPPER End of April

19. EMI INFSO-RI-261611 CERN Moving to production now Slides

20. EMI INFSO-RI-261611 NL-T1

21. EMI INFSO-RI-261611 France CC-IN2P3 publish summaries by direct database insert. Are still considering options of SSM, apel- parser/client or publishing via a central French site which

22. EMI INFSO-RI-261611 Next One by one move systems from test broker network to production. Quarantine records for comparisons Swing production publishing When all existing database inserters have migrated, move database to new schema. – Don’t need to wait for new publishers – Apel-client will continue to work – Introduce apel-cleint based on SSM.

23. EMI INFSO-RI-261611 Other accounting issues?

24. EMI is partially funded by the European Commission under Grant Agreement RI-261611