Accounting Repository Adrian Coveney OMB – November 2016

Contents Cloud Accounting Storage Accounting CPU Accounting

New Cloud Schema V0.4 of the Usage Record Contains extra fields like benchmark and clusters within a site Scripts to support this tested for oneacct and caso Clients ready to deploy in production Originally planned to deploy alongside long-running VM support but will now do it first separately

Long Running VM Support Current cloud accounting assigns all the usage of a VM to the month in which it started (https://accounting-next.egi.eu/cloud/sum_cpu/SITE/DATE/2015/11/2016/11/egi/onlyinfrajobs/) Have a method to process all the daily records sent by sites to produce totals for usage by a VM in each month during its lifetime Reprocessing of all the old cloud records takes a very long time and we are re-considering the feasibility of this method. It should be achievable but we need to improve the efficiency of processing. Current summarising processes the latest record for all VMs back to the start of time. Processing ALL records for ALL VMs may not be achievable.

Cloud Accounting Plan Currently working on a revised release that just includes the new v0.4 schema Long running VM support needs more work and with current effort available, this will not be possible before the end of the year

Storage Accounting APEL Storage Accounting is available for dcache and dpm This covers most sites We have a handful of sites successfully testing We believe the data published on LogicalCapacityUsed are correct Still checking that the portal correctly interprets them in all cases Still talking with storage system developers about allocated capacity.

Testing Storage Sites http://bit.ly/2gjBBnr

All Sites Plots

All Sites Plots

Site View LHC VOs Just like CPU, one can use the tree on the left to navigate among NGIs and Sites.

Site View All VOs (all supported, even if no data present)

Plan for Storage Accounting We believe the software is robust enough to encourage more sites to start publishing Still hosted on APEL test server and dev portal Both of these can be changed transparently (to sites) later May change the broker queue name to a production one first https://wiki.egi.eu/wiki/APEL/Storage

CPU Accounting Sites publishing large numbers of jobs put a load on our servers. Please do not publish more than the last couple of days data without negotiating with the APEL Team first i.e. if you fix an error that stopped you publishing or just want to try fixing a gap in the Sync tests We encourage big sites to use summary publishing Set option send_summaries = true in client.cfg

Summary Aim to release new cloud schema v0.4 before end of year Will work on long running VM support at start of new year Storage accounting is ready for more sites to publish Sites should liaise with APEL team if they are re-publishing a large amount of data. Moving to summaries would help.