1. DGAS Accounting – toward national grid infrastructures HPDC workshop on Monitoring, Logging and Accounting, (MLA) in production Grids 10/06/2009, Munich Andrea Guarise 1

2. Introduction The switch from the current EGEE–centric toward the EGI/NGI grid infrastructure poses some new requirements to a grid accounting system, such as: – National grids needs to deploy their own accounting infrastructures, which in turn should be able to inter-communicate. – Business partners willing to join the grid have strict requirements concerning accounting information. – The future EGI business model requires economic accounting. Then there are open issues which are not directly related to the switch to NGI infrastructures, but that have to be addressed: – How to treat accounting information for non-grid jobs. – VO-based accounting. – Storage accounting. – Normalization of CPU related metrics in heterogeneous farms. – Standardization. 7/2/20162

3. In this talk we illustrate the Distributed Grid Accounting System, developed within INFNGrid during the European DataGrid and the EGEE (I-II-III) projects. – First we will briefly illustrate DGAS key functionalities, highlighting those that makes it ready for National Grid Infrastructures. – Then we will focus on the open issues and on how we plan to address them. 7/2/20163 Introduction

4. DGAS - Key functionalities Sensors available for PBS,LSF and Condor batch systems Works with LCG and CREAM CEs Provides an highly customizable storage accounting toolkit Local Site manager has full access to the Usage Records (up to single job record), archived in an RDBMS Optional pricing of computing resources. Implementing economic accounting Distributed and scalable architecture, ready for regional grids and VO accounting, allowing for flexible deployment Authorization policies highly customizable, with both static or VOMS based roles CLI and WEB interfaces are available Capability to optionally account also local (non-grid) resource usage 7/2/20164

5. Basic site deployment Site Batch farm (LSF,PBS,Condor) Batch farm (LSF,PBS,Condor) CE (LCG,CREAM) HLR node Site HLR (DGAS server) Site HLR (DGAS server) DGAS sensors Site Manager 7/2/20165 Storage Element HLR: Home Location Register – the DGAS service that stores the Usage Records. CE: Computing Element – The GRID front-end to a batch system. Two implementation are available the older LCG, LHC computing GRID, and the newer one known as CREAM.

6. EGEE- Grid INFNGrid Regional Grid Deployment Regional grid Site 2 Site HLR Site 1 Site HLR HLR 2 Regional grid manager GOC 7/2/20166 Site 3 Site HLR Site 2 Site HLR

7. DGAS in INFNGrid DGAS sensors in nearly 50 sites. 12 Site HLRs deployed on T1,T2 and other major sites. 2 Multi-site HLRs for minor sites not willing to manage dedicated servers. 1 Second Level HLR collecting all the records and forwarding aggregates to GOC. Roughly 17 Million Records per Year. Information-consistency has been checked several times in the past years, by cross-checking with LRMS logs and third-party accounting tools, with highly satisfying results. GOC

8. Grid Regional grid 2 Distributed architecture ready for regional grids and VO accounting Regional grid 1 Site 2 Site HLR Grid Manager Site 1 Site HLR Site 3 Site HLR HLR 2 Regional grid manager HLR 2 VO 1 VO 2 HLR 2 VO1 Records VO2 Records ALL VOs Records 7/2/20168

9. Flexibility Sites VO VO HLR 2 (VO specific records) VO HLR 2 (VO specific records) HLR 2 Record for all VOs HLR 2 Record for all VOs VO Manager VO managers can perform queries on both general purpose or VO specific HLRs. Authorization levels are highly customizable. Static or VOMS based mappings are available. Little VOs do not need to have dedicated accounting servers. 7/2/20169

10. Grid Regional grid 2 Possibility to deploy accounting servers in order to ensure performance, redundancy and high availability Regional grid 1 Site 2 Site HLR Grid Manager Site 1 Site HLR HLR 2 Regional grid manager HLR 2 VO 1 HLR 2 VO HLR 2 Site 3 Site HLR 7/2/201610

11. Pricing DGAS allows to optionally deploy pricing servers, known as Price Authorities, which can be used to assign prices to the grid Computing Resources. Prices can be statically assigned by the resource managers or dynamically (price varies according to the CE load for example). A ‘virtual cost’ can thus be assigned to jobs executed on Computing Elements where economic accounting is activated. This cost will then be part of the Job’s Usage Record. The system allows for flexible implementation of different economic models, such as, as an example: – Pay per access: a fixed cost is assigned to each job, without taking care of the amount of resource consumed. – Pay per usage: the job cost is computed taking care of the amount of resources (e.g. wall clock time ) consumed by the job. DGAS does not provide any automatic charging mechanism. It is up to the site to eventually bill (if this is the case) the user. It just provides the needed information to the site managers. 7/2/201611

12. Pricing workflow Site CE (LCG,CREAM) HLR node Site HLR (DGAS server) Site HLR (DGAS server) PA DGAS sensors Site Manager 1. Plain Usage Record to HLR server. 2. HLR asks the valid CE price to the PA. 3. The HLR computes the job Cost using the algorithm provided by the site manager. 4. The Usage Record containing the assigned job cost is inserted in the HLR database. HLR database 7/2/201612

13. Storage accounting DGAS provides a storage accounting functionality which enables the site to keep track of VO space consumption on their storage elements. Due to the highly heterogeneous environment in which Storage Accounting has to be operated, DGAS just provides a toolkit that sites can leverage to implement their accounting sensors. The whole storage accounting toolkit has been designed to be as simple as possible and highly customizable. Sensors, provided by Site Managers, have been produced for dCache, StoRM, DPM and Castor. 7/2/201613

14. Storage accounting The workflow is really simple. A command line tool can be used to send a record to the HLR server, which stores it in its database. The contents of the database can then be queried via the DGAS query interfaces ensuring proper authentication and authorization. Although a pre-defined schema is provided, it is always possible for the site managers to define their own tables in the database and populate them with new metrics thus allowing a complete customization of the system. 7/2/201614

15. Storage accounting workflow HLR-listener RDBMS Ping Query getRecor d dgas-sendRecord DGAS Monitoring DGAS Monitoring UR- forward Hlr::voStorageRecords Hlr::sysDef* Hlr::voStorageRecords Hlr::sysDef* HAD Site HLR node Storage Element Storage Element Storage Usage Sensor

16. Open technical issues There are a set of technical issues which are currently being analyzed about DGAS accounting: Punctual normalization with WN-based benchmarks. Transport Protocol. Standardization. 7/2/201616

17. Punctual Normalization The requirement is to store the punctual value of Worker Node benchmark metrics used to normalize CPU usage instead of the average value of the entire farm. Server side the HLR is ready to support this requirement. The capability to manage user defined tables and metrics can be used to implement WN-based normalization. An open issue is how to organize and implement the database that is needed to store the values of the benchmarks. – Should it be within the information system? – Should it be a relational database somewhere? – Should it treat single WNs or class of them? – Is there the need to keep historical information on the values? 7/2/201617

18. WN-based normalization WN database – a tentative approach Site CE (LCG,CREAM) HLR node DGAS sensors Site Manager 1. Plain Usage Record to HLR server. 2Sys manager fills the WN database with up-to-date benchmark metrics 3. Joning the info on the Usage Records and WN databases results in the per- WN normalization requested. 7/2/201618 HLR database WN database Usage records DB WN Normalized records

19. Usage Records transport system Re quirement: GOC plans to implement a different data transport system, APACHE activeMQ. This could be a first step toward integration of different accounting systems, provided the fact that standard protocols and UR schema are needed as well. Issues: – Scalability and performances need to be investigated. – In DGAS we actually use our legacy transport system among our sites, which consists in a GSI-secured channel over TCP. We do not need to replace this. – Record aggregates are being sent to GOC via direct SQL insertion in their DB, this is where we plan to use activeMQ instead. – New transport mechanisms to forward Usage Records to GOC will be investigated as soon as they are available.

20. Standardization The adoption of a common agreed transport mechanism does not necessarily insure interoperability. Standards concerning Usage Records and interfaces are needed as well. Two working groups within OGF have been working on this, UR- WG and RUS-WG. It must however be highlighted the fact that those standards have yet to prove to be really usable in real world practice, and be usable also when local requirements have to be addressed.

21. Conclusion The switch from the current EGEE–centric toward the EGI/NGI grid infrastructure poses some new requirements to a grid accounting system. DGAS is a robust and mature, self contained GRID accounting system which already satisfies most of such new requirements. It is ready for regional grids and VO based accounting. New requirements are being addressed and open issues being investigated.