1. DGAS A.Guarise April 19th, Athens www.eu-egee.org
EGEE is a project funded by the European Union under contract INFSO-RI

2. DataGrid Accounting System
The Data Grid Accounting System was originally developed within the EU Datagrid Project and is now being maintained and re engineered within the EU EGEE Project.
The Purpose of DGAS is to implement Resource Usage Metering, Accounting and Billing in a fully distributed Grid environment. It is conceived to be distributed, secure and extensible.
The system is designed in order for Usage Metering, Accounting and Billing to be indipendent layers.

3. Usage Metering and Accounting
The usage of Grid Resources by Grid Users is registered in appropriate servers, known as HLRs (Home Location Registers) where both users and resources are registered.
In order to achieve scalability, there can be many independent HLRs. At least one HLR per VO are foreseen, although a finer granularity is possible.
Each HLR keeps the records of every grid job executed by each of its registered users or resources, thus being able to furnish usage information with many granularity levels:
Per user or resource, per group of users or resources, per VO.

4. Economic Accounting
The DGAS software is concieved mainly as an Economic Accounting system. Within this approach, grid users and grid resources are treated as consumers and producers of goods like computational power or storage capacity. The consumers can use these goods paying an apropriate amount of Grid Credits to the producers.
The idea behind this approach is that an Economic Accounting system may have a major advantage over a plain, bookkeeping style accounting system: it deploys a virtual economic market over the grid so that the natural behviour of this market can be used to drive the workload management system in balancing the resource load.

5. Example of economic accounting
Check Economic
Authorization
VO 1
VO 2
WMS
HLR 1
HLR 2
The user submits a job to the WMS (edg-job-submit)
The WMS selects the CE (matchmaking) and contact the User HLR to check if the user has a valid account and enough credits to pay the job or if he hasn’t exceed his usage quotas.
If this authorization is passed the job is submitted to the CE.
Once the job is finished, the Usage Records collected by the metering system are sent to the User HLR where the records are stored and, if economic accounting is required by the resource, the job cost is computed.
Then the User HLR contacts the Resource HLR and instantiates a transaction. The user account is debited, the resource account is credited.
In any moment the HLR administrator of VO2 can redistribute the resource earnings among the VO users. CE CE

6. HLR: Available Information
For each job executed on a CE, the following information are available querying the HLR:
GridJobID
User Certificate Subject
Local user Id
Grid Unique Computing Element Id
User VO
Cpu Time
Wall clock time
Physical memory
Virtual Memory
Job assigned nodes (cpus)
In principle all information available to the LRMS can be published in the HLR server.

7. HLR: Standard queries
An administrator that has local access to the HLR server can retrieve many information about the status of the accounts:
For example:
edg-wl-dgas-hlr-queryUser –Ua
Retrieves the information about the account of all the registered users.
edg-wl-dgas-hlr-queryUser –UaT
Retrieves also all the transactions performed for each user.
edg-wl-dgas-hlr-queryUser –U –e guariseATto.infn.it
|guarise|andrea.guariseATto.infn.it|Andrea Guarise|/C=IT/O=INFN…|torino|infngrid|10000|0|0|wall_t ime=0,cpu_time=0,job_number=0|
Retrieves only information about the user with the given address
Resource Queries
Similar queries are available for resources accounts via the command: edg-wl-dgas-hlr-queryResource

8. HLR: Advanced queries
It is also possible to perform more complex queries to get, for example, statistics of the usage of a set of resources, or to have a snapshot of the amount of work performed by users dividing them by groups or VOs.
Statistics of all the user jobs accounted by an HLR:
edg-wl-dgas-hlrAdvancedQuery –Q userAggregate
Jobs| totCost| totCpuTime|totWallTime|totMem(MB)|totVMem(MB)|
15| | | | | |
Statistics of all the user jobs accounted for the group ‘torino’:
edg-wl-dgas-hlrAdvancedQuery –Q userAggregate –g torino
5| | | | | |
Aggregates are available for resources, groups, VOs. It is also possible to have job lists and specify a time interval for the queries.

9. Metering Infrastructure: GIANDUIA
DGAS furnishes the APIs that can be used by each site to develop its own metering infrastructure. So that a site can freely decide how to measure its own Usage Records.
Two APIs are available. These both provide a C++ library and a CLI tool. Their purpose is to push the information to the User HLR once the Usage Record for a job is available.
These two API only differs in terms of the schema adopted to describe the Usage Records: The first implements a simple, proprietary, schema. The second implements the GGF URWG schema, more powerful, but rather complex to manage.
A minimum set of information need to be available in the usage record to make grid accounting possible:
The User HLR Location (comes with the job JDL)
A valid User Proxy (Needed to secure the communication)
The CE ID (Needed to unambiguously assign the Usage)
The Grid Job Id (For grid accounting the LRMS job id is not enough)

10. Metering Infrastructure: GIANDUIA
GIANDUIA workflow
(Gianduia Is A Nice Distributed Usage-metering Infrastructure for Accounting) WN
HLR_LOCATION
USER_PROXY
CE_ID
LRMS_JOB_ID
GRID_JOB_ID
jobWrapper CE
LRMS
LOG
gianduia
conf
User HLR
(HLR_LOCATION)
GIANDUIA
gianduiottiBox

11. References
Further information and documentation about DGAS can be found at: