A pre-alpha accounting architecture The “Computational Economy” model applied to DataGrid accounting

The accounting problem in brief Services required from the accounting architecture: Optimization of queues load. Determining the priorities for accessing execution queues on the resources (allocation policies). Meeting of job requirements (cpu speed, storage space, deadlines for receiving the output back). Obviously this accounting architecture will have to cope with the heterogeneity of the resources and of the local accessing policies.

What is Computational Economy? In recent years the problem of accounting has been investigated by many theoretical groups giving birth to various theoretical models and algorithms. The most flexible and capable of meeting our needs seems to be an economical analogy model. In this picture resources (i.e. cpu time) are exchanged between the groups participating in the project using a kind of virtual money (that we'll call Grid Credits, GC). Each time a group allows the use of its resources gains some GC that can be spent to get access to other Grid resources when needed.

A theoretical model What distinguish among the various theoretical models (and the extent of the economical analogy) is the price-setting mechanism used to decide the price of resource access and the type of algorithm used to reach market equilibrium. This last goal is of prime importance to avoid the trigger of undesirable effects like ‘virtual’ inflation or ‘virtual’ finance speculation.

Price-setting policies Here's a list of possible price-setting policies we found in literature: 1. Commodity market (flat or demand & supply driven pricing) model 2. Posted-price model 3. Bargaining model 4. Tendering/contract net model 5. Auction Model 6. Bid-based proportional resource sharing model 7. Community/coalition/bartering model

Price-Oriented and Resource-Oriented Algorithms (1) As we noticed before, the possibility of bulding a satisfying model heavily relies on the certainty of avoiding the typical undesireable effects of real world markets. This goal is within reach if the system is constrained to reach and keep equilibrium (while real world markets usually expand). In literature this kind of model is usually called 'multy-commodity market' (where each Grid site is the single commodity). The search for general equilibrium in such a system has been performed with basically two families of algorithms:

Price-Oriented and Resource-Oriented Algorithms (2) 1. Price-Oriented Algorithms. This algorithms guarantee that the price of each commodity is equal for every group participating in the Grid, but some groups may be unable to access Grid resources. 2. Resource-Oriented Algorithms. This ones are based on the resource constraint which guarantees that allocation is always feasible, since resources are only transferred between groups participating in the Grid.

Basic Principles In our preliminary work we started from the following basic principles: 1. A Grid structure with FIFO, FCFS or other 'simple' priority schemes won't fit the needs. 2. No one would participate in the Grid if there are no advantages in doing it (the flop of experiences like Popular Power seems to demonstrate this assumption quite clearly).

Proposals for guidelines (1) 1. Establish a biunique correspondance between Grid Credits and the amount of resources shared (no one is allowed to coin GC). This forces the model into a much simpler pure exchange and resource oriented system. 2. The virtual account which manages each user's GC is located at her/his Home Location Registry (HLR, in analogy with the one used by mobile phones networks). 3. Manage GC exchange with debit card like mechanism. This guarantees no-one could easily forge GC and lets taking into account GC that are gained by the user while using 'foreign' Grid resources.

Proposals for guidelines (2) 4. At each Grid resource, the cost for a job submitted to a particular queue is expressed in some simple way (regardless of the complexity of the issue of calculating it) 5. Since we want to assign cost to resources in an homogenous way, each time a new resource enters the Grid a benchmarking utility is automatically run and ends sending (in a sealed way) the benchmarking results to some appropriate Information Service. To help the end user in estimating realistic computing costs this benchmarks can also include typical applications (maybe like the reconstruction of a HEP event?).

A possible accounting flow (1) A job request arrives to the queue manager that responds to the submitter giving: a) lower cost queue where the job should be executed within the time limit specified by the submitter; b) cost of execution assuming estimated time should be associated with a tolerance or overrun, such as “2 hrs + 1 hr tolerance”. User replies confirming or canceling reservation within, say, ten minutes. If no reply arrives after above time, job is definitely inserted in queue and data transfer requested to WP2, if necessary. The HLR is checked to control that account balance is sufficient to cover the computing cost (estimated + tolerance).

A possible accounting flow (3) The amount of GC corresponding to the maximum duration (estimated + tolerance) is 'frozen' before the job is finally submitted. When the job finishes the frozen amount is released and the real cost is deducted from GC balance. GC which come into computing facilities associated with a HLR will be distribuited according to local criteria, not part of the Grid middleware.

References R. Buyya, D. Abramson, J. Giddy - “A Case for Economy Grid Architecture for Service Oriented Grid Computing” http://www.csse.monash.edu.au/~rajkumar/papers/ecogrid.pdf F. Ygge / H. Akkermans “Duality in Multi-Commodity Market Computations” - http://www.soc.hk-r.se/research/1997/dmcmc.ps An example of a ‘Grid like’ environment with price setting policy based on a community/coalition/bartering model - http://www.mojonation.net The first ‘Grid flop’ we know of: http://www.popularpower.com