1 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Enforcing resource allocations with the SweGrid Accounting System (SGAS) European Grid Conference (EGC), Amsterdam February 15, 2005 Peter Gardfjäll Umeå University Joint effort with Erik Elmroth (Umeå University) Lennart Johnsson (KTH) Olle Mulmo (KTH) Thomas Sandholm (KTH)

2 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Outline Background  Grid accounting  SGAS in SweGrid SGAS  Architecture  Components SGAS demo

3 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Grid accounting - tracking Grid resource usage Maintaining a (consistent) Grid-wide view of the resources utilized by VO members Measure and control users’ total resource usage on the Grid  Assuming absence of central point of control  Resource owners should retain local control

4 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Why accounting? Accounting information can be used for several purposes  Economic compensation  Tracking of resource usage  Evaluation/forecasting of resource usage  Resource brokering decisions  Assign scheduling priorities to jobs based on previous resource utilization  Pricing & creating economic markets for resource sharing  Enforcement of resource allocations  Etc…

5 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE SGAS in SweGrid SweGrid is a Swedish computational Grid  Connects six computer clusters (Umeå, Göteborg, Uppsala, Stockholm, Lund, Linköping) with a total of 600 processors Swedish National Allocation Committee  Allocates CPU time (measured in node hours) on SweGrid to research projects  Grid-wide allocations can be spent arbitrarily among Grid sites SGAS has been developed to  Enforce project allocations across all SweGrid sites Prevent project members from overspending  Store detailed information on each Grid job’s resource usage

6 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE SweGrid Accounting System (SGAS) Decentralized resource allocation enforcement system SGAS performs soft real-time enforcement of allocations  Real-time enforcement: Resources can, at the time of job submission, deny access if project quota has been used up  Soft: enforcement is subject to local resource policies (strict enforcement not always appropriate) Primarily targeted towards allocation enforcement in SweGrid  Not restricted to SweGrid use Developed with an emphasis on easy integration into different Grid environments  In SweGrid: deployed on top of NorduGrid middleware

7 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE SGAS (cont.) Service-oriented architecture  main components exposed as Web Services Java implementation  Based on Web and Grid services technology  Globus Toolkit 3 (GT3) primitives Based on open Grid standards (OGSA, GGF-UR) Transparent to (most) end-users  Single account  fully transparent Single-point-of integration Flexible and customizable  No assumptions about the types of resources accounted for  Abstract “currency” – Grid credits  Charge for arbitrary resource usage  Resources transform usage into Grid credits before charging account  System can be configured by customizing policies on three different levels User: “only run jobs if sufficient quota is available” Resource owner: “run quota-exceeding jobs with low priority” Allocation authority: “allow 10 % account overdraft”

8 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE SGAS component overview Four main components  Bank Online service Manages project accounts (resource allocations) Provides Grid users/resources with consistent information about resources consumed by Grid projects  JARM (Job Account Reservation Manager) Intercepts job requests on resources Makes account reservation prior to job execution Charges project account after job completion Single-point-of-integration  LUTS (Logging and Usage Tracking Service) Collects and publishes usage records which can be queried by users  PAT (Policy Administration Tool) Client tool to manage Bank and LUTS policies

9 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Component interactions 1.Contact resource 2.Authenticate/authorize (delegate credentials) 3.Submit job request 4.JARM intercepts request 5.Make account reservation 6.Run job 7.Collect usage info 8.Charge project account and log usage info

10 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Bank component Composed of three OGSA-compliant services  Bank Service Creates and locates Account services  Account Service Represents a project’s resource allocation Authorized users make soft-state reservations on account allocation. If granted it results in a...  Hold Service Time-limited reservation on the account Overdraft policy can be associated with each account Each account manages a set of time- stamped allocations  Each allocation valid for a limited time period  Allows total allocation to be spread out in time  Implements a "use-it-or-lose-it" policy … Bank Account … Hold >

11 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Allocation strategy example Picture from:

12 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Logging and Usage Tracking Service (LUTS) Collects and publishes usage records compliant with GGF- UR specification  XML-based format for storing detailed information about the resources consumed by Grid jobs CPU time, memory, storage, network, … Authorized users are allowed to run XPath queries directly against LUTS URs can be extended to hold additional information only understood by a subset of users/resources without modifying LUTS URs can be logged in batches  Improved performance and scalability XSLT-based transformation infrastructure to allow sites to easily convert their non-XML usage data to a UR- compliant format

13 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Job Account Reserv. Manager (JARM) Integration-point between SGAS and underlying Grid env.  Decoupled from workload manager NorduGrid integration  configuration of plug-in scripts called from NG job submission state machine Plugged into workload manager at each cluster  Makes account reservations prior to job execution Done in parallel with job preparation (to incur less overhead)  Collects usage data from batch system when job has finished  Converts usage into Grid credits, charges account and logs a usage record in LUTS Charging & logging of jobs usually deferred and performed in batches Local site policies can be enforced by overloading the default Site Policy Manager  Default Site Policy Manager let job through even if bank cannot be reached; log and charge later overdraft violation detected: run job with lower priority

14 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE SGAS demo Self-contained demo  Runs out of the box Can be downloaded from the SGAS website  Download and try it out! Runs on Windows and Unix/Linux  Requires Java (JDK/JRE 1.4.2) Runs against actual Bank and LUTS services  Service container runs embedded in demo A sample run

15 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Project milestones & future directions Sep 2003: SGAS project initiated Sep 2003: SweGrid site survey Oct 2003: SGAS white paper  Investigated existing work on Grid accounting  Presented an accounting sys architecture proposal for SGAS Jan 2004: Finished proof-of-concept prototype Feb 2004: Started working on production code base Apr 2004: Version alpha 0.1 was released Nov 2004: Version beta 0.2 was released very soon: Version 1.0 release  Improved stability/scalability  Simplified installation process  Improved administration client Spring/summer 2005: Planned GT4/WSRF transition  GT4 scheduled for release in April 2005

16 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Authorization framework Fine-grained authorization framework (contributed to Globus)  Authorization specified on a per-operation basis Associate authorization policy and engine with service  Authz policy managed through ServiceAuthzManagement porttype Implemented by several SGAS services (Bank, Account, LUTS)  Service orthogonal: transparent to service implementation  Customizable: allows different backend engines/policy languages Current authorization engine based on XACML  Not SGAS-specific

17 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTING SCIENCE DEPARTMENT OF COMPUTING SCIENCE Policy enforcement overview PAP = Policy Administration Point - set up policies PIP = Policy Information Point - retrieve policies PDP = Policy Decision Point - make policy decisions/manage policy PEP = Policy Enforcement Point - intercept request and query PDP(s)