Accounting for the Grid Usage Records and a Resource Usage Service

Acknowledgements... Work presented is the output from two Global Grid Forum Working Groups –Usage Record Working Group (UR-WG) –Resource Usage Service Working Group (RUS-WG) I was involved mainly in the RUS-WG Work was funded through UK e-Science Markets for Computational Services (MCS) Project The recent implementation of the RUS, and much of the material presented today is from John Ainsworth, University of Manchester.

Accounting on the Grid? Q. Why is it different from HPC Center accounting? A.Like accounting for a HPC Center, we need to track usage on more than one machine, but: –users have single sign-on – need to work with X509 Distinguished Names... –...so usernames may differ –Also, some machines are at (and run by) different organizations

How do we do this? (1) We know that different batch systems produce different accounting records –As many formats as batch systems (similar content) –But aggregating these directly is hard Also, need to cope with single sign-on (X509) So first, we create a standard accounting record representation (Usage Record) Defined by the GGF UR-WG This is defined as an XML Schema. The spec. and XML Schema are at: – The work of this group is nearly completed Specification is now stable

Example Usage Record green wwmarko CN=john ainsworth, L=MC, OU=Manchester, O=eScience, C=UK

...continued! completed PT1800S T06:47:30 1 cs5015 green PT0.0S PT1S T06:48: T06:48:34 green wren normal contract1234 escience

How do we do this? (2) Next, we need somewhere to store the records Something that we can push records into, and pull them back out of So first, we now define a standard Web Service interface (Resource Usage Service) Defined by the GGF RUS-WG Service interface is based on “plain” Web Services, i.e. it is compliant with the WS-I Basic Profile 1.0 This is defined as WSDL with XML Schema. The spec. is being updated prior to going to the GGF Editor – The work of this group is nearly completed Specification is now stable

How do I work this? Specs are all very well, but what about running it? There is an implementation of the RUS Also a record spooler for uploading records Built at ESNW in Manchester Will be maintained by LeSC in London Will receive continued support through the UK’s Open Middleware Infrastructure Institute (OMII) Current version is downloadable: –

How do I generate records? This is trickiest part... To some extent, this is scheduler specific Platform LSF can generate UR format directly For OpenPBS/PBSPro, you can use SourceForge’s PBSAccounting – Complex part is getting the X509 Distinguished Name into the record (for Grid jobs) Need to tweak Globus jobmanagers

Implementation Info

WS-I reformulation of OGSI specification Based on draft-ggf-rus-service-4-public Service Interface is largely the same Service data elements have been removed Incorporated our experience of implementing a RUS –Removed ambiguities –Identified items that are implementation choices –Completed specification where necessary

Service Interface (1) Write Operations –insertUsageRecords(UsageRecord[]), replaceUsageRecords(RecordAndId[]) –deleteRecords(XpathQuery), deleteSpecificRecord(RecordId[]) –modifyUsageRecordPart, updateUsageRecordPart (not implemented) Read Operations –extractRecords(XpathQuery), extractSpecificRecords(RecordId[]) –extractUsageByGlobalUserId, extractUsageByMachineName, extractUsgaeBySubmitHost,

Service Interface (2) Management –retrieveConfiuration –updateConfiguration Faults –RUSProcessingFault –RUSUserNotAuthorised –RUSInputFault

Security Model Role based security –Specified through access control file (XML) (Cached) –Administrator Unrestricted read/write authorization –ResoureManager Restricted read/write authorization Requires a ResourceDescription to specify the resources for which the RM has permission ResourceTypes are urwg:MachineName, urwg:SubmitHost, urwg:ProjectName and Domain Authorization for a record determined by Logical AND between different ResourceTypes, logical OR within values of same ResourceType –All other users denied both read and write access

Configuration  Mandatory Record Elements  A record must contain these elements for it to be valid for this RUS  Resolves “everything is optional” problem inherent in Usage Record specification

RUSUsageRecord Internal wrapper around UsageRecord Adds elements –RUSId –RecordHistory Audit trail of record insertion and modification Records who and when in StoredBy and ModifiedBy elements

Architecture

InsertUsageRecords Check user authorization for record Validate record against schema Check mandatory elements are present Check the record is not a duplicate Insert into database

Implementation notes Started with WS-Security, but moved to TLS –More widely available Extended set of error codes –Added InvalidRecord and DuplicateRecord (used in response for insert and replace) Database stores each record as a document –Xindice single document size limitation Developed web-based query client Developed a Perl usage record spooler

Test Machine Specification

Test Server 1

Test Server 1 (Restricted Data Set)

Test Server 2

Test Server 2 (Restricted Data Set)

Final Comments This is mature work, which is being deployed in the UK, at Manchester, and other sites The work is based on emerging standards from the Grid community The implementation has a future, including development and support Any questions?