1 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTER SCIENCE DEPARTMENT OF COMPUTING SCIENCE 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers TS10 – Service Applications Thomas Sandholm Olle Mulmo Peter Gardfjäll Erik Elmroth Lennart Johnsson

2 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Key Question ? How do we share national Grid compute resources in a fair, secure, open, and scalable way

3 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

4 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Fairness vs. Utilization Trade-off: Fair resource distribution and optimal resource utilization Soft real-time quota enforcement User preferences Resource policies Allocation authority policies Fair Distribution Maximum Utilization Secure Operation Scalable Efficiency 

5 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Security vs. Scalability Integrity & Privacy Single Sign-on/Impersonation DoS/Replay Attack prevention Privilege Delegation Message Level vs. Transport Level Policy Driven Authorization: PDP, PAP, PIP, PEP Scale: National Grid No single point of failure but coordinated allocation enforcement Fair Distribution Maximum Utilization Secure Operation Scalable Efficiency 

6 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Openness & Interoperability Systems Integration Platform Scheduler/Workload Manager Agnostic Programming Language/Model Agnostic Portable (100% pure Java) XML Based Standards: XPath, XQuery, XSLT, GGF- UR, XML-Signature, XML-Encryption, XACML Web/Grid Services Standards: SOAP, WSDL, WS- Security, OGSA, GGF-UR, GSI, GSSAPI, OGSI/WSRF

7 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

8 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Open Grid Services Architecture Global Grid Forum Standardization Initiative Architecture extending SOA and WSA to dynamically share stateful resources across organizational boundaries (=realizing the Grid vision) “… defining, within a service-oriented architecture, a set of core capabilities and behaviors that address key concerns in Grid systems.” OGSAv1 Assumes state modeling according to OGSI/WSRF  Core Infrastructure offering Inspection, Discovery, Lifetime Management, Notifications, Fault Handling WS-Resource = stateful resource and associated Web service.  Provide context for message exchange Addresses Grid security requirements such as Delegation and Single Sign-On

9 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Role of Accounting in OGSA Accounting foundational service to: Job Execution  Make sure that only jobs with sufficient quota can be executed on the compute resource  Decide queue priority based on available funds and usage history QoS/SLA Management  Negotiate pricing based on resource usage  Optimizing Utilization  SLA Attainment/Policing Security  Auditing  Access Control  PEP/PIP

10 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

11 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 SweGrid SweGrid connects 600 compute nodes (Intel P4) across 6 Swedish HPC centers interconnected by 10Gbs GigaSunet network 400 HPC users at all centers (some overlapping) Inaugurated March 2004 ~50 currently active researchers Up to 10k jobs per month per site

12 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 SweGrid Continued Resource quotas allocated by Swedish National Allocations Committee (SNAC) after peer-review of promising research projects with high computational demands (c.f. NRAC) Initially homogeneous hardware but heterogeneous scheduling, security, and accounting environment (policies, tools, data, processes, etc) Wanted: Uniform resource quota use & allocation

13 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

14 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 SweGrid Accounting System (SGAS) Key Design Points 1.Decentralized accounting solution based on standard, open protocols in compliance with the proposed OGSA 2.3-party (user, resource, allocation authority) policy customization 3.Non-intrusive to local site accounting systems 4.All components governed by a scalable cross- organizational authorization framework

15 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 SGAS Component Overview PAT BankLUTS Resource Scheduler Resource Manager Broker User WSDL JARM Policy Administration Tool Logging and Usage Tracking Service Job Account Reservation Manager SubmitJob Reserve/Release PublishUR Query AddUser

16 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 SGAS Security Design PAT Bank Resource Scheduler Resource Manager Broker User JARM PEPPEP PDPPDP PAPPAP Site Policy Manager PIPPIP PDPPDP LUTS PIPPIP PAPPAP External Authorization Service PDPPDP Membership/Community Service PIPPIP Policy Administration Point Policy Decision Point Policy Information Point Policy Enforcement Point Credential Delegation WS-SecureConversation XML-Signature XML-Encryption PKI Kerberos

17 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

18 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Overdraft XACML Policy <Condition FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-less-than-or- equal"> <Apply FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-one-and-only"> <EnvironmentAttributeDesignator AttributeId= "sgas:overdraw:percent:requested" DataType= " <AttributeValue DataType= " 175

19 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Overdraft Fuzzy Logic Policy R1: overdraft is low  allocation left is much  allow reservation R2: overdraft is high  allocation left is little  disallow reservation R3: allocation proximity is soon  overdraft is high  allocation left is much  allow reservation R4: allocation proximity is soon  overdraft is low  allocation left is little  allow reservation

20 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Super Computing 2004 Demonstration

21 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

22 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Conclusions Document centric communication in conjunction with semi-structured native XML databases is a very flexible combination Batch charging and eager prepare reservation needed for scalability Timestamp based allocations distributed in a staggered monthly flow result in the best trade-off between fairness and utilization Generic PEP/PDP/PIP/PAP model useful for encapsulating and evolving authorization code OGSI/WSRF state management ideal for controlling fine grained service state such as account quotas, reservations and policies in a standard way

23 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Future Work With large-scale flexibility and configurability comes complexity and it becomes hard to optimize high-level goals and to realize detailed user QoS requirements – development of an SLA Management framework and user/resource goal driven optimizing agents (WS-Agreement, ContractNet) Initial focus has been on scientific community resource sharing - support economic brokering and for-profit banks Multi jobs may overload the bank - SAML assertions (c.f. cheques) as a multi-allocation payment and reservation method

24 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Learn more…

25 2 nd International Conference on Service Oriented Computing, New York, NY, November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A