1. 1 CENTER FOR PARALLEL COMPUTERS DEPARTMENT OF COMPUTER SCIENCE DEPARTMENT OF COMPUTING SCIENCE 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers TS10 – Service Applications Thomas Sandholm sandholm@pdc.kth.sesandholm@pdc.kth.se Olle Mulmo Peter Gardfjäll Erik Elmroth Lennart Johnsson

2. 2 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Key Question ? How do we share national Grid compute resources in a fair, secure, open, and scalable way

3. 3 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

4. 4 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Fairness vs. Utilization Trade-off: Fair resource distribution and optimal resource utilization Soft real-time quota enforcement User preferences Resource policies Allocation authority policies Fair Distribution Maximum Utilization Secure Operation Scalable Efficiency 

5. 5 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Security vs. Scalability Integrity & Privacy Single Sign-on/Impersonation DoS/Replay Attack prevention Privilege Delegation Message Level vs. Transport Level Policy Driven Authorization: PDP, PAP, PIP, PEP Scale: National Grid No single point of failure but coordinated allocation enforcement Fair Distribution Maximum Utilization Secure Operation Scalable Efficiency 

6. 6 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Openness & Interoperability Systems Integration Platform Scheduler/Workload Manager Agnostic Programming Language/Model Agnostic Portable (100% pure Java) XML Based Standards: XPath, XQuery, XSLT, GGF- UR, XML-Signature, XML-Encryption, XACML Web/Grid Services Standards: SOAP, WSDL, WS- Security, OGSA, GGF-UR, GSI, GSSAPI, OGSI/WSRF

7. 7 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

8. 8 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Open Grid Services Architecture Global Grid Forum Standardization Initiative Architecture extending SOA and WSA to dynamically share stateful resources across organizational boundaries (=realizing the Grid vision) “… defining, within a service-oriented architecture, a set of core capabilities and behaviors that address key concerns in Grid systems.” OGSAv1 Assumes state modeling according to OGSI/WSRF  Core Infrastructure offering Inspection, Discovery, Lifetime Management, Notifications, Fault Handling WS-Resource = stateful resource and associated Web service.  Provide context for message exchange Addresses Grid security requirements such as Delegation and Single Sign-On

9. 9 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Role of Accounting in OGSA Accounting foundational service to: Job Execution  Make sure that only jobs with sufficient quota can be executed on the compute resource  Decide queue priority based on available funds and usage history QoS/SLA Management  Negotiate pricing based on resource usage  Optimizing Utilization  SLA Attainment/Policing Security  Auditing  Access Control  PEP/PIP

10. 10 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

11. 11 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 SweGrid SweGrid connects 600 compute nodes (Intel P4) across 6 Swedish HPC centers interconnected by 10Gbs GigaSunet network 400 HPC users at all centers (some overlapping) Inaugurated March 2004 ~50 currently active researchers Up to 10k jobs per month per site

12. 12 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 SweGrid Continued Resource quotas allocated by Swedish National Allocations Committee (SNAC) after peer-review of promising research projects with high computational demands (c.f. NRAC) Initially homogeneous hardware but heterogeneous scheduling, security, and accounting environment (policies, tools, data, processes, etc) Wanted: Uniform resource quota use & allocation

13. 13 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

14. 14 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 SweGrid Accounting System (SGAS) Key Design Points 1.Decentralized accounting solution based on standard, open protocols in compliance with the proposed OGSA 2.3-party (user, resource, allocation authority) policy customization 3.Non-intrusive to local site accounting systems 4.All components governed by a scalable cross- organizational authorization framework

15. 15 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 SGAS Component Overview PAT BankLUTS Resource Scheduler Resource Manager Broker User WSDL JARM Policy Administration Tool Logging and Usage Tracking Service Job Account Reservation Manager SubmitJob Reserve/Release PublishUR Query AddUser

16. 16 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 SGAS Security Design PAT Bank Resource Scheduler Resource Manager Broker User JARM PEPPEP PDPPDP PAPPAP Site Policy Manager PIPPIP PDPPDP LUTS PIPPIP PAPPAP External Authorization Service PDPPDP Membership/Community Service PIPPIP Policy Administration Point Policy Decision Point Policy Information Point Policy Enforcement Point Credential Delegation WS-SecureConversation XML-Signature XML-Encryption PKI Kerberos

17. 17 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

18. 18 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Overdraft XACML Policy <Condition FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-less-than-or- equal"> <Apply FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-one-and-only"> <EnvironmentAttributeDesignator AttributeId= "sgas:overdraw:percent:requested" DataType= "http://www.w3.org/2001/XMLSchema#integer"/> <AttributeValue DataType= "http://www.w3.org/2001/XMLSchema#integer"> 175

19. 19 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Overdraft Fuzzy Logic Policy R1: overdraft is low  allocation left is much  allow reservation R2: overdraft is high  allocation left is little  disallow reservation R3: allocation proximity is soon  overdraft is high  allocation left is much  allow reservation R4: allocation proximity is soon  overdraft is low  allocation left is little  allow reservation

20. 20 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Super Computing 2004 Demonstration

21. 21 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A

22. 22 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Conclusions Document centric communication in conjunction with semi-structured native XML databases is a very flexible combination Batch charging and eager prepare reservation needed for scalability Timestamp based allocations distributed in a staggered monthly flow result in the best trade-off between fairness and utilization Generic PEP/PDP/PIP/PAP model useful for encapsulating and evolving authorization code OGSI/WSRF state management ideal for controlling fine grained service state such as account quotas, reservations and policies in a standard way

23. 23 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Future Work With large-scale flexibility and configurability comes complexity and it becomes hard to optimize high-level goals and to realize detailed user QoS requirements – development of an SLA Management framework and user/resource goal driven optimizing agents (WS-Agreement, ContractNet) Initial focus has been on scientific community resource sharing - support economic brokering and for-profit banks Multi jobs may overload the bank - SAML assertions (c.f. cheques) as a multi-allocation payment and reservation method

24. 24 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Learn more… http://www.sgas.se http://www.swegrid.se

25. 25 2 nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004 Outline Requirements on Software Qualities Open Grid Services Architecture SweGrid National Grid Testbed SweGrid Accounting System Results Lessons Learned Q & A