Open Science Grid OSG Accounting System Matteo Melani SLAC 9/28/05 Joint OSG and EGEE Operations Workshop Culham, UK

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 2 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 3 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 4 What do I hope to get out of this workshop?  Present OSG view of the accounting problem:  OSG accounting requirements  Understand the EGEE/LCG view of accounting  Terminology  Requirements: goals, scope, main functionalities  Monitoring, accounting, logging, auditing differences and communalities  Understand what are the interoperability requirements for OSG Accounting  Understand what parts of APEL/DGAS we can re-use  Understand what are the technical difficulties in designing an accounting system for Grid environment. Interoperability requirements Leverage EGEE/LCG people experience

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 5 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 6 Who are we?  Matteo Melani – SLAC  Philippe Canal – Fermilab  Sudhir Borra –Fermilab

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 7 And we are talking to  Ian Foster, Frank Siebenlist, Martin Stuart - Globus  Sebastien Goasguen, Michael Shapiro - TeraGrid  David Kant, APEL developer - LCG  Andrea Guarise, Rosario Piro, DGAS developers - EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 8 What are the Activity Goals?  To provide the OSG stakeholders with a reliable and accurate set of views of the OSG grid resources usage  The OSG Accounting Activity objectives are:  To design (or choose) the accounting system data model  To design (or choose) a software system for gathering, storing, managing and presenting accounting data  To define and deploy repositories and access tools for the reporting and analysis of the OSG grid wide accounting information  To ensure interoperability with other Grids (LCG, TeraGrid,...)

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 9 What are we doing? Requirements Design Implementation Testing Requirements v-1.0 Working on v-2.0 Working on a DRAFT Project Charter Project Charter v1.0 Done first pass In progress Not there yet

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 10 Learning from others  Talking to the stakeholders  OSG stakeholders  Other Grids stakeholders: LCG, TeraGrid, EGEE, Globus…  Reviewing other Accounting Systems  DGAS, APEL, SGAS, GridBank, AMIE and TGAccounting  GGF groups RUS/UR/GESA/OGSA-WGs,  IETF’s AAA, Telecommunications Network

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 11

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 12

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 13 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity Presentation  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 14

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 15 Accounting is not monitoring Accounting:  User request to a service  Resource consumed Monitoring:  Service availability  Services performances  Alerting  Debugging

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 16 Overview  Goals  OSG Accounting Activity  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 17 Main Functionalities (1) The ability to collect services (CE and SE) usage data at each resources and services provider site. The ability to combine authentication, authorization and resource usage data related to a Grid user request to produce detailed accounting records. The ability to store and manage accounting records and accounting data. The ability to present accounting records in tabular format at the Grid wide and site level Track resource Usage per user “after the fact”

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 18 Main Functionalities (2) The ability to apply various (statistical) functions to analysis accounting records and data to facilitate the spotting of trends and patterns in the usage of resources and services. The ability to present the results of this analysis with graphical views. The ability to support auditing processes by storing the raw usage data and maintaining links between the accounting records and the raw usage data used to generate them. Accounting Information Presentation Graphic and Tabular View

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 19 Non-Functionalities  No user charging system,  No resources or services pricing  No support for an economic model for resource allocation But we have to keep them in mind! No Economic Model (for now)

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 20 System Properties(1)  Interoperability The Accounting System should leverage existing standards to maximize interoperability with other Grids and Accounting Services.  Fault Tolerance The Accounting System must reduce as much as possible usage data and accounting records loss. The Accounting System must be resilient to temporary communication failures over LAN and WAN. The Accounting System must be resilient to the failure of one of its component. EGEE is a stakeholder (because of USCMS and USATLAS)

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 21 System Properties(2)  Security The Accounting System will guarantee integrity and non– repudiation of the accounting records at the site level. The Accounting System must protect accounting information using secure communication channels (mutual authentication, message integrity, confidentiality) and access control lists.  Scalability and Performance Not really an issue  Other  The Accounting System should leverage existing tools and infrastructures to solve related problems. Dependable

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 22 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity  View of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 23 Design (1)

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 24 Design (2)

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 25 Design (3)  Service a distributed objects so meter are distributed objects  Meter support pull/push of accounting information from a service  Meter-Collector communication using WS  Accounting Records based on GGF UR  Once the Accounting Record is published is available to clients that communicate with the sites accounting record store through a WS interface Distributable Meter Accounting Records Storage at each site Separation of info gathering and info presentation Web Service based

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 26 Overview  What do I hope to get out of this workshop?  OSG Accounting Activity  Our view of accounting, monitoring, logging and auditing  OSG Accounting System Requirements  OSG Accounting System Design  Questions about Accounting in EGEE

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 27 Questions about Accounting in EGEE  APEL-DGAS relationship  How are they similar/different?  Why two systems?  What is the directions for the future?  Are they difficult to install, configure and manage?  Lessons learned for DGAS/APEL  What are the complains?  What are the praises?

9/28/2005 Joint OSG and EGEE Operations Workshop, Culham, UK 28 What are your accounting needs?