APEL as a Global Accounting Repository Alison Packer STFC 2/25/2019 APEL Accounting Repository – EGI Technical Forum (September 2012)

Introduction APEL Central Repository redevelopment Other Accounting Clients APEL Client Development Regional APEL Server

APEL – old system EGI Accounting Portal APEL client BROKER APEL client ActiveMQ APEL client APEL client MySQL CONSUMER http://goc-accounting.grid-support.ac.uk MySQL MySQL Summaries created External clients

New System - Deployment Stage 1 CONSUMER MySQL http://goc-accounting.grid-support.ac.uk APEL client BROKER ActiveMQ External clients MySQL MySQL Summaries created MySQL CPU JobRecords CPU Summaries External clients EGI Message Brokers Record loader Receiving SSM EGI Accounting Portal Sending SSM

New System - Deployment Stage 2 APEL client BROKER ActiveMQ http://goc-accounting.grid-support.ac.uk CONSUMER DBunloader MySQL EGI Message Brokers MySQL CPU JobRecords CPU Summaries Converted JobRecords New APEL client External clients EGI Message Brokers Record loader Receiving SSM EGI Accounting Portal Sending SSM

APEL – new system External clients EGI Accounting Portal http://goc-accounting.grid-support.ac.uk nagios pub/sync New APEL client DBunloader MySQL CPU JobRecords CPU Summaries New APEL client EGI Message Brokers EGI Message Brokers External clients EGI Accounting Portal Record loader Receiving SSM Sending SSM

External clients - production CERN publishing Job Records OSG publishing Summary Records CERN Job Records summarised on new system, including local jobs New system records included in Summaries retrieved by Accounting Portal

External Clients - testing INFN/DGAS have tested sending JobRecords and Summary records, ready to move to production. IN2P3 have successfully tested sending Summary records, but still some date errors to resolve. NDGF/SGAS have successfully tested sending Summary records, ready to move to production.

New External Clients SGAS (CH & Finland) – testing Nikhef – successfully tested, ready for production IGE/GridSafe – testing QosCoSGrid/MAPPER – testing EDGI – have a plan, awaiting testing UNICORE – successfully tested, implementing new SSM protocol

New Server - development New Summaries: By CE, to enable nagios monitoring by CE as well as by site Accounting of parallel and multicore jobs CAR support (XML) Additional types of accounting records Publishing to Portal plus additional consumers

EMI 3 Client - parsers Support for LSF (versions 5,6,7,8 ) PBS (version independent), GridEngine (version independent) Condor – is this still required? Parser code complete and tested with LSF, PBS and GridEngine logs. SLURM – we can do this if we have a collaborating production site

EMI 3 Client – in development Support for local jobs. Accounting of parallel and multicore jobs, Job Record modelled on CAR, and includes NodeCount and Processors fields. Publishing summaries from client.

APEL Regionalisation Once migration to new system complete a regional version of the repository will be provided. NGI can then collect data from APEL clients, create summaries and publish – to local portal and central APEL repository.

APEL - summary Migration ongoing Wider range of sites publishing in production Additional accounting types (cloud, storage) EMI 3 client supporting a range of new features Regional APEL server