APEL Architecture Alison Packer

Overview Grid jobs accounting tool APEL Client software - installed in sites (CEs, gLite- APEL node) APEL Server accepts data from APEL client plus external clients Data is processed by APEL Server system and made available by the Accounting Portal:

Current Architecture MySQL PORTAL BROKER CONSUMER CLIENT MySQL RGMA REGISTRY FLEXIBLE ARCHIVER CLIENT python Java perlbash Java EXTERNAL CLIENTS Closed on1 st March

Current System R-GMA transport mechanism now replaced with Apache ActiveMQ (broker) However: –Many machines and many databases –Many different languages/scripts –If the database is down clients can not publish –Some clients have direct access to the database, some publish via the APEL client

New System APEL CLIENT MESSAGE BROKERS – located across Europe (ActiveMQ) CENTRAL SERVER (MySQL, python) CENTRAL SERVER (MySQL, python) APEL Monitoring PORTAL EXTERNAL CLIENTS SSM RECORD LOADER Job Records, Summary Job Records, Sync Records Job Records, Summary Job Records, Sync Records SUMMARISER REGIONAL APEL SERVER SSM

New System features Database improvements: o removed duplication of information (many date/time fields in same record) o added important fields (number of nodes, processors – for MPI jobs) o partitioning, indexing All communication sent via Message Broker service using Secure Stomp Messenger (SSM) SSM is available even if the database is not, so clients can still send messages Regional APEL Server system

New System – showing Use Cases EXTERNAL CLIENTS MESSAGE BROKERS – located across Europe (ActiveMQ) CENTRAL APEL SERVER PORTAL APEL CLIENT REGIONAL APEL SERVER Job Records Summaries Summaries & Job Records Summaries plus

Use Cases 1.APEL Client - publishes Job Records to central APEL server 2.Regional APEL server – uses APEL Client and publishes Summaries to central APEL server 3.Site has own accounting system, publishes Summaries to central APEL server NOTE: All publishing is via Message Brokers

Messages A message is one file which is sent and received by the SSM. A message will contain a number of records (e.g. 1000) A record corresponds to one row in the database The header in each message tells the server which type of records are in that message Whole message is encrypted before it is sent and messages are signed so we know their origin

Message Types Currently 3 types of message defined: 1. Job Records – from APEL client 2. Summary Records – from external clients and regional APEL server 3. Sync Records – for APEL monitoring Additional message types will be defined to accommodate new record types as required.

Summary Record message APEL-summary-job-message: v0.1 Site: RAL-LCG2 Month: 3 Year: 2010 GlobalUserName: /C=whatever/D=someDN VO: atlas Group: /atlas Role: Role=production EarliestEndTime: LatestEndTime: WallDuration: CpuDuration: 2345 NormalisedWallDuration: NormalisedCpuDuration: 2500 NumberOfJobs: 100 %...another summary job record... %... %

Deployment External Clients (Dec, 2011) –External clients migrate to publishing their summaries to the new system although old system continues to receive Job Records, creates summaries and publishes to portal. New System in Production (Feb, 2012) –The data from the old database is migrated to the new database; –Data is now summarised and published from the new system; –All external clients must have migrated to the new system at this point; –The old APEL client continues to publish as usual but the data is transferred to the new system using SSM and the message brokers. New APEL client released (Apr, 2012) –New and old APEL clients both supported. APEL Regional server (Summer, 2012)

Deployment Current APEL CLIENT BROKER CONSUMER PORTAL MESSAGE BROKERS (ActiveMQ) CENTRAL SERVER (MySQL, python) APEL Monitoring External CLIENTS SSM New APEL CLIENT SSM Regional APEL SERVER SSM