1. Conditions Metadata for TAGs Elizabeth Gallas, (Ryan Buckingham, Jeff Tseng) - Oxford ATLAS Software & Computing Workshop CERN – April 19-23, 2010

2. April 2010Elizabeth Gallas - COMA2 Outline Overview of Oracle Schemas for TAGs 1.TAG DB (event-wise metadata) 2.CATALOG (Dataset Catalogue Metadata) 3.COMA (Conditions MetadatA) Evolution of Conditions Metadata “Current” COMA Schema Uses of Conditions Metadata ELSSI usage Decoding bit packed event-wise TAG attributes (Trigger) runBrowserReport runBrowser Upload procedure and COMA Loading Status COMA Browser - Interface Status Documentation and Links Plans Summary

3. April 2010Elizabeth Gallas - COMA3 TAG DB Event-wise metadata tables Stores Event-wise attributes: electron (Et, eta, phi …) muon (Et, eta, phi …) … and references to RAW, ESD, AOD files Official data processing chain: RAW  ESD  AOD  TAG files Data Catalogue tables Stores information on file and dataset processing and location Project name AMI tag (what processing occurred)… Sources: AMI (ATLAS Metadata Catalogue, Tier0 … ‘COMA’ (Conditions MetadatA) tables Conditions of data taking Beam conditions Trigger and DAQ conditions Magnetic field … Various sources: Conditions DB, Log files, xml files, email… Oracle Database: TAG DB and associated metadata tables RUNS 3. COMA tables …EventLBRun 1. TAG DB: Event-wise metadata ORACLE DB 2. DATA Catalogue Tables

4. April 2010Elizabeth Gallas - COMA4 History of this ATLAS TAG sub-project 2007: first COMA tables were filled for simulation tests/prototype development Streaming Test and (FDR) Full Dress Rehearsal exercises Data was collected from log files and other sources And INSERTED into new folders in Conditions database (COOL) Which formed the prototype for the ATLAS Luminosity calculation And formed the basis for some of the current COOL structures 2008: COMA tables were used by the ELSSI prototype Event-wise selection and analysis using TAGs requires Run and Lumiblock level information that is not practically stored event-wise Examples: temporal, detector, trigger, DQ … information 2009: we began extracting Run and Lumiblock level information from COOL to facilitate efficient access to select Conditions (Conditions Metadata) to ELSSI runBrowser was developed Initially to check data integrity and relationships between the data We realized this runBrowser would be more generally useful  Separating Run-browsing from event-browsing (ELSSI)  As a stand-alone tool runBrowserReport emerged Which will help ELSSI and runBrowser describe underlying COMA info and provides more links to other ATLAS systems 2010:

5. April 2010Elizabeth Gallas - COMA5 Evolution in the last year TAG DB / ELSSI evolved to a distributed architecture Not possible to upload all TAGs at any one Oracle site ELSSI – also needs to know which TAGs are uploaded at which voluntary sites So additional relational schema has been added: ATLAS TAGS CATALOG (Elisabeth Vinek) Contains information needed to deploy distributed TAG services on the grid The COMA tables use this CATALOG Upload only Run/LB metadata for Runs in CATALOG This reduces handling of conditions anomalies, allowing us to focus on Runs of ‘event analysis interest’ Next slide … Oracle DB contains: 1. TAG DB Tables, 2. CATALOG Tables, 3. COMA Tables.

6. April 2010Elizabeth Gallas - COMA6 Upload Procedure and COMA Loading Status Steps in Database loading – happens within hours of reconstruction TAGs uploaded to Oracle CATALOG tables updated COMA tables updated COMA: Loading programs Run upload Look in CATALOG tables for new Runs – upload Trigger/Prescale upload Adds trigger and prescale information for those Runs New: Aggregate prescale loading in final testing Calculates Run-wise summary: prescale, passthrough, rerun per triggger  Useful to ELSSI to display active/passive/disabled triggers per Run  So that users do not select triggers in a Run which would yield 0 events  To be discussed tomorrow afternoon Data Quality upload Run when DQ assessments into LBSUMM are loaded, tagged and locked  Includes Detector and Physics DQ assessments  Does not include virtual flags at this point

7. April 2010Elizabeth Gallas - COMA7 COMA runBrowser: History and Evolution A web-based browser was developed for COMA tables to facilitate checking data integrity and relationships in the relational tables March2009: Features of this initial browser were presented … we realized that this kind of browser could be a Run/LB wise component of ELSSI … but also could be a stand-alone browser for Conditions data http://indico.cern.ch/materialDisplay.py?contribId=1&materialId=slides&confId=54409 The name of this browser: runBrowser (or COMABroswer) Not yet released to the collaboration Not a replacement for runQuery (the web based browser to all online Runs in the Conditions DB) May2009: TAG group developed first DTD for the GoodRunList XML(GRL): This xml will be how runBrowser communicates to ELSSI the Run/LB selection. This DTD has since been taken up by the ATLAS experiment to communicate good Run/LB ranges between subsystems.

8. April 2010Elizabeth Gallas - COMA8 COMA runBrowser: Interface Status As data has been added to COMA tables runBrowser has evolved accordingly Current selection criteria includes: Data Source (online,simulation), Data Type, Run Number, Duration, Number of LBs, temporal, DAQ configuration, Filename tag (Project Name), Trigger Super Master Key, Trigger LVL1 and/or HLT Prescale Key. Useful for initial purpose and query development But initial implementation was not generally well constructed … October 2009: Ryan Buckingham joined project Oxford graduate student – 1 st year. Move to CERN Aug 1, 2010 Working on runBrowser Improved internal communication object oriented structure introducing further classes - allow criteria to be added in modules Will facilitate better syntax alignment with runQuery and inclusion of criteria to the GRL xml to pass to ELSSI Improve look/feel of interface: using php and jQuery widgets

9. April 2010Elizabeth Gallas - COMA9 Documentation and Links The current project documentation can be found at: https://gallas.web.cern.ch/gallas/Conditions_to_TAGs.html Current Status: Conditions metadata is being uploaded to COMA tables In use by ELSSI for online and simulation data - triggers Current Plans: Elizabeth – loading COMA data, runBrowserReport Ryan – focussing on runBrowser2 Consulting with Jeff Tseng (at Oxford) on program architecture Discussions this week: Trigger and Prescale – Tuesday afternoon (with Joerg) Metadata for file based TAGs - ??

10. April 2010Elizabeth Gallas - COMA10 Conclusions This talk reflects an evolving system Current information in the system is growing Adding Conditions Metadata to a relational database Making that criteria available in a useable interface We want to insure the Metadata is complete enough to satisfy use cases while Reflecting accurately its limitations Interfaces are being constructed to use selection syntax, criteria, and communication in common use in ATLAS i.e. runQuery, GoodRunList xml … This facilitates cross checks with other systems Continuous process: talking with various experts on insuring data integrity and completeness