Metadata Progress GridPP18 20 March 2007 Mike Kenyon

Metadata Team Glasgow –Mike Kenyon –Caitriana Nicholson –Tom Doherty –Paul Millar CERN/Glasgow –Helen McGlone

ATLAS Event-level Metadata Navigation from Tag to events in AOD/ESD/RAW format Produced during AOD production –Oracle DBs at Tier0, Tier1 –MySQL (probably) at Tier2 ATLAS are implementing a Tag database to allow efficient identification and selection of events for analysis First-level cuts will be possible via queries on relational databases

Tag Navigator Tool Need to integrate Tag DB with Distributed Data Management (DQ2) and Distributed Analysis (Ganga) tools Tag Navigator Tool (TNT) developed to plug that gap Aims to take user from query on Tag DB to analysis using results of query

TNT Process Summary Runs a query on the database and outputs a local Root collection Divides this into a number of sub- collections Submits user jobs to LCG, one per sub- collection Output files can be registered as new DQ2 dataset

TNT Process Summary

Ganga Plugin TNT has also been implemented as a plugin to Ganga Ganga already allowed use of Tag files –TNT plugin is complementary to this Makes use of existing Ganga functionality for job submission, handling etc Will be available in Ganga release 4.3

Future Plans Continue development of TNT and GangaTnt in response to user feedback Performance testing of Tags in TNT Scalability and performance measurements on Tag DB Continue integration of Tags with AMI

AMI-VOMS solution 1.The grid user authenticates themselves to AMI using a grid certificate. 2.AMI allows the user to launch a Java web start application to create a VOMS proxy. 3.The user chooses the x509 certificate to be sent to the VOMS server 4.The VOMS server sends back a VOMS proxy certificate to the Java application. 5.The VOMS proxy file is uploaded to the server, which hosts a proxy delegation service, via a secure web service attachment. 6.The users VO, group and role can be extracted from the proxy and used for authorisation purposes. SourceForge project: Acacia

DQ2/AMI integration Use of delegated VOMS proxy certificate - Contact application on behalf of user, e.g. DQ2. Register a dataset simultaneously in AMI and DQ2 Other dq2 commands possible Tags/AMI integration Preliminary work on prototype database to allow physics predicate cut on a collection and catalogue the results within AMI

pyAMI AMI Web Service Client for Python AMI tutorial available at: yAMI.html Future development and maintenance Providing AMI command wrappers for groups/tools such as POOL Conditions, Muon and Tag collector Code optimisation – replace SAX with DOM parsing of AMI output Reformatting output – interface to format XML

The present: MonAMI v0.6 Universal sensor framework –Capable of monitoring many different services –Can send data to any number of information systems –Plugin structure v0.6 deployed at all ScotGrid sites & Grenoble Configuration details available at –

Upcoming: MonAMI v0.7 v0.7 –Support for 64-bit (AMD64) architecture –Improved documentation –Major improvements to R-GMA support Robust against temporary R-GMA failures Support for TLS-based secure upload of data Support for proxy-certificate based authentication Drop in deployment –Support for event monitoring (e.g. DPM/GridFTP) Released pending solution of minor SL4- related issues

The Future: MonAMI v0.8 Planned features –New python plugin API –Web & RSS feeds –PostGreSQL monitoring