ATLAS Distributed Analysis Dietrich Liko

Thanks to … pathena/PANDA: T. Maneo, T. Wenaus, K. De DQ2 end user tools: T. Maneo GANGA Core: U. Edege, J. Mosckiki, K. Harrison, A. Murao, A. Soroko GANGA GUI C.L. Tan Athena AOD J. Elmshäuser User Production F. Brochu TAG Navigator C. Nicholson, M. Keynon EGEE/gLite: H.C. Lee, D. Liko Nordugrid B. Samset, K. Pajchet OSG/Panda D.Liko + PANDA team Dashbord Monitoring B. Gaidioz, S. Reddy, J. Wu AMI Integration F. Fassi, C.L. Tan + AMI team

ATLAS Analysis Model

Different scenarios Inside the Framework ◦ Athena ◦ AOD & ESD ◦ Grid based (GANGA, PANDA etc) Outside the framework ◦ ROOT ◦ DPD (ntuple) ◦ ROOT & PROOF Development aims to minimize the difference ◦ Migrate programs between ROOT and Athena ◦ AOD access from ROOT

Some recent activities pathena/PANDA ◦ See Tadashi’s talk GANGA ◦ EGEE ◦ Nordugrid ◦ PANDA (based on pathena) GANGA statistic and use cases

Migration to DQ2 0.3 The new release GANGA is available as of today. It supports DQ2 0.3 All users have to update their GANGA versions, as the changes of DQ2 are not backward compatible

EGEE – new features Support for very large sandboxes Support for group areas (EventViews) Support for LFC bulk operations AOD to ESD backnavigation Support for xrootd in Lyon and GridKa dCache readahead buffer Automatic patches for ROOT access to DPM and dCache More backends for AthenaMC

EGEE – gLite WMS At CERN we have started to use the gLite WMS ◦ It will be available also at sites in some time Bulk submission Better sandbox handling Will be aware of job priority settings at the site when available

Nordugrid upgraded to the latest ARC middleware, 0.6.0, distributed with made sure Athena-related code works with DQ2 0.3 done some work on ensuring proper use of credentials and backend commands, especially for systems without AFS made several internal changes for better job monitoring etc. Next Steps ◦ get DQ2 user dataset registration to work, using the new DQ2 tools ◦ connect NG backend jobs with arda dashboard

PANDA New interface to PANDA ◦ Newly developed ◦ Based on pathena Now it is possible to send a job to all ATLAS grids ◦ Not all options are supported yet … Can be the basis of further integration between the projects Will be available in the next release (4.3.6)

GANGA Users ATLAS: 435 users, 60 to 90 Users per week

GANGA ATLAS Jobs Data collected by ARDA dashboard

GANGA Jobs at T1 sites Data collected by ARDA dashboard

WMS Site efficiency at IN2P3

Job Distribution over the Tiers Data collected by ARDA dashboard

Where are the GANGA users?

Some examples of GANGA Usage Spain ◦ IFIC Valencia Italy ◦ INFN-Naples UK ◦ Lancaster & Manchester Germany ◦ Munich

IFIC Valencia Top Physics ◦ AANT Ntuples + ROOT Exotic Physics ◦ AOD + Athena ◦ HPTV Ntuples + ROOT Marcos Use GANGA ◦ AOD analysis ◦ Ntuple production Investigate StoRM with Lustre as disk based SE

INFN Naples

LMU Munich Production of Signal MC sample with AthenaMC additional events: ◦ event generation: 5 jobs or use already validated evgen samples where only a small fraction has officially been simulated/reconstructed ◦ simulation: 1000 jobs ◦ reconstruction: 50 jobs From this exercise a prototype for automatic job submission has evolved which will eventually will be part of Ganga Statistics from the dashboard: 11/4-11/6: 79% Grid eff. * 77% Application eff. = 53% overall eff. Distributed Analysis as part of the CSC homework: ◦ process signal and background MC samples at well known and maintained sites: GridKa, Lyon and LRZ ◦ This involves: SUSY signal, ttbar (5200), Di-boson backgrounds, etc. - Using SUSYView

UK Use Cases 1. GANGA FOR PHYSICS VALIDATION Require rapid validation of new releases as soon as they are deployed (usually reconstruction. EVGEN and DIGITS prepared for a variety of bench-mark channels some months back; AthenaMC used to reconstruct the digits using the new release/cache. AODs analyzed via Athena or sometimes copied directly to a local disk with DQ2 if there are a small number of files.Typical statistics: ~ events 2. GANGA FOR SMALL PRIVATE PRODUCTIONS Require small non-official samples for people developing new reconstruction code. Typical example: Eva Bouhova and Rob Henderson developing the V0 finder - this work has required several samples not included in the official production. Events generated, simulated and digitized using AthenaMC. The files are then copied to the local site with DQ2 so that they can be scanned using the reconstruction code under development Typical statistics: ~5000 events 3. GANGA FOR CSC PHYSICS ANALYSIS Running physics analysis code over CSC datasets with the aim of producing CSC notes GANGA used to submit analysis jobs on CSC AOD datasets; resulting ntuples merged and copied to local disks for analysis in ROOT. Typical statistics: events 4. GANGA FOR ANALYSIS TOOLS DEVELOPMENT 4. Using GANGA to test new tools for vertexing, tagging etc. GANGA used to submit analysis jobs on CSC AOD datasets, running new analysis tools; resulting ntuples merged and copied to local disks for analysis in ROOT. Typical statistics: events

Summary Distributed Analysis Use Cases in ATLAS are growing ◦ I was showing GANGA, you will hear soon about PANDA While we have two projects, we try to integrate them close and increase the collaboration Datamanagment and data distribution are the crucial aspects 2 nd week of July: GANGA Developer Days at CERN ◦ All developers will be locked in a room …