ATLAS Physics Analysis Framework James R. Catmore Lancaster University.

Slides:



Advertisements
Similar presentations
31/03/00 CMS(UK)Glenn Patrick What is the CMS(UK) Data Model? Assume that CMS software is available at every UK institute connected by some infrastructure.
Advertisements

Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
Batch Production and Monte Carlo + CDB work status Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
6/4/20151 Introduction LHCb experiment. LHCb experiment. Common schema of the LHCb computing organisation. Common schema of the LHCb computing organisation.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL March 25, 2003 CHEP 2003 Data Analysis Environment and Visualization.
ATLAS Analysis Model. Introduction On Feb 11, 2008 the Analysis Model Forum published a report (D. Costanzo, I. Hinchliffe, S. Menke, ATL- GEN-INT )
A tool to enable CMS Distributed Analysis
Analysis demos from the experiments. Analysis demo session Introduction –General information and overview CMS demo (CRAB) –Georgia Karapostoli (Athens.
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
December 17th 2008RAL PPD Computing Christmas Lectures 11 ATLAS Distributed Computing Stephen Burke RAL.
3 Sept 2001F HARRIS CHEP, Beijing 1 Moving the LHCb Monte Carlo production system to the GRID D.Galli,U.Marconi,V.Vagnoni INFN Bologna N Brook Bristol.
David Adams ATLAS ATLAS Distributed Analysis David Adams BNL March 18, 2004 ATLAS Software Workshop Grid session.
03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.
Choosing Delivery Software for a Digital Library Jody DeRidder Digital Library Center University of Tennessee.
CSE 548 Advanced Computer Network Security Document Search in MobiCloud using Hadoop Framework Sayan Cole Jaya Chakladar Group No: 1.
3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
8th November 2002Tim Adye1 BaBar Grid Tim Adye Particle Physics Department Rutherford Appleton Laboratory PP Grid Team Coseners House 8 th November 2002.
Nick Brook Current status Future Collaboration Plans Future UK plans.
ATLAS and GridPP GridPP Collaboration Meeting, Edinburgh, 5 th November 2001 RWL Jones, Lancaster University.
F. Fassi, S. Cabrera, R. Vives, S. González de la Hoz, Á. Fernández, J. Sánchez, L. March, J. Salt, A. Lamas IFIC-CSIC-UV, Valencia, Spain Third EELA conference,
7April 2000F Harris LHCb Software Workshop 1 LHCb planning on EU GRID activities (for discussion) F Harris.
November SC06 Tampa F.Fanzago CRAB a user-friendly tool for CMS distributed analysis Federica Fanzago INFN-PADOVA for CRAB team.
Introduction Advantages/ disadvantages Code examples Speed Summary Running on the AOD Analysis Platforms 1/11/2007 Andrew Mehta.
Architecture and ATLAS Western Tier 2 Wei Yang ATLAS Western Tier 2 User Forum meeting SLAC April
…building the next IT revolution From Web to Grid…
T3 analysis Facility V. Bucard, F.Furano, A.Maier, R.Santana, R. Santinelli T3 Analysis Facility The LHCb Computing Model divides collaboration affiliated.
ATLAS is a general-purpose particle physics experiment which will study topics including the origin of mass, the processes that allowed an excess of matter.
A PanDA Backend for the Ganga Analysis Interface J. Elmsheuser 1, D. Liko 2, T. Maeno 3, P. Nilsson 4, D.C. Vanderster 5, T. Wenaus 3, R. Walker 1 1: Ludwig-Maximilians-Universität.
Integration of the ATLAS Tag Database with Data Management and Analysis Components Caitriana Nicholson University of Glasgow 3 rd September 2007 CHEP,
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
Kyle Cranmer (BNL)HCP, Isola d’Elba, March 23, The ATLAS Analysis Architecture Kyle Cranmer Brookhaven National Lab.
The ATLAS TAGs Database - Experiences and further developments Elisabeth Vinek, CERN & University of Vienna on behalf of the TAGs developers group.
Using Ganga for physics analysis Karl Harrison (University of Cambridge) ATLAS Distributed Analysis Tutorial Milano, 5-6 February 2007
2 June 20061/17 Getting started with Ganga K.Harrison University of Cambridge Tutorial on Distributed Analysis with Ganga CERN, 2.
ATLAS-specific functionality in Ganga - Requirements for distributed analysis - ATLAS considerations - DIAL submission from Ganga - Graphical interfaces.
TAGS in the Analysis Model Jack Cranshaw, Argonne National Lab September 10, 2009.
INFSO-RI Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.
1 LHCb computing for the analysis : a naive user point of view Workshop analyse cc-in2p3 17 avril 2008 Marie-Hélène Schune, LAL-Orsay for LHCb-France Framework,
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,
ATLAS Distributed Analysis Dietrich Liko IT/GD. Overview  Some problems trying to analyze Rome data on the grid Basics Metadata Data  Activities AMI.
The ATLAS Computing & Analysis Model Roger Jones Lancaster University ATLAS UK 06 IPPP, 20/9/2006.
Distributed Analysis Tutorial Dietrich Liko. Overview  Three grid flavors in ATLAS EGEE OSG Nordugrid  Distributed Analysis Activities GANGA/LCG PANDA/OSG.
K. Harrison CERN, 21st February 2005 GANGA: ADA USER INTERFACE - Ganga release Python client for ADA - ADA job builder - Ganga release Conclusions.
David Adams ATLAS ATLAS Distributed Analysis (ADA) David Adams BNL December 5, 2003 ATLAS software workshop CERN.
Finding Data in ATLAS. May 22, 2009Jack Cranshaw (ANL)2 Starting Point Questions What is the latest reprocessing of cosmics? Are there are any AOD produced.
DZero Monte Carlo Production Ideas for CMS Greg Graham Fermilab CD/CMS 1/16/01 CMS Production Meeting.
David Adams ATLAS ATLAS Distributed Analysis and proposal for ATLAS-LHCb system David Adams BNL March 22, 2004 ATLAS-LHCb-GANGA Meeting.
A GANGA tutorial Professor Roger W.L. Jones Lancaster University.
Simulation Production System Science Advisory Committee Meeting UW-Madison March 1 st -2 nd 2007 Juan Carlos Díaz Vélez.
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
ATLAS TIER3 in Valencia Santiago González de la Hoz IFIC – Instituto de Física Corpuscular (Valencia)
Acronyms GAS - Grid Acronym Soup, LCG - LHC Computing Project EGEE - Enabling Grids for E-sciencE.
Joe Foster 1 Two questions about datasets: –How do you find datasets with the processes, cuts, conditions you need for your analysis? –How do.
Seven things you should know about Ganga K. Harrison (University of Cambridge) Distributed Analysis Tutorial ATLAS Software & Computing Workshop, CERN,
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Starting Analysis with Athena (Esteban Fullana Torregrosa) Rik Yoshida High Energy Physics Division Argonne National Laboratory.
ATLAS Distributed Computing Tutorial Tags: What, Why, When, Where and How? Mike Kenyon University of Glasgow.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL May 19, 2003 BNL Technology Meeting.
ANALYSIS TRAIN ON THE GRID Mihaela Gheata. AOD production train ◦ AOD production will be organized in a ‘train’ of tasks ◦ To maximize efficiency of full.
ATLAS Distributed Analysis S. González de la Hoz 1, D. Liko 2, L. March 1 1 IFIC – Valencia 2 CERN.
Simulation Production System
Moving the LHCb Monte Carlo production system to the GRID
Artem Trunov and EKP team EPK – Uni Karlsruhe
Introduction to Athena
ATLAS DC2 & Continuous production
Production Manager Tools (New Architecture)
Presentation transcript:

ATLAS Physics Analysis Framework James R. Catmore Lancaster University

Introduction  Physics analysis is the final link in the ATLAS chain  It is the point at which data is “handed over” from technical experts to the physics community  As such it sets significant challenges on the software infrastructure  Getting the data to the physicists across the world  Packaging it in a readily readable form  Providing tools which enable users to perform the sophisticated studies which will be necessary to discover new physics  Providing computing resources to enable large-scale data processing  In this talk:  Overview of analysis framework components  Why should we be using the Grid to do physics analysis?

Overview of the analysis framework (steady state) Monte Carlo Detector Reconstruction Digits AOD AOD Building ESD TIER 1 SITES ESD AOD TIER 2 SITESGRID MIDDLEWAREUSERS Analysis jobs Results TIER 0 (CERN) / PRODUCTION SYSTEM

Terms and tools  ESD: Event Summary Data  Will not in general be used for analysis  AOD: Analysis Object Data  This is the data which will be used by physicists for most analyses  Persistency: format in which the data is written to disk  ATLAS uses a format known as POOL  Athena: the overarching software framework within which all tasks are run  Analysis code is implemented as Athena algorithms  Analysis tools: software within Athena which expedite efficient physics analysis  “The Grid”: distributed network of storage and computing facilities where ATLAS data will be stored and on which analysis will be performed  Also includes “middleware” which provides the interface between the user and the grid sites  Jobs: Athena tasks set by the users to be performed on the Grid  Job submission tools: software which facilitates the submission of jobs to the Grid  ATLAS and LHCb share a common tool - GANGA  The ATLAS tool for managing Grid files is called DQ2 (Don Quichote)

Analysis Object Data (AOD)  Distilled information from the event reconstruction  Makes strong use of inheritance  Ultimately inherit from a four-momentum implementation  AOD objects are designed to reflect the physical objects which they represent  Muon, electron, photon, track, missing Et, tau-jet, b-jet  Common “look and feel” to all classes  Data is grouped into events and packaged in STL containers  MuonContainer, ElectronContainer, TrackParticleContainer etc  The basic operation in the analysis code is therefore looping over the AOD objects in these containers and interrogating them for information

Analysis Tools and event selection  Software which facilitates the writing of clean and transparent analysis code  Sorting, selecting, filtering, combining, calculating common physical quantities, removing overlaps  Principal tools  AOD analysis tools  EventView  B-physics analysis package  Event selection  Data will not come out of ATLAS nicely packaged according to the signal event type  Users will need to select events according to simple criteria  E.g. “event contains two muons with pt > 6GeV”  This information is known as “metadata” is implemented as a “TAG” on each event  Accessed through AMI, the Atlas Metadata Interface

“Post analysis” tools

Using the Grid to do physics analysis

Why use the Grid for analysis?  “Why can’t I just copy the files to my University and run analysis locally?”  Well, at the moment, with conveniently packaged sets of Monte Carlo, that’s possible  Once ATLAS starts to produce data, there will simply be too much of it for local analysis  Storage space  Time for copying files across to local institutes  So whilst it may not be immediately necessary to use the Grid, it makes sense to learn how to use it in advance of it becoming essential  The Grid provides immense computing resources which enable a user to run hundreds of jobs simultaneously  Users do not have to worry about installing software at their institutes  The DQ2 database provides an easy method of locating the required data  Users don’t have to worry about where it is  GANGA provides a very simple interface to the Grid; anyone who can run Athena can use the Grid  Graphical user interface provided

What do I need to use the Grid 1.A Grid Certificate 2.Membership of the ATLAS Virtual Organisation 3.A computer set up as a Grid User Interface (UI) Comprehensive workbook instructions (S. Lloyd):

About GANGA  Gaudi, Athena and Grid Alliance  Joint ATLAS/LHCb grid job submission tool  Minimizes user contact with Grid fabric  Principally designed for analysis but also for small-scale private Monte Carlo  Automatically retrieves and registers files  Can operate on either the Grid or a local batch system  Provides a python command line or a GUI  Easily installed on local machines Main page and installation instructions: Most recent tutorial (September 2006):

A complete example Set up Grid UI and DQ2: Search for the dataset:

DQ2 web interface

A complete example (ii) Set up GANGA and CMT: Check out analysis package: Setup in cmt directory of package:

A complete example (iii) Start GANGA:

A complete example (iv) Define the job:

A complete example (v)

A complete example (vi) Get the n-tuple: …which drops the results into the /afs home directory. These can then be analyzed in ROOT as normal

Other facilities and current issues  Job splitting  TAG analysis  Local backend switch  User-defined Monte Carlo production  Seamless registration onto DQ2  Ability to read old LFC data  GUI  Issues  DQ2 can only copy with datasets at one site  Not a GANGA issue but affects the way we need to work at the moment  Request to have the facility to view jobs as they run

Ganga GUI

Conclusions  The Physics Analysis Framework for ATLAS is now in an advanced state and will be ready for data taking  Physicists will need to use the Grid to do their analysis  Most of the tools are ready to be used - it makes sense to learn how to use them now  A large quantity of high-quality documentation is available  A tutorial will be held in the UK within a few months - announcements will be made shortly