AliRoot survey: Analysis P.Hristov 11/06/2013. Are you involved in analysis activities?(85.1% Yes, 14.9% No) 2 Involved since 4.5±2.4 years Dedicated.

Slides:



Advertisements
Similar presentations
The LEGO Train Framework
Advertisements

– Unfortunately, this problems is not yet fully under control – No enough information from monitoring that would allow us to correlate poor performing.
Parallel Reconstruction of CLEO III Data Gregory J. Sharp Christopher D. Jones Wilson Synchrotron Laboratory Cornell University.
Trains status&tests M. Gheata. Train types run centrally FILTERING – Default trains for p-p and Pb-Pb, data and MC (4) Special configuration need to be.
Struts 2.0 an Overview ( )
Introducing python into industrial environment applications Fabio Pliger SIA s.r.l.
The ALICE Analysis Framework A.Gheata for ALICE Offline Collaboration 11/3/2008 ACAT'081A.Gheata – ALICE Analysis Framework.
Alexandre A. P. Suaide VI DOSAR workshop, São Paulo, 2005 STAR grid activities and São Paulo experience.
Central Reconstruction System on the RHIC Linux Farm in Brookhaven Laboratory HEPIX - BNL October 19, 2004 Tomasz Wlodek - BNL.
David N. Brown Lawrence Berkeley National Lab Representing the BaBar Collaboration The BaBar Mini  BaBar  BaBar’s Data Formats  Design of the Mini 
Costin Grigoras ALICE Offline. In the period of steady LHC operation, The Grid usage is constant and high and, as foreseen, is used for massive RAW and.
Introduction Advantages/ disadvantages Code examples Speed Summary Running on the AOD Analysis Platforms 1/11/2007 Andrew Mehta.
EGEE is a project funded by the European Union under contract IST HEP Use Cases for Grid Computing J. A. Templon Undecided (NIKHEF) Grid Tutorial,
PWG3 Analysis: status, experience, requests Andrea Dainese on behalf of PWG3 ALICE Offline Week, CERN, Andrea Dainese 1.
Andrei Gheata, Mihaela Gheata, Andreas Morsch ALICE offline week, 5-9 July 2010.
Analysis trains – Status & experience from operation Mihaela Gheata.
5/2/  Online  Offline 5/2/20072  Online  Raw data : within the DAQ monitoring framework  Reconstructed data : with the HLT monitoring framework.
AliRoot survey P.Hristov 11/06/2013. Offline framework  AliRoot in development since 1998  Directly based on ROOT  Used since the detector TDR’s for.
CERN – Alice Offline – Thu, 27 Mar 2008 – Marco MEONI - 1 Status of RAW data production (III) ALICE-LCG Task Force weekly.
PROOF and ALICE Analysis Facilities Arsen Hayrapetyan Yerevan Physics Institute, CERN.
Why A Software Review? Now have experience of real data and first major analysis results –What have we learned? –How should that change what we do next.
MICE CM28 Oct 2010Jean-Sebastien GraulichSlide 1 Detector DAQ o Achievements Since CM27 o DAQ Upgrade o CAM/DAQ integration o Online Software o Trigger.
Predrag Buncic Future IT challenges for ALICE Technical Workshop November 6, 2015.
Online Monitoring System at KLOE Alessandra Doria INFN - Napoli for the KLOE collaboration CHEP 2000 Padova, 7-11 February 2000 NAPOLI.
A. Gheata, ALICE offline week March 09 Status of the analysis framework.
Gustavo Conesa ALICE offline week Gamma and Jet correlations analysis framework Short description, Status, HOW TO use and TO DO list 1/9.
Performance DPDs and trigger commissioning Preparing input to DPD task force.
1 Offline Week, October 28 th 2009 PWG3-Muon: Analysis Status From ESD to AOD:  inclusion of MC branch in the AOD  standard AOD creation for PDC09 files.
October Test Beam DAQ. Framework sketch Only DAQs subprograms works during spills Each subprogram produces an output each spill Each dependant subprogram.
OPERATIONS REPORT JUNE – SEPTEMBER 2015 Stefan Roiser CERN.
Pavel Nevski DDM Workshop BNL, September 27, 2006 JOB DEFINITION as a part of Production.
Data processing Offline review Feb 2, Productions, tools and results Three basic types of processing RAW MC Trains/AODs I will go through these.
M. Gheata ALICE offline week, October Current train wagons GroupAOD producersWork on ESD input Work on AOD input PWG PWG31 (vertexing)2 (+
Analysis Trains Costin Grigoras Jan Fiete Grosse-Oetringhaus ALICE Offline Week,
Physics selection: online changes & QA M Floris, JF Grosse-Oetringhaus Weekly offline meeting 30/01/
ATLAS Distributed Analysis Dietrich Liko IT/GD. Overview  Some problems trying to analyze Rome data on the grid Basics Metadata Data  Activities AMI.
Analysis experience at GSIAF Marian Ivanov. HEP data analysis ● Typical HEP data analysis (physic analysis, calibration, alignment) and any statistical.
Analysis train M.Gheata ALICE offline week, 17 March '09.
M. Gheata ALICE offline week, 24 June  A new analysis train macro was designed for production  /ANALYSIS/macros/AnalysisTrainNew.C /ANALYSIS/macros/AnalysisTrainNew.C.
Offline Weekly Meeting, 24th April 2009 C. Cheshkov & C. Zampolli.
Calibration algorithm and detector monitoring - TPC Marian Ivanov.
AliRoot survey: Reconstruction P.Hristov 11/06/2013.
Meeting with University of Malta| CERN, May 18, 2015 | Predrag Buncic ALICE Computing in Run 2+ P. Buncic 1.
Some topics for discussion 31/03/2016 P. Hristov 1.
Joe Foster 1 Two questions about datasets: –How do you find datasets with the processes, cuts, conditions you need for your analysis? –How do.
AliRoot survey: Calibration P.Hristov 11/06/2013.
AAF tips and tricks Arsen Hayrapetyan Yerevan Physics Institute, Armenia.
MAUS Status A. Dobbs CM43 29 th October Contents MAUS Overview Infrastructure Geometry and CDB Detector Updates CKOV EMR KL TOF Tracker Global Tracking.
29/04/2008ALICE-FAIR Computing Meeting1 Resulting Figures of Performance Tests on I/O Intensive ALICE Analysis Jobs.
ANALYSIS TRAIN ON THE GRID Mihaela Gheata. AOD production train ◦ AOD production will be organized in a ‘train’ of tasks ◦ To maximize efficiency of full.
Offline week meetings CERN – June 25, 2009 CAF user experience Maria Nicassio (INFN Bari)
Monthly video-conference, 18/12/2003 P.Hristov1 Preparation for physics data challenge'04 P.Hristov Alice monthly off-line video-conference December 18,
CALIBRATION: PREPARATION FOR RUN2 ALICE Offline Week, 25 June 2014 C. Zampolli.
THIS MORNING (Start an) informal discussion to -Clearly identify all open issues, categorize them and build an action plan -Possibly identify (new) contributing.
Calibration: preparation for pa
Data Formats and Impact on Federated Access
Analysis trains – Status & experience from operation
Developments of the PWG3 muon analysis code
ALICE analysis preservation
ALICE FAIR Meeting KVI, 2010 Kilian Schwarz GSI.
Tree based validation tool for track reconstruction
Experience in ALICE – Analysis Framework and Train
Analysis Trains - Reloaded
AliRoot status and PDC’04
Analysis framework - status
TPC status - Offline Q&A
ALICE Computing Upgrade Predrag Buncic
Simulation in a Distributed Computing Environment
Case Study 1 By : Shweta Agarwal Nikhil Walecha Amit Goyal
Data Challenge 1 Closeout Lessons Learned Already
Presentation transcript:

AliRoot survey: Analysis P.Hristov 11/06/2013

Are you involved in analysis activities?(85.1% Yes, 14.9% No) 2 Involved since 4.5±2.4 years Dedicated time (51±27)%

What is your current level of experience with ALICE analysis tools? 3 Mean 3.4

User Profile 4

Types of analysis 5

Analysis frameworks 6

How often do you run your analysis in the following way? ( 1 - rarely, 5 - very often) 7  Locally: 3.4  CAF or other analysis facility: 2  On a batch system: 2.7  On GRID: 3.4  In a LEGO train: 2.7

What do you use to deploy GRID analysis? 8

Quality of services (1 - lowest, 5 - highest): 9  easiness to develop a new task: 3.5  documentation of the analysis framework: 2.4  documentation of services: 2.3  alien handler functionality: 3.1  easiness to deploy analysis and run on large data sets: 3.2  analysis tutorials: 2.9  existing example code or code of others: 4.2  available analysis pages: 2.9  analysis mailing list: 3.4

Time to complete analysis (hours) 10  Achieved: 22±19  Expected: 10±10  Is it fast enough?

Are you aware of the LEGO framework? 11 Yes: 77.8% No: 6.4% No answer: 15.8%

Do you use the LEGO framework? 12 Yes: 53% No: 42.9% No answer: 4.1%

Reasons why people do not use LEGO trains 13  Code under development/ not stable/ not ready for the LEGO trains.  Missing documentation/instructions  No ESD trains/Only AOD centric analysis  Don't believe in AOD analysis and use a batch farm  So far because my needs could not be satisfied  For new analysis code, I am running the analysis task in 'let-me-start' way, i.e. the output of my jobs is a big tree. I would check as much as possible cutting variables.

Reasons why people do not use LEGO trains 14  Local batch farm is more performing, easier to use  Better debugging possibilities  higher flexibility  faster processing  better control  Because most of what I do can be done off-grid, on (S)AF.  My analysis activities mostly involve detector studies and are mostly done on my laptop or batch systems  Running on CAF; there was not yet the lego train framework; new analysis testing  Merging of large outputs, more flexibility needed  Frequent change of parameters and code

How useful do you consider the LEGO framework (1 - not useful, 5 - very useful)? 15 Mean: 4.7

Do you need extra functionality from the analysis framework? 16

Required functionality 17  Easy possibility to write custom streams.  more details for basic selection  Larger size of the buffer for event mixing ? (Do not really know if this is doable easily)  Mainly I need better documentation. I think it's all written but the documentation directly linked to on the offline pages is all >3 years old. Several features are completely undocumented - or at least the documentation is so obscure as to effectively not exist.  Trigger information at analysis level. Smarter merging. Now it only starts once all jobs are finished. Why not start merging when the first files are created? And then just do bookkeeping. Now we are in a situation where people doe their own merging on local clusters since that saves you 2 days!

Required functionality 18  Have access from within AODs of general things useful for all analysis, like trigger configuration (to get scalers, downscaling, etc..) and LHC information (to get lumi, etc...). Those things are mostly run-based and not event-based.  While running the merging (with a user job) a subjob can fail due to a corrupted/inaccessible file. This implies a significant loss of statistics that could be avoided if this error could be automatically detected and the job resubmitted skipping that specific file.  Embedding of real data and MC digits is missing. We run custom jobs which simulates events, read ESD of real data, convert ESD objects to digits, merge data digits with MC digits, run reconstruction, produce a new ESD, then filter to AOD. Only after that we can use the analysis framework to analyzed such embedded AOD.

Priorities for the analysis framework (1 - low priority, 5 -high priority) 19  better user support: 3.6  more functionality: 2.8  development of common analysis tools: 3.6  better speed: 3.5  better documentation and examples: 4.4  better PROOF support: 3.0  better LEGO support: 3.1

Conclusions 20  The most important issue is the documentation  Some of the requests show that not everybody is ready to use AODs in the analysis. This needs additional investigations