Browsing Data with GaudiPython Disclaimers: 1.I am an apprentice! 2.The porpoise: we think GaudiPython could be quite useful for a LHCb physicist… Browsing.

Slides:



Advertisements
Similar presentations
Ideas on the LCG Application Architecture Application Architecture Blueprint RTAG 12 th June 2002 P. Mato / CERN.
Advertisements

6/4/20151 Introduction LHCb experiment. LHCb experiment. Common schema of the LHCb computing organisation. Common schema of the LHCb computing organisation.
LHCb Software Vanya BELYAEV Vanya BELYAEV. Preface It is NOT a tutorial for beginners It is NOT a tutorial for beginners If you need – the tutorial will.
Usage of the Python Programming Language in the CMS Experiment Rick Wilkinson (Caltech), Benedikt Hegner (CERN) On behalf of CMS Offline & Computing 1.
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
DaVinci status Juan Palacios LHCb Software Week March, 2009.
Introduction to Hall-D Software February 27, 2009 David Lawrence - JLab.
JAS3 + AIDA LC Simulations Workshop SLAC 19 th May 2003.
LC Software Workshop, May 2009, CERN P. Mato /CERN.
LHCb Core Software Meeting, 18 Jan /4 Containers in GaudiPython E. Rodrigues, NIKHEF Why >>> for i in range( hits.size() ): >>> for i in range(
Overview of LHCb applications and software environment LHCb software tutorial - March
Nick Brook Current status Future Collaboration Plans Future UK plans.
As of 28 Juni 2005Getting Starged with GEM - Shuei Yamada 1 Getting Started with GEM Shuei YAMADA ICEPP, University of Tokyo What is GEM? Before you start.
The report on the current situation of the BESIII framework zhangxiaomei maqiumei 10/3/2004.
G.Corti, P.Robbe LHCb Software Week - 19 June 2009 FSR in Gauss: Generator’s statistics - What type of object is going in the FSR ? - How are the objects.
Introduction to Gaudi LHCb software tutorial - September
Update on the new reconstruction software F. Noferini E. Fermi center INFN CNAF EEE Analysis Meeting - 15/06/20151.
Use of Gaudi in Reconstruction Weidong Li 23/06/2004.
Gaudi Framework Tutorial, April Algorithm Tools: what they are, how to write them, how to use them.
Alexander Richards, UCL 1 Atlfast and RTT (plus DCube) Christmas Meeting 18/12/2007.
Jose A. Hernando Trigger Gaudies Reconstruction Tools & Algorithms Inspectors MC & Data Algorithms Template preserved container Jose A. Hernando.
Reconstruction Configuration with Python Chris Jones University of Cambridge.
LHCb-ATLAS GANGA Workshop, 21 April 2004, CERN 1 DIRAC Software distribution A.Tsaregorodtsev, CPPM, Marseille LHCb-ATLAS GANGA Workshop, 21 April 2004.
Vanya BELYAEV Vanya BELYAEV (Syracuse) Vanya BELYAEV Tutorial Gaudi/DaVinci/LoKi/Bender.
Argonne Jamboree January 2010 Esteban Fullana AOD example analysis.
EGEE is a project funded by the European Union under contract IST “Interfacing to the gLite Prototype” Andrew Maier / CERN LCG-SC2, 13 August.
INFSO-RI Enabling Grids for E-sciencE Ganga 4 – The Ganga Evolution Andrew Maier.
GLAST LAT Offline SoftwareCore review, Jan. 17, 2001 Review of the “Core” software: Introduction Environment: THB, Thomas, Ian, Heather Geometry: Joanne.
Mantid Manipulation and Analysis Toolkit for ISIS data.
Gaudi Framework Tutorial, Interfacing Gaudi with Python.
Bender & Visualization (s) Smart&Transparent Python/LoKi-based Physics Analysis +4 demo Vanya Belyaev CERN & ITEP/Moscow.
1 SICBDST and Brunel Migration status and plans. 2 Migration Step 1: SICBMC/SICBDST split  Last LHCb week: Split done but not tested  Software week.
LHCb Software Week, 26th April /23 Tracking in LHCb E. Rodrigues, NIKHEF LHCb Software Week A Status Report.
Open project in Microsoft Visual Studio → build program in “Release” mode.
Tracking Event Model, Status 1.The plan, and the classes (again) 2.The packages modified or to be modified 3.Interactive reconstruction 4.Some ideas 5.Conclusion.
INFSO-RI Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.
CERN Tutorial, September Overview of LHCb applications and software environment.
Gaudi Framework Tutorial, Getting started with Gaudi and building tools.
Overview Methodology Design Architecture Outline of future work Ideas for discussion.
1 HLT (confirmation, generic)  Idea Reconstruct only a fraction tracks In hand:  better PT estimation  signal (secondary) vertices  Data TDR DaVinci.
27/02/04 ATLAS weekTileCal Athena tutorial, part 21 TileCal Athena tutorial Part 2: Reading GEANT hits from ZEBRA and POOL Reading HLT and testbeam ByteStream.
Python -based Physics Analysis Environment for LHCb G. Barrand, M. Frank, P. Mato, E. de Oliveira, A.Tsaregorodtsev, I. Belyaev CHEP 2004, Interlaken,
Bologna Tutorial, June Overview of LHCb applications and software environment.
Thomas Ruf, CERN EP Experience with C++ and ROOT used in the VX Beam Test Thomas Ruf, CERN, EP  Why? Event structure for VX-data rather complex: raw hits.
Nikhef Bfys Workshop Day One: Exploring a (m)DST Juan Palacios
Use of CMT in LHCb CMT Workshop, LAL (Orsay) 28 th February - 1 st March 2002 P. Mato / CERN.
Starting Analysis with Athena (Esteban Fullana Torregrosa) Rik Yoshida High Energy Physics Division Argonne National Laboratory.
Fermilab Scientific Computing Division Fermi National Accelerator Laboratory, Batavia, Illinois, USA. Off-the-Shelf Hardware and Software DAQ Performance.
ST Analysis: Introduction M. Needham EPFL. Outline Aims of the meeting Releasing code Performance Monitoring Results (IT and TT): Active fraction, noise.
Update on the new reconstruction software F. Noferini E. Fermi center INFN CNAF EEE Analysis Meeting - 28/09/20151.
LHCb Software Week 25/11/99 Gonzalo Gracia Abril 1 r Status of Geant4 in LHCb. r Ideas on how to populate the LHCb Detector Description Data Base (LHCb.
AOD example analysis Argonne Jamboree January 2010
Development Environment
PowerShell Introduction Copyright © 2016 – Curt Hill.
Migration of reconstruction and analysis software to C++
The LHCb Software and Computing NSS/IEEE workshop Ph. Charpentier, CERN B00le.
LHCb Software Tutorial
Status of CVS repository Production databases Production tools
1. Open Visual Studio 2008.
SICB under CMT Why? What is CMT? How to work with CMT? Package layout
Introduction to Athena
SCEC-VDO vtk Summer 2018 Objectives.
2 Getting Started.
2 Getting Started.
2 Getting Started.
Summary Computing Model SICb Event Model Detector Description
Java Analysis Studio and the hep.lcd classes
Status and plans for bookkeeping system and production tools
Tracking Event Model, Status
Production client status
Presentation transcript:

Browsing Data with GaudiPython Disclaimers: 1.I am an apprentice! 2.The porpoise: we think GaudiPython could be quite useful for a LHCb physicist… Browsing data with GaudiPython H. Dijkstra, JA. Hernando, T. Ruf,

GaudiPython: a gate… LHCb code is in Python GaudiPython: (most) Gaudi framework accessible via Python (GaudiPython v*) + the Event Model classes (Data) “Bender”: Extra Gaudi-classes + DaVinci Tools + ‘LoKi’ for analysis Why Python?: It is a simple, intuitive, powerful OO language Ads: ie.: no pointers, dynamically typed, heterogeneous containers, dictionaries It supports interactivity and introspection It is one for all: use for scripting, for programming, for interactive data-analysis GaudiPython (we have used for… but we can do more…): XXX?Enviroment: we use PanoramixEnv Creating a simple Application Browsing Data Quick way to develop monitoring algorithms (histograms)… > source PanoramixEnv vXrY > cd ~/cmtuser/Vis/Panoramix/vXrY/cmt > source setup.csh

An illuminating example (T.Ruf): anaSpillOver.py Import Gaudi Create a Gaudi Application Select a Data file Get the Event/Histo Svc Book histos

An example: anaSpillOver.py, histogram monitoring algo Create your Algorithm Execution per event Get the clusters And fill the Histos Add your algorithm to the Application Run!!! Final Print out

An example: anaSpillOver.py: 1,2,3, how to run? ~>python anaSpillOver.py ~>python Python (#1, Jun , 11:39:13) >> import gaudimodule >> gaudi = gaudimodule.AppMgr(outputLevel = 2, jobptions = ‘spillOverDicts.opts) As an script! (console) At the prompt: (console) From the Emacs menu: # get the prompt: (menu) Python->start interpreter # Execute a region (fraction) or buffer (all): (menu) Python->execute region/buffer # define a fucntion, execute region and use it in the promt! > source PanoramixEnv vXrY > cd ~/cmtuser/Vis/Panoramix/vXrY/cmt > source setup.csh

The option files and the result, an Hbook file Dictionaries Algo and options Note: You can see histos and analyze ntuple from Python!! PyROOT (HIPYS)

GaudyPython extension in Bender: Vanya’s similar example From Bender import Gaudi Creating the ‘GaudiHisto’Algorithm Getting the data plot = (book + fill) histogram Configuring Gaudoi Creating my algorithm NOTE: Vanya’s will exports GaudiAlgorithms from Bender to GaudiPython

Interactivity and Introspection: ginspector.py  But…  What data is there?,  What methods has my data?  What are the options?  …  Exploring the data  What methods?  Module inspection  What functions?  What classes? >> evh = evt[‘Header’] >> dir(evh) >> print evh.evtNum() >> import ginspector >> help(ginspector)

ginspector.py: What is in the DST & TES?  Inspecting Data:  PATH “Header”  TYPE “EventHeader”  Methods: evtNum() runNum() >>ginspector.inspectData(evt,”dst_data.txt”) NOTE: in the next version: we can have also the arguments of the methods (gracias Pere)

Ginspector: What properties?  Inspecting Properties  Name and Type AppMgr iAlgorithm  List of properties >> ginspector.inspectProperties(gaudi,’dst_properties.txt’)

A repository of scripts and catalog, and tests  Different uses of Python: (one for all..) Scripting: 1.create joboptions Hugo’s stripping 2.stage data in castor 3.send jobs Programming: 1.Physics Analysis: Vanya’s Bender Interactive Analysis: Monitoring: spillOver (T.Ruf,H.D.) With Panoramix Reconstruction: (plans) debug: event by event Histos/Ntuple Analysis: PyROOT (HIPYS) it is time to replace your “kumacs”  In the repository :  A repository of packages python modules /python directory in the packages CMT will take care of PYTHONPATH  I catalog in the web: Python script name, author, description, comment  Test before a release (Brunel, DaVinci)  Some problems (linux): i.e RTTC a simple script did not (almost solved now)!  A set of meaningful GaudiPython test scripts before a release?

Other uses of GaudiPython  Visualization:  Steer Panoramix from a Python script What to draw and how: T. Ruf examples  Testing/debugin: Click in a object in Panoramric and get it in the Python prompt (i.e track) Operate it in Python: i.e remove measurements and refit  Tracking  Debuging/Prototyping the reconstruction event by event In the Python prompt: tracks, extrapolator, fitter  Geometry  Debugin/testing detector geometries/aligment From Bender we can import to and update DetectorDescription classes Debug/test Volumes, DetectorElements…

Enlarging the community LHCb code is in Python GaudiPython + Event Model classes (Data) + “Bender” Making it more powerful (Pere, Vanya): exposing to GaudiPython more features of Gaudi: NTuples importing fron Bender common modules: GaudiAlgorithm extending to DetDescription classes We think it is useful for the Physicist: Simple, intuitive, powerful, interactive language: scripts, programming, analysis Allows easy browse of the data: create histograms, ntuples convenient for: prototyping, debugging, analysis event per event Enlarging the community: a repository with pythons scripts and a catalog on the web (TR, JAH) standard meaningful test with some GaudiPython scripts before a release? (check of data size, spill over…etc)