1. DAQ Data Processing Chain Vasilis Vlachoudis 25-27 Feb 2015

2. Account: ntofdaq@lxplus runDSTcreator.cc runDSTcreator.cc : acrontab every 5min C++ program parses the INPUT.INP and submits an lsf batch run DSTcreator with arguments the INPUT.INP parameters switch#.bat switch#.bat: acrontab in between the runDSTcreator shell script to overwrite the INPUT.INP with new parameters INPUT.INP INPUT.INP: input file with variable=value, parameters of runs to process DSTcreator DSTcreator: C program checking all castor files (within the range defined in INPUT.INP) if they are processed or not if not then submit another lsf batch calling the “proc2014.bat” script proc2014.bat proc2014.bat: shell script running the raw2dst program copy the output in castor and in /afs Current Status [1/2]

3. Account: ntofuser@lxplus raw2dst raw2dst: C++ program reading the raw files with ntoflib output: root: pulse fitting pulses routine (generic Petar’s UserInput.h, BaF 2, ppac’s) dst files creation ntoflib ntoflib: C helper library to access the data structures of the raw format with the rfio castor libraries UserInput.h UserInput.h: Fitting parameters for all detector types for the generic Petar’s routine perl scripts perl scripts merging together the root/segment files in a single root file per run Current Status [2/2]

4. UserInput.h All detector type parameters are in a single file

5. Complicated chain of shell/C++/C/perl programs, result of evolution over time (needs cleanup) Synchronization needed between users, to know which version/parameters are used for the pulse fitting  Needs a key person to launch the process. Users cannot easily test new parameters. All detectors (even the ones they are not interested needs reprocessing) DST files are huge. Are they useful? Do we need to automatically generate and keep them all? raw2dst access the raw data as stream of bytes  needs modification for 16bit or other type of data. Slow parameters not inside the root files. Needs correlation with the timestamp Primitive Versioning control: One subdirectory for every new version with all libraries copied inside. Multiple copies of libraries and programs. Verification of the process data. Who is checking them? Which version/parameters are the correct ones? Issues

6. Cleanup / rewrite all processing scripts chain up to raw2dst to provide the same/similar functionality SVN repositories ntoflib responsibility moves to EN-STI provides us with an API to access the data with a common way, regardless of the format they are stored benefit: we are accessing our data with a set of routines and we don’t mess with the RAW format. All our processing programs are transparent in the format changes Proposal: Phase I nTOFLIB -getFile() -parseFile() Changes: -C  C++ -Update CASTOR protocol rfio  xrootnTOFLIB -getFile() -parseFile() Changes: -C  C++ -Update CASTOR protocol rfio  xroot CASTORCASTOR EN-STI nTOF raw2dst -new lib Changes: -C  C++ -use API to access dataraw2dst -new lib Changes: -C  C++ -use API to access data scripts Changes: -scan /afs directory for input parameters -flag them when donescripts Changes: -scan /afs directory for input parameters -flag them when done

7. Rewrite the raw2dst  raw2root eliminate the dst from the automatic output (if not commonly used). Only on demand output one root file per run per segment per detector type End users, will provide (through some interface, web, script) the range of runs they want to process with which parameters ONLY for their detector Similar perl scripts will merge the data in a single root file if needed. Proposal: Phase II

8. Create a suite of validity checks of the data Correlations between the data for possible errors Integral quantities between detectors, pulse intensity, etc… Automatic creation of standard plots Available directly on the daq web site Proposal: Phase III