1. Low Level Processing of MINOS Data. Andy Blake Cambridge University Friday March 31 st 2006

2. Overview Andy Blake, Cambridge UniversityCandMorgue Package, slide 2 The atmospheric analysis has been using data quality software: – “DcsFilter” package vetoes bad Far Detector data (HV or Coil trips). – “FarDetDataQuality” package calculates detector live time by counting the number of timeframes processed. This package also collates data quality information from raw data and passes it through the framework. This software seems to have performed well! – Analysed >1 year of Far Detector data and published the results! I wanted to develop (+ tidy up!) the software for wider use. – e.g. make it work on the Near Detector + useable for other analyses. I have now committed some new and improved software. – “CandDataQuality” and “CandDeadChip” classes in “CandMorgue” package. – “DataQualityFilter” module in “Filtration” package.

3. Overview Andy Blake, Cambridge UniversityCandMorgue Package, slide 3 RAW DATA GLOBAL FILTER MODULE DATA QUALITY MODULE LIVE TIME RAW DIGIT DATA BLOCK DATA QUALITY CANDIDATE DATABASE CAND DIGITS DAQSNARL LIGHT INJECTION DAQ MONITOR Design of New Data Quality Software: GLOBAL FILTER MODULE: Pass/Fail data each second based on status of detectors, beam etc… Calculate live time by counting number of timeframes that pass these conditions. DATA QUALITY MODULE: Extract information from raw data blocks and package into a new candidate.

4. Global Filter Andy Blake, Cambridge UniversityCandMorgue Package, slide 4 Global filter: pass events if a specified set of data quality tests are satisfied (e.g. require that both beam and detector are in a good state). PHYSICS DATA BEAM DETECTOR GOOD PASS / FAIL BAD PASS

5. Global Filter Andy Blake, Cambridge UniversityCandMorgue Package, slide 5 Filter=0; if( ( !CheckBeam || this->GetBeamStatus(vldc)==1 ) && ( !CheckDetector || this->GetDetectorStatus(vldc)==1 ) ){ Filter=1; } Check the Coil / HV tables: Check the BeamMonSpill table: const BeamMonSpill& BMspill =...; BMSpillAna BMana; BMana.SetSpill(BMspill); BMana.SetTimeDiff(...); return BMana.SelectSpill(); // waiting to be filled (1) Pass or fail events depending on outcome of specified data quality tests:

6. Live Time Calculation Andy Blake, Cambridge UniversityCandMorgue Package, slide 6 (2) Write out detector and beam status after each second to calculate live time: currentTime=vldc.GetTimeStamp().GetSec(); If( currentTime>Time){ Time=currentTime; this->WriteOutResults(); } ************************************************************************************************************* * TimeFrame * DetectorStatus * BeamStatus * LiveTime * ProtonsOnTarget * CheckBeam * CheckDetector * Filter * ************************************************************************************************************* * 1 * 1 * 1 * 1 * 21.607900 * 0 * 1 * 1 * * 2 * 1 * -1 * 1 * 0 * 0 * 1 * 1 * * 3 * 1 * 1 * 1 * 21.464899 * 0 * 1 * 1 * * 4 * 1 * -1 * 1 * 0 * 0 * 1 * 1 * * 5 * 1 * 1 * 1 * 21.450599 * 0 * 1 * 1 * * 6 * 1 * -1 * 1 * 0 * 0 * 1 * 1 * Output File (Live time = number of entries that pass specified data quality tests) :

7. Running the Global Filter Andy Blake, Cambridge UniversityCandMorgue Package, slide 7 { gSystem->Load(“libFiltration”); JobC j; j.Path.Create( “reco”, “DataQualityFilter::Ana “ // REST OF RECONSTRUCTION CHAIN // GOES HERE ); j.Path(reco).Mod(“DataQualityFilter”).Set(“FilterOnOff=1”); j.Path(reco).Mod(“DataQualityFilter”).Set(“FilterBadDetector=1”); } Configure filter to veto events when detector is bad Run filter at start of reconstruction

8. Data Quality Module Andy Blake, Cambridge UniversityCandMorgue Package, slide 8 The “DataQualityModule” runs at beginning of reconstruction chain. The module watches the raw data blocks, extracts the monitoring information, and packages up this information as a new candidate. – It outputs a “CandDataQualityHandle” with each RawDigitDataBlock. A collection of data quality classes are used to process each of the different types of raw data block and extract the useful information. – The data quality objects are member variables of DataQualityModule. – The data quality objects are passed in CandContext to AlgDataQuality which collates all the data quality information. The overall data quality is assessed and a list of “CandDeadChipHandles” is produced detailing each bad chip (hot, cold, busy, in error etc…).

9. DataQualityModule Andy Blake, Cambridge UniversityCandMorgue Package, slide 9 TIter momitr(mom->GetFragmentIter()); while( momobject = momitr() ){ RawRecord* rawrec = dyanamic_cast (momobject); this->ProcessRawHeader( rawrec->GetRawHeader() ); TIter rawitr = rawrec->GetRawBlockIter(); while( rawobject = rawitr() ){ this->ProcessRawBlock( rawobject ); } These methods pass the raw headers and data blocks to internal data quality objects which extract the data quality information. (1) The Data Quality Module processes the Raw Data Blocks:

10. DataQualityModule Andy Blake, Cambridge UniversityCandMorgue Package, slide 10 Header Raw Digits Light Injection Hot/Cold Electronics Spill Server DaqSnarlHeader DaqHeaderBlock SnarlHeaderBlock RawDigitDataBlock RawLIAdcSummaryBlock RawLiTpmtDigitsBlock RawTpSinglesSummaryBlock RawSpillServerMonitorBlock Time, Run, Snarl, Timeframe number, Trigger source, Trigger time etc… Number of pre-trigger digits, Number of post-trigger digits, Busy chips, Readout errors. LI time, LI Pulser Box, LED, LI calib type, pulse height etc… Number of crates in readout, Cold chips, Hot chips. Spill type, status etc… Spill GPS error. RAW DATA BLOCKSDATA QUALITY INFO INTERNAL DATA QUALITY OBJECTS

11. CandDataQuality Andy Blake, Cambridge UniversityCandMorgue Package, slide 11 HeaderRaw Digits Light Injection Hot/Cold Electronics Spill Server AlgDataQuality CandDataQuality (2) Use internal data quality objects to make a CandDataQualityHandle: Collate the data quality info (N.B: some quantities need to be calculated using more than one of the internal data quality objects e.g. the LI time requires the TPMT times along with the snarl trigger time + some chips can be simultaneously hot, busy and in error!)

12. Int_t GetTime() const; Int_t GetRun() const; Int_t GetSnarl() const; Int_t GetTriggerSource() const; Int_t GetErrorCode() const; // etc... Int_t GetPreTriggerDigits() const; Int_t GetPostTriggerDigits() const; // etc... Int_t GetSpillStatus() const; Int_t GetSpillType() const; Int_t GetSpillTimeError() const; // etc... Int_t GetLiTime() const; Int_t GetLiPulserBox() const; Int_t GetPulserLed() const; // etc... Int_t GetColdChips() const; Int_t GetHotChips() const; Int_t GetBusyChips() const; // etc... CandDataQuality Andy Blake, Cambridge UniversityCandMorgue Package, slide 12 Information from header Information from raw digits Monitoring of spills Monitoring of light injection Monitoring of data quality (3) Here is some information available from the CandDataQualityHandle:

13. CandDataQuality Andy Blake, Cambridge UniversityCandMorgue Package, slide 13 typedef enum EDataQuality { kOkay = 0x00, kBad = 0x01, kLowMultiplicity = 0x02, kHighMultiplicity = 0x04, kLightInjection = 0x08, kTimeFrameBoundary = 0x10, kManyColdChips = 0x20, kManyHotChips = 0x40, kManyBusyChips = 0x80, kManyReadoutErrors = 0x100 } DataQuality_t < 10 raw digits > 1000 raw digits < 100 microsecs from TPMT hit < 100 microsecs from TF boundary > 20 cold chips > 20 busy chips Only the events with many cold chips, hot chips, busy chips or readout errors are considered bad. (4) The CandDataQuality contains an assessment of the overall data quality:

14. CandDeadChips Andy Blake, Cambridge UniversityCandMorgue Package, slide 14 typedef enum EChipStatus { kOkay = 0x00, kBad = 0x01, kCold = 0x02, kHot = 0x04, kBusy = 0x08, kError = 0x10 } ChipStatus_t (5) Each CandDataQuality contains a list of “CandDeadChipHandles” These record the details of any bad chips in the detector at the time of the event. < 50 Hz singles > 5000 Hz singles analyse pre-trigger. FAR DETECTORNEAR DETECTOR no definitions yet! stored in raw digits (6) Can ask the CandDataQualityHandle about its CandDeadChipHandles: How many bad chips are there? cdh->GetColdChips(), cdh->GetHotChips() etc. Is a given channel good or bad? cdh->GetChipStatus(RawChannelId rawch) etc.

15. Running the Data Quality Module Andy Blake, Cambridge UniversityCandMorgue Package, slide 15 { gSystem->Load(“libFiltration”); gSystem->Load(“libCandMorgue”); JobC j; j.Path.Create( “reco”, “DataQualityFilter::Ana “ “DataQualityModule::Reco “ // REST OF RECONSTRUCTION CHAIN // GOES HERE ); j.Path(reco).Mod(“DataQualityFilter”).Set(“FilterOnOff=1”); j.Path(reco).Mod(“DataQualityFilter”).Set(“FilterBadDetector=1”); } run the global filter and data quality module at the start of the reconstruction

16. Summary Andy Blake, Cambridge UniversityCandMorgue Package, slide 16 The first versions of data quality software are now in CVS. – Supercedes previous software used for the atmospheric analysis. The code will probably require further development. – Design and implementation may need a little tweaking (or a lot!) – Assessment of Near Detector data quality needs work. – Need to add in HV and Coil checks. Hopefully software has now been developed for more wider use.