SCT DQ Training DQ Training 2011 Dr. Petra Haefner

SCT DQ Training Introduction What has not changed The tools / plots to check the data quality DQMD SCTDQWebTool DCS elog … What has changed The DQ flags were dismissed We have a defects database now The shifter only checks if defects are present The shifter does not decide on the severity of a defect (tolerable / intolerable) New Duty 36h calibration loop checks 2 Dr. Petra Haefner

SCT DQ Training Data Quality Flags 5 types of DQ flags GreenData good YellowData has recoverable problem RedData bad GreyToo few statistics / undefined BlackDisabled in Atlas partition 3 DQ Regions in SCT Barrel Endcap A Endcap C Dr. Petra Haefner 3

SCT DQ Training Data Quality Flags 5 types of DQ flags GreenData good YellowData has recoverable problem RedData bad GreyToo few statistics / undefined BlackDisabled in Atlas partition 3 DQ Regions in SCT Barrel Endcap A Endcap C Dr. Petra Haefner 4

SCT DQ Training DQ Duties Shifters: Look at every run to check for data flaws Decide if data is recoverable (e.g. during reprocessing)  yellow flag Decide if problem is severe enough that data should not be used anywhere  red flag Experts: Define guidelines for shifters Help in cases of doubt Follow up unclear problems Make sure “recoverable” problems are recovered during reprocessing Dr. Petra Haefner 5

SCT DQ Training Limits of DQ Flags Most of the problems are reoccuring ones Should have a one-click mechanism for flagging Unusual problems not always clear if they cause issues or not (e.g. in tracking) Could be ok for some analyzers but not for others Difficult to handle new problems Difficult to change policy Dr. Petra Haefner 6

SCT DQ Training Idea of Defects Database Put problem = defect itself into a database Decide further downstream (virtual flags) if defect causes issues or not Easily create new defects “on the fly” Changes of policy much easier Combined performance / physics groups can require “perfect” data, i.e. no defects at all or allow for certain flaws Created defects based on 2010 experience Dr. Petra Haefner 7

SCT DQ Training Defect “Definition” A defect is everything that is out of the nominal data taking What happens if “the nominal” changes (e.g. HV settings, large detector losses)? This changes the baseline! It is not a defect! A new data period has to be started The new nominal has to go into the MC … Dr. Petra Haefner 8

SCT DQ Training Defect Policy Numerical defects will have two limits The “perfect” data limit (no defects) The “crap” data limit (intolerable defects) The operation policy is accordingly No need to restart run for tolerable defects (e.g. 1 ROD off) For intolerable defects action should be taken to loose as few as data as possible Operation shifters must be aware of intolerable defects! Decision is always taken by Run Coordinators! 9 Dr. Petra Haefner

SCT DQ Training Intolerable Defects We did not implement any defects (in advance) that were not present in 2010 That means most of the intolerable ones! Will give us time to set limits in accordance with e.g. tracking studies List is prepared and a new defect can be created within a day! Created “UNKNOWN” defect to assess these cases by the shifter Expert intervention needed! 10 Dr. Petra Haefner

SCT DQ Training Defect Definitions Each defect consists of An identifier (“name”) A (short) description A virtual flag (tolerable / intolerable) Recoverable flag is set case by case Intolerable means: should not be used by any analysis (unless for very good reasons) Tolerable means: can be used by most analyses without problems 2 Defect Types Boolean: either present or not (e.g. standby) Numerical: limit on certain numerical value (e.g. hit efficiency) Dr. Petra Haefner 11

SCT DQ Training Common Detector Defects SCT_UNCHECKED intolerable Run has not been checked for defects, yet Has to be cleared by the shifter with sign-off, otherwise data is not usable! Necessary to distinguish from defectless runs SCT_GLOBAL_UNKNOWN intolerable 2 reasons A) new defect that is not in the database, yet B) shifter does not know which defect is appropriate Needs expert help A) create new defect, decide on tolerable or not B) map to existing defects (i.e. change entry) SCT_GLOBAL_LOWSTAT tolerable Not enough statistics for DQ checks (< 200 tracks) Old “grey” flag SCT_GLOBAL_DISABLED intolerable (part of) SCT not in Atlas partition (should never happen!) Dr. Petra Haefner 12

SCT DQ Training SCT Configuration Defects SCT_GLOBAL_STANDBY intolerable The “classic” one ;-) SCT_HV_NOTNOMINAL tolerable HV neither 50 V nor 150 V Put actual voltage in comment field SCT_THR_NOTNOMINAL tolerable Threshold not at 1 fC Put actual threshold in comment field SCT_TIMINGSCAN tolerable Can probably be used for most analyses Let tracking decide if they see flaws Put description (e.g. coarse timing scan) in comment field Dr. Petra Haefner 13

SCT DQ Training SCT Numerical Defects SCT_RODOUT_1 tolerable one or more RODs off during data taking Let tracking look for inefficiencies SCT_EFF_LT99 tolerable Efficiency < 99.5 % Put rounded efficiency in comment field (e.g. 99%) Yes, we have a very good detector! Certainly no problem for most analyses Efficiency is normally a symptom for another defect!  find out what’s the underlying problem Dr. Petra Haefner 14

SCT DQ Training SCT 2010 Efficiencies ; ECA; 1 ROD off ; SCT; compressed mode ; B; high # of errors layer 1, ; SCT; toroid off, 50 ns bunch spacing, high beam backgrounds Barrel ECAECC Dr. Petra Haefner 15

SCT DQ Training SCT Module Defects - Noise SCT_MOD_NOISE_GT40 tolerable # of noisy modules > 40 Limit: Noise Occupancy >100 x Put actual number of modules in comment field Dr. Petra Haefner 16

SCT DQ Training SCT Module Defects - Errors SCT_MOD_ERR_GT40 tolerable # of modules with bytestream errors > 40 Limit: >50% errors Put actual number of modules in comment field Dr. Petra Haefner 17

SCT DQ Training SCT Module Defects - Disabled SCT_MOD_OUT_GT40 tolerable # of modules excluded in DAQ > 40 Put actual number of modules in comment field Dr. Petra Haefner 18

SCT DQ Training Settings / Cuts for Defects Realized that settings/cuts were different for different offline tools (DQMD / web display) Different settings as well between online / offline Documentation contains inconsistencies as well Unify all settings/cuts on same/similar plots/quantities and update documentation Please tell me if you find inconsistencies! 19 Dr. Petra Haefner

SCT DQ Training Suggested Limits 20 Dr. Petra Haefner QuantityYellow LimitRed LimitCut Noisy Modules x Error Modules % Disabled Modules40120 Efficiencies99.5%97% Should be implemented in DQMD (online / offline), OHP, SCTGUI, web display,… Yellow limits correspond to tolerable defects Red limits correspond to intolerable defects Limits are not fixed, yet  maybe subject to change!

SCT DQ Training SCT Defects List 21 Dr. Petra Haefner

SCT DQ Training SCT Prepared Defects List 22 Dr. Petra Haefner As said before, the limits are suggestions! Limits will be fixed if serious defects occur! Implemented lower limits in DQ checks to have a “pre-warning”

SCT DQ Training “Expert” Work Add new defects to database Overwrite defect flagging (i.e. go from defect “present” to “absent”) Decide what happens with “recoverable” defects Decide what happens to “unknown” defects Create new DB defect change to already existing defects NOTHING should remain in this category, even if you think it can never happen again!!! Dr. Petra Haefner 23

SCT DQ Training Dr. Petra Haefner Defect Entry Tool How-To 24

SCT DQ Training Defect Entry Tool 25 Dr. Petra Haefner

SCT DQ Training When you start… 26 Dr. Petra Haefner You are prompted for your nice account Choose database Test for test purposes (training / development) Production the “real” thing for DQ assessment Choose the tag Head for normal operation with Tier-0 data

SCT DQ Training Defects Overview 27 Dr. Petra Haefner Check if there are already defects present From automatic checks (e.g. DCS flags) SCT_UNCHECKED always present before sign off If you agree with all defects in (or none present) Go to “Sign off a run” directly If you do not agree / if there are defects missing Go to “Upload”

SCT DQ Training A “Real” Run Overview 28 Dr. Petra Haefner All 2010 LBs that had yellow / red DQ flags got a “2010NONGREEN” defect

SCT DQ Training Removed Defects (“absent”) 29 Dr. Petra Haefner If a defect was deleted (  “absent”), you will see it stroke through in the list of defects

SCT DQ Training Defect Entry Tool Filter “SCT” shows only the SCT defect subtree Expand the subtree if necessary Select the defect you want to add by clicking its checkbox Enter all information on the right side of the window  30 Dr. Petra Haefner

SCT DQ Training Enter Defect Information 31 Dr. Petra Haefner Select Run for which the defect was present Only one run can be entered at a time! Enter LBs for which the defect was present Several LBs can be entered like this: “1-12,42,43,415-End” “1-End” selects the full run Note: LB ranges are inclusive

SCT DQ Training Run Information 32 Dr. Petra Haefner As soon as you enter a run number, you will see the run information pop-up in the window Project tag Run start / stop time # of LBs, # of evts Stable beams? Online integrated luminosity

SCT DQ Training Enter Defect Information 33 Dr. Petra Haefner Enter a comment like # of modules / RODs disabled (e.g. “42”) Efficiency (e.g. “Barrel, 96%”, “ECA, Disk 7, 93%”) Not nominal setting (e.g. “coarse timing scan”, “HV=100 V”, “threshold = 1.2 fC”) No trivial comments needed! (e.g. “Standby”)

SCT DQ Training Enter Defect Information 34 Dr. Petra Haefner Select “present” (default) or “absent” Shifters should only use “present” “absent” is for overwriting / deleting defects (should be only done by experts!) Check “Expected recoverable?” box, if you think the defect can be recovered later (e.g. during reprocessing)  old “yellow” flag

SCT DQ Training Upload Defect to Database 35 Dr. Petra Haefner Enter system password (see next page) Popup window summarizes defect info Click “OK” to store defect in database

SCT DQ Training Sign Off a Run 36 Dr. Petra Haefner Type in the Run number you wish to sign off Select “SCT” from drop-down menu Enter system password “Inner Detector” at the moment Should be changed to an SCT password soon!

SCT DQ Training Dr. Petra Haefner “Homework” 37

SCT DQ Training Dr. Petra Haefner Test Defect Entry Tool Take some of the 2010 runs with defects and load them into the defects (test) database! 38

SCT DQ Training List of 2010 Problems Runquery command: find time and ready and ev 20k+ only 14 / 194 runs have defects other than Standby! RunLBRegionReasonFlag Timing Scangreen no significant physics effectgreen B1 ROD offgreen EA1 ROD offgreen EA5.1% bad due to errorsred , , same as DCSred , same as DCSred Pix test, SCT cluster size small, zero eff EA, ECred SCT L2 stress test, non-standard datared need to check eff calculation, ignoring 100 contaminationyellow Berror ROD (many modules)red B1 ROD offgreen B1 ROD offgreen , same as DCSred , same as DCSred not enough statsgrey not enough statsgrey EC1 module with timeout errorsgreen BBarrel green but EA, EC red!green Dr. Petra Haefner 39

SCT DQ Training Outlook – Planned Changes Include automatic checks into defects DB e.g. copy DCS red flag to standby defect e.g. automatic defects based on DQMD flags What CAN we implement? What do we WANT to implement? Needs thorough control by DQ shifters! Create automatic history plots e.g. efficiencies, noise occupancies,… Monitor long-term changes Again, what do we want? Stopless recovery Less / shorter cases of RODs in error / excluded Automated module recovery Less modules with errors 40 Dr. Petra Haefner

SCT DQ Training Dr. Petra Haefner Backup 41

SCT DQ Training ID Subsystem Defects 42 Dr. Petra Haefner test

SCT DQ Training Documentation Database concept & design ATL-COM-DAPR Talk by Peter Onyisi at DQ workshop slides&confId= See following slides for a summary Demonstration of new tools in DQ Meeting SCT Defects TWiki Dr. Petra Haefner 43