1. SCT DQ Training 11.02.11 DQ Training 2011 Dr. Petra Haefner

2. SCT DQ Training 11.02.11 Introduction What has not changed The tools / plots to check the data quality DQMD SCTDQWebTool DCS elog … What has changed The DQ flags were dismissed We have a defects database now The shifter only checks if defects are present The shifter does not decide on the severity of a defect (tolerable / intolerable) New Duty 36h calibration loop checks 2 Dr. Petra Haefner

3. SCT DQ Training 11.02.11 2010 Data Quality Flags 5 types of DQ flags GreenData good YellowData has recoverable problem RedData bad GreyToo few statistics / undefined BlackDisabled in Atlas partition 3 DQ Regions in SCT Barrel Endcap A Endcap C Dr. Petra Haefner 3

4. SCT DQ Training 11.02.11 2010 Data Quality Flags 5 types of DQ flags GreenData good YellowData has recoverable problem RedData bad GreyToo few statistics / undefined BlackDisabled in Atlas partition 3 DQ Regions in SCT Barrel Endcap A Endcap C Dr. Petra Haefner 4

5. SCT DQ Training 11.02.11 2010 DQ Duties Shifters: Look at every run to check for data flaws Decide if data is recoverable (e.g. during reprocessing)  yellow flag Decide if problem is severe enough that data should not be used anywhere  red flag Experts: Define guidelines for shifters Help in cases of doubt Follow up unclear problems Make sure “recoverable” problems are recovered during reprocessing Dr. Petra Haefner 5

6. SCT DQ Training 11.02.11 Limits of DQ Flags Most of the problems are reoccuring ones Should have a one-click mechanism for flagging Unusual problems not always clear if they cause issues or not (e.g. in tracking) Could be ok for some analyzers but not for others Difficult to handle new problems Difficult to change policy Dr. Petra Haefner 6

7. SCT DQ Training 11.02.11 Idea of Defects Database Put problem = defect itself into a database Decide further downstream (virtual flags) if defect causes issues or not Easily create new defects “on the fly” Changes of policy much easier Combined performance / physics groups can require “perfect” data, i.e. no defects at all or allow for certain flaws Created defects based on 2010 experience Dr. Petra Haefner 7

8. SCT DQ Training 11.02.11 Defect “Definition” A defect is everything that is out of the nominal data taking What happens if “the nominal” changes (e.g. HV settings, large detector losses)? This changes the baseline! It is not a defect! A new data period has to be started The new nominal has to go into the MC … Dr. Petra Haefner 8

9. SCT DQ Training 11.02.11 Defect Policy Numerical defects will have two limits The “perfect” data limit (no defects) The “crap” data limit (intolerable defects) The operation policy is accordingly No need to restart run for tolerable defects (e.g. 1 ROD off) For intolerable defects action should be taken to loose as few as data as possible Operation shifters must be aware of intolerable defects! Decision is always taken by Run Coordinators! 9 Dr. Petra Haefner

10. SCT DQ Training 11.02.11 Intolerable Defects We did not implement any defects (in advance) that were not present in 2010 That means most of the intolerable ones! Will give us time to set limits in accordance with e.g. tracking studies List is prepared and a new defect can be created within a day! Created “UNKNOWN” defect to assess these cases by the shifter Expert intervention needed! 10 Dr. Petra Haefner

11. SCT DQ Training 11.02.11 Defect Definitions Each defect consists of An identifier (“name”) A (short) description A virtual flag (tolerable / intolerable) Recoverable flag is set case by case Intolerable means: should not be used by any analysis (unless for very good reasons) Tolerable means: can be used by most analyses without problems 2 Defect Types Boolean: either present or not (e.g. standby) Numerical: limit on certain numerical value (e.g. hit efficiency) Dr. Petra Haefner 11

12. SCT DQ Training 11.02.11 Common Detector Defects SCT_UNCHECKED intolerable Run has not been checked for defects, yet Has to be cleared by the shifter with sign-off, otherwise data is not usable! Necessary to distinguish from defectless runs SCT_GLOBAL_UNKNOWN intolerable 2 reasons A) new defect that is not in the database, yet B) shifter does not know which defect is appropriate Needs expert help A) create new defect, decide on tolerable or not B) map to existing defects (i.e. change entry) SCT_GLOBAL_LOWSTAT tolerable Not enough statistics for DQ checks (< 200 tracks) Old “grey” flag SCT_GLOBAL_DISABLED intolerable (part of) SCT not in Atlas partition (should never happen!) Dr. Petra Haefner 12

13. SCT DQ Training 11.02.11 SCT Configuration Defects SCT_GLOBAL_STANDBY intolerable The “classic” one ;-) SCT_HV_NOTNOMINAL tolerable HV neither 50 V nor 150 V Put actual voltage in comment field SCT_THR_NOTNOMINAL tolerable Threshold not at 1 fC Put actual threshold in comment field SCT_TIMINGSCAN tolerable Can probably be used for most analyses Let tracking decide if they see flaws Put description (e.g. coarse timing scan) in comment field Dr. Petra Haefner 13

14. SCT DQ Training 11.02.11 SCT Numerical Defects SCT_RODOUT_1 tolerable one or more RODs off during data taking Let tracking look for inefficiencies SCT_EFF_LT99 tolerable Efficiency < 99.5 % Put rounded efficiency in comment field (e.g. 99%) Yes, we have a very good detector! Certainly no problem for most analyses Efficiency is normally a symptom for another defect!  find out what’s the underlying problem Dr. Petra Haefner 14

15. SCT DQ Training 11.02.11 SCT 2010 Efficiencies 160954; ECA; 1 ROD off 166658; SCT; compressed mode 167844; B; high # of errors layer 1,2 167963; SCT; toroid off, 50 ns bunch spacing, high beam backgrounds Barrel ECAECC Dr. Petra Haefner 15

16. SCT DQ Training 11.02.11 SCT Module Defects - Noise SCT_MOD_NOISE_GT40 tolerable # of noisy modules > 40 Limit: Noise Occupancy >100 x 10 -5 Put actual number of modules in comment field Dr. Petra Haefner 16

17. SCT DQ Training 11.02.11 SCT Module Defects - Errors SCT_MOD_ERR_GT40 tolerable # of modules with bytestream errors > 40 Limit: >50% errors Put actual number of modules in comment field Dr. Petra Haefner 17

18. SCT DQ Training 11.02.11 SCT Module Defects - Disabled SCT_MOD_OUT_GT40 tolerable # of modules excluded in DAQ > 40 Put actual number of modules in comment field Dr. Petra Haefner 18

19. SCT DQ Training 11.02.11 Settings / Cuts for Defects Realized that settings/cuts were different for different offline tools (DQMD / web display) Different settings as well between online / offline Documentation contains inconsistencies as well Unify all settings/cuts on same/similar plots/quantities and update documentation Please tell me if you find inconsistencies! 19 Dr. Petra Haefner

20. SCT DQ Training 11.02.11 Suggested Limits 20 Dr. Petra Haefner QuantityYellow LimitRed LimitCut Noisy Modules40120100 x 10 -5 Error Modules4012050% Disabled Modules40120 Efficiencies99.5%97% Should be implemented in DQMD (online / offline), OHP, SCTGUI, web display,… Yellow limits correspond to tolerable defects Red limits correspond to intolerable defects Limits are not fixed, yet  maybe subject to change!

21. SCT DQ Training 11.02.11 SCT Defects List 21 Dr. Petra Haefner

22. SCT DQ Training 11.02.11 SCT Prepared Defects List 22 Dr. Petra Haefner As said before, the limits are suggestions! Limits will be fixed if serious defects occur! Implemented lower limits in DQ checks to have a “pre-warning”

23. SCT DQ Training 11.02.11 “Expert” Work Add new defects to database Overwrite defect flagging (i.e. go from defect “present” to “absent”) Decide what happens with “recoverable” defects Decide what happens to “unknown” defects Create new DB defect change to already existing defects NOTHING should remain in this category, even if you think it can never happen again!!! Dr. Petra Haefner 23

24. SCT DQ Training 11.02.11 Dr. Petra Haefner Defect Entry Tool How-To 24

25. SCT DQ Training 11.02.11 Defect Entry Tool 25 Dr. Petra Haefner https://atlasdqm.cern.ch/defectentry-dev/?filter=SCT

26. SCT DQ Training 11.02.11 When you start… 26 Dr. Petra Haefner You are prompted for your nice account Choose database Test for test purposes (training / development) Production the “real” thing for DQ assessment Choose the tag Head for normal operation with Tier-0 data

27. SCT DQ Training 11.02.11 Defects Overview 27 Dr. Petra Haefner Check if there are already defects present From automatic checks (e.g. DCS flags) SCT_UNCHECKED always present before sign off If you agree with all defects in (or none present) Go to “Sign off a run” directly If you do not agree / if there are defects missing Go to “Upload”

28. SCT DQ Training 11.02.11 A “Real” Run Overview 28 Dr. Petra Haefner All 2010 LBs that had yellow / red DQ flags got a “2010NONGREEN” defect

29. SCT DQ Training 11.02.11 Removed Defects (“absent”) 29 Dr. Petra Haefner If a defect was deleted (  “absent”), you will see it stroke through in the list of defects

30. SCT DQ Training 11.02.11 Defect Entry Tool Filter “SCT” shows only the SCT defect subtree Expand the subtree if necessary Select the defect you want to add by clicking its checkbox Enter all information on the right side of the window  30 Dr. Petra Haefner

31. SCT DQ Training 11.02.11 Enter Defect Information 31 Dr. Petra Haefner Select Run for which the defect was present Only one run can be entered at a time! Enter LBs for which the defect was present Several LBs can be entered like this: “1-12,42,43,415-End” “1-End” selects the full run Note: LB ranges are inclusive

32. SCT DQ Training 11.02.11 Run Information 32 Dr. Petra Haefner As soon as you enter a run number, you will see the run information pop-up in the window Project tag Run start / stop time # of LBs, # of evts Stable beams? Online integrated luminosity

33. SCT DQ Training 11.02.11 Enter Defect Information 33 Dr. Petra Haefner Enter a comment like # of modules / RODs disabled (e.g. “42”) Efficiency (e.g. “Barrel, 96%”, “ECA, Disk 7, 93%”) Not nominal setting (e.g. “coarse timing scan”, “HV=100 V”, “threshold = 1.2 fC”) No trivial comments needed! (e.g. “Standby”)

34. SCT DQ Training 11.02.11 Enter Defect Information 34 Dr. Petra Haefner Select “present” (default) or “absent” Shifters should only use “present” “absent” is for overwriting / deleting defects (should be only done by experts!) Check “Expected recoverable?” box, if you think the defect can be recovered later (e.g. during reprocessing)  old “yellow” flag

35. SCT DQ Training 11.02.11 Upload Defect to Database 35 Dr. Petra Haefner Enter system password (see next page) Popup window summarizes defect info Click “OK” to store defect in database

36. SCT DQ Training 11.02.11 Sign Off a Run 36 Dr. Petra Haefner Type in the Run number you wish to sign off Select “SCT” from drop-down menu Enter system password “Inner Detector” at the moment Should be changed to an SCT password soon!

37. SCT DQ Training 11.02.11 Dr. Petra Haefner “Homework” 37

38. SCT DQ Training 11.02.11 Dr. Petra Haefner Test Defect Entry Tool Take some of the 2010 runs with defects and load them into the defects (test) database! 38

39. SCT DQ Training 11.02.11 List of 2010 Problems Runquery command: find time 31.03.2010-06.12.2010 and ready and ev 20k+ only 14 / 194 runs have defects other than Standby! RunLBRegionReasonFlag 15277763-88 Timing Scangreen 159224418-814 no significant physics effectgreen 16080052-87B1 ROD offgreen 160954226-273EA1 ROD offgreen 160958215-226EA5.1% bad due to errorsred 1657321-86, 500-542, 565-589same as DCSred 1657671-193, 576-606 same as DCSred 1660551-9 Pix test, SCT cluster size small, zero eff EA, ECred 1660561-21 SCT L2 stress test, non-standard datared 16796364-231 need to check eff calculation, ignoring 100 contaminationyellow 1698377Berror ROD (many modules)red 1698378-53B1 ROD offgreen 1698381-57B1 ROD offgreen 1659271-250, 832-847 same as DCSred 1699611-77, 410-428 same as DCSred 1599271-26 not enough statsgrey 1599341-101 not enough statsgrey 155569224-470EC1 module with timeout errorsgreen 169750274BBarrel green but EA, EC red!green Dr. Petra Haefner 39

40. SCT DQ Training 11.02.11 Outlook – Planned Changes Include automatic checks into defects DB e.g. copy DCS red flag to standby defect e.g. automatic defects based on DQMD flags What CAN we implement? What do we WANT to implement? Needs thorough control by DQ shifters! Create automatic history plots e.g. efficiencies, noise occupancies,… Monitor long-term changes Again, what do we want? Stopless recovery Less / shorter cases of RODs in error / excluded Automated module recovery Less modules with errors 40 Dr. Petra Haefner

41. SCT DQ Training 11.02.11 Dr. Petra Haefner Backup 41

42. SCT DQ Training 11.02.11 ID Subsystem Defects 42 Dr. Petra Haefner test

43. SCT DQ Training 11.02.11 Documentation Database concept & design ATL-COM-DAPR-2010-002 Talk by Peter Onyisi at DQ workshop http://indico.cern.ch/getFile.py/access?contribId=12&resId=0&materialId= slides&confId=117855 See following slides for a summary Demonstration of new tools in DQ Meeting http://indico.cern.ch/conferenceDisplay.py?confId=121253 SCT Defects TWiki https://twiki.cern.ch/twiki/bin/view/Sandbox/SCTDefectsDatabase Dr. Petra Haefner 43