1. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Quality Control Loop and Daytime Checks Reinhard Hanuschik, Head QC Garching Data acquired on Paranal transferred, processed and scored by QC Garching results fed back to Paranal  closed QC loop explain the tools & processes explain the shared roles & responsabilities

2. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013

3. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Two main QC tools for closed-loop QC: save last night: calibration completeness  calChecker save next night: calibration quality, instrument health  HC monitor

4. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Mission #1: calibration completeness (“save last night”) calibrations incomplete: science OBs to be repeated Completeness of calibrations: calibration plan –known, predictable, configured –can be checked automatically 1. calChecker

5. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 www.eso.org/CAL exists for all instruments evaluation for 7 last days (columns) and all science setups (rows) technically: works on headers (metadata), no pipeline processing needed running every 30 min as cronjob driven by science data [a second page exists for cal4cal, later] checks: –all science setups –follows the calibration plan –knows about validity and required number of files

6. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 evaluates each box in colors: –green (OK): all calibrations complete and within validity (e.g.: 5 BIAS & 3 FLATs & 1 ARC & 1 STD, all within validity) –yellow (NOK): calibrations complete but some outdated (e.g.: as above but 3 FLATs are 2 days old instead of 1 day) –red (MISS): calibrations incomplete, at least one missing (e.g.: 5 BIAS & 3 FLATs ok, 1 STD missing)

7. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 links to details –for each box: complete list of science files –prog_id, OB_id etc. for easy reference –OB grade and comment: useful for analysis

8. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 ANALYSIS of NOK or MISS cases –use analysis editor to provide analysis –you can override automatic NOK/MISS results to become OK you cannot turn OK into NOK/MISS!

9. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 calScores –quick overview of calChecker issues –if green: skip checking that instrument!

10. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 shiftleader page –quick overview of calChecker issues –blinking if analysis is missing Overview of calChecker scores

11. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 cal4cal (“calibrations for calibrations”) –same logics as for ‘science’ –checks e.g. skyflats required for STD data –also supported by calScores and analysis editor cal4cal page

12. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 long-term calibrations –maintenance (like detmon, darks etc.): not driven by science –taken less frequently than 7 days –monitoring by reminders: soft (validity not yet expired), hard (expired) –has also ‘complete overview’ and ‘how to execute’

13. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 tellTracker –module for certain instruments (X-Shooter, KMOS) –monitors telluric standard stars (having two hours validity only) –checks airmass and age during the night –can be used to follow the calib needs of science OBs –refreshed at high frequency (15 mins) tellTracker

14. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Links –raw data report –NLT: nightlogs –products: all processed calibration data [later] –last seven days, current date highlighted –yellow: science night number of science files indicated last date: daytime calibs … –status of daytime calibration queue

15. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Mission #2: calibration product quality (“save next night”) calCheckerHC monitor completeness calChecker quality 2. HC monitor

16. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 goes beyond completeness, includes quality checks requires pipeline processing incremental processing at QC Garching: 1x per hour all calibrations, no science after processing: extract QC1 parameters calculate quality scores for most important instrument components: put results on HC monitor page 2 flavours: full graphics (trending report, current value plus history) quick overview (scoring report) www.eso.org/HC

17. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 full version: trending plot (VIMOS BIAS stability) same report, score version (quick-look)

18. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 organisation: –many pages per instrument –sorted by main instrument components –attempt to have same look and feel across instruments e.g. VIMOS: –‘Common’: detector and global instrument properties –‘Imaging’ mode: filters, zeropoints –‘MOS’ mode: dispersion, resolution –‘IFU’ mode: same

19. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 navigation: vertical and horizontal –items from vertical navigation bar split into groups of reports –each of those links to a HC report –a report consists of one or several trending plots navigation

20. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 closeup views –click on any report to see the closeup closeup view

21. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 report features: –upper/lower thresholds –average values (mean or median) –outliers marked –date range marked Report features upper threshold lower threshold average (mean or median)

22. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Report may contain one or several parameter sets –distinguished by color, size, or symbol –HC setup: a calibration taken regularly to monitor the instrument, not driven by science data –typically feeds the HC monitor Report features for instance: red: values for a HC setup black: all other values

23. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Opslog datasets –if available: included for last 5 days –only serve as backup if data transfer is disturbed –have lower quality than final QC1 parameters –are usually overplotted by the final data Opslog data opslog data accessible here

24. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Comments: –general news: applies to HC monitor in general –instrument news: apply to all HC reports for that instrument –Report news: apply to this particular report (web interface for editing, see later) Comments

25. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Further information: –“This plot”  tutorial info for this plot –“General information”  short info about HC reports in general –also: link to QC pages with more tutorial information (later) This plot/General information

26. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Result table: –for each report one or several lines –listing: statistics (average, thresholds, number of data points) name of QC1 parameter, options for data downloads Result table

27. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Data downloads: –all plot data can be downloaded –date range: this plot, last year, all –also: link to QC1 database table with plotter/browser Result table: downloads

28. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Navigation –links to history, tutorial, scores (quick-look) history plots QC tutorial pages click for email contact to QC scientist switch to quick-look

29. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Navigation –same report etc. links as on calChecker –useful for deeper analysis

30. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 History –use navigation bar to browse through entire history of this report –some HC reports also have FULL plot

31. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 FULL reports –entire data history for that report –useful to discover long-term trends invisible in normal reports

32. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 QUICK-LOOK version: scores focusing on last 7 days reduced information: ok or nok best to start with quick-look pages and investigate red scores only if available, comments by QC scientist are displayed

33. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Score propagation –Scores have several levels (plot, report, group) –Scores are propagated upwards –all higher-level scores turn red if at least one lower-level score is red –plot score: last one counts!

34. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Score overview page –group and report scores collected but no details –you can focus on comments: - red scores only, or - all comments

35. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 big advantage of score hierarchies: within a second –instrument score green  you know within a second: all is OK! –if red: the score overview can tell you more

36. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Comments –use ‘edit’ button –dialog with options to create, update, or delete comment –accessible to you and QC  exchange of information enter the dialog either here:or on any HC report page: create comment:

37. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Comments update or delete a comment:

38. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Shiftleader’s page –as for calChecker –know about instrument issues in quick overview –has also versions collecting scores for last 7 and 30 days

39. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 HC monitor: focus on instrument health product quality monitor (“AB monitor”): each pipeline job scored Paranal: reduction block RB, Garching: association block AB basically the same one AB monitor per day linked to calChecker and to HC monitor: 3. AB monitors: details about quality calCheckerHC monitor products/ AB monitor

40. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 AB product monitor

41. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 automatic evaluation scheme: compliance with thresholds (similar to OB grading but automatic) counts outliers score report: all details about scores entered by QC scientist Scores on product monitor

42. Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 certification / rejection: interaction & analysis by QC scientist replaces visual checks for Paranal: no need to dig deep into this, it is mainly for QC Garching as long as scores are green, this page doesn’t need to be inspected update for new ABs: within one hour