European Southern Observatory

Slides:

Advertisements

Similar presentations

New Challenges in M&E Lets go. Scaling Up Monitoring & Evaluation Strategic Information PROGRAM GUIDANCE RESULT NEEDS OPPORTUNITIES Resources New directions.

Advertisements

Complexity Metrics for Design & Manufacturability Analysis

July 2006Instrument Scientist Retreat, Santiago Quality Control: closing the loop Reinhard Hanuschik, DFO/QC 1. Principles 2. Processes 3. Critical components.

Optics in Astronomy - Interferometry - Oskar von der Lühe Kiepenheuer-Institut für Sonnenphysik Freiburg, Germany.

Cornerstone Base Station for Calibration Management Applied System Technologies, Inc N.W. 53rd Street, Suite 101 Fort Lauderdale, Florida

Fast and Extensible Building Modeling from Airborne LiDAR Data Qian-Yi Zhou Ulrich Neumann University of Southern California.

Cornerstone Cal Station for Calibration Management Applied System Technologies, Inc N.W. 53rd Street, Suite 101 Fort Lauderdale, Florida

1 Databases in ALICE L.Betev LCG Database Deployment and Persistency Workshop Geneva, October 17, 2005.

Searching for correlations in global environmental noise Karl Wette, Susan Scott and Antony Searle ACIGA Data Analysis Centre for Gravitational Physics.

FMOS Observations and Data 14 January 2004 FMOS Science Workshop.

Software Metrics II Speaker: Jerry Gao Ph.D. San Jose State University URL: Sept., 2001.

CIRSGSFC April 2004 dej-1 CIRS Instrument Overview CIRS Team Meeting Observatoire de Paris - Meudon April 26 – CIRS Instrument Group presented.

VISTA pipelines summit pipeline: real time DQC verified raw product to Garching standard pipeline: instrumental signature removal, catalogue production,

Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark.

Short Course on Introduction to Meteorological Instrumentation and Observations Techniques QA and QC Procedures Short Course on Introduction to Meteorological.

18 April 2007 Second Generation VLT Instruments 1 VIRCAM & CPL: Lessons Learned Jim Lewis and Peter Bunclark Cambridge Astronomy Survey Unit.

Top Tactics for Maximizing GMP Compliance in Blue Mountain RAM Jake Jacanin, Regional Sales Manager September 18, 2013.

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence From Data Mining To Knowledge.

Data Quality Monitoring of the CMS Tracker

Physical Modelling of Instruments Activities in ESO’s Instrumentation Division Florian Kerber, Paul Bristow.

Memorandam of the discussion on FMOS observations and data kicked off by Ian Lewis Masayuki Akiyama 14 January 2004 FMOS Science Workshop.

Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 The big picture I Why do we take calibrations? SCIENCE observations.

Detector Monitoring as part of VLT Science and Data Flow Operations Wolfgang Hummel DMO/DFO/QC group Lander de Bilbao, SDD/pipeline group Andrea Modigliani,

STIS Closeout Plan Paul Goudfrooij 2005 HST Calibration Workshop, 10/26/2005.

Hearing Conservation Program Administration Part 4: Training & Evaluation Thomas W. Rimmer, ScD, CIH Fay W. Boozman College of Public Health University.

Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.

Quality assurance / Quality control system for the Greek GHG emissions inventory Yannis Sarafidis, Elena Georgopoulou UNFCCC Workshop on National Systems.

Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group.

Automated CBC Parameters

Automated CBC Parameters

Use of Hierarchical Keywords for Easy Data Management on HUBzero HUBbub Conference 2013 September 6 th, 2013 Gaurav Nanda, Jonathan Tan, Peter Auyeung,

Proposal for a self-calibrating and instrument-independent MOS DRS Carlo Izzo.

Solutions for Quality Control of multi-detector instruments SPIE Marseille, 25 June 2008 Solutions for Quality Control of multi-detector instruments and.

Andrew S. Budarevsky Adaptive Application Data Management Overview.

Investigative Study for NSS Chemistry Curriculum Raymond Fong 21 Nov 2007.

SCIOPS 2013 Reinhard Hanuschik, ESO Garching The VLT Quality Control Loop.

Improved and automated QC process [calChecker] [HC monitor] [histo]calCheckerHC monitorhisto April 27, 2009 Improved and automated QC process Reinhard.

Statistical Expertise for Sound Decision Making Quality Assurance for Census Data Processing Jean-Michel Durr 28/1/20111Fourth meeting of the TCG - Lubjana.

Any data..! Any where..! Any time..! Linking Process and Content in a Distributed Spatial Production System Pierre Lafond HydraSpace Solutions Inc

GLAST LAT Project LAT Instrument Analysis Workshop – Feb 27, 2006 Hiro Tajima, TKR Data Processing Overview 1 GLAST Large Area Telescope: TKR Data Processing.

Investigative Study for NSS Chemistry Curriculum Raymond Fong 3 November 2009.

C2d Data flow diagram BCD from SSC Texas SAO Quality Analysis and Improved Calibrated Data Mapping team.

Paul Bristow (ESO Instrumentation) Thanks to: Andrea Modigliani, Joël Vernet & Florian Kerber, Sabine Moehler (ESO) Paolo Goldoni, Frédéric Royer & Régis.

Configuration database status report Eric van Herwijnen September 29 th 2004 work done by: Lana Abadie Felix Schmidt-Eisenlohr.

Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 Quality Control Loop and Daytime Checks Reinhard Hanuschik,

Data Processing & Quality Control Group [ Faculty Meeting Sept Data Processing & Quality Control Quality Control Group head: Reinhard.

Why is the VLT very efficient? Fernando Comerón On behalf of Francesca Primas and Martino Romaniello.

November 1, 2004 ElizabethGallas -- D0 Luminosity Db 1 D0 Luminosity Database: Checklist for Production Elizabeth Gallas Fermilab Computing Division /

The DCS Databases Peter Chochula. 31/05/2005Peter Chochula 2 Outline PVSS basics (boring topic but useful if one wants to understand the DCS data flow)

1 Creating Situational Awareness with Data Trending and Monitoring Zhenping Li, J.P. Douglas, and Ken. Mitchell Arctic Slope Technical Services.

1 GlueX Software Oct. 21, 2004 D. Lawrence, JLab.

Quality Control of Soil Moisture and Temperature For US Climate Reference Network Basic Methodology February 2009 William Collins USCRN.

Device Security in Cognitive Radio

Precipitation Data: Format and Access Issues

From LSE-30: Observatory System Spec.

reduction data treatment for ARCS

Data Flows in ACTRIS: Considerations for Planning the Future

GLAST Large Area Telescope:

The JWST Exposure Time Calculator

Systems Medicine Automated Real-Time Quality Control of LC-MS Metabolomics Data: QC4Metabolomics stanstrup.github.io.

Proposal for LAT Year 1 Data Release Plan

Descriptive Statistics

CMS Pixel Data Quality Monitoring

Chapter 13 Quality Management

Assessing Quality of Paradata to Better Understand the Data Collection Process for CAPI Social Surveys François Laflamme Milana Karaganis European Conference.

CMS Pixel Data Quality Monitoring

SOA initiatives at Istat

Product Training Program

H.I. Reuter – DG EUROSTAT - E4 - GISCO

Presentation transcript:

European Southern Observatory Scoring: a novel approach towards automated certification of pipeline products Reinhard Hanuschik, Wolfgang Hummel, Mark Neeser, Burkhard Wolff Data Processing and Quality Control Group European Southern Observatory

Complexity and mass production VLT: 11 instruments VLTI: 2 instruments 2009/10: 2 survey telescopes roughly 50 modes about 150 data types most of them pipeline-supported

Complexity and mass production each data type or instrument component controlled by QC1 parameters metrics for instrument performance extracted by data pipelines stored in database data frequency varies on timescales between a day and a semester

Complexity and mass production two challenges … … the complexity challenge!

Complexity and mass production current VLT instruments: avg: 30 GB/night … the volume challenge! VIRCAM: OMEGACAM: 400 GB/night

The solution: The solution: scoring scoring: well-established outside astronomy … QC process: measure quality compare quality assess quality: score and certify

Scoring: the solution measure quality: done by automatic procedures pipelines other QC procedures implemented for CALIB data progress on SCIENCE data

Scoring: the solution compare quality: trending first step towards assessment: same/different behaviour as others?

Scoring: the solution assess quality: scoring compare new result to trending pick out outliers and flag them

Scoring: the solution required: thresholds statistical thresholds: e.g. +/- 3 sigmas + independent from specs - sensitivity variable control charts specified thresholds: e.g. lower/upper level + stable thresholds - requires careful configuration and thinking saturation noise

Scoring of pipeline products each new pipeline product is scored most relevant QC1 parameters: compare to trending, score as OK: 0 NOK: 1 (outlier) count scores, assign total score, e.g.

Scoring of pipeline products score report:

Scoring of pipeline products score reports for multi-detector instruments (CRIRES: N=4) … detailed information on demand!

Scoring of instrument health all scores go into database re-arrange them by instrument properties detector health calibration lamp performance system efficiency score-based instrument QC cal file1 scores FLAT instrument scores cal LAMPs cal file2 scores ARC

Scoring of instrument health example: detector properties for GIRAFFE instead of …

Scoring of instrument health and finally: www.eso.org/HEALTH

Conclusions scoring: automatic flagging of outliers useful to auto-certify pipeline products quick-look monitoring of instrument health powerful concept to handle large and complex data sets hierarchical approach: from global overview (instrument score) to QC1 parameter per product and detector (6 levels)