QARTOD III Scripps Institution of Oceanography La Jolla, CA Quality Control, Quality Assurance, and Quality Flags Mark Bushnell, NOAA/NOS/CO-OPS November.

Slides:



Advertisements
Similar presentations
Future Directions and Initiatives in the Use of Remote Sensing for Water Quality.
Advertisements

NOS – National Ocean Service NWS – National Weather Service What is QARTOD? IOOC DMAC Steering Team Second Meeting January 2012 Washington, D.C.
Review and Rating Discharge Measurements David S. Mueller Office of Surface Water March 2010.
Hernan E. Garcia (U.S. NODC, IODE Group of Experts on Biological and Chemical Data Management and Exchange Practices) EDM Workshop 2014, Silver Spring,
National Commission for Academic Accreditation & Assessment Preparation for Developmental Reviews.
MARE 250 Dr. Jason Turner Hypothesis Testing II. To ASSUME is to make an… Four assumptions for t-test hypothesis testing:
QARTOD II Currents and Waves Charge to the Breakout Groups QARTOD II February 28 – March 2, 2005.
QARTOD III November 2-4, 2005 Metadata in the IOOS Community Julie Bosch NOAA Coastal Data Development Center QARTOD III November 2–4, 2005.
Operational Quality Control in Helsinki Testbed Mesoscale Atmospheric Network Workshop University of Helsinki, 13 February 2007 Hannu Lahtela & Heikki.
DATA INTEGRATION DEMONSTRATION QA/QC Focus Quality Assurance of Real Time Ocean Data QARTOD II Feb 28-March 2, 2005 National Federation Of Regional Associations.
QARTOD II Currents and Waves IN-SITU WAVE GROUP Kent Hathaway, Moderator QARTOD II February 28 – March 2, 2005.
QARTOD 2: Remote Currents Working Group Definitions: Level 1 – refers to radials Level 2 – refers to total vectors Level 3 – refers to higher level data.
QARTOD II Currents and Waves In-Situ Currents: Breakout Group Report Out QARTOD II February 28 – March 2, 2005.
SOWK 6003 Social Work Research Week 10 Quantitative Data Analysis
1 Presenter: Chien-Chih Chen Proceedings of the 2002 workshop on Memory system performance.
Senior Design – Acceptance Test Plan Review The goal is to: define the criteria for approving the application. Tightly coupled to the Requirements document.
Chapter 10 Verification Procedures. Objective In this module, you will learn: u How to define verification u What functions are part of HACCP plan verification.
Quality Control Standards for SeaDataNet Review status at 1 st Annual Meeting (March 2007) Review developments over last year Current status Future work.
IQuOD An International Quality Control Effort Tim Boyer EDM workshop September 10, 2014.
Guide to Using Message Maker Robert Snelick National Institute of Standards & Technology (NIST) December 2005
1 Quality Control Review of E3 Calculator Inputs Comparison to DEER Database Brian Horii Energy and Environmental Economics, Inc. November 16, 2006.
Towards a Standard for Real-time Quality Control Procedures for in situ Ocean Waves Richard Bouchard 1 and Julie Thomas 2 1.NOAA’s National Data Buoy Center.
Standardization and Test Development Nisrin Alqatarneh MSc. Occupational therapy.
Best Practices for Preparing Data Sets Non-CO2 Synthesis Workshop Boulder, Colorado October 2008 Compiled by: A. Dayalu, Harvard University Adapted.
Quality Control Project Management Unit Credit Value : 4 Essential
Understanding MYP Criteria
NEPTUNE Canada Workshop Oceans 2.0 Project Environment NEPTUNE Canada DMAS Team Victoria, BC February 16, 2009.
Workshop on QC in Derived Data Products, Las Cruces, NM, 31 January 2007 ClimDB/HydroDB Objectives Don Henshaw Improve access to long-term collections.
Editing RT QC flag in delayed mode ? Virginie Thierry DMQC 4 Toulouse, 28 septembre 2009.
Technical Working Group, II Teruko Manabe Steven Worley Miroslaw Mietus Shawn Smith Simon Tett Volker Wagner Scott Woodruff David Berry Liz Kent.
IV-3.1 JCOMMOPS SOT Technical Coordinator. 2 JCOMMOPS structure Programmes currently supported –Ship Observations Team (30% Mathieu Belbeoch) –Argo Profiling.
ARM Aerial Facilities BAECC Kickoff Meeting Feb Aircraft Data Local Share Drive ARM IOPshare (at Oak Ridge) ARM IOPshare (at Oak Ridge) Corrections.
Disclosure risk when responding to queries with deterministic guarantees Krish Muralidhar University of Kentucky Rathindra Sarathy Oklahoma State University.
Data Buoy Cooperation Panel Scientific and Technical Workshop Scripps Institution of Oceanography La Jolla, CA Recent Buoy Developments at NOAA/NOS/CO-OPS.
Chapter 1 Measurement, Statistics, and Research. What is Measurement? Measurement is the process of comparing a value to a standard Measurement is the.
Copyright © 2007 Pearson Education Canada 1 Chapter 21: Completing the Audit.
Quality control of daily data on example of Central European series of air temperature, relative humidity and precipitation P. Štěpánek (1), P. Zahradníček.
DATA QUALITY CONTROL Julie Thomas Coastal Data Information Program (CDIP) Scripps Institution of Oceanography CI Design Workshop October 2007 La.
Quality Control for the World Ocean Database GSOP Quality Control Workshop June 12, 2013.
CBS-SSB STATISTICS NETHERLANDS – STATISTICS NORWAY Work Session on Statistical Data Editing Oslo, Norway, September 2012 Jeroen Pannekoek and Li-Chun.
Sub-regional Workshop on Census Data Evaluation, Phnom Penh, Cambodia, November 2011 Evaluation of Age and Sex Distribution United Nations Statistics.
Statistical Expertise for Sound Decision Making Quality Assurance for Census Data Processing Jean-Michel Durr 28/1/20111Fourth meeting of the TCG - Lubjana.
Project quality management. Introduction Project quality management includes the process required to ensure that the project satisfies the needs for which.
2 nd Knowledge Area : Project Scope Management. Importance of Good Project Scope Management 1995 CHAOS study cited user involvement, a clear project mission,
1 Implementing QA/QC Standards for In Situ Ocean Sensors Using OGC-Sensor Web Enablement a.k.a. QARTOD to OGC a.k.a. Q2O Project Status Brief to NOAA IOOS.
Universiteit Antwerpen Conference "New Frontiers in Evaluation", Vienna, April 24th-25th Reliability and Comparability of Peer Review Results Nadine.
Analyzing wireless sensor network data under suppression and failure in transmission Alan E. Gelfand Institute of Statistics and Decision Sciences Duke.
Characterizing Rural England using GIS Steve Cinderby, Meg Huby, Anne Owen.
Special Considerations for Archiving Data from Field Observations A Presentation for “International Workshop on Strategies for Preservation of and Open.
Summary of QARTOD and Recent Workshops on QA/QC March 9-10, 2006.
Hernan E. Garcia (U.S. NODC, IODE Group of Experts on Biological and Chemical Data Management and Exchange Practices) 2nd IQuOD Workshop 2014, Silver Spring,
N ational C limatic D ata C enter Development of the Global Historical Climatology Network Sea Level Pressure Data Set (Version 2) David Wuertz, Physical.
Rick Walker Evaluation of Out-of-Tolerance Risk 1 Evaluation of Out-of-Tolerance Risk in Measuring and Test Equipment Rick Walker Fluke - Hart Scientific.
XBT fall rate corrections Summary of pre-meeting discussions Step 1: Making recommendations for future XBT data collection Step 2: Correction of the historical.
Norwegian Meteorological Institute met.no QC2 Status
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Data Quality Control and Visualization with Ocean Data View 4.
Data Sharing and Data Quality of Ocean Currents and Temperature: Towards an Operational Data Management System for the Southeast Region Southeast Regional.
SeaDataNet Technical Task Group meeting JRA1 Standards Development Task 1.2 Common Data Management Protocol (for dissemination to all NODCs and JRA3) Data.
Chapter 10 Verification Procedures.
Quality Assurance of Real-Time Oceanographic Data
Water Quality Monitoring -Quality Assurance/Quality Control-
Data Sharing and Data Quality of Ocean Currents and Temperature:
Marine Meteorology Quality Control at the Florida State University
Chapter 10 Verification Procedures.
Ocean Data View 4 - Data Quality Control Reiner Schlitzer
6.1 Quality improvement Regional Course on
GODAE Quality Control Pilot Project
Ocean Data View 4 - Data Quality Control Reiner Schlitzer
Revised Art 12 reporting format
A handbook on validation methodology. Metrics.
Presentation transcript:

QARTOD III Scripps Institution of Oceanography La Jolla, CA Quality Control, Quality Assurance, and Quality Flags Mark Bushnell, NOAA/NOS/CO-OPS November 2, 2005

QUALITY ASSURANCE Processes employed a priori to support the generation of high quality data  Sufficiently accurate, precise, reliable sensor w/ adequate resolution  Calibration, calibration checks, and/or in-situ verification  Proper deployment considerations (rugged, anti-foulant, corrosion, etc.)  Robust data communications  Maintenance intervals  Creation of QC process 

QUALITY CONTROL Follow on steps supporting the delivery of high quality data  Format, checksum  Timely arrival  Threshold checks (min, max, rate of change)  Neighbor checks, climatology checks, model comparisons  Signal/noise ratios  Verification of user satisfaction  Generation of data flags

QARTOD I – BREAKOUT GROUP FLAG STATEMENT It is important that an Aggregate Quality Level be delivered with each real-time data Record collected. While many scientific researchers will want to use the test-specific quality information, more common users (the workshop group postulated that their number would be about 85% of the total anticipated users of IOOS-type systems) of the real-time data systems may only be interested in a general or aggregate quality level. The group of “casual” users would probably want a very simple quality level of only a few choices. The data provider would map the test-specific flags to the aggregate quality level, using product- and system-specific mapping which is adequately described in the metadata. The group recommended a 4-level aggregate quality level designation.

QARTOD I – Summary The following minimum standards for QA/QC were agreed upon by the workshop participants for real-time ocean observations: 1.Every real-time observation that is distributed to the ocean community must be accompanied by a quality descriptor. 2. All observations should be subject to automated real-time quality tests. 3. The real-time quality is best described by an aggregate quality flag (a simple overall descriptor) and a detailed quality test record (indicative of the results of any individual quality tests applied) to suit both the common user and the real-time scientist. The Aggregate quality levels were recommended to be few (although the order is arbitrary): -9 = missing value 0 = quality not evaluated 1 = bad 2 = questionable/suspect 3 = good 4. The quality flags and quality test descriptions must be sufficiently described in the accompanying metadata.

QARTOD I Examples of such flagging schemes abound, including these sources: CSIRO XBT QC flags : GTSPP/document/codetbls/gtsppcode.htm WOCE QC flags: RVSMDC/woce/qccodes_NetCDF.shtml ARGO, IUGG also cited

QARTOD II – In Situ Currents breakout group QUESTION 2: What categories of real-time quality descriptor flags should be applied? The group agreed the categories of real-time quality descriptor flags should follow the QARTOD I recommendation: -9 = missing value; 0 = quality not evaluated; 1 = bad; 2 = questionable/suspect; 3 = good.

QARTOD II – Remote Currents breakout group QUESTION 2: What categories of real-time quality descriptor flags should be applied? The quality descriptor flags defined in QARTOD I are appropriate; however, an additional flag is required to identify interpolated (radial or total vector) data. “Interpolation” refers to filling values within Doppler spectra or within a range cell (at the radial level). This data quality flag does not apply to interpolations of Level-3 data products. Rather it indicates an interpolation technique was used to create the radial or total vector since this information may influence the utility or application of the data. -9 = missing value 0 = quality not evaluated 1 = bad 2 = questionable/suspect 3 = good 4 = interpolated data

QARTOD II – In Situ Waves breakout group The group generally concurred with the descriptor flags suggested by QARTOD I, with one modification (shown in bold): -9 = missing value 0 = quality not evaluated 1 = bad 2 = questionable/suspect 3 = passed real time QA/QC (as opposed to the suggested flag of “good”) Data with a flag of “3” should not be considered final since it has only passed real time QA/QC. We may need an additional flag for final QA’d data to be included with metadata, although this additional flag would not be considered real time. It should be noted that Datawell has its own set of descriptor flags, where 0 = good, 1 through 8 are different quality levels. The user can define a further set (e.g. 9+). However, these are not incompatible with Q1 standards and can be mapped to those flags for standardization.

QARTOD III - PREPARATION Global Temperature & Salinity Profile Program (GTSPP) flags International Standard Flag Set?

CodeMeaning 0No QC was performed 1Good data 2Probably good data 3Bad data that are potentially correctable 4Bad data 5Value changed 6Not used 7 8Interpolated value 9Missing value The whole set of GTSPP flags? Too many flags for real time operations?

NOAA/NOS/CO-OPS PORTS® PORTS® UNIFORM FLAT FILE FORMAT (PUFFF) Each file, besides the basic observed data, has several fields containing information regarding the quality of the data as determined on a real-time, single sample basis. These fields are the Data Quality Assurance (DQA) bit mask, Data Quality Class Code (DQCC), and Data Quality Action Codes (DQAC). The DQA bit mask immediately follows the data fields on line 7. It consists of 32 digits, each digit either a zero or a one (0 or 1). Bit 0 (zero) is the first character, bit 31 is the last character. If the character is ‘1', this means it failed a particular test. The meaning of the bits varies according to the data type. The DQCC and the DQAC follow the DQA bit mask. The DQCC is a three digit code. The first digit is ‘3' or ‘4'. If it is ‘3', there are no failure codes relating to real-time use of the data. If it is ‘4', the data failed the DQA in some way for real-time use. The next two digits are the number of DQAC’s following the DQCC. The DQAC’s are defined in Appendix IV.

NOAA/NOS/CO-OPS PORTS® PORTS® UNIFORM FLAT FILE FORMAT (PUFFF) Line 7: Water elevation (relative to mean lower low water (MLLW)) Standard deviation Outlier count DQA bit mask DQCC DQAC Note, that because the DQCC was 300, there were no DQAC’s following (3=no errors, 00=number of DQAC’s).

Death & taxes Gravity & corrosion Hints & allegations Maximum influence Noise Subtle suggestions QARTOD recommendations

The Life Aquatic with Steve Zissou – Team Zissou