Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Data Quality Control and Visualization with Ocean Data View 4.

Slides:

Advertisements

Similar presentations

OBSERVATIONS & PRÉVISIONS CÔTIÈRES Training Workshop – Ostende – October 2009 REFORMATTING TOOLS Med2MedSDN and NEMO M. Fichaut.

Advertisements

th Avenue, Suite 200 Honolulu, HI (808) Software Services Systems Oceanic Imaging Consultants, Inc. Professional.

EDMODnet Chemistry - DIVA Products from the Baltic Region Örjan Bäck, Lotta Fyrberg, Martin Hansson, Magnus Wenzer Swedish Meteorological and Hydrological.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV Gridding Methods.

Hernan E. Garcia (U.S. NODC, IODE Group of Experts on Biological and Chemical Data Management and Exchange Practices) EDM Workshop 2014, Silver Spring,

QARTOD III Scripps Institution of Oceanography La Jolla, CA Quality Control, Quality Assurance, and Quality Flags Mark Bushnell, NOAA/NOS/CO-OPS November.

11 MEDATLAS 2002: database and data management system for the long term monitoring of Mediterranean and Black seas EC-MAST Concerted Action (MAS3-CT /ERBIC20-CT

Marine GIS Applications using ArcGIS Global Classroom training course Marine GIS Applications using ArcGIS Global Classroom training course By T.Hemasundar.

SDN2 First Training Course, Oostende IODE-PO, 2-6 July 2012 Metadata Directories Management Sissy Iona, HCMR/HNODC.

First Data Management Training Workshop, February, 2007, Oostende, Belgium 1 Quality control checks description First Data Management Training Workshop.

Quality Control Standards for SeaDataNet Review status at 1 st Annual Meeting (March 2007) Review developments over last year Current status Future work.

Synthesis of Incomplete and Qualified Data using the GCE Data Toolbox Wade Sheldon Georgia Coastal Ecosystems LTER University of Georgia.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View - Available Data Collections and Data Model.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View 4 - Data Quality Control.

2 nd Training Workshop 4 – 5 June 2007 Common Data Index - CDI By Dick M.A Schaap Technical Coordinator SeaDataNet.

WP2: Data products generation in Mediterranean Sea EMODNET Chemistry Data Products, Experts Workshop Split (Croatia), June 2014 Sissy Iona, HCMR-HNODC.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 rd SeaDataNet training course – Ostende – June 2008 NEMO reformatting tool v1 M. Fichaut.

SeaDataNet2 Plenary meeting, 19th-20th September 2012, Rhodes SeaDataNet2 Plenary meeting THE EMODNET CHEMISTRY LOT. FROM THE EXPERIENCE OF THE THREE YEARS.

1 The IODE Ocean Data Portal - current status and future Nikolai Mikhailov, Chair of IODE/JCOMM ETDMP National Oceanographic Data Centre, Russia Four Session.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View.

Common Data Index CDI V1 How to proceed By Dick M.A. Schaap – technical coordinator Madrid, March 09.

QUT Library EndNote : Managing images. Adding images to EndNote records With EndNote Version 7, images may be embedded within records The Figure, Chart.

OceanBrowser viewing service, overview and upgrades Alexander Barth (1), Charles Troupin (2), Aida Alvera Azcárate (1), Jean-Marie Beckers (1) (1) University.

TTG, March 2014, Barcelona WP3: Capacity building and training.

Reiner Schlitzer & Stephan Heckendorff Alfred Wegener Institute for Polar and Marine Research Ocean Data View – New Developments.

ODV for Data Products Reiner Schlitzer, Alfred Wegener Institute, Bremerhaven, Germany.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet annual meeting, Madrid, March 2009 How to prepare data for integration in SeaDataNet.

NOAA’s CENTER for OPERATIONAL OCEANOGRAPHIC PRODUCTS and SERVICES WALI Demo Colleen Fanelli Wednesday, October 23, 2012 FY13 Hydro Kick-off Meeting.

BOĞAZİÇİ UNIVERSITY DEPARTMENT OF MANAGEMENT INFORMATION SYSTEMS MATLAB AS A DATA MINING ENVIRONMENT.

Chemical lot - HOW: Infrastructure set up based on SeaDataNet V1 efficient distributed Marine Data Management Infrastructure; Principle of “ADOPTED AND.

Applications of XML Oz OceanAtlas Software and NOAA/PMEL.

EMODnet Chemistry 2 Service Contract MARE/2012/10 S How to make EMODnet Chemistry fit for purpose at system level By Dick M.A. Schaap – Technical.

Worldwide Protein Data Bank Common D&A Project Sequence Processing Modular Demo May 6, 2010 Project Deliverable.

SeaDataNet Pan-European infrastructure for Ocean & Marine Data management An Integrated research Infrastructure Initiative (I3) NA2: TA system Monitoring.

Hernan E. Garcia (U.S. NODC, IODE Group of Experts on Biological and Chemical Data Management and Exchange Practices) 2nd IQuOD Workshop 2014, Silver Spring,

GLOSS Training Workshop Course Japan Meteorological Agency May 15-26, 2006 Sea Level Data Processing with SLPR2 1. Introduction.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV – New Developments.

2 nd training course – Ostende – May, 2014 What’s new in NEMO 1.5?

Ocean Data View Course Objective: Import, quality control and visualize own or external datasets. What you learn: Large publicly available datasets exist.

Metadata V1 By Dick M.A. Schaap – technical coordinator Oostende, June 08.

The SeaDataNet data products regional temperature and salinity historical data collections S. Simoncelli 1, C. Coatanoan 2, O. Bäck 3, H. Sagen 4, S.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View Achievements and Future Developments.

EDMODnet Chemistry, Baltic Sea - Oxygen, Chlorophyll and contaminants - General issues Magnus Wenzer, Lotta Fyrberg, Örjan Bäck Swedish Meteorological.

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View and itsRole in SeaDatanet and its Role in SeaDatanet.

Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level 1 SI O S Svalbard Integrated Arctic.

IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal.

Validation of Metadata XML files SeaDataNet Training, June 2008 Presented by with contributions from Karen Vickers (BODC) Presented by Michèle Fichaut.

SeaDataNet Technical Task Group meeting JRA1 Standards Development Task 1.2 Common Data Management Protocol (for dissemination to all NODCs and JRA3) Data.

DATA AGGREGATION AND PRODUCTS GENERATION

SeaDataNet tools NEMO, OCTOPUS, MIKADO

NEMO – Reformating tool

Partner contribution:

Using Ocean Data View for EMODnet Chemistry Reiner Schlitzer

OCTOPUS – SeaDataNet Format conversion tool

Introduction to Ocean Data View

WP2 Products in Mediterranean Sea

ISRAMAR Work Flow for SeaDataNet

Tools for generation of SDN NetCDF (CFPOINT) files

Ocean Data View Reiner Schlitzer

Ocean Data View 4 - Data Quality Control Reiner Schlitzer

Pima Medical Institute Online Education

Ocean Data View 4 - Data Quality Control Reiner Schlitzer

Data aggregation and products generation

ODV Gridding Methods Reiner Schlitzer

Ocean Data View Spreadsheet Format Reiner Schlitzer

Ocean Data View Reiner Schlitzer

Generation of data products in the Norwegian and Barents Seas

Progress with transition to new SDF

EMODnet Chemistry 3 Generation of data products in the Baltic Sea

Presentation transcript:

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Data Quality Control and Visualization with Ocean Data View 4

Wish: All data are of highest quality. Reality…

Objective: Extend and utilize ODV‘s extensive graphical display capabilities and interactive controls for automatic and visual quality control and data quality flagging.

General data analysis and visualization softwareGeneral data analysis and visualization software >10,000 registered users>10,000 registered users

Ocean Data View - Version 4 Complete software re-coding to overcome many limitations of ODV3. Specifically…

LOCAL_CDI_ID EDMO_code Bot. Depth [m] … etc. required optional required by SeaDataNet …allow an unlimited number of numeric and text metavariables. Cruise Station Type YYYY-MM-DDTHH:MM:SS.SSS Longitude [degrees_east] Latitude [degrees_north]

Flag Description ODVGENERICQF GTSPP ARGO SEADATANET ESEAS WOD WODSTATION WOCEBOTTLE WOCECTD WOCESAMPLE QARTOD BODC PANGAEA SMHI no quality control Q*blank good value blank probably good value blank probably bad value K?? bad value K/B changed value R*| value below detection <<< value in excess >>> interpolated value T*| missing value N*B value phenomenon uncertain100A Q*B QFSetID= SEADATANET Set Description: SeaDataNet quality flags Reference: …support the SeaDataNet quality flag scheme.

Flag Description ODVGENERICQF GTSPP ARGO SEADATANET ESEAS WOD WODSTATION WOCEBOTTLE WOCECTD WOCESAMPLE QARTOD BODC PANGAEA SMHI no quality control Q*blank good value blank probably good value blank probably bad value K?? bad value K/B changed value R*| value below detection <<< value in excess >>> interpolated value T*| missing value N*B value phenomenon uncertain100A Q*B QFSetID= SEADATANET Set Description: SeaDataNet quality flags Reference: …support all important quality flag schemes and mappings between them.

ODV Integration into QC Process Data exchange via extended ODV spreadsheet format adopted by SeaDataNet NODCDataArchive QCwithODV

Data Editing and Quality Control easy spotting and identification of outliers or offsets easy spotting and identification of outliers or offsets painless editing of data value and quality flag painless editing of data value and quality flag logging of all value or flag modifications logging of all value or flag modifications automatic range checks and manual or automatic editing automatic range checks and manual or automatic editing

Manual Data Editing and Quality Control

Automatic Range Checks and Outlier Removal

All changes logged in file.

Using quality flags to filter the data by quality.

Integration of DIVA gridding software into ODV ODV will automatically… create all files for DIVAcreate all files for DIVA run DIVA mesh generation and estimationrun DIVA mesh generation and estimation read and display the gridded fieldread and display the gridded field

Example (3) - Filling large gaps…

Training: Total of 4.5 days of presentations and exercises at two SeaDataNet training courses at IOC Oostende.

Deliverable No Deliverable titleTaskLeaderForeseen Delivery date Nature 1Specifications for ODV Preliminary Version SDN1JRA3.2P15M6report 2ODV – Version SDN1 + documentation ready for beta-tests and first training course JRA3.2P15M10Software + electronic manual 3Specifications for ODV Version SDN2JRA3.3P15M18report 4ODV – Version SDN2JRA3.3P15M22Software + manual 5Specifications for remote sensing moduleJRA3.4P15M30report 6ODV - Version SDN3JRA3.4P15M34Software + manual 7ODV final documented version including maintenance after feedback from the training workshops JRA3.1P15M47Software + manual 8Contribution to Final ReportJRA3.1P15M59report JRA3 Deliverables:    

Future: Use of ODV/DIVA in Web Applications Web Application ODV/DIVA X/Y/Z Data Image Gridded Values

Component of Version 2 viewing servicesComponent of Version 2 viewing services Identify partner PIs and data centersIdentify partner PIs and data centers Start discussion of interface designStart discussion of interface design