The SeaDataNet data products regional temperature and salinity historical data collections S. Simoncelli 1, C. Coatanoan 2, O. Bäck 3, H. Sagen 4, S.

Slides:



Advertisements
Similar presentations
WP1 - Data collection and metadata compilation in sea regions Sissy Iona (HCMR/HNODC) EMODNET Chemistry th Steering Committee, 2-3 December 2014,
Advertisements

ECOOP Data Management System (T2.2/WP2) Declan Dunne 13 th February 2008, Athens.
1 NODC, Russia GISC & DCPC developers meeting Langen, 29 – 31 March E2EDM technology implementation for WIS GISC development S. Sukhonosov, S. Belov.
Status of CDI service (connections, NetCDF introduction, CDI ingestion service and set-up of CDI ingestion pilot) By Dick M.A. Schaap – Technical Coordinator.
11 MEDATLAS 2002: database and data management system for the long term monitoring of Mediterranean and Black seas EC-MAST Concerted Action (MAS3-CT /ERBIC20-CT
Status of advanced access and viewing services following SeaDataNet D5.6 and D8.7 By Dick M.A. Schaap – Technical Coordinator Barcelona – Spain, 19 – 20.
Quality Control Standards for SeaDataNet Review status at 1 st Annual Meeting (March 2007) Review developments over last year Current status Future work.
2 nd Training Workshop 4 – 5 June 2007 Common Data Index - CDI By Dick M.A Schaap Technical Coordinator SeaDataNet.
WP2: Data products generation in Mediterranean Sea EMODNET Chemistry Data Products, Experts Workshop Split (Croatia), June 2014 Sissy Iona, HCMR-HNODC.
Status of upgrading CDI service (user interface, harvesting via GeoNetwork, CDI interoperability options following SeaDataNet D8.7) By Dick M.A. Schaap.
Marine Data Management within EMODNet Chemistry project: data aggregation, quality control and products preparation A.S. Iona 1, P. Karagevrekis 1, S.Balopoulou.
JRA6: Black Sea Products. Objectives and expected impact: Preparation and dissemination of regional statistical products like mean, seasonal and monthly.
WP1: Data collection and metadata compilation in Mediterranean Sea EMODNET Chemistry 2, Kick-off meeting Trieste (Italy), 3-5 June 2013 Sissy Iona, HCMR-HNODC.
EMODnet Chemistry 2 Advanced services Progress by Deltares Service Contract MARE/2012/10 S By Dick M.A. Schaap – Technical Coordinator Istanbul.
OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 rd SeaDataNet training course – Ostende – June 2008 NEMO reformatting tool v1 M. Fichaut.
SeaDataNet2 Plenary meeting, 19th-20th September 2012, Rhodes SeaDataNet2 Plenary meeting THE EMODNET CHEMISTRY LOT. FROM THE EXPERIENCE OF THE THREE YEARS.
Needs for Data and Information Managing living and non-living resources, monitoring environmental changes in the sea and protecting the marine environment,
From Emodnet Chemical Pilot to Emodnet Chemistry 2 Matteo Vinci, Istituto Nazionale di Oceanografia e di Geofisica Sperimentale – OGS – NODC group, Dipartimento.
MyOcean2 First Annual Meeting – April 2013 WP 15 THEMATIC & ASSEMBLY CENTRE for In Situ MyOcean2 First Annual Meeting – Cork /16-17 April 2013.
EMODnet Chemistry 2 Service Contract MARE/2012/10 S Progress of the CDI service By Dick M.A. Schaap – Technical Coordinator Istanbul – Turkey,
Opendap dev - meeting, Boulder, Feb 2007 OPeNDAP infrastructure in European Operational Oceanography T Loubrieu (IFREMER) T Jolibois (CLS)
TTG, March 2014, Barcelona WP3: Capacity building and training.
Emodnet Chemistry Lot Guidelines for data products.
From Emodnet Chemical Pilot to Emodnet Chemistry 2 Matteo Vinci and Alessandra Giorgetti, – OGS – NODC group, OCE - MODEG meeting, Copenhagen, 4 and 5.
ODP Interoperability Package Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal MINCyT, Buenos Aires, Argentina, 7 – 11 October.
ODV for Data Products Reiner Schlitzer, Alfred Wegener Institute, Bremerhaven, Germany.
OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet annual meeting, Madrid, March 2009 How to prepare data for integration in SeaDataNet.
Annual Meeting, June , Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet.
Chemical lot - HOW: Infrastructure set up based on SeaDataNet V1 efficient distributed Marine Data Management Infrastructure; Principle of “ADOPTED AND.
EMODnet Chemistry 2 Service Contract MARE/2012/10 S How to make EMODnet Chemistry fit for purpose at system level By Dick M.A. Schaap – Technical.
Hellenic Centre for Marine Research (HCMR) MedOBIS - Ocean Biogeographic Information System for the Eastern Mediterranean and Black Sea.
EMODnet Chemistry 2 Technical progress and new challenges Service Contract MARE/2012/10 S By Dick M.A. Schaap – Technical Coordinator Venice,
Marine Strategy 2012, 14th-16th May 2012, Copenhagen Marine Strategy 2012 THE EMODNET CHEMISTRY LOT. AN OVERVIEW ON THE EXPERIENCE OF THE THREE YEARS PILOT.
1 1 Arctic and North Seas Products Coordinated by IMR P17, Helge Sagen JRA 8.
EMODNet Chemistry (MARE/2012/10). The portal should collect the following groups of chemicals: - in 3 matrices: water column, biota, sediment. - in all.
Data Product Catalogue for SeaDataNet and Emodnet Chemistry M. Treguer, T. Loubrieu, IFREMER.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV – New Developments.
Download Manager software Training Workshop Ostend, Belgium, 20 th May 2014 D.M.A. Schaap - Technical Coordinator.
EMODNet Physics + My Ocean – SeaDataNet Dick Schaap (MARIS) – Brussels – 1 March 2013.
Metadata generation, control and updating Peter Thijsse - MARIS.
Data Product Catalogue for SeaDataNet and Emodnet Chemistry M. Treguer, T. Loubrieu, C. GUYOT, IFREMER.
Data management dedicated for assessing and modeling of the Mediterranean and Black Seas ecosystem changes (SESAME integrated project). Isaac Gertman,
Metadata V1 By Dick M.A. Schaap – technical coordinator Oostende, June 08.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View Achievements and Future Developments.
Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research Ocean Data View and itsRole in SeaDatanet and its Role in SeaDatanet.
CDI Data Discovery and Access Service Dick Schaap (MARIS) – SeaDataNet Technical Coordinator RDA – Paris - Sept 2015.
DATA AGGREGATION AND PRODUCTS GENERATION
MIKADO – Generation of ISO – SeaDataNet metadata files
SeaDataNet tools NEMO, OCTOPUS, MIKADO
EMODnet Chemistry 3 Upgrading of the portal and its services By
SEXTANT PRODUCTS CATALOGUE
Data aggregation and products generation in the Mediterranean Sea
ODP Interoperability Package
By Dick M.A. Schaap – Coordinator
EMODnet – Bathymetry By
Using Ocean Data View for EMODnet Chemistry Reiner Schlitzer
Flanders Marine Institute (VLIZ)
WP2 Products in Mediterranean Sea
Dick M.A. Schaap – Technical Coordinator SeaDataNet Training Workshop
Dick M.A. Schaap – Technical Coordinator
Data aggregation and products generation
Data aggregation and products generation in the Mediterranean Sea
EMODNet Physics + My Ocean – SeaDataNet
Data collection and metadata population – harvesting
Upgrading the portal and its services
Updating the existing advanced viewing services
Steering Committee Meeting Amsterdam, Dec 2014
Generation of data products in the Mediterranean Sea
Status on Products Catalogue service
EGU Conference, Vienna, Austria, 8-13April 2018
Presentation transcript:

The SeaDataNet data products regional temperature and salinity historical data collections S. Simoncelli 1, C. Coatanoan 2, O. Bäck 3, H. Sagen 4, S. Scory 5, V. Myroshnychenko 6, D. Schaap 7, R. Schlitzer 8, S. Iona 9,10, M. Fichaut 2 (1) INGV, (2) Ifremer, (3) SMHI, (4) IMR, (5) RBINS, (6) METU, (7) MARIS, (8) AWI, (9) HCMR, (10) Univ. of Thessaly CENTRAL CDI 1. Data Harvesting 2. File and parameter aggregation 3. QC analysis 4. Analysis of data anomalies QCS INTRODUCTION SeaDataNet Quality Control Strategy (QCS) Quality Control Analysis Conclusions The data anomalies were organized in files by EDMO code and sent to the data providers asking for inspection and correction. Guidelines were provided to explain the information in the anomalies files, how to make corrections on the data and how to send back the resulting information in detailed reports. The confirmed anomalies were corrected within the CDI improving the quality of the database. Data Harvesting of T and S files was performed through a CDI Robot, a robot user that uses the CDI (Common Data Index) Data Discovery and Access Service to query, shop and retrieve data sets from the SDN distributed data centres in automatic way. Data query consisted in searching for all data sets with T&S, whose access was unrestricted or under SDN License. The Robot was triggered to start harvesting the related ODV files from the distributed data centres through the general CDI shopping mechanism RSM-DM (Request Status Manager and Download Manager). Quality Control analysis of T&S collections has been conducted at regional level by Regional Coordinators (RC) and an harmonized procedure was designed in collaboration with MyOcean In-situ TAC (Thematic Assemble Centre) to facilitate data and information flow. RC and In-situ TAC both identified data anomalies through their QC procedures and defined a common plan to improve the quality of SDN database content and consequently of the next data collection release. Data (SDN/ODV format) and csv files with CDI metadata were processed using ODV software in 2 steps. File aggregation: all data files were analysed determining the type of data (profile, trajectory or time-series) and the set of parameters. Data were added to the metadata enriched ODV collection importing the full set of CDI metadata, instrument information and references. Parameter aggregation: ODV software aggregated parameters using up-to-date versions of the P35 vocabulary and the AWI unit conversion database. SeaDataNet II project implemented and continuously refined a Quality Control Strategy aiming at improving the quality of the database content and creating the best products deriving from it. The QCS was originally implemented in collaboration with MyOcean2 and MyOcean Follow On projects in order to develop a true synergy at regional level to serve operational oceanography and climate change communities. The QCS consists of four main phases: The approach is iterative to facilitate the upgrade of the database and it allows versioning of data products through the release of new data collections at the end of each QCS loop and the generation of the derived climatological products after a certain time lag. During SDN II project the QCS loop was completed three times. A first trial loop produced V1 regional data collections that were not released. A second loop permitted to refine each technical phase and improve the quality of both the central CDI content and the data collections that were released as version V1.1. A third QCS loop gave origin to the V2 regional aggregated datasets. Temperature and Salinity data collections, one per each European marginal sea (Arctic Sea, Baltic Sea, North Sea, North Atlantic Ocean, Mediterranean Sea, Black Sea) were analysed at regional level to assess and report on their quality. A common and basic QC analysis was performed using ODV software ( Analysis of data distribution and data density to identify possible spatial gaps and missing data sets (FIG.1); Analysis of temporal (annual/seasonal) data distribution (FIG.1); TS scatter plots of the entire dataset and range check; Visual control of scatter plot of observations considering various SDN Quality Flags (QF) to identify wrong profiles (outliers, spikes)(FIG.2); Analysis Quality Flag statistics; Identification of stations falling on land; Identification of wrong or missing data. The scatter plots of the regional data sets highlighted the necessity of applying specific sub-regional checks (FIG.3), per areas and per depth, and stability check on density to point out observations with depth, T and S out of reasonable values. Some good data (QF=1,2) presented values out of ranges and their QFs were modified accordingly. Many data resulted not QCed (QF=0) but looked reasonable and were further analyzed. Qualified historical data collections of all unrestricted temperature and salinity measurements contained within SeaDataNet database were produced for the European sea basins (Mediterranean Sea, Black Sea, Baltic Sea, North Sea, North Atlantic Ocean, Arctic Sea) covering the time period Two versions of the data collections have been published within the framework of SeaDataNet II project and they represent a snapshot of the database content at two different times: V1.1 (January 2014) and V2 (March 2015). They are available through the SeaDataNet web catalog at and are all minted by permanent doi. V1.1 regional data sets have been used to compute regional monthly climatologies of T and S fields also available on the web catalogue ( FIG.1 Mediterranean V2 Data Collection: Data density map and annual time distribution FIG.2 Black Sea V2 Data Collection: (left) Temperature and Salinity (right) scatter plots of measurements with QF=1 before and after additional QC analysis. TEMPERATURE SALINITY QC FIG.3 North Atlantic V2 Data Collection: ODV example of visual data inspection (map,  0 versus depth, temperature versus depth, potential temperature versus salinity and salinity versus depth). SDN data population increased due to 3 QCS iterations that produced V1, V1.1 and V2 data collections (FIG.4). V2 regional collections show a general increment, especially in the Atlantic and North Sea regions due to the insertion of new data. The Black Sea presents a reduction of number of stations from V1.1 and V2 due to the removal of data from the central CDI after the Ukrainian crisis and duplicates elimination. The data decrease in the Arctic Ocean (-35%) is due to an error in the ODV format of the Norwegian time series detected through a more strict control by ODV software. Norwegian files have been corrected to be completely compliant with format specifications. FIG.4 Histogram representing SeaDataNet population increase per each sea basin computed analysing the number of data for V1, V1.1 and V2 regional data collections. The production of SDN historical data collections was an extensive and constructive exercise to manage more than 1 Million data and involving many people and institutions, 62 data centres and more than 300 data originators. The implemented Quality Control Strategy permitted to identify and correct lots of data and it highly improved the quality of SDN database. The deriving data collections present an increasing quality thanks to the additional QC analysis performed at regional level and can be used for further applications. V1.1 collections have been used to compute regional Temperature and Salinity climatologies (V1.1) already available through the SDN web catalogue at V2 climatologies are in progress.