Hovedside

CDI and QC at IMR Ann Kristin Østrem Sjur Ringheim Lid Tittelside

SeaDataNet (Maris) Download Manager CDI Mikado SeaDataNet DB User requests download Send by e-mail to Maris Download Manager CDI Lookup in coupling table and create file for download Mikado SeaDataNet DB Create CDI Unique local_id Copy of data (finished QC process) Sea2Data DB Data is collected from different sources such as ships, buoys etc. It is then put into our internal Sea2Data database and quality controlled. Calibrated and QC data is copied to the SeaDataNet database that is accessible for the Download Manager. We use the automatic generation of CDI files in Mikado with an xml configuration that connects to the SeaDataNet database. When the CDI’s are created we send them to Maris. User requests comes to the Download Manager and the DM then uses the coupling table in the SeaDataNet database to find the correct select statements to generate the files requested. QC data Ships Buoys …

SeaDataNet Database Contains Tables for CDI Coupling table Data table Extendable with different data types Currently hydrography and chemistry The SeaDataNet database has tables for the CDI metadata information, a coupling table used by the download manager and data tables for hydrography and chemistry. Other data tables can be added on in the future when new data types will become available. SeaDataNet DB

Sea2Data Database Internal database for data handling Contains quality controlled data Has information about cruises and the actual data collected on cruises Currently hydrographic, biological (fish), chemical, fisheries acoustic and cruise tracks Other data types planned for the future (e.g. plankton, underway data, fisheries statistics) The Sea2Data database is the database used at the Institute of Marine Research for storage scientific data and making them available to scientists working there. This database contains multiple version of data (raw data, calibrated data and quality controlled data) for different data types. It has been developed during the last years and will continue to be extended with new data types as time goes by. Sea2Data DB

Handling updates Copies in the SeaDataNet DB is checked against the Sea2Data DB (last_edited field) If the data has been updated we regenerate the data and generate a new CDI Same unique key Same local_cdi_id Sea2Data DB SeaDataNet DB Both databases contains a last_edited column that is used when checking for updates to the data. If the data available internally is updated this will be discovered the next time CDI’s are generated and a new version of the CDI will be generated.

Data handling procedure (TS) The Quality control procedure is depicted in the blue box on the right hand side of this data handling procedure. In it you can see that we run a program called VIKKHY and programs for calibration of the data based on watersamples take on board the ship.

Tools used in the process Quality control VIKHHY – IMR developed tool for QC (SAS Institute) Laboratory corrections – IMR developed tool (Pascal) Creation of CDI’s Populate local SeaDataNet database - IMR developed Java tool Generate CDI’s – Mikado Generate data files when ordered - Download Manager

Many thanks to BODC who shared their database schema and Mikado xml document for creating CDI’s We now have a PostgreSQL version of the Oracle schema that BODC uses so if anyone is interested please let us know (nmd@imr.no)