1. Hovedside

2. CDI and QC at IMR
Ann Kristin Østrem
Sjur Ringheim Lid
Tittelside

3. SeaDataNet (Maris) Download Manager CDI Mikado SeaDataNet DB
User requests download
Send by to Maris
Download Manager
CDI
Lookup in coupling table
and create file for download
Mikado
SeaDataNet DB
Create CDI Unique local_id
Copy of data (finished QC process)
Sea2Data DB
Data is collected from different sources such as ships, buoys etc.
It is then put into our internal Sea2Data database and quality controlled.
Calibrated and QC data is copied to the SeaDataNet database that is accessible for the Download Manager.
We use the automatic generation of CDI files in Mikado with an xml configuration that connects to the SeaDataNet database.
When the CDI’s are created we send them to Maris.
User requests comes to the Download Manager and the DM then uses the coupling table in the SeaDataNet database to find the correct select statements to generate the files requested.
QC data
Ships
Buoys …

4. SeaDataNet Database Contains Tables for CDI Coupling table Data table
Extendable with different data types
Currently hydrography and chemistry
The SeaDataNet database has tables for the CDI metadata information, a coupling table used by the download manager and data tables for hydrography and chemistry.
Other data tables can be added on in the future when new data types will become available.
SeaDataNet DB

5. Sea2Data Database Internal database for data handling
Contains quality controlled data
Has information about cruises and the actual data collected on cruises
Currently hydrographic, biological (fish), chemical, fisheries acoustic and cruise tracks
Other data types planned for the future (e.g. plankton, underway data, fisheries statistics)
The Sea2Data database is the database used at the Institute of Marine Research for storage scientific data and making them available to scientists working there.
This database contains multiple version of data (raw data, calibrated data and quality controlled data) for different data types.
It has been developed during the last years and will continue to be extended with new data types as time goes by.
Sea2Data DB

6. Handling updates
Copies in the SeaDataNet DB is checked against the Sea2Data DB (last_edited field)
If the data has been updated we regenerate the data and generate a new CDI
Same unique key
Same local_cdi_id
Sea2Data DB
SeaDataNet DB
Both databases contains a last_edited column that is used when checking for updates to the data.
If the data available internally is updated this will be discovered the next time CDI’s are generated and a new version of the CDI will be generated.

7. Data handling procedure (TS)
The Quality control procedure is depicted in the blue box on the right hand side of this data handling procedure.
In it you can see that we run a program called VIKKHY and programs for calibration of the data based on watersamples take on board the ship.

8. Tools used in the process
Quality control
VIKHHY – IMR developed tool for QC (SAS Institute)
Laboratory corrections – IMR developed tool (Pascal)
Creation of CDI’s
Populate local SeaDataNet database - IMR developed Java tool
Generate CDI’s – Mikado
Generate data files when ordered - Download Manager

9. Many thanks to BODC who shared their database schema and Mikado xml document for creating CDI’s We now have a PostgreSQL version of the Oracle schema that BODC uses so if anyone is interested please let us know