OBSERVATIONS & PRÉVISIONS CÔTIÈRES Metadata and Data Management Activities winthin NA 3-7 S. Iona (HCMR), D. Schaap (MARIS), L. Rickards (BODC), F. Nast (BSH)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 2 Bologna, 19 September 2008  Outlines SeaDataNet Objectives Training and Capacity building Discovery System Maintenance Upgrade Current Status Summary

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 Bologna, 19 September 2008  SeaDataNet Objectives To network existing oceanographic data centres already nationally funded To develop an efficient distributed pan-European marine data management Infrastructure (a “unique Virtual Data Centre”) To provide on-line access to integrated databases of standardised quality by using adapted communication & information technology

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 4 Bologna, 19 September 2008  NA3-Training and Capacity building Aims: to ensure that the data and metadata to be integrated in the system will be formatted, checked for quality and disseminated according to the common protocols developed during the project to transfer expertise and to train IT experts of the SeaDataNet data centers in the basics, installation and operation of the SeaDataNet technical components

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 5 Bologna, 19 September 2008  NA3-Training and Capacity building Training Workshops, IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 1. February : focused on data re-formatting, quality control, and processing using ODV software. training on project metadata and related web services.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 6 Bologna, 19 September 2008  NA3-Training and Capacity building Training Workshops, IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 2. June : dedicated on generating XML records with the use of MIKADO tool

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 7 Bologna, 19 September 2008  NA3-Training and Capacity building Training Workshops, IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 3. June : use of the new V1 formats, interfaces and maintenance tools (MIKADO, online CMS, Web services, Validation services, Vocabularies) data quality control and assessments, using ODV software analysis and data presentations, using ODV – DIVA software

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 8 Bologna, 19 September 2008  Discovery System The SeaDataNet Discovery System is an integrated catalogue service, aiming at facilitating marine data searching, location and retrieval. The Objectives are to: Maintain and expand the national metadata-bases. Standardize the information using common vocabularies and reference tables. Interconnect the national inventories in a common Pan-European directory.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 9 Bologna, 19 September 2008  Discovery System It is composed of several thematic inventories of different levels: European Directory of Marine Environmental Datasets (EDMED) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Environmental Research Projects (EDMERP) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 10 Bologna, 19 September 2008  Discovery System Structure

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 11 Bologna, 19 September 2008 High Level Directories – EDMED, CSR, EDIOS for describing collected observation datasets (by ships, by laboratories, by continuous observing system)  Discovery System Structure

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 12 Bologna, 19 September 2008 Common Reference Tables – EDMERP, EDMO hold research projects and organizations metadata common to higher directories  Discovery System Structure

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 13 Bologna, 19 September 2008 The Common Data Index (CDI) provides access to the data, information and products by data type and/or any other field distributed by the TA platforms  Discovery System Structure

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 14 Bologna, 19 September 2008  Maintenance Version 0 – Continuation and maintenance of existing Sea-Search system :  the data access needs several different requests to each data centres  and the data sets are delivered in different formats

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 15 Bologna, 19 September 2008  Discovery System Upgrate Version 1 – Setup of the integrated online data services to users :  networking of 10 “interoperable” data centres of the Technical Task Team  unique request to the interconnected data centres  and the data sets are delivered with a unique format Progressive integration of 10 data centres by end of 2008

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 16 Bologna, 19 September 2008  Current Status Frozen directories Content upgrade from Version 0 to Version 1 using on-line (like CMS forms) and off-line tools that produce XML ISO compliant exchanges and developed by the Technical Task Team in the joint research activities JRA1,JRA2:

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 17 Bologna, 19 September 2008 Cruise Summary Reports (CSR = former ROSCOPs) are the usual means for reporting on cruises or field experiments at sea. Traditionally, it is the Chief Scientist's obligation to submit a CSR to his/her National Oceanographic Data Centre (NODC) not later than two weeks after the cruise. This provides a first level inventory of measurements and samples collected at sea.  Cruise Summary Reports

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 18 Bologna, 19 September 2008  New features in SDN V1 The major differences to previous V0 version are: CSR Local ID given by Data Centre for future updates -  makes it easier to modify/improve existing reports Most entities now have defined vocabularies eg. EDMO for organisation, EDMERP for projects and many more  no more spelling/typing mistakes or deviant interpretations Mandatory fields  improves quality of report

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 19 Bologna, 19 September 2008  Structure of Report Similar to the former ROSCOP forms the Cruise Summary Report has 4 basic parts: General Cruise Information Mooring Description Sampling/Measurement Description Information on Geographical Coverage

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 20 Bologna, 19 September 2008  How to submit CSRs? 2 methods for generating CSRs Online for individual entries (CMS) XML files for bulk submission with the use of MIKADO tool Both tools can be applied for new entries as well as updates !

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 21 Bologna, 19 September 2008  Online Content Management System Link to CSR Discovery

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 22 Bologna, 19 September 2008  How to update existing reports? Online update: contact BSH-DOD requesting CSRs to be updated BSH-DOD loads requested CSRs to entry database with list of CSR ref. no. and passwords modify CSRs in entry database save and submit

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 23 Bologna, 19 September 2008  How to update existing reports? Offline – XML update: contact BSH-DOD requesting CSRs to be updated BSH-DOD sends requested CSRs as XML V0 files - 1 file/cruise using BSH CSR ref. no. as file name - all free text information included submit modified CSRs in XML V1 format to BSH-DOD - use BSH CSR ref. no. as central identifier (CSR ID) - include local identifier (from NODCs) for future updates

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 24 Bologna, 19 September 2008  User Interface - Discovery Results

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 25 Bologna, 19 September 2008  User Interface - Report Download in V0 format

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 26 Bologna, 19 September 2008  Content Conversion into V1 Status of preliminary conversion for cruise reports: Responsible Laboratories (EDMO codes)~ 20% EDMO codes of Chief Scientists ~ 20% Ship/Platform (ICES codes)~ 60% Port of departure/return (C381):~ 30% General Ocean Area (C161+C162):~ 90% Sampling units (common vocab. L181)~ 20% EDMO codes of Principal Investigators ~ 3%

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 27 Bologna, 19 September 2008  Continuing operation and population Heraklion/Triest we got 1071 CSRs Online 33 % XML 34 % ICES 25 % Others 8 % Since Triest we got 1263 CSRs Online 52 % XML 44 % ICES 4 % Others 0 %

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 28 Bologna, 19 September 2008  Status Number of entries (in an Oracle DB): Characteristics: From , 2105 ships from 48 countries

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 29 Bologna, 19 September 2008  CSR Contents 2156 CSR Entries during SDN Project ( ) V1 in operation since 21. July 08 Russia and Spain with online upgrades 42 already using new V1 from 8 countries, Lithuania first after the 3rd SDN workshop in Oostende,

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 30 Bologna, 19 September 2008  European Directory of Marine Organizations The directory lists the organization profiles of all (1000+) Data Holding Centres, Research Institutes, Monitoring Agencies and Research Vessel operators, that have an active role in one or more of the SeaDataNet Discovery services (EDMED - data sets, EDMERP - research projects, CSR - research cruises, EDIOS - observing stations/ systems, and CDI - index to data). Direct crosslinks are provided to their entries in these directories. The organization entries are maintained online per country by the SeaDataNet partners. new Web service for retrieving EDMO entries in XML

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 31 Bologna, 19 September 2008  Online Content Management System EDMO CMS geo-locator via Google maps

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 32 Bologna, 19 September 2008  EDMO V1 search and retrieval

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 33 Bologna, 19 September 2008 EDMO  EDMO Contents Number of EDMO entries is quite stable As part of producing CDI records a lot of new organisations have been added; also existing entries have been altered due to re- organisations, double entries etc. Content upgrade from V0 to V1 is finished

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 34 Bologna, 19 September 2008 EDMO EDMED is a directory of data sets relating to the marine environment. It covers a wide range of disciplines including marine meteorology; physical, chemical and biological oceanography; sedimentology; marine biology and fisheries; environmental quality monitoring; coastal and estuarine studies; marine geology and geophysics etc.  European Directory of Marine Environmental Data

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 35 Bologna, 19 September 2008  EDMED Contents Currently, EDMED describes: over 3500 datasets from 700 data holding centres

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 36 Bologna, 19 September 2008  Total No. of Datasets and Data Holding Centres

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 37 Bologna, 19 September 2008  Progress Content Management System will be launched soon (Content upgrade will start then) Web interface is under development

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 38 Bologna, 19 September 2008  European European Directory of Marine Environmental Research Projects EDMERP is a European directory of research projects relating to the marine environment. It covers a wide range of disciplines including marine meteorology; physical, chemical and biological oceanography; sedimentology; marine biology and fisheries; environmental quality; coastal and estuarine studies; marine geology and geophysics etc.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 39 Bologna, 19 September 2008  EDMERP developments double entries have been taken out, because projects are entered and maintained by the country of the coordinator, who can add via EDMO entries all related partners capability of creation of sub-accounts for institutes in the NODC’s country, while the NODC safeguards the quality by having the chief editor role before publishing new Web service EDMERP entries in XML (export and import).

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 40 Bologna, 19 September 2008  EDMERP Retrieval Browse list Additional details

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 41 Bologna, 19 September 2008  EDMERP – CMS

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 42 Bologna, 19 September 2008  EDMERP Contents Number of EDMERP entries reduced during upgrading because of removal of duplicates

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 43 Bologna, 19 September 2008  European Directory of Ocean Observing Systems EDIOS is the European Directory of the Ocean-observing System, a unique searchable metadatabase of observing systems operating repeatedly, regularly and routinely in European waters. It contains metadata on European observing systems such as platforms, repeated ship-borne measurements, buoys, remote imagery, etc.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 44 Bologna, 19 September 2008  Progress No new input has been requested over the year Awaiting new technological developments (XML schema and new version of Mikado) BODC has produced an improved Oracle (database) schema Supported by common vocabularies, EDMO and EDMERP

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 45 Bologna, 19 September 2008 New User Interface:

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 46 Bologna, 19 September 2008  Common Data Index The CDI provides an index (metadatabase) to individual data sets. For comparison: the present European Directory of Marine Environmental Datasets (EDMED) gives an overview of datasets at a high metalevel. Each EDMED data set description covers a broad set of individual measurement data. The CDI gives references to these individual measurement data, providing a more detailed insight into the available datasets.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 47 Bologna, 19 September 2008  New interface

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 48 Bologna, 19 September 2008  CDI Contents Partners are now working with the new Mikado to upgrade V0 to V1 The IIrd training workshop has been very useful for transferring expertise and instructing partners in use of CDI format and tools.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 49 Bologna, 19 September 2008  Summary 2006/2007 EDMED: CSR: EDMERP: EDMO: CDI: /2008 EDMED: CSR: EDMERP: EDMO: CDI: A very important increase in the CDI contents but extra coverage of national data is need