OBSERVATIONS & PRÉVISIONS CÔTIÈRES Synthesis of the data management activities (NA) S. Iona (HCMR), R. Lowry (BODC), F. Nast (BSH), D. Schaap (MARIS)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 2 3 rd Plenary Annual Meeting, Madrid, March 2009  Overview Training and Capacity building Metadata Services in V1 System status and data integration in V1 - EDMED - CSR - EDIOS - EDMERP - EDMO - CDI Summary

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 3 rd Plenary Annual Meeting, Madrid, March 2009  Training and Capacity building Aims: to ensure that the data and metadata to be integrated in the system will be formatted, checked for quality and disseminated according to the common protocols developed during the project to transfer expertise and to train IT experts of the SeaDataNet data centers in the basics, installation and operation of the SeaDataNet technical components

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 4 3 rd Plenary Annual Meeting, Madrid, March 2009  Training and Capacity building Training Workshops, at IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 1. February : focused on data re-formatting, quality control, and processing using ODV software. training on project metadata and related web services.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 5 3 rd Plenary Annual Meeting, Madrid, March 2009  Training and Capacity building Training Workshops, IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 2. June : dedicated on generating XML records with the use of MIKADO tool

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 6 3 rd Plenary Annual Meeting, Madrid, March 2009  Training and Capacity building Training Workshops, IOC Rroject Office for IODE, Oostende, (Belgium) : (Training material and presentations available on SDN Extranet) 3. June : use of the new V1 formats, interfaces and maintenance tools (MIKADO, online CMS, Web services, Validation services, Vocabularies) data quality control and assessments, using ODV software analysis and data presentations, using ODV – DIVA software

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 7 3 rd Plenary Annual Meeting, Madrid, March 2009  Metadata Services The objectives are to: Maintain and expand the national metadata-bases. Standardize the information using common vocabularies (as Web Services) Interconnect the national inventories in a common Pan- European directory. One of the SDN Online Services to facilitate the marine data searching, location and retrieval.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 8 3 rd Plenary Annual Meeting, Madrid, March 2009  Metadata Services Structure It is composed of several thematic inventories of different levels: High Level Directories – EDMED, CSR, EDIOS (for describing observational datasets collected by ships, by laboratories, by continuous observing systems) Common References - EDMERP, EDMO and Common Vocabularies (available as Web Services) that interconnect the directories The Common Data Index (CDI) that gives access to the data distributed by the TA platforms (NODCs, Satellite Centers)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 9 3 rd Plenary Annual Meeting, Madrid, March 2009  System Maintenance – V0 Version 0: Continuation and maintenance of past Sea-Search system :  the data access needed several different requests to each data centre  and the data sets were delivered in different formats

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 10 3 rd Plenary Annual Meeting, Madrid, March 2009  System Upgrade to V1 Version 1: Setup of the integrated online data service to users :  networking the distributed data centres  unique request to the interconnected data centres  and the data sets are delivered with a unique format

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 11 3 rd Plenary Annual Meeting, Madrid, March 2009  Maintenance modalities and governance Maintenance and content upgrade from Version 0 to Version 1 by using on-line (like CMS forms) and off-line tools (MIKADO) that produce XML ISO compliant exchanges and developed by the Technical Task Team in the joint research activities JRA1, JRA2: Reformatting Tools now available: - Med2MedSDN - NEMO Generation of coupling table for DM, mapping download: - NEMO - Mikado V1.6 Training hour tomorrow

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 12 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Environmental Research Projects (EDMERP) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 13 3 rd Plenary Annual Meeting, Madrid, March 2009  European Directory of Marine Environmental Datasets (EDMED) V0 to V1 content changes - Content developed to be ISO19115-compliant through a profiling exercise Major change was that fields such as parameters were converted from plaintext to vocabulary term lists that could be implemented as ISO19115 keywords Some fields needed for mandatory ISO19115 core were added A small number of fields rendered redundant by plaintext replacement were dropped

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 14 3 rd Plenary Annual Meeting, Madrid, March 2009  European Directory of Marine Environmental Datasets (EDMED) Content conversion plan -A CMS has been provided by Maris ( - Partners upload V0 records into the CMS - CMS template guides conversion to V1 - V1 records exported as XML and ingested into BODC database

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 15 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Infrastructure Status Infrastructure Work Completed - V1 Oracle storage schema - V1 XML ingestion software - V1 Web Service export Infrastructure Work To Do - Upgrade presentation schema to V1 - Storage to presentation schema translator - V1 User Interface

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 16 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Content Conversion Status BODC (out of 3357!) V0 records are the responsibility of BODC Initial conversion completed by junior member of staff List of issues to be resolved by Lesley - Status of other partners’ progress checked on

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 17 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Content Conversion Status Countries with entries completed or nearing completion

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 18 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Content Conversion Status Countries with entries apparently not converted

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 19 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Content Conversion Status SUMMARY Records upgraded (as of 23 March): 1448 (45%) Countries supplying V1 records in XML (i.e. will not be converted in CMS): France, Germany, Ireland, Italy, Netherlands CLS (France) will contribute with V1 records in XML Romania, Portugal: large collections with no upgrade or submission of XML files Remaining records to be upgraded: ~430 (excluding BODC’s 700) Upgrade needs to be done very soon, assistance is available

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 20 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) European Directory of Marine Environmental Research Projects (EDMERP) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 21 3 rd Plenary Annual Meeting, Madrid, March 2009  EDMERP content upgrade Using CMS for upgrading V0 to V1 and also for adding new records Using MIKADO and XML import [available as webservice for XML V1 files (export and import) ]

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 22 3 rd Plenary Annual Meeting, Madrid, March 2009  EDMERP Web Interface - Retrieval Browse list Additional details

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 23 3 rd Plenary Annual Meeting, Madrid, March 2009  EDMERP CMS

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 24 3 rd Plenary Annual Meeting, Madrid, March 2009 STATUS OF EDMERP UPGRADING FROM V0 TO V1 LEVEL, 17-Mar % of the upgrade is done - the rest should finish the next months

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 25 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) European Directory of Marine Environmental Research Projects (EDMERP) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 26 3 rd Plenary Annual Meeting, Madrid, March 2009  CSR Status l ~ CSRs ( ) l 17 partners submitting new entries in V1 l in an Oracle DB l 16 entries from 2009 (Skandinavia)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 27 3 rd Plenary Annual Meeting, Madrid, March 2009  V1 Maintenance New entries (and updates) -Online for individual entries (CMS) -XML files for bulk submission with the use of MIKADO tool

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 28 3 rd Plenary Annual Meeting, Madrid, March 2009  CSR Content upgrade to V1 - Online contact BSH-DOD requesting CSRs to be updated BSH-DOD loads requested CSRs to entry database modify CSRs in entry database - Offline contact BSH-DOD requesting CSRs to be updated BSH-DOD sends requested CSRs as XML V0 files (free text entries) Use of MIKADO tool and submit modified CSRs in XML V1 format to BSH-DOD

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 29 3 rd Plenary Annual Meeting, Madrid, March 2009  Status of Upgrading to V1 7 Partners upgrading systematically using online application 17 partners submit new online entries in V1 5 partners submit new entries and upgrades in XML V1 Slovenia (NIB), Italy (OGS), UK (BODC), Turkey (IMS-METU) and France (IFREMER) altogether around 4800 completed CSRs in V1 format only 10% is in V1 !!! Remark: many project information which were originally in the V0 format get lost due to missing EDMERP entries

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 30 3 rd Plenary Annual Meeting, Madrid, March 2009 lbig new entries and upgrades of XML V1 submissions by 5 partners: France, UK, Italy, Turkey, Slovenia loperational new online entries in V1 by 17 partners

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 31 3 rd Plenary Annual Meeting, Madrid, March 2009 ISSUE lonly part (10%) of the total CSRs are now upgraded, mainly from the last decade lnot all partners started upgrading lnot all partners started using new online entries in V1 ACTION Suggestion lBegin with the latest data and go backwards lPartners get export to update those CSRs either online or by Mikado lPartners get a reminder, may need help?

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 32 3 rd Plenary Annual Meeting, Madrid, March 2009  New features in CSR User Interface Platform link to EurOcean RV Infobase CSR Online Entry - uploading of track charts, station lists and additional reports directly into database Organisation link to EDMO record XML download in V1 format for upgraded CSRs (being tested)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 33 3 rd Plenary Annual Meeting, Madrid, March 2009 CSR V1 Retrieval EurOcean RV Infobase link EDMO link Track chart

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 34 3 rd Plenary Annual Meeting, Madrid, March 2009 CSR V1 CMS Upload station list Upload reports Upload track charts

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 35 3 rd Plenary Annual Meeting, Madrid, March 2009 CSR V1 Retrieval

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 36 3 rd Plenary Annual Meeting, Madrid, March 2009  Planned Actions lInterconnecting Meta-catalogues: Link from CSR to EDMERP Link from EDMERP to CSR to CDI ? lPrintable form for CSR lPlanning CSR Web Service for use together with MIKADO lUse of POGO ship schedule information for preparing initial CSRs

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 37 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) European Directory of Marine Environmental Research Projects (EDMERP) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 38 3 rd Plenary Annual Meeting, Madrid, March 2009  Status No new input has been requested over the last year Improved Oracle (database) schema Supported by common vocabularies, EDMO and EDMERP ISO content model: Description of the format and XML tags available XML example file available Programme and Series XML Schema available Upgrade is underway MIKADO V1.6: for new entries and updates New user Interface available

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 39 3 rd Plenary Annual Meeting, Madrid, March 2009 New User Interface:

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 40 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) European Directory of Marine Environmental Research Projects (EDMERP) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 41 3 rd Plenary Annual Meeting, Madrid, March 2009  Maintenance Update of existing entries via CMS forms Access for NODC‘s New records only to be created via MARIS (because EDMO = vocabulary) Available as webservice (export only)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 42 3 rd Plenary Annual Meeting, Madrid, March 2009  European Directory of Marine Organizations Currently, EDMO lists and describes more than organizations. There is direct crosslink with EDMED - data sets, EDMERP - research projects, CSR - research cruises, EDIOS - observing stations/ systems, and CDI V0 and V1 - index to data. Content upgrade from V0 to V1 is finished

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 43 3 rd Plenary Annual Meeting, Madrid, March 2009  EDMO V1 search and retrieval

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 44 3 rd Plenary Annual Meeting, Madrid, March 2009  EDMO CMS EDMO CMS geo-locator via Google maps

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 45 3 rd Plenary Annual Meeting, Madrid, March 2009  System status and data integration in V1 European Directory of Marine Environmental Datasets (EDMED) European Directory of Marine Environmental Research Projects (EDMERP) Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing Systems (EDIOS) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 46 3 rd Plenary Annual Meeting, Madrid, March 2009  Common Data Index: gate to data access CDI V0 System: -More than CDI entries - Data requests through different requests at data centres - Data delivered at different formats CDI V1 System: -Unique request -order and access for downloading via a shopping basket mechanism and a communication between the SeaDataNet portal and the local systems -Data delivery in unique format

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 47 3 rd Plenary Annual Meeting, Madrid, March 2009  CDI V1 – Quick Search

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 48 3 rd Plenary Annual Meeting, Madrid, March 2009  CDI V1 – Extended Search

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 49 3 rd Plenary Annual Meeting, Madrid, March 2009 Integration of data centres in CDI V1 9 partners fully connected, CDI files loaded in the production environment 15 partners waiting the DM/interim tests, CDI files loaded in the import system 18 partners still to come, assistance is available

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 50 3 rd Plenary Annual Meeting, Madrid, March 2009  Summary The system upgrade from V0 to V1 is making good progress, but needs to be completed very soon and especially CDI (considering that the EMODNET chemical pilot is starting) : Directory Status of upgrade % ACTIONS EDMED1448/335745%Acceleration of the work is needed CSR4777/ %Acceleration of the work is needed EDIOS EDMERP1663/176795%Very good progress EDMO1607/ %Completed CDI in the production system Good progress