OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet Pan-European Infrastructure for Ocean and Marine Data Management Gilbert Maudire and the SeaDataNet Consortium SeaDataNet mid-term review – Bologna – September 2008, 19

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 2 SeaDataNet mid-term review – Bologna – September 2008, 19  Data is of vital importance for marine research and support to public decision  Marine observation and data sets status oobservation conducted by more than 1100 active scientific laboratories in the countries of partners, using various sensors and platforms onot always quality controlled using comparable procedures onot always easily accessible  Necessity of large background work in order to :  secure them in long term archives  document available observations and data sets (metadata)  improve coherence of data sets (references, quality, redundancy)  make data sets accessible easily (formats, on line distribution)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 SeaDataNet mid-term review – Bologna – September 2008, 19  What is SeaDataNet?  EU Sixth Framework Programme  Integrated Research Infrastructure Initiative (I3)  1 April 2006 to 31 March 2011 (5 years)  Objectives oTo network existing oceanographic data centres already nationally funded oTo develop an efficient distributed pan-European marine data management Infrastructure (a “unique Virtual Data Centre”) oTo provide on-line access to distributed databases of standardised quality by using adapted communication & information technology  This project integrates and develops several EU past initiatives and international cooperation, in particular Medar/Medatlas, Edmed, Edios, Sea-Search etc.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 4 SeaDataNet mid-term review – Bologna – September 2008, 19  49 Partners  National Oceanographic Data Centres odivisions of major national marine research institutes in 35 countries oexperienced as leaders of data Management of many scientific projects and of concerted actions (e.g. MTPII-MATER, CANIGO, OMEX, EDIOS, MERSEA, MEDAR/MEDATLAS, SEA- SEARCH)  3 satellite data centres  3 scientific modelling centres  2 SMEs experts in software development  3 International organisations with strong regional interest/cooperation

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 5 SeaDataNet mid-term review – Bologna – September 2008, 19  Primary focus on water column parameters oPhysical oChemical oBio-chemical (non species dependant)  Projects will be proposed for other domains by involved communities in strong relationships with SeaDataNet Several SeaDataNet partners are already involved in those domains oGeophysical and geological data oBiology (species dependant)  SeaDataNet managed data

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 6 SeaDataNet mid-term review – Bologna – September 2008, 19  SeaDataNet services  Online services : web portal and web services oDiscovery services Metadata directories Common vocabularies : parameters, units, … oDelivery services Security services : Authentication, Authorization & Administration (AAA) Data download and access including reformatting in standardized formats oViewing services Web map services, data visualization.  Offline services : « back office » services oSafeguarding within the national repositories (NODCs) oQuality Control and desktop viewing services ODV analysis software oProduct generation adapted to each regions oMonitoring and statistics : usage & system performance

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 7 SeaDataNet mid-term review – Bologna – September 2008, 19  Infrastructure versions & planning  Version 0 – Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats o Operated at the start of the project  Version 1 – Setup of the integrated online data services to users : networking of 10 “interoperable” data centres of the Technical Task Team unique request to the interconnected data centres and the data sets are delivered with a unique format o Presently under test and progressive integration of 10 data centres during 2008  Version 2 – Improved version of the system (ease of use, data visualisation, data access) and by step, all data centres connected for providing online trans-national access to the data, metadata and products o Operational in 2010

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 8 SeaDataNet mid-term review – Bologna – September 2008, 19  Discovery services All catalogues (European metadata directories) are compliant with ISO19115 standard family for spatial metadata make use of common vocabularies and references are accessible and linked through web services ease data access using various selection criterions adapted to different users (scientists, observatory and fleet managers, …)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 9 SeaDataNet mid-term review – Bologna – September 2008, 19  Common data index : gate to data access  online searchable directory where?, when?, what?, …  tool for managing user’s “Shopping Basket”, that is : oTo prepare a list of distributed datasets matching user’s request oTo ask concerned data centres to serve the user, oTo display the status of user’s request oTo route the requested data from the distributed data centres to the user

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 10 SeaDataNet mid-term review – Bologna – September 2008, 19  SeaDataNet data policy and data licence inspired by INSPIRE Directive to facilitate the data flow takes into account the local rules and the SeaDataNet users needs  Metadata ofree and open access, no registration required oeach data centre should provide the meta-data in standardized format to populate the catalogue services  Data and products ovisualisation freely available odata download, in general, free for academic purposes with mandatory registration oa “SeaDataNet role” (partner, academic, commercial etc.) is attributed to individual user using the Authentication, Authorization and Administration (AAA) Service Each NODC attributes the roles to the users of its of country Out of the partnership, the roles are assigned by SeaDataNet user-desk oa common data licence has been adopted and users must agree on when registering oeach data centre node delivers data according to the user’s role and its local regulation oeach data centre should provide the data sets necessary to develop the common products

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 11 SeaDataNet mid-term review – Bologna – September 2008, 19  QC procedures and tools  SeaDataNet protocol : o Coherent automatic and visual checks are performed on the data by each NODC o A unique quality flag scale has been adopted  The automatic checks and the visualization could be performed using either by Ocean Data View (ODV) or existing partner software (if the same protocol and checks are applied) Example: Using derivatives for spike & discontinuity detection in ODV

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 12 SeaDataNet mid-term review – Bologna – September 2008, 19  Data products  SeaDataNet develops added-value products at regional scales: oMediterranean Sea oBlack Sea oBaltic Sea oArctic Seas oAtlantic and global ocean The preparation of these products allows the checking of the compatibility of the QCs performed on the data in each NODC.  First products => gridded climatologies and trends  The comparability of the different regional products is assessed by using a new version of DIVA (Data Inverse Variational Analysis) developed (University of Liege)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 13 SeaDataNet mid-term review – Bologna – September 2008, 19  Capacity Building is a main issue  Training program main objectives  to ensure that management procedures are coherent within the consortium  to transfer expertise and to train IT experts of the SeaDataNet data centers in the installation and operation of the SeaDataNet technical components

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 14 SeaDataNet mid-term review – Bologna – September 2008, 19 Please contact :

OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet Project management SeaDataNet mid-term review – Bologna – September 2008, 19

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 16 SeaDataNet mid-term review – Bologna – September 2008, 19  Role of partners (1/4)  Coordination o Ifremer : project coordinator o Maris : technical coordinator animator of the Technical Task Group with the help of : NERC/ BODC : standard development AWI : data quality control and desktop viewing services oUniversity of Liege : scientific coordinator in charge of the regional products development with the help of : JRC : Product harmonization INGV : coordinator of Mediterranean products METU : coordinator of Black Sea products NERI : coordinator of Baltic products IMR & VLIZ : coordinators of Artic & North Seas products Ifremer : coordinators of Atlantic & Global Ocean products

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 17 SeaDataNet mid-term review – Bologna – September 2008, 19  Role of partners (2/4)  Coordination (cont.) o HCMR : Coordinator of the networking activities i.e. metadata and data management, system monitoring with the help for coordinating the distributed network of Maris : management of Common Data Index (CDI) and of Directories of Marine Projects (EDMERP) and Marine Organizations (EDMO) NERC/ BODC : management of Directories of Marine Environmental Datasets (EDMED) and of Ocean Observing System (EDIOS) BSH : management of Cruise summary reports Ifremer : Management of SeaDataNet users

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 18 SeaDataNet mid-term review – Bologna – September 2008, 19  Role of partners (3/4)  Technical task team (11 partners : Western Europe + Russia) o Technical specifications, o Check of the defined standards o Integration and system setup, test and validation of technical components o Running of System V1  Capacity building and training oUNESCO/ IOC Project office with the help of Technical Task Team  Information and communication oENEA

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 19 SeaDataNet mid-term review – Bologna – September 2008, 19  Role of partners (4/4)  All partners which are mostly NODC’s (except international organizations) have to o generate metadata : organizations, projects, cruises, observing systems, data sets omanage data from their country : quality, format, … according to SeaDataNet agreed standards oprovide access to their data repository, automatically using SeaDataNet system or manually as a first stage omanage users of their respective countries by providing information and help

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 20 SeaDataNet mid-term review – Bologna – September 2008, 19  External relationships / Collaborations  Advisory board o MEDS Canada (represents IODE/WMO JCOMM, GTSPP and Argo international programs) o US – NODC (represents North America initiatives like DMAC) oEuroGOOS (represents Operational Oceanography community)  Standard summit (last January on behalf of COI) oNorth America, Europe, Japan  “Memorandum of understanding” with other consortiums oUnidata, Mersea-MyOcean (GMES), EuroBis-Marbef (FP6), GENESI (FP7)  International conference on Marine Data and Information Systems (IMDIS) oApril 2008, more than 200 participants from about 50 countries.

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 21 SeaDataNet mid-term review – Bologna – September 2008, 19  Project management tools  SeaDataNet web sitewww.seadatanet.orgwww.seadatanet.org oDistributed management, based on a Content Management system (EZPublish) oGeneral information : Description of Work, Meetings, Conferences, Education (several languages) oProject services (discovery, data access, …) oFinalised documents (applicable version of standards) and software tools  Mailing lists oBased on Sympa software oOne list per group (technical, …) owned by the chair person o+ user desk (SeaDataNet users point of contact)  Document management oBased on BSCW software (Basic Support for Cooperative Work - Germany) oCooperative work on documents (non finalised versions) and software  Software management (versionning tool) oBased on GForge

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 22 SeaDataNet mid-term review – Bologna – September 2008, 19  System monitoring  focused on system “availability”  Availability of SeaDataNet user services oavailability of web pages and web services general information, metadata browsing (EDMED, CSR, CDI), data selection users’ interfaces, web services oavailability of web data access services data centre download services oService “available” if monitoring system get an answer Search of keywords within web pages, Web services can be reached : WSDL returned  Monitoring odone by IFREMER (coordinator) and HCMR in order to avoid biases due to network troubles oUsing NAGIOS Open Source software Polling of services (web pages, web services)

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 23 SeaDataNet mid-term review – Bologna – September 2008, 19  Polling of system components

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 24 SeaDataNet mid-term review – Bologna – September 2008, 19  Monitoring board  Example of one-month report for Ifremer http access Electrical failure during 6 hours

OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet Administrative and financial aspects SeaDataNet mid-term review – Bologna – September 2008, 19

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 26 SeaDataNet mid-term review – Bologna – September 2008, 19  Known issues  IOC Project Office partner oIn charge of training courses and capacity building oReport at 0 up to now even if work done (3 training courses organised yet) no agreement on US $ / € change rate between UNESCO and EC) oIOC is an « Additional Cost partner » not well adapted to training course organization (additional personnel)  Transnational activities report oPartners with very small web site and/or very little audience oAudience measurment : KiloPageViews unit  difficult to report accurately o80% of partners report less than expected for TA

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 27 SeaDataNet mid-term review – Bologna – September 2008, 19  Transnational Activities amendment  Change reporting unit from KiloPageViews to KiloHits oTo take into account all hits on online services : web pages (for any usages), web services, ftp servers, … oBut, cost / unit must decrease (Project Officer comment)  However, most audiences will remain far less than expected oCost are not linearly proportional to the audience « Entry costs » to set up a web site  funds transfer to JRA (web site maintenance) oWho? How much?  All must remain in the same budget envelope

OBSERVATIONS & PRÉVISIONS CÔTIÈRES SeaDataNet Planning SeaDataNet mid-term review – Bologna – September 2008, 19

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 29 SeaDataNet mid-term review – Bologna – September 2008, 19  Infrastructure versions & planning  Version 0 – Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats o Operated at the start of the project  Version 1 – Setup of the integrated online data services to users : networking of 10 “interoperable” data centres of the Technical Task Team unique request to the interconnected data centres and the data sets are delivered with a unique format o Presently under test and progressive integration of 10 data centres during 2008  Version 2 – Improved version of the system (ease of use, data visualisation, data access) and by step, all data centres connected for providing online trans-national access to the data, metadata and products o Operational by 2010

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 30 SeaDataNet mid-term review – Bologna – September 2008, 19  Project planning and budget  High level of effort during the first three years (80% of the budget) o Common standard adoption, set up and training, o Development / Adaptation of common tools o System specifications, development and setup (V1 and then V2) o Development of first release of regional products and initial harmonization  Lower level of effort during the last two years o Continuous data management o Exploitation of the system, system maintenance and upgrade o Upgrade of regional products

OBSERVATIONS & PRÉVISIONS CÔTIÈRES 31 SeaDataNet mid-term review – Bologna – September 2008, 19  More information about budget  Funds breakdown / year / partner after second report oFunds breakdown : breakdown  Transnational access audience oEvolution for TA- Seadatanet.xlsEvolution for TA- Seadatanet.xls