Helen Glaves (NERC- BGS), Dick Schaap (MARIS), Robert Arko (LDEO) and Roger Proctor (IMOS)

Slides:



Advertisements
Similar presentations
Rolling Deck to Repository: Transforming the United States Academic Fleet Into an Integrated Global Observing System Suzanne M. Carbotte, Robert Arko,
Advertisements

International Research Cruise Database and Website Status October ‘07.
ECOOP Data Management System (T2.2/WP2) Declan Dunne 13 th February 2008, Athens.
National facility concerned with looking after and distributing marine data Part of NOC National Marine Facilities Serve science, education and industry,
DMAC ST and the Activities of the IOOS PO Derrick Snowden DMAC Steering Team
First Marine Board Forum – 15 May Oostende Marine Data Challenges: from Observation to Information UNESCO/IOC’s project.
Ocean Data Interoperability Platform EU-US-Australia collaborative project Grant Number: Call: FP7-INFRASTRUCTURES INFSO Activity: INFRA :
FP7-Infra : Design studies for European Research Infrastrutures 1st October 2011 – 31st December 2014 Duration 39 months – Periods : 2 (month.
Geo-Seas e-infrastructure A pan European infrastructure for the management of marine and ocean geological and geophysical data Colin Graham, Dick Schaap.
OneGeology-Europe - the first step to the European Geological SDI INSPIRE Conference 2010, Session Thematic Communities: Geology Krakow, June 24 th 2010.
1 NODC, Russia GISC & DCPC developers meeting Langen, 29 – 31 March E2EDM technology implementation for WIS GISC development S. Sukhonosov, S. Belov.
Ocean Data Interoperability Platform (ODIP) ODIP Background and Prototype 1 development EGU 2014 – ESSI 2.5 Session – Vienna – Austria – 1 st May 2014.
SDN2 First Training Course, Oostende IODE-PO, 2-6 July 2012 Metadata Directories Management Sissy Iona, HCMR/HNODC.
NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.
EDMED and EDIOS Roy Lowry, Karen Vickers (Technical) Lesley Rickards, Liz Bradshaw (Content) British Oceanographic Data Centre.
Ocean glider data management: Argo concepts, GROOM, OGC sensor observation services Justin Buck + many collaborators British.
SeaDataNet, pan-European infrastructure for ocean and marine data management A key component of EMODNet A possible building block for GEOSS By Dick M.A.
Serving society Stimulating innovation Supporting legislation Meeting on OGC Sensor Observation Service (SOS) for INSPIRE Mickael.
A common global framework for marine data management Helen Glaves.
2 nd Training Workshop 4 – 5 June 2007 Common Data Index - CDI By Dick M.A Schaap Technical Coordinator SeaDataNet.
Metadata (for the data users downstream) RFC GIS Workshop July 2007 NOAA/NESDIS/NGDC Documentation.
Status of upgrading CDI service (user interface, harvesting via GeoNetwork, CDI interoperability options following SeaDataNet D8.7) By Dick M.A. Schaap.
MEDIN Data Guidelines. Data Guidelines Documents with tables and Excel versions of tables which are organised on a thematic basis which consider the actual.
GEO Work Plan Symposium 2012 ID-05 Resource Mobilization for Capacity Building (individual, institutional & infrastructure)
THEME[ENV ]: Inter-operable integration of shared Earth Observation in the Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014 Total EC.
Helen Glaves (NERC- BGS), Dick Schaap (MARIS), Robert Arko (LDEO) and Roger Proctor (IMOS)
Ocean Data Interoperability Platform EU-US-Australia collaborative project Grant Number: Call: FP7-INFRASTRUCTURES INFSO Activity: INFRA :
SeaDataNet2 Plenary meeting, 19th-20th September 2012, Rhodes SeaDataNet2 Plenary meeting THE EMODNET CHEMISTRY LOT. FROM THE EXPERIENCE OF THE THREE YEARS.
WP 9 (former Task 1b of WP 1): Data infrastructure Robert Huber UNI-HB Esonet 2nd all regions workshop, Paris
1 The IODE Ocean Data Portal - current status and future Nikolai Mikhailov, Chair of IODE/JCOMM ETDMP National Oceanographic Data Centre, Russia Four Session.
SeaDataNet A Pan-European Infrastructure for Ocean and Marine Data Management: Achievements and challenges.
AGU Fall Meeting, San Francisco, USA, 13 – 17 December 2010 Geo-Seas – building a unified marine geoscientific data infrastructure for Europe Helen Glaves,
Design central EMODnet portal Objectives, Technical Proposal and Consultation Process.
IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal MINCyT,
NERC DataGrid NERC DataGrid Vocabulary Server Use Cases Vocabulary Workshop, RAL, February 25, 2009.
WP5: DATA MANAGEMENT AND DISTRIBUTION Overview and Status: months Caterina Fanara 1, Loic Petit De La.
THEME[ENV ]: Inter-operable integration of shared Earth Observation in the Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014 Total EC.
IODE Ocean Data Portal – from data access to integration platform Sergey Belov, Tobias Spears, Nikolai Mikhailov International Oceanographic Data and Information.
Jamie Hall (ILL). SciencePAD Persistent Identifiers Workshop PANData Software Catalogue January 30th 2013 Jamie Hall Developer IT Services, Institut Laue-Langevin.
MEDIN Work Plan for By March 2011 MEDIN will be 3 years into the original 5 year development plan started in Would normally ask for continued.
IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.
EMODnet Chemistry 2 Service Contract MARE/2012/10 S How to make EMODnet Chemistry fit for purpose at system level By Dick M.A. Schaap – Technical.
SeaDataNet Harmonizing and optimizing the metadatabases and controlled vocabularies, incl maintenance & retrieval systems.
NESC Worshop – 07 September 2005 Development of a Marine Metadata Standard Greg Reed Executive Officer Australian Ocean Data Centre Joint Facility.
International Oceanographic Data and Information Exchange - Ocean Data Portal (IODE ODP) Enabling science through seamless and open access to marine data.
Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.
EMODNet Physics + My Ocean – SeaDataNet Dick Schaap (MARIS) – Brussels – 1 March 2013.
Helen Glaves 1 and Dick Schaap 2 1 British Geological Survey, United Kingdom 2 MARIS, The Netherlands.
Data Product Catalogue for SeaDataNet and Emodnet Chemistry M. Treguer, T. Loubrieu, C. GUYOT, IFREMER.
Metadata V1 By Dick M.A. Schaap – technical coordinator Oostende, June 08.
Sensor Web Enablement (SWE) developments for fixed monitoring platforms and research vessels By Dick M.A. Schaap – SeaDataNet Technical Coordinator with.
Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level 1 SI O S Svalbard Integrated Arctic.
International Planetary Data Alliance Registry Project Update September 16, 2011.
Workshop Sensor Web Enablement (SWE) for oceanography - introduction Dick M.A. Schaap – MARIS March 2016.
EarthCube Integrative Activities: Cross-Domain Observational Metadata Environmental Sensing Network (X-DOMES) September 2015 – August 2017 Focusing on.
IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal.
Ocean Data Interoperability Platform I & II
Supported by the European Commission
SEXTANT PRODUCTS CATALOGUE
Task 2.6 Eric Delory PLOCAN
Linked Data for Field Deployments
Progress EMODnet Ingestion
Management - coordination
Justin Buck OceanSITES data Incentives for participation: Data citation & data services Justin Buck
Flanders Marine Institute (VLIZ)
Distributed Marine Data System:
Dick M.A. Schaap – Technical Coordinator SeaDataNet Training Workshop
Design central EMODnet portal Objectives and Technical description Initial draft prepared by the Flanders Marine Institute.
EMODNet Physics + My Ocean – SeaDataNet
ODIP: an international effort promoting the development a global framework for marine data management to support large-scale ocean research Helen Glaves.
Presentation transcript:

Helen Glaves (NERC- BGS), Dick Schaap (MARIS), Robert Arko (LDEO) and Roger Proctor (IMOS)

Paradigm shift from traditional discipline based marine research Multidisciplinary ecosystem level approach: promoted in Europe by Marine Strategy Framework directive MSFD (2008) and Marine Knowledge 2020 Requires large amounts of good quality, interoperable data from a range of disciplines and organisations

Barriers to re-use of marine data Use of different Formats Standards Best practice Co-ordinate systems Technologies National and organisational data access policies 3

E-infrastructures A number of regional initiatives have made significant progress in addressing these barriers by developing marine data management infrastructures Development of these infrastructures is being promoted and supported by international organizations such as UNESCO‘s Intergovernmental Oceanographic Commission (IOC) USA Europe Australia

Ocean Data Interoperability Platform EU-US-Australia collaborative project Grant Number: Call: FP7-INFRASTRUCTURES INFSO Activity: INFRA : International co-operation with the USA on common e-infrastructure for scientific data Start date: 1 October 2012 Duration: 36 months Funded in parallel by European Commission, National Science Foundation (NSF) and Australian Government

Europe: 10 EU funded partners: 6 countries NERC-BGS/BODC, MARIS, OGS, IFREMER, HCMR, ENEA, ULG, CNR, RBINS-MUMM, TNO ODIP partners

USA: NSF funded partners (supplement to existing R2R project) San Diego Supercomputer Center (SDSC) Scripps Institution of Oceanography (SIO) Woods Hole Oceanographic Institute (WHOI) Lamont-Doherty Earth Observatory (LDEO) Florida State University: Center for Ocean-Atmospheric Prediction Studies (FSU) Australia University of Tasmania (IMOS) International UNESCO IOC-IODE

Associate partners Europe Alfred Wegener Institute for Polar Research (AWI) MARUM USA NOAA US-IOOS, NOAA US-NODC, NOAA NGDC UNIDATA Australia Australian National Data Service (ANDS) Geoscience Australia (GA) CSIRO

ODIP: Objectives To establish an EU/USA/Australia/IOC-IODE co-ordination platform to facilitate the interoperability of ocean and marine data management infrastructures To demonstrate this co-ordination through the development of several joint prototype interoperability solutions that allows effective sharing of marine and ocean data across scientific domains, organisations and national boundaries Development of a common approach to marine data management that can be extended to other regions and organisations beyond the original project consortium

Overall strategy Development of a collaboration platform to facilitate organised dialogue between key organisations in Europe, USA and Australia involved with marine data management. Achieved by: Creating inventories of existing standards and policies Publication of these existing standards and best practice through ODIP portal and Research Data Alliance Regular joint workshops to develop interoperability solutions and/or agree on common standards Development of prototypes for testing and evaluating potential interoperability solutions Dissemination /promotion of ODIP activities to encourage wider participation and adoption of the outcomes of the project

Progress to date………. 1 st ODIP workshop: Ostende, Belgium (February 2013) Addressed 6 discussion topics Resulted in an extensive list of actions Formulated into three prototype interoperability solution development tasks 2 nd ODIP workshop: San Diego, USA (December 2013) Addressed three discussion topics and implementation plans for the prototype development tasks Initiation of 3 prototype interoperability solution development tasks

ODIP 1: objective Establishing interoperability between the SeaDataNet, IMOS and NODC data discovery and access services using brokering services Lead by European partners via SeaDataNet Initially addressing use of brokers at the metadata level Progress to data access services (possibly including authentication and authorisation systems)

ODIP 1: the plan make use of the (Euro)GEOSS Brokerage harmonise the 3 regional services to a common level SeaDataNet (Europe) IMOS (Australia) NODC (USA) start at metadata level, but progress to data access, including providing solutions for possible AAA systems use the broker to facilitate access to data from the regional services by the GEOSS and Ocean Data Portals

ODIP 2: objective ODIP 2: Establishing interoperability between cruise summary reporting systems in Europe, the USA and Australia Lead by Rolling Deck to Repository (R2R) partners (USA) Improvement of delivery and exchange of cruise summary information through the use of common formats and vocabularies Use of GeoNetWorks for routine harvesting of cruise data for delivery via the Partnership for Observation of Global Oceans (POGO) portal

ODIP 2: the plan Publish ISO Cruise Summary Reports at regional nodes: Marine National Facility (Australia) SeaDataNet (Europe) R2R (USA) Deploy GeoNetwork catalogues at regional nodes providing both a GUI (web portal) and API (CSW service) Harvest GeoNetwork nodes into POGO global catalogue

ODIP 3: objective Establishment of a prototype for a Sensor Observation Service (SOS) for selected sensors installed on vessels and in real-time monitoring systems using sensor web enablement (SWE) Lead by AODN (Australia) regional initiatives progress towards the adoption of SWE allowing direct standardised access to the data from operational sensor systems

ODIP 3: the plan establish a collaboration tool (Github) compile inventory of SOS services and their endpoints compile inventory of instrument SensorML records & O&M structures compile inventory of vocab and registry services Working groups to assess SOS performance propose templates for SensorML/StarFl and O&M profiles examine vocabulary services and potential mappings Set-up a test bed

Establish a collaboration tool Github: compile inventory of SOS services and their endpoints compile inventory of instrument SensorML records & O&M structures compile inventory of vocab and registry services Working groups to assess SOS performance propose templates for SensorML/StarFl and O&M profiles examine vocab services and potential mappings Set up testbed ODIP 3: the plan

StarFL OGC discussion document *FL Starfish Fungus Language for Sensor Description

Testbed - Progress so far… (Scott Bainbridge) Set up two public open SOS servers, one running V4.0 of the 52North SOS server, the other running V3.6; Both servers are fully open, that is anyone can undertake the full SOS set of requests including registering sensors and adding data; Currently data from the IMOS FAIMMS Sensor Network is going into the servers, more data can be added to look at load testing if need be; Both servers form an ideal platform for client development and service chaining experiments.

SOS v4.0 Base Address: XML Address: So far two Features of Interest: IMOS/DAVIES/SF1= Sensor Float 1 at Davies Reef IMOS/HERON/RP8= Relay Pole 8 at Heron Island With two Parameters: Water temperature (Deg. C.) Depth (m)

SOS v3.6 Base Address: XML Address: Will be set up with the same Features of Interest and Parameters as the V4 server; Supports both SOS V1 and SOS V2.

Activities underway SeaDataNet (Jordi Sorribas) SensorML_profiles_for_specific_ marine_observations_data-final.pdf Example: RONIMAR program station located in Cadiz with a tide gauge. SensorML (using the SensorML 2.0 schema) and O&M files are online:

Activities underway CNR (Alessandro Oggioni, Mauro Bastianini, Cristiano Fugazza) VMs with 52°North SOS implementations: a) version 3.2 linked to some data streams from the Venice Acqua Alta Tower b) version 4.0 linked to some data streams from the LTER ecological data (also for 5 marine stations) c) version 4.0 linked to the tidal network stations Examples of SensorML for version 3.x and 4.0: a) weather stations; b) vessel sensors; c) probe profile (eg CTD) An inventory of SOS: collected 420 SOS endpoints (site to share this list?) A SOS client to upload metadata (SensorML format) and Observations, visualize (also in realtime), and download data from the sensors ‘Starter’ kit

Activities underway OGS (Elena Partescano, Alessandra Giorgetti) Testing a SOS service linked to the meteo- oceanographic MAMBO1 buoy (in the North Adriatic Sea) and the observatory site E2M3A (in the South Adriatic Sea) Installed a OGC 52° North Sensor Observation Service 3.5 ( and a SOSClient ( For testing: MAMBO1 - all CTD and pH sensors – and E2M3A - pH sensor - (pH data for both buoys, stored using O&M format). The SensorML 1.0 are compiled using, where possible, the SeaDataNet vocabularies and EDIOS catalog.

Activities underway AWI (Hans Pfeiffenberger, Ana Macario) Research vessel SOS SensorML and O&M SeaDataNet profiles are being adopted – lots of work! AWI hands-on experience will be communicated in next ODIP workshop AWI is also keen on testing StarFL SensorML and O&M will be „opened“ for RV Polarstern and RV Heincke during Q1/2015 Dynamic vessel tracking On-going efforts towards integration in EUROFLEETS 2 prototype Cooperation with 52North (SOS 4.0) is planned Optimization of current near real time services

Activities underway Ifremer (Thomas Loubrieu, Mickael Treguer) Made some progress on oceanotron which now implements SOS protocol. Output is encoded in XML, JSON, and netCDF. It works for online results or off-line (or asynchronous) results for big volumes. Working together with Jordi Sorribas on templates of sensorML and O&M and shared core rules on them (on vocabulary's references,unique identifiers, contacts,...) Doing some experimentation with 52north.

Activities underway US-IOOS (Derrick Snowden) All IOOS regions now have functioning IOOS milestone 1.0 SOS services in place. Strongly urging to adopt the 52North 4.0 branch as the basis for ODIP. Inventory of SOS services and endpoints - SetId=&componentId=&serviceType=SOS&serviceStatus=&serviceU rl=&search=List+Collection+Sources SetId=&componentId=&serviceType=SOS&serviceStatus=&serviceU rl=&search=List+Collection+Sources Inventory of sensorML and O&M structures - %2Ftemplates%2FMilestone1.0 %2Ftemplates%2FMilestone1.0 Vocab and registry services –

Activities underway Two test instances of 52N and ncSOS (four services - latest stable build and continuous integration build) 52N: ncSOS: linkshttps://github.com/asascience-open/ncSOS/wiki#wiki-quick- links Test data sets for further development: For ncsos - open/ncSOS/blob/master/resources/datasets.ziphttps://github.com/asascience- open/ncSOS/blob/master/resources/datasets.zip For IOOS 52N - An early version of a python client for the IOOS SOS profile (and several other web service interfaces): open/pyooshttps://github.com/asascience- open/pyoos

Activities underway BODC (Justin Buck) Planning to implement a prototype ncSOS system for glider data at BODC (based on IOOS but need to integrate access control).

What next? Use github repository to compile inventory of SOS services and their endpoints compile inventory of instrument SensorML records & O&M structures compile inventory of vocab and registry services Initiate Working groups to assess SOS performance Justin Buck, Scott Bainbridge propose templates for SensorML/StarFl and O&M profiles Paolo Diviacco, Irina Bastrakova, Thomas Loubrieu*, Cristiano Fugazza*, Hans Pfeiffenberger* examine vocab services and potential mappings Kim Finney / Sebastien Mancini, Cristiano Fugazza Add more SOS to testbed, add more datasets

In time for ODIP#3 workshop When? 4-8 August 2014 Where? Australian Institute of Marine Science, Townsville, Australia Townsville will be warm and sunny in August, max around 26C., min around 14C Participation: Accommodation: IBIS TOWNSVILLE HOTEL (“ AIMS0814”)

Thank you!

Dissemination of ODIP outcomes Project website Social media International conferences Other related initiatives Ocean Data Portal (ODP) Research Data Alliance Belmont Forum

Github repository Users requesting to be registered for Github Paulo Diviacco - Thomas Loubrieu (tloubrieu) - Mickael Treguer (mtreguer) - Kim Finney - Scott Bainbridge - Alessandro Oggioni - Mauro Bastianini - Cristiano Fugazza - Justin Buck - Ana Macario - Derrick Snowden - Jordi Sorribas - Dominic Lowe - Elena Partescano -