Download presentation
Presentation is loading. Please wait.
Published byPaul Hodges Modified over 8 years ago
1
ENVRIPLUS week, May 2016 Zandvoort Background, ocean observation e-infrastructure Who is the community/project the use case belongs to? ENVRIPLUS EU project : Ocean observations : Euro-Argo What's the timeline for development, tests and large-scale operation? First implementation in 2016: Euro-Argo, Copernicus marine service Large scale operation from 2017 EMSO, SeaDataNet, SOCAT Solid earth and atmosphere RIs What's your role in the use case? Any experience/link to EGI? Implement Euro-Argo use case Involve Copernicus, SeaDataNet, SOCAT and ESONET in a next step Link with EGI through ENVRIPLUS
2
ENVRIPLUS week, May 2016 Zandvoort Users Who will be the users of the planned community-specific e-infrastructure? How many of them? Environmental monitoring and forecasting : EU ocean-atmosphere models Calibration and validation with in-situ data : SMOS, Sentinel3 satellite missions Trans-disciplinary communities (ocean - atmosphere - biology - solid earth) Efficient workplace to combine storage and CPU for advanced services (such as geo-spatial analysis) Input stream for data providers Digital collaborative workplace for researchers
3
ENVRIPLUS week, May 2016 Zandvoort Current status: psuh RI data on EU E- infrastructure Push Argo observations on EU E-infrastructures Observations data A flat table of 12 columns and 5 billion lines Observations metadata JSON collections of metadata platforms, parameters, data types, institutions, scientists, countries, projects, cruises Observations are daily pushed on workplaces such as n Hadoop, NoSQL or MySQL n The use of in-memory features should provide instant answers n Metadata are indexed with ElasticSearch n The workplace is activated on a virtual server, replicated on the cloud
4
ENVRIPLUS week, May 2016 Zandvoort Current status: Ocean observation e-services, discovery Euro-Argo data discovery on EUDAT infrastructure A monthly snapshot of the whole Argo dataset is published with a DOI on : http://www.argodatamgt.org/Access-to-data/Argo-DOI-Digital-Object-Identifier http://www.argodatamgt.org/Access-to-data/Argo-DOI-Digital-Object-Identifier Each Argo data snapshot has a DOI The master DOI: http://doi.org/10.17882/42182http://doi.org/10.17882/42182 A monthly snapshot, April 2016: http://doi.org/10.17882/42182#43181http://doi.org/10.17882/42182#43181 Seanoe is the publisher of Argo data sets. Seanoe provides an OAI-PMH service its publication catalogue. http://www.seanoe.org/oai/ http://www.seanoe.org/oai/ B2FIND harvests and publicizes Argo and ocean data on EUDAT community http://eudat6c.dkrz.de/group http://eudat6c.dkrz.de/group
5
ENVRIPLUS week, May 2016 Zandvoort Current status: Ocean observation e-services, discovery aaa
6
ENVRIPLUS week, May 2016 Zandvoort ENVRIPLUS data subscription use case The user subscribes with his criteria time and spatial coverage, parameter, data type update period for delivery (daily, monthly, yearly, on the spot) The relevant data are Extracted from ENVRIPLUS cloud Converted/transformed on ENVRIPLUS grid WPS ? The user’s cloud account is regularly updated with the new data files An accounting of data delivery is performed MDC ? A citation scheme is attached to the delivered data DOI ? bibliographic surveys can track the use of these data in publications reproducibility is possible A users identification scheme is implemented Federation of identities : Marine-ID, Shibboleth, OpenID
7
ENVRIPLUS week, May 2016 Zandvoort Digital collaborative workplace for trans-disciplinary communities (ocean - atmosphere - biology - solid earth) Environmental monitoring and forecasting : EU ocean-atmosphere models Calibration and validation with in-situ data : SMOS, Sentinel3 satellite missions Efficient workplace for advanced services Input stream for data providers Space Generalization to other RIs : E-infrastructures data VM my cloud VM Solid Earth Atmosphere Infrastructure Ocean Research infra.
8
ENVRIPLUS week, May 2016 Zandvoort ENVRIPLUS e-infrastructure architecture Data and metadata workplace Drill, HBase, Spark Géographique index and facets ElasticSearch, Drill Data discovery and visualization OpenLayer3, AngularJS, BootStrap, html5/css3, Material design Data extraction Drill, SQL, PIG Data subscription service OwnCloud, B2SHARE, Onedata Logs management accounting Kibana CSV & JSON data upload Drill
9
ENVRIPLUS week, May 2016 Zandvoort Ocean observation e-services Data services to be developed around Euro-Argo cloud Ocean observations API : profiles, time-series, trajectories Metadata visualization : map wms services Data visualization : graphics Data products such as mixed-layer depths maps Agile and incremental implementation Step 1 : Euro-Argo data file on the cloud (1to file daily updated) Step 2 : VM for indexation of data file (ElasticSearch) Step 3 : data file generation service (CSV then NetCDF) Step 4 : data subscription/distribution service to OwnCloud accounts Step 5 : replicate data and VM in mirror sites (EU, US, AU, JP) Next steps : promote the development of various services around an ENVRIPLUS cloud
10
ENVRIPLUS week, May 2016 Zandvoort Current status Which components/services already exist in your architecture? The data and metadata files are available The ElasticSearch index is available Which components/services are under development (and by who)? The web interface for discovery and data access (Ifremer) The data processing services (Ifremer) Which components/services do you expect to get from EGI? Data storage and distribution Data processing Data subscription service: PIDs, citation in aggregated datasets
11
ENVRIPLUS week, May 2016 Zandvoort Plans for today What questions would you like to get answered today? Let’s organize the contributions and people involved What issues you would like to solve today? There is no particular blocking issue Other outcomes that you would like to get out from the workshop?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.