Discovering and accessing data from a distributed network of data centres S. Mazzeo (ESA)

Outline Data portals providing "one-stop shop" for heterogeneous products: Co-location of satellite, models and in-situ observations Interoperability issues Data processing GEO: interoperability with GEOSS – why and how to contribute

Discovering heterogeneous products Insert the search text string Set the bounding box as desired Click on Search to start the query Insert Start Date and Stop Date In Situ Collections of dataset corresponding to the search criteria are listed here Remote sensing Datasets belonging to the selected collection (access URL is also provided) High level products

OpenSearch A federation of catalogues: One at the level of the portal containing the metadata at series collection level Many catalogues at the level of the federated resources containing the metadata at dataset product level All the catalogues can be accessed using OpenSearch protocol, a collection of technologies allowing websites and search engines to publish search results in a standard and accessible format Data/products remain at their original location, i.e., where the Data Provider stores them; the Catalogues provide the user with the link to directly access the data The metadata model is based on RDF -Resource Description Framework Can be expanded as needed according to the specific needs of the communities to include additional information (quality, uncertainty, specificities of the sensor…)

Sustainability of the proposed discovery approach Highly sustainable solution: Only the portal and the central catalogues are “central elements” The central catalogue can be easily duplicated (is a simple registry) New portals can be developed by the community The federated catalogues should be deployed at the premises of the single data providers.

Sustainability of the proposed discovery approach In a Mall Centre An info-wall ……with a store listing….. and the shops If one shop closes, you can still enter in the other shops. If the info-wall is dismissed you can still use the store listing, despite less user friendly Out of service or you can look for another info-wall in the mall centre. It may even look better. Not available In the worst case the info-wall and the store listing are dismissed, you can still go directly to the shops, maybe you still remember where they are located from your last visit. The shop owners will decide soon to put in place a new store listing and info-wall. This is an easy operation The portal The central catalogue The federated resources (data centers and catalogues)

ENVRI VRE: What is this? It is a Virtual Research Environment*, i.e. an innovative, web-based, community-oriented, comprehensive, flexible, and secure working environment conceived to serve the needs of modern science It is oriented to provide the “ENVRI Community” with a set of facilities including data access and processing facilities developed in ENVRI It is available at https://www.d4science.org/group/envri Subscription required It is hosted by the D4Science infrastructure and build by relying on gCube technology * L. Candela, D. Castelli, P. Pagano (2013) Virtual Research Environments: An Overview and a Research Agenda. Data Science Journal, Vol. 12, p. GRDI75-GRDI81 http://dx.doi.org/10.2481/dsj.GRDI-013 2nd ENVRI Knowledge Transfer: Data Processing

Data Processing: Statistical Service An environment offering a rich array of data processing algorithms as-a-Service New algorithms can be easily added (programming skills) The GUI to execute each algorithm is automatically generated Algorithms benefits from the underlying distributed computing infrastructure 2nd ENVRI Knowledge Transfer: Data Processing

Data Processing: Occurrence Data Management An environment offering occurrence data processing algorithms as-a-Service E.g. Union, Intersection, Clustering Relying on Statistical Service back-end New algorithms GUI automatically generated Algorithms benefits from the underlying distributed computing infrastructure 2nd ENVRI Knowledge Transfer: Data Processing

Possible exploitations The ESFRIs have different possibilities for using the processing facilities: Use the algorithms already available with their “own” data (data should be in catalogues and compliant with the algorithm accepted format) Develop new algorithms (programming skills are needed) Statistical Service WPS North-52 processes Liaise with D4Science.org for the creation of dedicated VREs Data and processes specific Exploit specific services instances operated by D4Science.org Publish data in the D4Science SDI, e.g. harvesting metadata for GeoNetwork, publish products in THREDDS Data Server or GeoServer Exploit the underlying technology for the creation of a gCube-based infrastructure 2nd ENVRI Knowledge Transfer: Data Processing

Overview of GEO GEOSS GEO (Group on Earth Observations) is a voluntary partnership coordinating efforts to build the GEOSS. GEOSS, “System of Systems” involves Earth Observation and Environmental data collection systems and centers for creation and distribution of information products. GEOSS serves nine “Societal Benefit Areas”: disasters, health, energy, climate, water, weather, ecosystems, agriculture and biodiversity.

How to contribute to GEOSS and other infrastructures OpenSearch is a widely accepted Standard that would enable interoperability with: GEOSS – so making ICOS data discoverable to GEOSS users, and other GEOSS data discoverable and accessible to ICOS users; A large number of Environmental ESFRI projects, including EMSO, EuroArgo and Eiscat-3D. DLR, partner in IAGOS, is using OpenSearch for its IAGOS-precursor aircraft flight data; Space agencies - The Committee for Earth Observation Satellites (CEOS) is considering OpenSearch for interagency interoperability; Many other services and infrastructures - They could take advantage of the simplicity of the OpenSearch interface to easily discover and access ICOS data.