1. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
MSc. Vassilis TSIRONIS, Software Development Dr. Anna POLYCHRONIOU, Data Collection & Analysis
Dr. Haris KONTOES, Project Coordinator
Good Evening & thank you very much for your kind invitation to the current workshop.
My name is Vassilis Tsironis, I represent the National Observatory of Athens and I am going to present you the GEO-CRADLE Regional Data Hub tool of the GEO-CRADLE project.
2nd GEO Data Providers workshop (20th-21st April 2017, Florence, Italy)

2. Introduction to GEO-CRADLE project
The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Introduction to GEO-CRADLE project
…is a unique EU funded Coordination Action running at regional level
…is looking at the N. Africa, Middle East, and the Balkan territories
objectives
Promote the uptake of EO services and data in response to regional needs
Support the effective integration of existing Earth Observation Capacities in the region
Facilitate the engagement of the complete ecosystem of EO stakeholders in the region
Enhance the participation in and contribution to the implementation of GEOSS and Copernicus in North Africa, Middle East and the Balkans
Α key output of GEO-CRADLE that can contribute to the long-term uptake of EO activities in the region is the operation of the Regional Data Hub.
Provide access to region-related datasets and services, directly fed from the GCI, and at the same time being the centralised gateway for regional data providers to contribute easily and timely their products to GEOSS
Α successful example of being implemented Community Portal leveraging GEO DAB APIs.
Funded under H Climate action, environment, resource efficiency and raw materials
ACTIVITY: Developing Comprehensive and Sustained Global Environmental Observation and Information Systems
CALL IDENTIFIER: H2020 SC5-18b-2015 Integrating North African, Middle East and Balkan Earth Observation capacities in GEOSS
Project GA number:
Total Budget: 2,910, €
GEO-Cradle project is coordinated by Dr. Haris KONTOES of the Institute for Astronomy, Astrophysics, Space Applications, and Remote Sensing (IAASARS)of the National Observatory of Athens (NOA).
The various software components (GEO-CRADLE Portal, Network of Stakeholders and Regional Data Hub) are development from the NOA IT team.
The project has brought together 25 partners from 3 continents, to work in a highly-complementary team that combines a strong background in EO coordination activities with proven scientific excellence in four key thematic areas
GEO-CRADLE is a regional project which aims to support, promote and facilitate the EO data & services in the region of Balkans, Middle East and North Africa, and especially enhance GEOSS and Copernicus Implementation it the region.
GEO-CRADLE is funded by the European Commission under H2020.
The Institute for Astronomy, Astrophysics, Space Applications & Remote Sensing of the National Observatory of Athens leads the project. Dr. Haris Kontoes is the coordinator and the most important software components are developed by the NOA IT team.
This project is a collaborative project which brought together 25 partners from across three regions of three continents (Balkans, Europe, Middle East, Asia & North Africa, Africa).
A key concept of the GEO-CRADLE’s regional data hub is to become a regional gateway to GEOSS portal by providing direct access to GEOSS datasets through the GCI and thus provide a successful example of a community portal built upon GEO DAB APIs

3. EO – stakeholders data sets
The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
GEOSS Portal
High Level Architecture
EO – stakeholders data sets
Geo Cradle pilots
Web Services
Regional Data Hub
DKAN UI
User n-2
User n-1
User n
User n+3
User n+2
User n+1
- This is the high level architecture of the integrated data hub which is harvesting datasets from multiple sources, not only the GEOSS portal, but also the pilots and third party data providers and makes these resources available to its users through a UI interface built upon DKAN.

4. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Print Screens

5. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Print Screens cont’d

6. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Print Screens cont’d

7. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Print Screens cont’d

8. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
A. Admin panel
Why DKAN?
B. Search & View
Powerful, ready to use admin panel
Easy to define content types
Easy to add content (e.g. Datasets, resources)
Manage users, accessibility, taxonomies, etc.
Ready to use/customize search and view of data
Fully customizable page layout through panels
A multitude of APIs in order to implement various functionalities
Faceted full-text search through Facet, Views and Search APIs
C. Big Community
Big open-source Community
DKAN is a complimentary offering to CKAN.
Integrates open data catalog features into Drupal CMS, which is build upon PHP.
PHP powers a significant percentage of Web, while Drupal powers ~2% of the Internet as a whole.
Has a wide community of active users/developers.
Ready to use powerful administration panel that allows to easily add content, whether it is a dataset comprised from files, remote files, sites and/or APIs.
Ready to use pluggable modules that allow to easily discover and access content, customize the display of the content, add facets, etc…
Big active, open-source community since it is developed in Drupal and PHP.

9. Innovation towards Challenges
The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Innovation towards Challenges
The Challenge
An integrated Search & Display mechanism that offers the users unified,
centralized & user-friendly
interface
DKAN
DKAN admin, search & display functionalities ready to use. A multitude of modules for the open data purposes.
Again the DKAN rendering mechanism uses the local
database.
GEOSS accessible through the GEO DAB APIs (the major remote source). Pilots will follow.
Mission
By default DKAN looks up for datasets and resources in a single local database
Intg. Search & display
GEO DAB APIs
The GEO DAB APIs are very easy to use. You just plug and play!
GEO-CRADLE Regional Data Hub
Access to region-related GEOSS datasets and services.
Centralised gateway for regional data providers to contribute easily and timely their products to GEOSS.
Display
on the fly
Two mature systems
However, the eventual integration of GEO DAB APIs into the DKAN ecosystem was a great challenge itself!
Remote sources
Multiple Sources
DKAN and GEO DAB APIs are two mature systems. However the integration of GEO DAB APIs into DKAN was a great challenge itself.
This challenge comprised of several smaller challenges was faced through several somehow innovative steps.
Initially we had to enable DKAN to search for data in multiple sources, since DKAN by default uses a local database a single source of data.
Then we had to enable the search of remote sources, such as the GCI and this is where the GEO DAB APIs came into the play.
In the next step (which is the most difficult one and not yet entirely accomplished) we had to make DKAN capable of displaying the remote data without storing it somewhere, i.e. display on the fly.
Then we had to build an integrated search & display mechanism that allows its users through a user-friendly interface to search for open EO data in a centralized and unified mode.
And our mission is almost accomplished which is the access of regional GEOSS datasets.

10. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Conclusions
GEOSS is a vital component for the creation of a universe of open EO data, together with other initiatives including NextGEOSS, DIAS platform, etc.
The GEOSS Common Infrastructure (GCI) comprises the backbone and the necessary toolbox for the uptake of EO services & applications.
The GEO Discovery and Access Broker (DAB) is a subtle component for accessing EO data.
Offers a rich suite of standard protocols (a.k.a. interfaces).
It’s really plug and play; once you read the documentation it requires the minimum effort to integrate those in your development.
Its discovery, access & semantics functionalities support properly the development of regional data hubs.
The development of GEO-CRADLE Regional Data Hub revealed issues that made evident the difficulty of integrating the GEO DAB APIs in DKAN, especially in our attempt to display “on-the-fly” the brokered information of GEOSS, because DKAN is a strict system designed to “ingest” content either manually or through harvesting other DKAN installations, whereas on the other hand GCI is an abstract system designed to broker datasets from “everywhere” and mostly in a distributed mode.
In addition DKAN uses taxonomies (taxonomy terms & vocabularies) in order to classify the content according to format, keywords, topics, etc. While GCI users free text.
The solution to problems like these should likely be the provision of tools that could bridge gap between those systems.
For example GCI could provide an analytics tool allowing applications to periodically poll for statistics regarding formats, topics, keywords and other meaningful data patterns related with the GEOSS brokered resources.
DKAN is suggested to be used because:
Provides powerful plug-in modules for access, discovery of open data, as well as for the visualization, registration of them.
Greatly decouples the data model from the data rendering mechanism.
Enhances existing GEO discovery and access mechanisms like the ones provided by the GEO DAB APIs.
So here are some of our conclusions:
GEOSS is a vital component in our attempts to build a universe of open EO data, along with other initiatives such as NextGEOSS, DIAS, etc. GCI is a core component towards this direction.
The GEO DAB APIs are really plug and play and offer significant functionalities which indeed provide the necessary interfaces for accessing and discovering GEOSS brokered resources.
DKAN is recommended to be used because of the powerful plugin modules, the great decoupling that succeeds between the data model and the data visualization and because it can really enhance the GEO DAB APIs.
The development of the GEO-CRADLE regional data hub using DKAN and the GEO DAB APIs revealed some issues regarding the integration of the APIs in another software application. The greatest issue is related with displaying on the fly the GEOSS brokered datasets. DKAN is a strict system, assuming that the data are stored in a local database, inserted manually or through harvesting another DKAN installation. Moreover it uses taxonomies in order to classify the data according to the format, the keywords, the topics, etc. On the other hand GCI is an abstract system design to broker datasets from everywhere, in a distributed mode and sometimes the information that returns is a bit fuzzy.
Maybe some tools could resolve such problems. An analytics tool that would keep track of all the formats of all the brokered resources and could be polled periodically might be an example.

11. The GEO-CRADLE Regional Data Hub tool: Utilizing the GEO DAB APIs for easy access and discovery of regional EO data
Roadmap
Continue the developments for the GEO-CRADLE Regional Data Hub, making it the gateway of GEOSS portal in the region and operating it as a focal node towards GEOSS and Copernicus implementation.
Use the Data Hub as a platform for making available the GEO-CRADLE pilots’ data and results and allow the regional stakeholders to harvest them.
Bring third-party data providers suitable for registration in GCI (e.g. JRC Danube Portal and some tenths of regional portals) in the course of the project. Connect Data Hub with these providers too.
Facilitate the exchange of knowledge on existing EO capacities (data, services, models, etc.) in the region.
Showcase concrete examples of tackling regional challenges related to adaptation of climate change, improved food security & water extremes management, better access to raw materials and energy.
Continue the development until we make GEO CRADLE Regional Data Hub an operational regional GEOSS focal node.
Connect the data hub with the pilots’ data.
Continue to bring third party regional data providers to register in GCI. JRC Danube is a success story.
Further facilitate the exchange of knowledge.
Showcase examples of how the data hub will help in tackling regional problems related with climate, water management, energy, etc.

12. THANK YOU FOR LISTENING
THE END
THANK YOU FOR LISTENING
I’m Happy to answer to your questions!