Pilot use of Linked Open Data technologies for publishing official statistics: current status in the ESS and Eurostat April 17th, 2018 GISCO WG

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

DIGICOM: Modernising communication and dissemination of European statistics User analysis Innovative and shareable products DIGICOM - WP 3 Open data dissemination Communication and promotion Open data dissemination User analysis LOD APIs PUF

Key benefits of Linked (Open) Data approaches The statistical data and metadata, based on RDF standards, become (web) addressable. This allows publishers and third parties to annotate and link to this data/metadata. Data can be flexibly combined across datasets. The statistical data becomes an integral part of the broader web of linked data. Publishing as linked-data offers a flexible, non-proprietary, machine programmable means for providing advanced and dynamic querying and visualisation capabilities

Joint framework for action and roadmap

Joint framework for action and roadmap Development of a proposal for a reference architecture for LOD and for the relevant governance aspects (2017) Elaboration of joint pilots involving several NSIs, focusing on data and metadata (2018) ESSnet Development of a community of (best) practice for all NSIs investing in LOD or wishing to learn (2018) ESSnet Engagement with users and stakeholders Training and knowledge sharing Guidance on how to design and build an LOD portal Implementation of standards Toolkit: evaluation and selection of a standard set of tools, recommendations on performance optimisation

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

About ESSnet Four NSIs Bulgaria (Project Coordinator), France, Ireland, Italy Subcontracting work to academic experts from research communities in France and Ireland Objectives of ESSnet To explore NSIs publishing statistical data as LOSD on the semantic web To allow users to engage with statistical data easily as linked open data To prepare NSIs in the wider ESS to publish LOSD To recommend a way forward for the ESS to adopt and exploit LOSD Launched November 2017, initial LOSD published April and October 2017, final deliverables in April 2019

Expected results of the ESSnet WP1: LOD pilot An extensive pilot (data and metadata) Assessment from user perspective WP3: Networking and capacity building Collaboration platform: build a community of practice within the international community of official statisticians Express your interest if you want to join Training (webinars) Toolkit

Clean CSO Statbank Staging 5* - Cork City Profile CSO Apps CSO Automatic eReleases CSO Horizontal eReport Clean CSO Statbank Staging HSE OSi 2013 GDP 2016 GDP CCC (Query Based API) Automatic JSON/RDF CSO Statbank INSEE Data Analyst UN ISTAT Release DATA GDP 0101 1010 Machine Readable Catalogue CKAN Lookup NSI BG Cloud based LOD platform Data Intermediary Bots European Data Portal KEY 4* Semantic Web Node Semantic Link Read / Copy European Data Portal IRELAND 5* LOD ESSnet WP1 PROTOTYPE

ESSnet principles Agile - initially linking one Census dataset then expanding further Link data locally, nationally and internationally incorporating non statistical datasets e.g. spatial, local government Each NSI hosts their own single source of RDF data NSI links RDF datasets to the European Data Portal (EDP) Data analyst accesses LOSD through a cloud based tool pulling from EDP Access to LOSD Open Cube toolkit (openly licensed LOSD, open source tools, freely available, cloud based) NSI data analysts can access and create cross domain products and services e.g. linking statistical, spatial and local government datasets for an easy to use citizen friendly view Capacity building in ESS once proven model is created

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

LOD in ESS context Capability definitions Relevant standards Building blocks High level Application Architecture LOD principles Template for Use cases

ESS LOD Reference Architecture Adopting a common base ontology and a common vocabulary expressing the meaning behind the data they expose will enable ESS members to publish that data on a «queryable» endpoint so that they can be linked across organisations

ESS LOD Reference Architecture Draft document for review and discussion with ESSnet LOD pilots/ESS EAB Aims at providing a template for statistical organisations in the development of their own LOD capabilities and infrastructure Not binding, but helps to spread common views and definitions and to foster collaboration between partners Basis for ESS LOD strategy and mapping for ESS developments

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

Purpose Mirror of ESSnet activities: Cross-domains, cross-institutions (EU agency) datasets exposed as linked data together with metadata Explore and evaluate data discoverability, data integration, data visualisation/analytics capabilities offered by the technology Building internal capabilities and demonstrate business benefits Identify existing infrastructures and reusable component at EC level

outline Background: ESS roadmap on LOD ESSNet of Linked Open Statistical Data ESS LOD reference architecture Eurostat LOD pilot Linked Open Statistical Metadata (ISA2 project)

EC ISA2 program Eurostat project proposal aiming at enhancing statistical data and metadata discoverability and analytics capabilities for users (policy DG's, external researchers, data scientists, ….)   2018 : Studies and PoC Architecture definition (business, information, application, security), including semantic aspects PoC using confidential data and "confidentiality on the fly" (table builder) 2019 -2020 : TBC Reference ontology Semantic repository exposing metadata assets (nomenclatures, concept definitions…) Basic services (UI, metadata management, …) -> successors of RAMON Advanced services : confidentiality on the fly, semantic query and data analytics