ESS.VIP.SERV Shared Services TF Meeting 28 Oct 2015 Agenda item B2 - 2 Eurostat

Agenda VISION 2020 and CSPA ESS.VIP.Shared Services Project objectives Roadmap Current Status Governance ESSNet

VISION 2020 Project Portfolio Portfolio of projects adopted by the ESSC – Feb 2015 SIMSTAT and REDESIGN Single market statistics and Redesign of Intrastat ESBRs European system of interoperable business registers VALIDATION Common data validation policy ADMIN Administrative data sources ESDEN European statistical data exchange network SERV Shared services DIGICOM Digital communication, User analytics and Innovative products BIGD Big data EVUG meeting 8-9 October 2015

CSPA - Problem statement

CSPA - Target architecture

Progress towards CSPA target architecture in ESS CSPA certified Progress towards CSPA target architecture in ESS Project ESS.VIP.SERV: Sharing common functionalities in the ESS ESS CSPA basecamp

ESS.VIP.SERV Objectives The purpose of the project is to: Implement governance structure to enable sharing of services Contribute to standardising the description of business needs by providing CSPA compliant guidelines for describing service definitions Host and maintain a CSPA service catalogue at the Commission/Eurostat containing also services identified at ESS level Provide guidelines and recommendations for the ESS enabling national projects to realize technological solutions for industrialisation and integration of processes and for the rationalisation of information systems. Provide an ESS/Eurostat SOA environment for sharing services in the ESS for which no replication is necessary. Provide Proof of Concepts statistical services that are re-used in various organizations

ESS.VIP.SERV Benefits Easier exchange of information about re-usable statistical services Flexible business processes Faster adoption of new production processes Cost reduction of development and maintenance of information systems Easier management of skills

2015 2016 2017 SERV ROADMAP |S2 |S1 |S2 ESS Service Catalogue 2015 2016 2017 |S1 |S2 |S2 ESS Service Catalogue SOA Infrastructure Example Services Service Candidates Service Guidelines and templates Open Source Struval Transformation Content Validation Time Series Questionnaire Generation Metadata Dissemination ESS Catalogue V1 … V1.1, V1.2 Configured Middleware Available for hosting ESTAT Services Available for hosting Member States Services List V1 List V2 Struv. ESTAT Struval for M.S. Transf. for ESTAT Transf. for M.S. EDIT-based V1 Content Validation V2 Implementation of Services Times Series, Questionnaire Generation,MD Dissemination in 3 Member States Current State Analysis Assessment Recomm. Delivery by ESSnet & TF Delivery by Eurostat Draft Final Coordination with Business Projects Provide guidance for business projects to deliver results in line with SERV CSPA Catal.

Implementation – Services in Business Projects Cooperation with ESS.VIPs: Validation – STRUVAL services ESBRS – Unique Identification service Cooperation among projects is the key

Start of the TF and ESSnet will bring the project to next gear. Project Status Deliverables already provided: Hosting of the CSPA Service Catalogue SOA Infrastructure for hosting services available Deliverables provided in the next 2 months: Structured validation service and NAPS-S POC Evaluation and start of ESSNet Project Reviews will follow: Jan 2016: By VIG May 2016: By ESSC Start of the TF and ESSnet will bring the project to next gear.

SOA Infrastructure MIDDLEWARE 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service implementation(s) Client Manages service interfacing Decouples/Abstracts the service interface from its implementation Manages 'light integration' -> protocol transformation (eg : HTTP -> JMS) Service throttling SLA management at runtime, audit trailing capabilities

Project Governance Project reports to VIG TF SERV is an integral part of the project Steering Group will be established soon: Core group – main involvement Wider group – for information

ESSnet Sharing Services in ESS From 2016 to 2017 Work packages: ESS guidelines and procedures Plan for a Centre of Excellence for sharing statistical services in ESS Apply services – provide real-life examples of re-use and integration of statistical services Identification of re-usable services and analysis of requirements Fostering open source solutions for statistical services

Annex / Backup slides

Main page https://webgate.ec.europa.eu/fpfis/wikis/display/CSPACatalog

Service description Follows the SERV service guidelines Split in three layers Service Definition Service specification Service implementation Entries must only have the "Service definition" part filled in

Service Definition

Service Specification

Service implementation (1/3)

Service implementation (2/3)

Service implementation (3/3)

Roles & Rights Catalogue is ECAS protected Main roles: users, editors, moderators Service owners are responsible to restrict access to their service descriptions

Discussion platform Comments are allowed on Definition, Specification and Implementation level Allows discussion between stakeholders Aims to get best out of the services before going towards the ral implementation

Limitations Not directly accessible, ECAS account is needed MediaWiki page has been created (freely accessible, but static content) Wiki based: no real databases for storage templates based labels driven difficult to implement business logic or cross connections

MediaWiki page Why? To have an authentication free entry point First place in Google result for search term "CSPACatalogue" Content: High level description of the CSPA project List of services included in the full catalogue Instruction to get access to the full catalogue https://webgate.ec.europa.eu/fpfis/mwikis/cspacatalogue

Main page

Service List

Virtualising services (1) 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service implementation Client Manages service interfacing Decouples/Abstracts the service interface from its implementation Manages 'light integration' -> protocol transformation (eg : HTTP -> JMS) Service throttling SLA management at runtime, audit trailing capabilities

Virtualising services (2) Abstracting impl. 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Weblogic Application Server Service V1 implementation Client Weblogic Application Server Service V2 implementation

Virtualising services (3) Throttling 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service V1 implementation Client Buffering requests Application Server Service V2 implementation

Virtualising services (4) Integration 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service V1 implementation RDBMS Client HTTP(S) SOAP Application Server Service V2 implementation JMS

Virtualising services (5) Integration 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service A-1 implementation Client HTTP(S) SOAP Application Server Service A-2 implementation e.g. employee information from two underlying system after a fusion

Virtualising services (5) Simplification(1) 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Client HTTP(S) SOAP Application Server Service A-1 implementation Client e.g. STruVal extracted from SDMXConverter HTTP(S) SOAP

Virtualising services (5) Simplification (2) 3rd parties EC/ESTAT (Intranet) Service Bus Service Mediation Application Server Service A implementation 1 Client HTTP(S) SOAP Application Server Service B implementation 2 Combining two simple actions in one. E.g. upload of data to a new data file which needs to be created first.

What changes with OSB Service Bus Mediation App1 Interface App2 Services F4 F5 F6 F12 App1 Interface App2 F1 F2 F3 F11 Service Bus Mediation F2 Services F1 F3 F11 F4 F5 F6 F12

OSB architecture (simple) Customer site Endpoint definition Binding http(s) web service interface (SOAP/REST) JMS (Java Message Service) File, email listener Pipeline Split in request and response pipeline Transformation of data Service/Java callouts Error handling Reporting SOAP = Simple Object Access Protocol REST = REpresentational State Transfer