1. ESS Validation State of Play and next steps
DIME/ITDG Steering Group Meeting
Item 5
Zagreb, 26 June 2017
Eurostat, Unit B1

2. Presentation Outline Goals and expected outcomes for ESS Validation
Business Architecture for ESS Validation
Validation rules (including VTL)

3. Goals and expected outcomes
for ESS Validation
Medium-term goals
Business Outcomes
Goal 1: Ensure the transparency of the validation procedures applied to the data sent to Eurostat by the ESS Member States.
Increase in the quality and credibility of European statistics
Reduction of costs related to the time-consuming validation cycle in the ESS ("validation Ping-Pong")
Goal 2: Enable sharing and re-use of validation services across the ESS on a voluntary basis.
Reduction of costs related to IT development and maintenance

4. BA for ESS Validation
Approved by TF Validation, ESS Entreprise Architecture Board and WG on Methodology
Clarifies:
To-be state for validation in the ESS
Roles of Eurostat and Member States in the validation of data
How common validation services could be used in ESS
Includes:
Validation principles
Business, Information and Application views
3 optional scenarios for implementation by countries
Detailed acceptance process

5. BA for ESS Validation: Principles
The sooner, the better
Trust, but verify
Well-documented and appropriately communicated validation rules
Well-documented and appropriately communicated validation errors
Comply or explain
Good enough is the new perfect

6. BA for ESS Validation: Severity levels
Errors: rules must necessarily be satisfied for mathematical or logical reasons. Data containing errors are not considered acceptable.
Warnings: highlight suspicious values. Data with warnings may be accepted upon further verification and/or justification.
Information: highlight potentially suspicious values which do not usually require further justification for acceptance.

7. BA for ESS Validation:
Tools and Services (1)

8. BA for ESS Validation: Tools and services (2)
STRUVAL available at Eurostat in Process Manager and as shared service
CONVAL-1 (no VTL) available at Eurostat in Process Manager
CONVAL-2 (with VTL) Q1/2018 for test and Q4/2018 as shared service
Validation Rule Manager Q3/2018 for test and Q4/2018 as shared service from COM website
Validation Rule Registry Q4/2018 (for all 3 scenarios)

9. BA for ESS Validation: Options for implementation in NSIs
Connect to central repositories
Installation of service/tool in own environment, access to repositories (SDMX DSDs, validation rules)
Connect to CSPA compliant service
Integration of service into own workflow
Connect to the process
Transmission of pre-validated data to Eurostat, Eurostat runs service and delivers report
Grants
Applications from 9 June – 8 Sept 2017
Action Jan/Feb (12 months)
€ total, € min/MS, EU 90% of eligible costs
ESSnet deliverables

10. Validation rules
Assessment of VTL for content validation rules in 5 domains
Need to harmonise description of validation rules, link to DSD,
VTL suitable, support needed for complex rules
Proof of concept of a VTL interpreter (VTL sandbox)
VTL 1.1 Q3/2017
Standard framework (template, guidelines, best practices, …) for describing and agreeing validation rules and roles Q3/2017
Roll-out to cover statistical domains Q4/2019

11. Communication The ESS Validation Video:
Business Architecture for ESS Validation: here
ESTP course on Validation Nov 2017

12. Do you have any QUESTIONS
Do you have any QUESTIONS ? What are your plans for implementing the ESS Validation architecture?

13. ESSnet Validat Integration
Timeline: Jan 2017 and March 2018
6 countries: DE, LT, NL, PL, PT, SE
Work Packages:
WP1: Finalisation of the Methodological Handbook for Validation
WP2: Creation of a common validation report for the ESS
WP3: Pilot implementation of the 3 scenarios of the Business and IT architecture and cost/benefit analysis of the implementations
WP4: Dissemination in the ESS

14. ESSnet Validat Integration
WP1: Methodolocical Handbook
Identified potential areas for improvement of Methodological Handbook
Started rewrite of some sections
Preparation of a survey of validation metrics in use in ESS
WP2: Common Validation Report for the ESS
Identified 126 items to report (reduced to 96)
(Re-)Definition of machine-readable report structure ongoing

15. ESSnet Validat Integration
WP3: Implementation of Business Architecture
Proof of Concept of VTL translator almost finished
Preparation of a model for cost/benefit analysis of implementation scenarios by end of June
Start pilot implementations in July
WP4: Four Regional Conferences
(PT) Lisbon
(NL) The Hague
(LT) Vilnius
(RS) Belgrade

16. The Hague, 11.-12.01.2018 Vilnius, 01.-02.02.2018 Belgrade,
Lisbon,