1. ESS Vision 2020: ESS.VIP Validation
DIME/ITDG plenary meeting
24th February 2016
Item 1 of the agenda

2. Contents
Summary of the objectives and achievements of the ESS.VIP Validation
Proposals on follow-up actions to the ESS.VIP Validation

3. What is validation? UNECE's Glossary on Statistical data editing:
"Data validation is "an activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values."
It is not:
An activity to check or assess process metadata: validation focuses on data
Editing or imputation: these are separate activities which may be performed based on the outcomes of validation
The ESS.VIP Validation took as a starting point for its activities the following definition given by the UNECE: Data validation is "an activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values."
Data validation is focused on checking the validity/consistency of data. Checking process or structural metadata is not within the scope of validation, though process or structural metadata may serve as input to validation procedures.

4. Focus of ESS.VIP Validation
NSI
ESTAT
Validation can take place in several points of the ESS statistical production process

5. As-is situation

6. Current problems in ESS validation
No clear picture of who in the ESS is doing what as regards validation:
Risk of validation gaps
Time-consuming validation "ping-pong"
Subjective assessment of data quality
Lack of standards and of a common architecture for shareable and reusable validation services:
Duplication of IT development costs within the ESS
Manual work due to low integration between the different tools

7. Medium-term goals for ESS validation
Business Outcomes
Goal 1: Ensure the transparency of the validation procedures applied to the data sent to Eurostat by the ESS Member States.
Increase in the quality and credibility of European statistics
Reduction of costs related to the time-consuming validation cycle in the ESS ("validation Ping-Pong")
Goal 2: Enable sharing and re-use of validation services across the ESS on a voluntary basis.
Reduction of costs related to IT development and maintenance

8. ESS.VIP Validation
The ESSC approved the ESS.VIP Validation project in November 2012 and gave the green light for the beginning of the execution phase in November The project ended in November Collaboration with the Member States was ensured through a Task Force during the course of 2014 and through an ESSnet (the ValiDat Foundation ESSnet) during the course of 2015.

9. As-is situation

10. To-be situation

11. Validation Rule Registry NSI production systems
Scenario 1
Member States receive the common rules and implement them in their own validation systems
Validation Rule Registry
ESS service platform
NSI environment
Data to be sent to Eurostat
Transmission to Eurostat
NSI production systems

12. Validation Rule Registry
Scenario 2
Member States use common ESS services in their process for validation
Validation Rule Registry
ESS service platform
Common ESS service
NSI environment
Transmission to Eurostat
Data to be sent to Eurostat
NSI production system
Common ESS service

13. Validation Rule Registry
Scenario 3
Member States use common ESS processes for validation
Validation Rule Registry
Common ESS service
ESS service platform
Common ESS service
NSI environment
Transmission to Eurostat
Data to be sent to Eurostat

14. High-level deliverables
Needed developments
Developments on five dimensions are needed to achieve the two medium-term objectives
Dimension
High-level deliverables
Methodology
Common definition, classifications and metrics for validation
Standards
Standard ESS validation syntax and validation reports IT
Common architecture for validation solutions and shareable services HR
Communication and training
Processes/Governance
Common validation policy

15. ESS.VIP Validation - Methodology Methodological handbook
The methodological handbook was written by the ESSnet.
It contains common definitions, classifications and metrics and provides a common business language to discuss validation in the ESS.
Many parts of the handbook are based on documents developed by Eurostat over the course of (i.e. classification in terms of validation levels, classification in terms of validation typology, etc…).
The methodological handbook would provide the basis for the definition of a template for the description and definition of validation rules by the different subject matter Working Groups.

16. ESS.VIP Validation - Standards
VTL
A new standard language for validation created by the SDMX community: VTL (Validation and Transformation Language)
Version 1.0 came out in March 2015
VTL is a potential standard syntax for validation created by the SDMX Sponsors. The aim of VTL is to provide an unambiguous language to communicate validation rules between different statistical organisations.
Through the ESS.VIP Validation Eurostat contributed to the development of VTL 1.0, which was published in March 2015.
The ESS.VIP Validation has also conducted a pilot translation of validation rules into VTL in selected domains (i.e. SIMSTAT, Animal Production Statistics, National Accounts coming soon). Through the ESSnet, the ESS.VIP Validation is also conducting a thorough review of VTL to see whether it can meet the criteria for a standard ESS validation language.

17. ESS.VIP Validation – Processes Business and IT architecture
The ESS.VIP Validation produced a Proposal for a Business and IT Architecture for the ESS
The Proposal details the medium-term changes in the business processes and IT infrastructure for validation in the ESS

18. ESS.VIP Validation - IT
IT developments

19. Contents
Summary of the objectives and achievements of the ESS.VIP Validation
Proposals on follow-up actions to the ESS.VIP Validation

20. Next steps: "The Wiesbaden consensus"
"The adoption of a common [validation] infrastructure is based on a clear business case for the member states of the ESS in terms of costs and benefits."
"Embrace the development of a standardised validation language as a first step towards a European end-to-end data validation policy."
"A standard specification language for validation rules is indispensable. […] VTL is an interesting candidate, but it needs considerable improvement. The ESS validation community will contribute to that goal."

21. Next steps: February ESSC conclusions
"Member States took note of the results of the Validation project and supported the proposed roadmap for future actions in this area."

22. Priorities for next steps
The ESS.VIP Validation concentrated on providing the architectural and methodological frameworks to achieve the two medium-term goals.
In order to realise the two goals, two priorities:
Implementing the available actionable deliverables
Transforming the available enabling deliverables in actionable ones

23. Priorities for next steps
Medium-term goal 1

24. Priorities for next steps
Medium-term goal 1
Short-term actions (up until Mid-2017)
Improvements to VTL 1.0 and piloting in statistical domains
Implementation of methodological handbook in statistical domains
ESS validation policy
Medium-term actions (Mid-2017 onwards)
Implementation of the ESS Validation policy

25. Priorities for next steps
Medium-term goal 2

26. Priorities for next steps
Medium-term goal 2
Short-term actions (up until Mid-2017)
Testing of available prototypes
Definition of requirements for shareable ESS validation services
Medium-term actions (Mid-2017 onwards)
Development of shareable ESS services

27. Collaboration with Member States
Two possible vehicles to collaborate with Member States (not mutually exclusive)
Task Force under Methodology Working Group
Functions as VTL User Group
Develops ESS validation policy
Development of shareable ESS services
Develops requirements for shareable ESS services
Pilots national implementations

28. Next steps Time Actor Action February 2016 DIME/ITDG
Creation of an informal high-level group of DIME/ITDG members to steer the definition of the follow-up actions
February-April 2016
Consultation of the informal high-level group on the follow-up actions to be taken
March 2016
VIN
Response to deployment questionnaire and Eurostat analysis of the results
VIG
Consultation of the VIG on the follow-up actions to be taken
May 2016
ESSC
Presentation of a set of follow-up actions for approval

29. Questions to DIME/ITDG participants
Do you agree with the priorities proposed as regards follow-up actions to the ESS.VIP Validation?
Do you think Member States should be involved via a Task Force, ESSnet or both? What should be the respective roles of these bodies?
Do you agree with the timeline and proposed course of action for the definition of the next steps? Would you be willing to participate in the informal high-level group of DIME/ITDG members proposed?