ESS Vision 2020: ESS.VIP Validation DIME/ITDG plenary meeting 24th February 2016 Item 1 of the agenda

Contents Summary of the objectives and achievements of the ESS.VIP Validation Proposals on follow-up actions to the ESS.VIP Validation

What is validation? UNECE's Glossary on Statistical data editing: "Data validation is "an activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values." It is not: An activity to check or assess process metadata: validation focuses on data Editing or imputation: these are separate activities which may be performed based on the outcomes of validation The ESS.VIP Validation took as a starting point for its activities the following definition given by the UNECE: Data validation is "an activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values." Data validation is focused on checking the validity/consistency of data. Checking process or structural metadata is not within the scope of validation, though process or structural metadata may serve as input to validation procedures.

Focus of ESS.VIP Validation NSI ESTAT Validation can take place in several points of the ESS statistical production process

As-is situation

Current problems in ESS validation No clear picture of who in the ESS is doing what as regards validation: Risk of validation gaps Time-consuming validation "ping-pong" Subjective assessment of data quality Lack of standards and of a common architecture for shareable and reusable validation services: Duplication of IT development costs within the ESS Manual work due to low integration between the different tools

Medium-term goals for ESS validation Business Outcomes Goal 1: Ensure the transparency of the validation procedures applied to the data sent to Eurostat by the ESS Member States. Increase in the quality and credibility of European statistics   Reduction of costs related to the time-consuming validation cycle in the ESS ("validation Ping-Pong") Goal 2: Enable sharing and re-use of validation services across the ESS on a voluntary basis. Reduction of costs related to IT development and maintenance

ESS.VIP Validation The ESSC approved the ESS.VIP Validation project in November 2012 and gave the green light for the beginning of the execution phase in November 2014. The project ended in November 2015. Collaboration with the Member States was ensured through a Task Force during the course of 2014 and through an ESSnet (the ValiDat Foundation ESSnet) during the course of 2015.

As-is situation

To-be situation

Validation Rule Registry NSI production systems Scenario 1 Member States receive the common rules and implement them in their own validation systems Validation Rule Registry ESS service platform NSI environment Data to be sent to Eurostat Transmission to Eurostat NSI production systems

Validation Rule Registry Scenario 2 Member States use common ESS services in their process for validation Validation Rule Registry ESS service platform Common ESS service NSI environment Transmission to Eurostat Data to be sent to Eurostat NSI production system Common ESS service

Validation Rule Registry Scenario 3 Member States use common ESS processes for validation Validation Rule Registry Common ESS service ESS service platform Common ESS service NSI environment Transmission to Eurostat Data to be sent to Eurostat

High-level deliverables Needed developments Developments on five dimensions are needed to achieve the two medium-term objectives Dimension High-level deliverables Methodology Common definition, classifications and metrics for validation Standards Standard ESS validation syntax and validation reports IT Common architecture for validation solutions and shareable services HR Communication and training Processes/Governance Common validation policy

ESS.VIP Validation - Methodology Methodological handbook The methodological handbook was written by the ESSnet. It contains common definitions, classifications and metrics and provides a common business language to discuss validation in the ESS. Many parts of the handbook are based on documents developed by Eurostat over the course of 2013-2014 (i.e. classification in terms of validation levels, classification in terms of validation typology, etc…). The methodological handbook would provide the basis for the definition of a template for the description and definition of validation rules by the different subject matter Working Groups.

ESS.VIP Validation - Standards VTL A new standard language for validation created by the SDMX community: VTL (Validation and Transformation Language) Version 1.0 came out in March 2015 VTL is a potential standard syntax for validation created by the SDMX Sponsors. The aim of VTL is to provide an unambiguous language to communicate validation rules between different statistical organisations. Through the ESS.VIP Validation Eurostat contributed to the development of VTL 1.0, which was published in March 2015. The ESS.VIP Validation has also conducted a pilot translation of validation rules into VTL in selected domains (i.e. SIMSTAT, Animal Production Statistics, National Accounts coming soon). Through the ESSnet, the ESS.VIP Validation is also conducting a thorough review of VTL to see whether it can meet the criteria for a standard ESS validation language.

ESS.VIP Validation – Processes Business and IT architecture The ESS.VIP Validation produced a Proposal for a Business and IT Architecture for the ESS The Proposal details the medium-term changes in the business processes and IT infrastructure for validation in the ESS

ESS.VIP Validation - IT IT developments

Contents Summary of the objectives and achievements of the ESS.VIP Validation Proposals on follow-up actions to the ESS.VIP Validation

Next steps: "The Wiesbaden consensus" "The adoption of a common [validation] infrastructure is based on a clear business case for the member states of the ESS in terms of costs and benefits." "Embrace the development of a standardised validation language as a first step towards a European end-to-end data validation policy." "A standard specification language for validation rules is indispensable. […] VTL is an interesting candidate, but it needs considerable improvement. The ESS validation community will contribute to that goal."

Next steps: February ESSC conclusions "Member States took note of the results of the Validation project and supported the proposed roadmap for future actions in this area."

Priorities for next steps The ESS.VIP Validation concentrated on providing the architectural and methodological frameworks to achieve the two medium-term goals. In order to realise the two goals, two priorities: Implementing the available actionable deliverables Transforming the available enabling deliverables in actionable ones

Priorities for next steps Medium-term goal 1

Priorities for next steps Medium-term goal 1 Short-term actions (up until Mid-2017) Improvements to VTL 1.0 and piloting in statistical domains Implementation of methodological handbook in statistical domains ESS validation policy Medium-term actions (Mid-2017 onwards) Implementation of the ESS Validation policy

Priorities for next steps Medium-term goal 2

Priorities for next steps Medium-term goal 2 Short-term actions (up until Mid-2017) Testing of available prototypes Definition of requirements for shareable ESS validation services Medium-term actions (Mid-2017 onwards) Development of shareable ESS services

Collaboration with Member States Two possible vehicles to collaborate with Member States (not mutually exclusive) Task Force under Methodology Working Group Functions as VTL User Group Develops ESS validation policy Development of shareable ESS services Develops requirements for shareable ESS services Pilots national implementations

Next steps Time Actor Action February 2016 DIME/ITDG Creation of an informal high-level group of DIME/ITDG members to steer the definition of the follow-up actions February-April 2016 Consultation of the informal high-level group on the follow-up actions to be taken March 2016 VIN Response to deployment questionnaire and Eurostat analysis of the results VIG Consultation of the VIG on the follow-up actions to be taken May 2016 ESSC Presentation of a set of follow-up actions for approval

Questions to DIME/ITDG participants Do you agree with the priorities proposed as regards follow-up actions to the ESS.VIP Validation? Do you think Member States should be involved via a Task Force, ESSnet or both? What should be the respective roles of these bodies? Do you agree with the timeline and proposed course of action for the definition of the next steps? Would you be willing to participate in the informal high-level group of DIME/ITDG members proposed?