ESS VIP project on Validation Presented by: Pedro Díaz EUROSTAT

CONTENT Scope & objectives Business case Link with previous projects & maturity Proposed approach & milestones Next steps

Scope & objectives Documentation / Standardisation Methodological analysis of Data Validation Distribution of responsibilities in the production chain Preparing for IS/IT solutions and architecture

Scope & objectives Coherent validation policy in the different statistical domains, in cooperation with MS. Standard validation language Solutions for complex validation actions to ensure: Coherence between data files Coherence between Member States Integrity of the ESS data

Scope & objectives Contribution to the VISION: More efficient production chain Clearly attributed responsibilities Standard Description of validation steps Description of validation language Development of functional specifications Common tools to be shared within the ESS Contribution to the VISION:

Business case: Need for change Two steps circular validation process Step 1: Validation in Member States Step 2: Validation in Eurostat Lack of harmonisation In documentation (validation procedures & rules) In coordination: "Validation ping pong" In software solutions Subjective assessment of data quality Risk of low quality results

Link with previous projects & maturity Deliverables of VIP –Validation (until end 2012) Guidelines for documentation of validation steps Structured inventory of validation rules used in Eurostat Template to deal with error messages Human Understandable Syntax for validation rules Functional specifications for adapting existing IT tools Awareness actions

Link with previous projects & maturity Favourable reaction of Member States in ITDG: Oct 2011 DIME: March 2012 ESSC: May 2012 Working Groups

Proposed approach & milestones Packages: 1 - VIPV IMPLEMENTATION Implementing methodological outcomes of VIPV Phase I 2 - VIPV DATA MODEL – Household Survey Data model and validation for Household micro data 3 - VIPV TOOLS Validation solutions and other related IT services 4 - VIPV EXTENSIONS AND GENERAL COORDINATION

Proposed approach & milestones Project team Steering Committee Workshops with users Assessment and monitoring per domain Decentralised (Working Groups) High level monitoring (ESSC) Methodological / IT Solutions Links with other ESS VIP Validation Warehouses SIMSTAT Shared services …

Next steps: 2013 - 2014 Data sent to Eurostat already validated (push mode, in-file validation, micro-macro) Enablers: Descriptive syntax Governance mechanism with Member States Implementation monitoring Adaptation of IT Tools Micro data specificities

Next steps: 2014 - 2020 Extended validation (across files, time series) Need for shared databases Need of shared tools Mirror checks: One country checking after countries' data Confidentiality treatment Enhancement of transmission tools Change roles (Member States, Eurostat) Pull and shared modes Common warehouses Census hub technologies

Next steps Why an ESS VIP? Role of NSI's in project phases 1 - Implementation 2 – Data model HHS 3 - Tools 4 – Extensions and General coordination Methodological Task Force Validate phase-1 deliverables Advise on implementation/monitoring Definition of ESSNet work ESSNet

Next steps Why an ESS VIP? Role of NSI's in project phases 1 - Implementation 2 – Data model HHS 3 - Tools 4 – Extensions and General coordination Methodological Task Force Validate phase-1 deliverables Advise on implementation / monitoring Definition of ESSNet work ESSNet