1. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Proposal of a revised approach for data validation within the European Statistical System Michel HENRARD European Commission – EUROSTAT

2. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 The current situation Validation process : 2 separated steps –Member States –Eurostat Double work ? Zero Work ? Ping-Pong ?

3. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 The current situation Harmonisation –Validation procedures Harmonised? Documented ? Formally agreed? Coordinated ? –Validation rules Common syntax ? Formalised ?

4. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 The current situation Harmonisation of software solutions –Standard building blocks within Eurostat Partial degree on integration Many ad-hoc solutions –High number of supported solutions Does not help streamlining of production processes High cost of development and maintenance Common validation solutions within the ESS ?

5. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 The current situation Compliance monitoring –Assessment of data quality Subjective interpretation Not structured

6. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Yielding efficiency gains Vision Infrastructure Project on Validation (end 2010) –Validation solutions that can be used both by the Member States & Eurostat –Validation as soon as possible –Definition of the roles in the validation chain

7. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Project proposals Better & standard documentation of the validation process –Should be part of the general documentation of the complete production system

8. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Project proposals Agreed & standard description of checking rules –Common language (syntax) –Guidelines for the selection of the validation rules

9. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Project proposals Availability of common software solutions –Common standard IT tools and associated services –Automation of procedures within the ESS

10. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Project proposals Clear & agreed distribution of responsibilities –Validation where it can be done in the most efficient way

11. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 How ? Process Description –Systematic standard description of The data validation The validation rules using a common syntax

12. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 How ? Validation review with Member States –Gradual introduction of objective measurement of data quality –Clear responsibility for each validation rule based on efficiency considerations (the sooner the better) –Calendar of introduction of the new mechanism

13. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 How ? Integration of validation in compliance assessment procedures –Based on the agreed set of validation rules –Reports on validation problems in the statistical domains

14. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 How ? Improve the coherence of validation solutions within Eurostat –Taxonomy of validation rules –Standard description of the validation rules (common language/syntax) –Good practices –Current validation building blocks adapted to the users’ requirements

15. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 How ? Provision of standard solutions to be shared in the ESS –Deployment of a standardized validation language –Adapting the IT tools to be shared in Eurostat & to be offered to Member States

16. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 What the VIP on Validation does provide ? Templates & Guidelines –Documentation of the validation process –Error messages and reporting –Selection criteria of validation rules ensuring minimum quality standard –Attribution of responsibility of validation tasks

17. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 What the VIP on Validation does provide ? Inventory of validation rules used in Eurostat Typology of validation rules Human understandable language for validation rules User requirements & functional specifications to adapt the IT tools to the needs identified

18. Statistical data editing - UNECE work session – OSLO 24-26 September 2012 Q&A Thank you for your attention