Quality Assessments of Statistical Production Processes in Eurostat Pierre Ecochard and Małgorzata Szczęsna

Quality assessment in the ESS: “Over the last three years ( ), how many of your statistical processes have been monitored using:”

Quality assessment in the ESS: “Over the next three years ( ), how many of your statistical processes do you plan to monitor using:”

Audits and self-assessments in the ESS Number of NSIs conducting: AuditsSAs 5 NSIs9 NSIs10 NSIs 7 NSIs

The European Statistics Code of Practice The Eurostat Quality Assurance Framework Documentation Measurement EvaluationConformity Quality reports Quality indics Process descriptions Process variables etc Quality AssessmentsLabelling Background

What is Eurostat Quality Assessment? A systematic review and evaluation of all stages of the statistical production process with the use of the DESAP- based Checklist IT conditions; Management, planning and legislation; Staff, work conditions and competence User needs Data collection Validation Confidentiality Dissemination Documentation Follow-up

Assessment Outputs (1) Assessment Report

Assessment Outputs (2) Coherence General coherence Timeliness Timeliness of final publication Accuracy Overall accuracy Relevance User satisfaction Accessibility and clarity Overall quality of metadata Comparability over time Comparability across countries Assessment Diagram

Assessment Outputs (3) Highlight of good practice

Categories of Eurostat assessments Self- Assessment Supported Self-Assessment Peer ReviewRolling Review Process 1 Process 4 Process 2 Process 3 Characteristics: - Periodicity - Legal Basis - Output - ESTAT intervention Similiarity: DESAP-based Checklist, outputs Difference: extent of external interventation in a review Process 5

Office-wide implementation plan Approach piloted in two domains in 2007 Most of the statistical process will benefit from an assessment within a three-year period reviews planned for 2008: –14 Supported Self-Assessments –14 Self-Assessments –4 Rolling Reviews –1 Peer Review Follow-up report to Eurostat management by the end of 2008

Benefits of quality assessment For production teams: An opportunity for a systematic analysis of the production process Identify and prioritize improvement actions Spread and benefit from the Good Practice For Eurostat: Identify horizontal problematic issues Foster standardisation of statistical processes Support resource allocation, planning and programming Show quality commitment

Feedback The general workflow works well, the Checklist is flexible and the assessments are considered useful by domain managers Heads of Unit should be involved earlier The diagram can be a red herring: it should be used with caution The ownership of the results should be made very clear More assessment should involve an external expert

Keys for success in implementing quality assessments Top management commitment Middle management acceptance Sound communication Long term perspectives Implementation and fine-tuning in pilot projects Standardised use of methods Clear responsibilities and ownership Sufficient resources allocated for the assessments