Quality assurance activities at EUROSTAT CCSA Conference Helsinki, 6-7 May 2010 Martina Hahn, Eurostat

2 9/21/2015 Structure of the presentation Code of Practice Institutional environment Process quality Output quality ESS Quality Architecture

3 European Statistics Code of Practice

4 Content of the Code of Practice 3 headings – 15 principles – 77 indicators Institutional environment Professional independence, Mandate for data collection, Adequacy of Resources, Quality commitment, Statistical confidentiality, Impartiality and objectivity Statistical processes Sound methodology, Appropriate statistical procedures, Non- excessive burden on respondents, Cost effectiveness Statistical output Relevance, Accuracy and reliability, Timeliness and punctuality, Coherence and comparability, Accessibility and clarity

5 Institutional environment ESS-wide peer reviews during –to monitor the implementation of the Code of Practice –to boost quality, enhance credibility –to share ESS best practices and transfer knowledge Establishment of the European Statistical Governance Advisory Board (ESGAB): –to provide an independent overview of the implementation of the Code of Practice by Eurostat –to advise Eurostat and the ESSC on updating and communicating the Code to users and data providers Establishment of general guidelines (e.g. confidentiality manual)

6

7 ESS Quality Coaching Two quality managers work together to address specific improvement actions Based on experience, dialogue and information exchange Agreement on commitments from both sides Joint working – not doctor/patient relation 17 coaching pairs took part Coaching exercise completed at the end of 2009 Appropriate where there are clear standards and guidelines

8 Process and output quality: the Eurostat Quality Assurance Framework A system of coordinated methods and tools to ensure a sustainable level of quality of processes and outputs in line with the Code of Practice: –product/ output quality requirements are explicitely documented –processes are defined and made known to all staff –the correct implementation of the processes is monitored –users are informed on the quality of the products and possible deficits –a procedure to guarantee that the necessary improvement measures are planned, implemented and evaluated

9 The Eurostat Quality Assurance Framework

10 The Eurostat Quality Assurance Framework

11 Process Quality Assessments (1) Conceptual framework (2) IT conditions (11) – Management, planning and legislation (12) – Staff, work conditions and competence (13) User needs (3) Data collection (4) Validation Country level (5) International level (6) Confidentiality (7) Dissemination (9) Documentation (8) Follow-up (10) Common tool: The Process Quality Assessment Checklist (based on the International ISPAC) Method: chronologically examining the process Outputs: –Assessment Diagram –Summary Assessment report with improvement actions –Highlight of a good practice

12 Process Quality Assessments (2) Each process is matched with an assessment category Self- Assessment Prod. unit: fills in Checklist, produces outputs B1: provides support on demand Supported Self-Assessment Prod. unit: in control of assessment, participates in all steps B1: pre-fills the Checklist, produces the outputs Peer review Same as Supported Self-Assessment with one or more peers (int/ext) taking part in all steps Rolling review A comprehensive review based on the checklist implemented by ext.contractor, involving partner & user surveys and cost-benefit analysis Process 1 Characteristics: - Periodicity - Legal Basis - Output - ESTAT intervention -availability of quality report Process 4 Process 2Process 3 Process 5

13 Process Quality Assessments Implementation Plan: Principles Minimize burden for units –Test of the approach in advance –Support from the Quality unit –Flexibility Build on existing information –Quality reports –Process analysis Integrate QA with other horizontal activities

14 Process Quality Assessment Outputs (1) Summary Assessment Report

15 Process Quality Assessment Outputs (2) Coherence General coherence Timeliness Timeliness of final publication Accuracy Overall accuracy Relevance User satisfaction Accessibility and clarity Overall quality of metadata Comparability over time Comparability across countries Assessment Diagram

16 The Eurostat Quality Assurance Framework

17 ESS Standard & Handbook for Quality Reports (1) Provide (mainly producer oriented) recommendations / guidelines for the preparation of comprehensive quality reports for statistical processes and their outputs by distinguishing between 6 types of statistical processes: –Sample survey –Census –Statistical process using administrative sources –Statistical process involving multiple data sources –Price or other economic index process –Statistical compilation

18 ESS Standard & Handbook for Quality Reports (2) Promote harmonized quality reporting across statistical processes and Member States => facilitate comparisons Are applicable to National Statistical Offices and Eurostat in their roles as producers of statistics Also include 21 quality and performance indicators Handbook: is more detailed, with lots of examples and explanations, describes the quality concepts, explains how the different quality documents relate to each other and is aligned with the 15 Principles of the Code of Practice

19 ESS Quality Declaration (national TQM framework) ESS Quality architecture ESS Statistical Law ESS levelnational / Eurostat level national Statistical Law National QAF: Description of practices and procedures for implementing the CoP QAF: ESS standards for implementing the CoP