1 Recent developments in quality related matters in the ESS High level seminar for Eastern Europe, Caucasus and Central Asia countries Claudia Junker, Eurostat, Head of unit “Statistical cooperation”

2 Content Major quality initiatives  The European Statistics Code of Practice (CoP) – peer reviews  The ESS Quality Assurance Framework  Quality assurance in Eurostat  Quality reporting Other related initiatives Conclusions

Major Quality Initiatives – the CoP  Revision of the CoP in 2011 →New round of peer reviews in the Member States  New methodology  All principles will be reviewed  Includes other producers of official statistics  Covers the coordinating role of the NSI  Covers the level of cooperation in and level of integration of the ESS  Self-assessment questionnaire has changed  Questionnaires for other producers of official statistics  Audit-like approach – experts study documents, identify risk areas and assess those, findings based on evidence  Different report – no assessment; qualitative conclusions with recommendations (NSI can publish diverging views)  NSI has to define improvement actions  Piloting in 2013, implementation March 2014 – May 2015  Report by the end of

4 The Quality Assurance Framework of the ESS

5 Quality Assurance Framework of the ESS  Describes activities, methods and tools facilitating the implementation of the CoP  Might be extended and completed in the future, a "living" document  For the time being, it focuses on principles:  4 - commitment to quality  7 - sound methodology  8 - appropriate statistical procedures  9 - non-excessive burden on respondents  10 – cost-effectiveness  11 – relevance  12 – accuracy and reliability  13 – timeliness and punctuality  14 – coherence and comparability  15 – accessibility and clarity

Quality framework and quality assurance in the ESS 6 Model with 3 levels Level 1 = Code of Practice: Principles (standards) Level 2 = Code of Practice: Indicators (how the standards can be demonstrated) Level 3 = Quality Assurance Framework: Methods and tools (how the indicators can be implemented in the practice)

7 Quality Assurance Framework - example Principle 4 Statistical authorities are committed to quality. They systematically and regularly identify strengths and weaknesses to continuously improve process and product quality. Indicator 4.1 Quality policy is defined and made available to the public. An organisational structure and tools are in place to deal with quality management. Methods at institutional level A quality commitment statement (is public, lays out principles and commitments, is consistent with the vision and the mission statement. An organisational structure for managing quality (exists, is visible in the organisational chart) Defition on quality guidelines (description of processes, documentation guidelines, description of methods to monitor quality Avaialibility of quality guidelines An infrastructure for documentation Training courses

8 Quality Assurance Framework - example Principle 4 Statistical authorities are committed to quality. They systematically and regularly identify strengths and weaknesses to continuously improve process and product quality. Indicator 4.3 Product quality is regularly monitored, assessed with regard to possible trade-offs, and reported according to the quality criteria for European statistics. Methods at institutional level Procedures to monitor product quality (procedures in place, results are analysed regularly, senior management is informed). User satisfaction surveys (implemented on a regular basis, results made public and incorporated into quality reports) Methods at product/survey level User oriented quality reports (made available to the public) Producer oriented quality reports (published regularly, depending what is in the legislation) Product quality monitoring (for regular monitoring over time)

9 Quality assurance in Eurostat  First round of quality assessments based on the tailor-made DESAP assessment checklist finalised ( )  Results analysed  New round focuses on process modernisation, based on a questionnaire  It analyses processes with other "like" processes and is in line with the implementation of the ESS vision  Assessment against the Eurostat QAF, a tailor-made version of the ESS one  Use of the BPM tool  Supports the implementation of the CoP  Pilot implementation on statistics on information and communication technology (2013)  Rolling-out in Eurostat ( )

10 Quality reporting ESS recommendations on quality reporting Distinction of producer and user oriented quality reports Single metadata structure to be used to derive both Manual to be prepared to support the single structure and the template Use of common ESS IT tools User oriented reports to be disseminated to the wide audience while producer oriented ones to the producers Regular interaction with the target groups Specific ESS task force set up to Develop and describe a single reporting structure to be used for reporting on quality and metadata, and to prepare a Technical Manual on its use Update the 2009 edition of the ESS Handbook for Quality Reports (including the single reporting structure, the updated quality indicators, etc)

11 Other related initiatives  Implementation of the vision of Eurostat – increase efficiency and streamline statistical production  Project on the census hub  Project on common data validation tools  Project on using data from the Internet (prices)  Project on creating a common business register  Project on data exchange network  Project on common data warehouses  Commitment on Confidence  Decision on fines for manipulated statistics

12 Conclusions  Quality becomes the decisive criteria for official statistics  Quality assessments of all kinds are needed to increase trust  Quality needs to be documented  Quality documentation needs to be public  Quality reports needs to be understandable for users  Quality commitment starts at the top  Quality is valid for every stage of the production and dissemination of statistics Thank You