1. Quality at a Glance: Documentation of Quality Indicators at Statistics Austria European Conference on Quality in Official Statistics Rome, 8-11 July 2008 © STATISTIK AUSTRIA www.statistik.at Thomas Burg, Peter Hackl, Werner Holzer

2. www.statistik.atpage 2 Content  Introduction, background  Objectives  The quality sheet – general remarks  The quality sheet – quality indicators in detail  Implementation

3. www.statistik.atpage 3 Introduction (1) Format of standard documentation or quality report  Primarily designed for documentation of sample surveys  Detailed information on all phases of the statistical production process  Description of the statistical concepts and the applied methodology  A special chapter deals with the various aspects of quality, according to the ESS quality dimensions  Beside broad descriptions; limited, unsystematic provision of quantitative and non-textual qualitative information (quality indicators) Background

4. www.statistik.atpage 4 Introduction (2) to extend quality information for users  Peer review report on Statistics Austria’s compliance with the principles of the European Statistics Code of Practice: “Principle 12: Accuracy and Reliability Extension of calculation of quality indicators to be included in quality reports”  The Austrian Statistical Council: ”Quality indicators should be applied systematically for all statistical products” Recommendations

5. www.statistik.atpage 5 Objectives  concise quantitative and non-textual qualitative information by means of a comprehensive set of quality indicators  In form of a quality sheet that contains all relevant information in condensed form. This “quality at a glance” should serve the following purposes: 1.Provision of the users with quick and easily understandable insights into the most relevant aspects of the product, following the ESS quality dimensions 2.Enabling the producers to identify improvement potentials within the relevant statistical processes 3.Keeping the size of the quality sheet in a manageable format To provide

6. www.statistik.atpage 6 The quality sheet – General remarks  The quantification problem  The variety of products  The “space” problem  The problem of condensing  The burden for the producer General problems and remarks  Quality indicators, quality sheets of other NSIs  Quality indicators proposed by EUROSTAT Consideration of similar concepts and experiences

7. www.statistik.atpage 7 The quality sheet - General information  Product, type of product  Reference period General information

8. www.statistik.atpage 8 The quality sheet – Accuracy (1)  Sampling errorsAccuracy

9. www.statistik.atpage 9 The quality sheet – Accuracy (2)  Non sampling effects (1)Accuracy  Data editing procedures  Evaluation of data sources used

10. www.statistik.atpage 10 The quality sheet – Accuracy (3)  Non sampling effects (2)Accuracy  Coverage (under coverage, over coverage)

11. www.statistik.atpage 11 The quality sheet – Accuracy (4)  Non sampling effects (3)Accuracy Non-Response  General information: voluntary or compulsory survey?

12. www.statistik.atpage 12 The quality sheet – Accuracy (5)  Non sampling effects (4)Accuracy Unit-Non-Response  Absolute numbers

13. www.statistik.atpage 13 The quality sheet – Accuracy (6)  Non sampling effects (5)Accuracy Unit-Non-Response  Unit-Non-Response Rate  Rate of eligible records

14. www.statistik.atpage 14 The quality sheet – Accuracy (7)  Non sampling effects (6)Accuracy Item-Non-Response, Imputation  Item-Non-Response Rates  Imputation Rates

15. www.statistik.atpage 15 The quality sheet – Accuracy (8)  Non sampling effects (7)Accuracy  Model based estimation

16. www.statistik.atpage 16 The quality sheet – Timeliness Timeliness  Preliminary results  Final results

17. www.statistik.atpage 17 The quality sheet – Comparability Comparability  Over time  Geographic  Other

18. www.statistik.atpage 18 Implementation (1)  Workshop in Statistics Austria (5 May 2008): - feedback from subject matter units (content) - discussion of the update procedures  Discussion in a meeting of the Top Management Team (27 May 2008) Internal discussion of the draft quality sheet

19. www.statistik.atpage 19 Implementation (2)  Pilot implementation (deadline: 10 July 2008)  Update, also by taking into account SDMX requirements  Final proposal; discussion with the Quality Assurance Committee of the Statistical Council (autumn 2008)  Final implementation/integration of Quality Indicators: - Metadata repository - Standard documentations/quality reports Next steps

20. Thank you for your attention !