Karin Blix, Statistics Denmark, kwb@dst.dk Raising awareness and continuously improving quality in Statistics Denmark Karin Blix, Statistics Denmark, kwb@dst.dk 28.06.2018 Session 5

Quality management in Statistics Denmark European frame ESS CoP and ESS QAF Peer Reviews Local frame – Quality policy Quality awareness in dissemination of statistics Dissemination of press releases and other publications Documentation of statistics, other metadata, user involvement Quality awareness in production of statistics EU-cooperation, guidance from the methods division, quality audits Extensive use of administrative registers In practice the management system builds on three pillars Quality assurance of documentation of statistics (quality reports) Quality audits – or quality reviews Process model

Starting point for quality reports – documentation of statistics The starting point is Code of Practise Indicator 4.3 – reporting of quality Indicator 15.5 – metadata are documented according to standardised metadata systems Standards: SIMS, ESQRS, ESMS, GSBPM Metadata system – Colectica – all SIMS fields are filled in for documentation of statistics Any new dissemination is followed by an updated version of the documentation of statistics – and goes through a quality assurance process Motivation is given by reflecting on this model for producing statistics:

Conceptualised reality Reality as presented by data ”Objective” reality (Ideal) statistical characteristics – what we are seeking information about (ideal) population (ideal) variables (true) values Conceptualised reality Reality as it is conceived and operationalised by designers Statistical target characteristics Target population Variables that can be measured Values that can be measures Reality as presented by data Reality as it is perceived by respondents and represented by data Observed object characteristics Observed objects Observed variables Observed values Statistics about reality as interpreted by users Reality as it is understood by users when interpreting data Interpreted statistics Discrepancies Coverage (first kind) Sampling Operational def. of variables deviate from ideal definitions Discrepancies Coverage (second kind) Respondents cannot be found Respondents refuse to answer Respondents misinterpret Respondents make mistakes (conscious or unconscious) Discrepancies Frames/references differ between users, designers and respondents Understanding of statistical methods From Bo Sundgren: Statistical systems – some fundamentals (2004)

Quality reports – documentation of statistics Help for the user to understand the statistics – giving the user information about the frame we have worked within Explain the content of the statistics History and purpose of the statistics Content – population, variables etc. Quality = Fitness for use Quality of contents: Relevance, Accuracy & reliability, Timeliness and punctuality, Coherence & comparability, Accessibility and clearness Three levels “Front page” to appear at www.dst.dk, with a short description of the 9 headlines in the Structure. From the front page one can open around 100 specified topics (SIMS) SIMS topics cover the more detailed quality report Annexes The idea is to cover all users (national, international, EU) in one product

Documentation of statistics on www.dst.dk

www.dst.dk – more details

Cycle for documentation of statistics Edit mode QA mode Changes made by subject matter responsible Return to QA Approved by QC Disseminated at www.dst.dk

Quality audits Started in SD in 2015 Introduction to the ”audit” and distribution of self assessment form and request for available documentation etc. Started in SD in 2015 In 2017 – six statistical products Tourism statistics, LFS, SILC, CPI and HBS Audit by team of experts Self assessment Review of user needs and the fulfilment of these Review of production processes – GSBPM used to assist Report with quality review and recommendations Action plan Self assessment form Documentation Meetings, visit from the review team Draft report Final meeting with the statistics division Final report Action plan Reports and plans to the Management of SD

Quality audits Choosing statistical products for quality audit Through the quality assurance process of documentation of statistics Through the wishes of the directors Self-assessment of compliance of CoP Each of the indicators of CoP from principle 4 are evaluated QAF is used for inspiration on the level of single statistics Degree of compliance A – Most of the demands fulfilled, including documentation B – Some of the demands fulfilled, but still some missing C – Only few of the demands fulfilled, much missing X – not relevant Review of the production process using GSBPM to assist

Statistical process model - why GSBPM? Quality of statistical output depends on two factors: The quality of the data from data providers (we can influence) The quality of the processes by which we treat these data (in our control) Major efficiency gains can be seen when ‘best practices’ are applied and standardised datasets and uniform production procedures are used for similar tasks Frame for analysis and gradual improvement “It is difficult to improve something which is not described” A common process model can assist us with a common conceptual, methodological and organisational reference for describing, analysing and disseminating our statistical products. It provides a tool to ease and facilitate the training of new employees and at the same time to extract knowledge from experienced experts before it’s too late.

Process model

Work processes and documentation of statistics

How GSBPM? The first thing we’ve done is ask our statisticians to organize their working documents and files etc. in a folder structure similar to the process model This way we create a common foothold for where what is allocated. Also, it makes it much more intuitive for employees to find data and other relevant material

How GSBPM? Dynamic documentation as HTML work as a local point-and-click website for the individual statistic

Karin Blix, Statistics Denmark, kwb@dst.dk Raising awareness and continuously improving quality in Statistics Denmark Karin Blix, Statistics Denmark, kwb@dst.dk