1. Karin Blix, Statistics Denmark, kwb@dst.dk
Raising awareness and continuously improving quality in Statistics Denmark
Karin Blix, Statistics Denmark,
Session 5

2. Quality management in Statistics Denmark
European frame
ESS CoP and ESS QAF
Peer Reviews
Local frame – Quality policy
Quality awareness in dissemination of statistics
Dissemination of press releases and other publications
Documentation of statistics, other metadata, user involvement
Quality awareness in production of statistics
EU-cooperation, guidance from the methods division, quality audits
Extensive use of administrative registers
In practice the management system builds on three pillars
Quality assurance of documentation of statistics (quality reports)
Quality audits – or quality reviews
Process model

3. Starting point for quality reports – documentation of statistics
The starting point is Code of Practise
Indicator 4.3 – reporting of quality
Indicator 15.5 – metadata are documented according to standardised metadata systems
Standards: SIMS, ESQRS, ESMS, GSBPM
Metadata system – Colectica – all SIMS fields are filled in for documentation of statistics
Any new dissemination is followed by an updated version of the documentation of statistics – and goes through a quality assurance process
Motivation is given by reflecting on this model for producing statistics:

4. Conceptualised reality Reality as presented by data
”Objective” reality
(Ideal) statistical characteristics – what we are seeking information about
(ideal) population
(ideal) variables
(true) values
Conceptualised reality
Reality as it is conceived and operationalised by designers Statistical target characteristics
Target population
Variables that can be measured
Values that can be measures
Reality as presented by data
Reality as it is perceived by respondents and represented by data Observed object characteristics
Observed objects
Observed variables
Observed values
Statistics about reality as interpreted by users
Reality as it is understood by users when interpreting data
Interpreted statistics
Discrepancies
Coverage (first kind)
Sampling
Operational def. of variables deviate from ideal definitions
Discrepancies
Coverage (second kind)
Respondents cannot be found
Respondents refuse to answer
Respondents misinterpret
Respondents make mistakes (conscious or unconscious)
Discrepancies
Frames/references differ between users, designers and respondents
Understanding of statistical methods
From Bo Sundgren: Statistical systems – some fundamentals (2004)

5. Quality reports – documentation of statistics
Help for the user to understand the statistics – giving the user information about the frame we have worked within
Explain the content of the statistics
History and purpose of the statistics
Content – population, variables etc.
Quality = Fitness for use
Quality of contents:
Relevance, Accuracy & reliability, Timeliness and punctuality, Coherence & comparability, Accessibility and clearness
Three levels
“Front page” to appear at with a short description of the 9 headlines in the Structure. From the front page one can open around 100 specified topics (SIMS)
SIMS topics cover the more detailed quality report
Annexes
The idea is to cover all users (national, international, EU) in one product

6. Documentation of statistics on www.dst.dk

7. www.dst.dk – more details

8. Cycle for documentation of statistics
Edit mode
QA mode
Changes made by subject matter responsible
Return to QA
Approved by QC
Disseminated at

9. Quality audits Started in SD in 2015
Introduction to the ”audit” and distribution of self assessment form and request for available documentation etc.
Started in SD in 2015
In 2017 – six statistical products
Tourism statistics, LFS, SILC, CPI and HBS
Audit by team of experts
Self assessment
Review of user needs and the fulfilment of these
Review of production processes – GSBPM used to assist
Report with quality review and recommendations
Action plan
Self assessment form
Documentation
Meetings, visit from the review team
Draft report
Final meeting with the statistics division
Final report
Action plan
Reports and plans to the Management of SD

10. Quality audits Choosing statistical products for quality audit
Through the quality assurance process of documentation of statistics
Through the wishes of the directors
Self-assessment of compliance of CoP
Each of the indicators of CoP from principle 4 are evaluated
QAF is used for inspiration on the level of single statistics
Degree of compliance
A – Most of the demands fulfilled, including documentation
B – Some of the demands fulfilled, but still some missing
C – Only few of the demands fulfilled, much missing
X – not relevant
Review of the production process using GSBPM to assist

11. Statistical process model - why GSBPM?
Quality of statistical output depends on two factors:
The quality of the data from data providers (we can influence)
The quality of the processes by which we treat these data (in our control)
Major efficiency gains can be seen when ‘best practices’ are applied and standardised datasets and uniform production procedures are used for similar tasks
Frame for analysis and gradual improvement
“It is difficult to improve something which is not described”
A common process model can assist us with a common conceptual, methodological and organisational reference for describing, analysing and disseminating our statistical products.
It provides a tool to ease and facilitate the training of new employees and at the same time to extract knowledge from experienced experts before it’s too late.

12. Process model

13. Work processes and documentation of statistics

14. How GSBPM? The first thing we’ve done is ask our statisticians
to organize their working documents and files etc.
in a folder structure similar to the process model
This way we create a common foothold for where
what is allocated. Also, it makes it much more
intuitive for employees to find data and other
relevant material

15. How GSBPM?
Dynamic documentation as HTML work as a local point-and-click website for the individual statistic

16. Karin Blix, Statistics Denmark, kwb@dst.dk
Raising awareness and continuously improving quality in Statistics Denmark
Karin Blix, Statistics Denmark,