1. Metadata on quality of statistical information
Petr Elias
Jitka Prokop
Czech Statistical Office

2. Issues Quality in statistics generally.
Quality regarding statistical data and metadata.
Producers and users – roles, needs.
What should metadata on quality cover?
How metadata should be structured?
For which purposes can metadata be used?
Which tools?
Issues

3. OUTLINE
Statistical Quality
Focus, Coverage
Methods, Tools
References

4. Purposes Producers Users need to => Quality in broad sense
Assure quality of statistical data
Provide data to users
Provide meta-information to users
Users need to
Know what they need
Gain good relevant data (do not care about process)
Use data properly
=> Quality in broad sense

5. Focus Outputs, Products: Processes, sub-processes, phases (GSBPM):
data and metadata, news releases
publications, web publishing, ad-hoc products
web-databases
Processes, sub-processes, phases (GSBPM):
needs specification, design, build, data collection, processing, analyses, dissemination, evaluation
Input data:
Micro-data from survey, registers, secondary data

6. Quality Statistical Quality Quality Management
Statistical outputs (Quality criteria and QPI)
Statistical processes (GSBPM; QPI mapping?)
Input data, (micro)data
Quality Management
TQM, EFQM, etc.

7. Quality Criteria (Principles)
Code of Practice: Principles 11 – 15
EU Regulation 223/2009: Quality Criteria
Principle 11: Relevance
Principle 12: Accuracy and Reliability
Principle 13: Timeliness and Punctuality
Principle 14: Coherence and Comparability
Principle 15: Accessibility and Clarity

8. Quality Criteria (Principles)
Relevance
EU Statistics must meet the needs of users.
Accuracy and Reliability
EU Statistics must accurately and reliably portray reality.
Timeliness and Punctuality
EU Statistics must be disseminated in a timely and punctual manner.
Coherence and Comparability
EU Statistics should be consistent internally, over time and comparable between regions and countries; it should be possible to combine and make joint use of related data from different sources.
Accessibility and Clarity
EU Statistics should be presented in a clear and understandable form, disseminated in a suitable and convenient manner, available and accessible on an impartial basis with supporting metadata and guidance.

9. Coverage Business statistics Social and demography statistics
Price statistics
National accounts
Administrative data statistics

10. Methods, tools Monitoring Q Q Reports (internal, external)
Self-assessments of a survey
Auditing (internal, external), CoP peer reviews
Quality reports for producers
Quality reports for users
Single Integrated Metadata Structure (SIMS)
Databases and SW applications

11. Self-assessments Natural character: Subjective
Self-reflection, supportive for audits
Form of checklists (eg. DESAP), e-form
Can be for public (partly)
Includes
Quantitative and qualitative info
Categorical and textual evaluations
Summaries, strengths and weaknesses
Suggested actions

12. Quality reporting Quality and Performance Indicators
Applicability: national level, EU level
Process and output quality
To evaluate quality criteria (dimensions)
EU Standard Q Reports & Euro SDMX Structure
Single Integrated Metadata Structure
For producers
For users

13. Single Integrated Metadata Structure
Dynamic inventory of ESS quality and reference metadata statistical concepts
Quality reporting 2in1 - for producers, for users
Includes Quality and Performance Indicators QPI
ESS reference metadata report structure (Euro SDMX Metadata Structure ESMS)
Optional tool for production, export of metadata: National Reference Metadata Editor (NRME)

14. SIMS Dimensions By scope By level By audience
Particular survey or a group of surveys
A statistical variable (e.g. IPI index)
By level
National
ESS
By audience
Users
Producers

15. Metadata: stability of values
Levels
General, group of surveys
Survey
Reference year
A processing (reffers eg. to one news release)
Statistical variable
Reference period

16. Types of metadata in Q application
Reference metadata
Info about process and its phases
Schedules, timing
Quality performance indicators
Calculations
Benchmark results
Evaluation, (self-)assessments, commentaries
Textual, Numerical, Date

17. Possible categories of Q metadata (content)
Basic information (about a survey).
User requirements agenda.
Methodology info.
Time schedules. Timeliness, punctuality.
Statistical process phases.
Data confidentiality and protection.
Data sources; Frame; Sample.
Outputs and dissemination.
Individual quality criteria (i.e. quality dimensions).
Quality performance indicators.

18. Q metadata application issues
Content, structure, links to databases and other apps
Metadata updates (items or their parameters, hierarchical structure)
Retrieving and updating data (new reference periods...) – i.e. Keep history & update
Validity within certain years, batches, reference periods
Alerts from the app; decision making in the application.
Multi-lingual features.
Benchmarking.

19. ESS quality reporting (references)
Technical Manual of the Single Integrated Metadata Structure (SIMS)
ESS Handbook for Quality Reports 2014
ESS Guidelines for the Implementation of the ESS Quality and Performance Indicators (QPI)
Generic Statistical Business Process Model. Version 5.0.
European Statistics Code of Practice
Quality Assurance Framework
The European Self Assessment Checklist for Survey Managers (DESAP)

20. Any questions?