Quality Reporting in CBS ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017 Quality Reporting in CBS Slavica Rogar Croatian Bureau of Statistics Industry, Energy and Information Society Department Luxembourg, 2017

Content Database of quality information (DBQI) Quality Reports Conclusion and future developments ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

Database of quality information (DBQI) Key tool for quality assessment, quality documentation and quality reporting for CBS surveys Established on the exhaustive list of quality information based on two ESS structure Part of the national IPA project A central place for storing: Direct information on quality estimation Reference metadata Documentation of survey processes DBQI is planned to become a key tool for quality assessment, quality documentation and quality reporting for CBS surveys It is established on the exhaustive list of quality information based on two widely accepted ESS structure-ESMS and ESQRS As part of the national IPA project, much progress has been achieved within the component on the production of survey reports relating quality reports across the statistical survey work in the CBS. QD is planned to be a central place for storing: Direct information on quality estimation Reference metadata Documentation of survey processes ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

DBQI Homepage/ Data Entering Homepage Data entering Research Num. Indicators Administrations Information Data entering List of surveys divided by sectors Here we can see the homepage for data entering into the DBQI (QD). It has a list of all surveys grouped by sectors that can enter data for its quality database and quality reports. You need to choose the name of the survey, and the reference period (month and year) for which you want to enter the data. Survey Reference period Enter data ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

DBQI for STS- Industrial Production (1/2) Monthly Survey on Industrial Production and Persons Employed (IND-1/KPS/M form) – 12/2016 Descriptive and numerical information/values A list of all indicators with descriptions / instructions (excel) Number Concept name Description Value/information Instructions Here we can see the database look for Monthly Survey on Industrial Production and Persons Employed for December 2016 All the information in the database can roughly divide into two parts: Descriptive (textual) information which we can see in this picture/slide and Numerical information (quality indicators)- shown later For help in filling in the textual information, we have an excel table with all indicators and theirs detailed descriptions and instructions to download. In the end, we can generate the quality report in Croatian and/or English. Quality report ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

DBQI Metadata Structure 1 Contact 1.1 Contact name 1.2 Contact organization/unit 1.3 Contact mail address 1.4 Contact phone number 1.5 Contact email address 1.6 Purpose, goal and subject of the survey (introduction) 2 Metadata update 2.1 Metadata last certified 2.2 Metadata last posted 2.3 Metadata last update 3 Statistical presentation 3.1 Data description 3.2 Classification system 3.3 Sector coverage 3.4 Statistical concepts and definitions 3.5 Statistical unit 3.6 Statistical population 3.7 Reference area 3.8 Time coverage 3.9 Base period 4 Unit of measure 5 Reference period 6 Institutional mandate 6.1 Legal acts and other agreements 6.2 Data sharing 7 Confidentiality 7.1 Confidentiality- policy 7.2 Confidentiality- data processing 7.3 Data protection treatment 7.4 Accessibility and clarity 8 Release policy 8.1 Release calendar 8.2 Release calendar access 8.3 User access 9 Frequency of dissemination 10 Dissemination format 10.1 News release 10.2 Publications 10.3 On-line database 10.4 Micro-data access 10.5 Other 11 Accessibility of documentation 11.1 Documentation on methodology 11.2 Quality documentation 12 Quality management 12.1 Quality assurance 12.2 Quality assessment 13 Relevance 13.1 Data users 13.2 User needs 13.3 User satisfaction 13.4 Completeness 14 Accuracy and reliability 14.1 Overall accuracy 14.2 Sampling error 14.3 Non-sampling error 14.4 Measurement error 14.5 Non-response error 14.6 Processing error 14.7 Coverage error 14.8 Model assumption error 15 Timeliness and punctuality 15.1 Timeliness 15.2 Punctuality 16 Comparability 16.1 Comparability- geographical 16.2 Asymmetry for mirror flows statistics 16.3 Seasonal adjustment 16.4 Comparability- over time 16.5 Reasons for time break series 17 Coherence 17.1 Coherence- cross domain 17.2 Coherence- internal 18 Cost and burden 18.1 Cost 18.2 Burden 19 Data revision 19.1 Data revision- policy 19.2 Data revision- practice 20 Statistical processing 20.1 Source data 20.2 Frequency of data collection 20.3 Data collection 20.4 Data validation 20.5 Data compilation 20.6 Adjustment 21 Comment Here is our metadata structure. As I have already mentioned, it has been established combining two accepted ESS structure: ESMS and ESQRS The Quality Reports on our website include only the numbers rounded in red. Unfortunately, it is available only in Croatian. We are doing are best to publish quality reports in English. ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

DBQI for STS- Industrial Production (2/2) Monthly Survey on Industrial Production and Persons Employed (IND-1/KPS/M form) – 12/2016 Handbook for Calculation of Quality Indicators Dimensions Quality and Performance Indicators (QPIs) Key/auxiliary indicators tag Data Description Relevance R1 Data completeness rate Punctuality On this slide, we can see the second part of the database, the numerical information. In order to fulfill as well as possible, we wrote a Handbook for Calculation of QIs, where for each of the indicator are provided: the definition of the indicator, calculations procedure and example(s) We also had several workshops on Quality Reporting and Quality Assessment Methods and Tools A2 Over-coverage rate A4 Unit non-response rate ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

DBQI Metadata Structure (QPIs) Relevance R1 Data completeness rate Accuracy A1 Sampling error indicators A8 Bias due to selection process A2 Over-coverage rate A4 Unit non-response rate A5 Item non-response rate A6 Data revision- average size A7 Imputation rate A9 Editing rate A10 Edit failure rate A11 Hit rate A12 Misclassification rate Timeliness and punctuality TP1 Time lag – first results TP2 Time lag – final results TP3 Punctuality Comparability CC1 Length of comparable time series CC2 Asymmetry for mirror flows statistics- coefficient Coherence CH1 Coherence- administrative sources CH2 Coherence- short-term and structural data CH3 Coherence- national accounts Here is the list of quality and performance indicators available in our Quality Database. Only the two of the listed indicators are not in the quality report. ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

Quality Reports on CBS website Reporting on Quality of Statistical Surveys Quality reporting template Handbook for calculating key quality indicators On our website under the domain Quality/ Quality Reporting, we have published 30 quality reports in Croatian. Currently, we are planning to publish 5 quality reports of industry department (such as Industrial Production, Industrial Turnover…) hopefully in Croatian and English. Survey results ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

Conclusion and future developments The system for disseminating information about the quality of statistics to their users should continue to be developed. Room for improving existing quality database The main challenge- data access for calculating QPIs The system for disseminating information about the quality of statistics to their users should continue to develop. There is always room to improve existing quality database so that published quality reports could become even better. The main challenge is to have the possibility of finding the data not to fill in and publish quality reports. That calls a great IT sector support and development of our applications for data processing so we could fill in all QPIs. ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

Thank you for your attention! Slavica Rogar email: rogars@dzs.hr ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017