Quality Reports Ukraine November 2014

Regulation 223/2009 on European Statistics European Statistics shall be produced  on the basis of uniform standards  using harmonized methods Users shall have access to metadata describing the quality of statistical output, in order to interpret and use the statistics correctly Why quality reporting 2

Here you will find: ESS Quality Reporting 3

 SDMXStatistical Data and Metadata Exchange  ESMSEuro-SDMX Metadata Structure  User-oriented format for quality reporting  ESQRS ESS Standard for Quality Reports Structure  Producer-oriented format for quality reporting All statistical concepts of ESMS and ESQRS have been included and streamlined in  SIMSSingle Integrated Metadata Structure Some abbreviations 4

ESMSSIMSESQRS 14Accuracy and reliabilityS.15Accuracy and reliabilityV 14.1Overall accuracyS.15.1Overall accuracyV.1Overall accuracy 14.2Sampling errorS.15.2Sampling error and A1. Sampling errors - indicators for UV.2Sampling error S A1. Sampling errors - indicators for PV.2.1Sampling errors - indicators 14.3Non-sampling errorS.15.3 Non-sampling error and A4. Unit non-response - rate for U and A5. Item non-response - rate for U V.3Non-sampling error Coverage errorV.3.1Coverage error S A2. Over-coverage - rateV.3.1.1Over-coverage - rate S A3. Common units - proportion S Measurement errorV.3.2Measurement error S Non response errorV.3.3Non response error S A4. Unit non-response - rate for PV.3.3.1Unit non-response - rate S A5. Item non-response - rate for PV.3.3.2Item non-response - rate S Processing errorV.3.4Processing error V.3.4.1Imputation - rate V.3.4.2Common units - proportion S Model assumption errorV.3.5Model assumption error V.3.7Seasonal adjustment Example: Accuracy and reliability 5

Handbook is addressed to  NSO for their own internal assessment of process and output quality  NSO as the starting point for preparing user-oriented quality reports  NSO for producer-oriented quality reports to Eurostat Single metadata structure should promote  Both user-oriented and producer-oriented should be derived from the same source  Maximum re-use of information in the metadata system  Reduction and simplification of documents  The user-oriented quality reports should be improved ESS handbook - Purpose 6

1.Introduction 2.Relevance, assessment of user needs 3.Accuracy and reliability 4.Timeliness and punctuality 5.Accessibility and clarity 6.Coherence and comparability 7.Cost and burden 8.Confidentiality 9.Statistical processing 2-6: output components7-9: process components ESS Handbook - Structure 7

The ESS Handbook applies to the following statistical processes: 1.Sample survey 2.Census 3.Statistical process using administrative source(s) 4.Statistical process involving multiple data sources 5.Price or other economic index process 6.Statistical compilation assembling a variety of primary sources (e.g. National Accounts) Statistical Processes 8

Guidelines for preparing quality reports  For all 9 headlines in the structure (Relevance, Accuracy …)  For all statistical process on the whole and where relevant  For the 6 types of statistical processes (Sample Survey, Census …) Part II: Guidelines for preparing detailed quality reports 9

1.ESS Quality and Performance Indicators 2.Technical Manual of the Single Integrated Metadata Structure (SIMS)  Annex 1. Relation between ESMS, SIMS and ESQRS (extract in slide 5)  Annex 2. Descriptions and guidelines for each item in SIMS 3.References and key documents Part III: Annexes 10

 Re-organisation 2014 following the ESS handbook  Three levels 1. “Front page” to appear at the homepage of Statistics Denmark, with a short description of the 9 headlines in the Structure. From the front page one can point on at and open around 100 specified topics (SIMS) 2. SIMS topics cover the more detailed quality report (See guidelines in Annex 2). From level 2 one can point at and open annexes for further description 3. Annexes  The idea is in one product to cover all customers (national, international, EU).  Prepared in Danish and English DK quality reports 11

16 quality indicators to be reported in the ESS Quality Reports to Eurostat  RelevanceR1  Accuracy and ReliabilityA1 – A7  Timeliness and PunctualityTP1 – TP3  Coherence and ComparabilityCC1 – CC2  Accessibility and ClarityAC1 – AC3 (cover the output components in slide 7) Quality and Performance Indicators 12

U ( user ) and P ( producer ) in SIMS  P-fields are producer-oriented and are normally just the figure(s) asked for  U-fields are user-oriented and are an annotated version of the figure(s) asked for 13 Indicator U SIMS ID P SIMS ID Indicator U SIMS ID P SIMS ID Indicator U SIMS ID P SIMS ID A A CC A TP AC A TP AC A CC AC A R A TP

A1Sampling error 14

A2Over-coverage 15

Mixed statistical processes where some variables or data for some units come from survey data and others from administrative sources  Measure for agreement between different sources  Definition: Units in both the survey and the administrative source Units in the survey A3Common units 16

A4Unit non-response 17

A5Item non-response 18

A6Data revision 19

A7Imputation 20