CZECH STATISTICAL OFFICE | Na padesatem 81, Prague 10 | Jitka Prokop, Czech Statistical Office SMS-QUALITY The project and application focused on metadata on quality European conference on Quality in Official statistics 3-5 June 2014, Vienna

2 ■Introduction - aims, features, coverage, current state ■Architecture - Q-attributes, hierarchical structure, design, preparation, data retrievals and inputs, functionality, stability of values and updates ■Benchmarking ■Challenges Topics

3 Starting points ■Horizontal way of management ■Demands for quality reporting & relevant metadata ‚standardisation‘ ■Standardisation of quality reports & adjustments for domain statistics Tool for managers ■Semi-interactive cross-cutting overviews about quality of a survey (incl. assessments…) ■Quality reporting ■Application using web-browser environment Reasons and Aims

4 ■Q reporting focused on a survey (of any kind) and groups of surveys ■Cross-cutting info on quality of statistical process, output data and products ■Data preferably retrieved from other source databases ■Data monitoring, comparisons, aggregations, assessment, benchmarking ■Flexibility of metadata content and possibility of survey’s adjustments ■Help to improve quality reporting and statistical quality itself ■Encourage self-assessment, support auditing Aims and main features

5 ■The application is integrated within the internal SIS and SMS systems ■Meta-data values retrieved from databases or manually inputted ■Design & preparation of various quality reports ■Hierarchical structure (refers to ESQR, GSBPM, DESAP) ■Public vs. non-public – individual items or complete reports ■Bilingual (multi-lingual) solution ■Usual output formats PDF, HTML, XLS, DBF, DOC, not SDMX ■User roles: admin, owner, editors, viewers (public vs. internal) Features

6 Other SMS subsystems can provide certain knowledge on quality criteria e.g. accuracy, relevance, accessibility, clarity, timeliness, punctuality. ■SMS SURVEYS: statistical processes (particular surveys) ■SMS REQUIREMENTS: management of main user requirements ■SMS DISSEMINATION, CATALOGUE OF PUBLICATIONS: dissemination, product quality, info service in some cases in relation to concrete surveys Interlinks with other SMS-applications

7 Any statistical process processed or at least with its data stored in the central DWH. ■Business statistics ■Social and demography statistics ■National accounts ■Price statistics ■Administrative data statistics. Coverage

8 Type of info on quality - quantitative and qualitative: ■Reference metadata ■Info about process and its phases ■Schedules ■Quality performance indicators ■Calculations ■Benchmark results ■Evaluation, (self-)assessments, commentaries ■Textual, Numerical, Date Q-attribute (item, meta-information, indicator)

9 ■Basic information (about a survey) ■User requirements agenda ■Methodology info ■Time schedules; Timeliness; Punctuality ■Statistical process phases ■Data confidentiality and protection ■Data sources; Frame; Sample ■Outputs and dissemination ■Individual quality criteria (i.e. quality dimensions) ■Quality performance indicators Categories of Q-attributes (info on…)

10 Relates to functionality (stability of values) ■General ■Statistical survey (key users, methodology, key statistical variables…) ■Reference year ■Processing (all reference periods processed or revised at one time) ■Reference periods Levels of Q-attributes

11 ■To provide relevant & up-to-date information ■Validity for certain years, batches (i.e. processings), ref.periods ■When generating data for new reference periods... ■Metadata updates on each level ■How to update the derived Q-Maps ■Managers informed and decide via the application ■Keeping history and updates Updates of metadata structures and values

12 ■Category ■Level of Detail (validity in time) ■Benchmarking ■Source of Values ■Nomenclature ■Data Format and Data Mask ■Parent-Child, Layout ■Multiplicity of Values Parameters of Q-attributes

13 ■Structure (hierarchy): Sections, Sub-sections, Q-attributes ■Q-Maps: monitoring, benchmarking General - > Specific -> Survey Q-Maps design, specifications Value Q-Maps output report ■Q-Forms: also comparisons and aggregations… General QM for Q-Forms -> Q-Form -> Value Q-Form Q-Forms use (not only) Value Q-Maps as the source of data Q-Maps & Q-Forms

14 Comparisons, aggregations over ■Statistical variables ■Reference periods ■Surveys ■Years… Which data ■Values ■Benchmark results Q-Forms - comparisons, aggregations

15 Design of a report ■General Q-Map -> A type of report. General design, pre-setting of parameters. ■Specific Q-Map -> A group of surveys. Selection of Q-attributes, way of benchmarking. ■Survey Q-Map -> A survey Statistical variables, benchmark scales, links to data. Output report ■Value Q-Map -> One reference period. Retrieval, editing, approval of values. Benchmarking. Levels of Q-Maps - Hierarchy

16 ■Primarily for internal management purposes ■Benchmarked values: numerical or textual ■Adjustments of scales (boundaries) for particular surveys ■Parameters ■To benchmark or not to benchmark? ■Manually (each value individually) or Automatically (pre-definitions) ■Categories’ definition – number of categories, and either definition of boundaries or assignment of values from a nomenclature ■Categories’ labelling – from a special nomenclature or directly in the app Benchmarking

17 ■Deeper relations between subsystems ■Revisions of quality attributes ■Involvement of domain statisticians ■Full implementation ■ESS standard quality reports in SDMX Challenges

CZECH STATISTICAL OFFICE | Na padesatem 81, Prague 10 | Thank you for your attention. Any questions?