GSBPM Giorgia Simeoni, Istat, simeoni@istat.it “Methods and tools for reference metadata documentation and communication” Project “Methods, quality and metadata” Division ESTP Training Course “Information standards and technologies for describing, exchanging and disseminating data and metadata” Rome, 19-22 June 2018 THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION

What is GSBPM? The Generic Statistical Business Process Model (GSBPM) is a international standard model that “describes and defines the set of business processes needed to produce official statistics” http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0

GSBPM Background Developed by the Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (METIS) Born as a documentation model, with the aim of standardising terminology First version released on April 2009 (4.0) Widely adopted by the global official statistics community Revised in 2013: to incorporate feedback based on practical implementation to improve consistency with Generic Statistical Information Model (GSIM), released in December 2012

GSBPM 5.0 GSBPM 5.0 released in December 2013 A cornerstone of the High-Level Group for the Modernisation of Statistical Production and Services (HLG) vision and strategy for standards-based modernisation Managed by the “Supporting Standards Group”, under the HLG Fully aligned with version 1.1 of the GSIM http://www1.unece.org/stat/platform/display/GSBPM/Generic+Statistical+Business+Process+Model

GSBPM has been approved as ESS Standard by the ESSC in February 2017. GSBPM as ESS Standard GSBPM has been approved as ESS Standard by the ESSC in February 2017. Definition of ESS standard: A normative document, established by consensus among ESS members and approved by a recognised body according to the procedure of ESS standardisation, that provides for common and repeated use by several actors in the ESS, rules, guidelines or characteristics for the development, production and dissemination of European Statistics, aimed at the achievement of the optimum degree of order in the context of the implementation of the mission and vision of the ESS.

Overarching processes The Generic Statistical Business Process Model Level 0 Overarching processes Level 1 Level 2 5.2. Classify and code - This sub-process classifies and codes the input data. For example automatic (or clerical) coding routines may assign numeric codes to text responses according to a pre-determined classification scheme.

Main characteristics Organised in 3 levels with increasing level of detail Detailed description of what is included in each sub-process Several overarching processes broadly classified in 2 categories: - With statistical component More general It is «Generic» National, domain-specific or other implementations (e.g. Checklist for SDMX Design Projects) are possible It is extremely flexible: Not all the subprocesses should be performed The order of subprocesses can be different from the one presented It is possible to repeat some steps more than once

Applicability “The GSBPM is intended to apply to all activities undertaken by producers of official statistics, at both the national and international levels, which result in data outputs” Surveys Censuses Processes based on administrative records Processes based on other non-statistical sources Processes based on mixed sources Revision of existing data Re-calculation of time-series Compilation of National Accounts Processes of international statistical organisations Development and maintenance of statistical registers

Overarching processes With statistical component General Quality management Human resource management Metadata management Financial management Data management Project management Process data management Legal framework management Knowledge management Organisational framework management Provider management Strategic planning Statistical program management Statistical framework management Customer management GAMSO

GAMSO v.1.1 The Generic Activity Model for Statistical Organisations

The Generic Statistical Business Process Model Planning Realisation

The planning sub-processes Dialog with users, Identification of needs (new or additional), Definition of high level solution, Get approval from senior management Definition of all methods and tools that will be used in the realisation of the statistical process Set up and test of all methods and tools defined in the design stage

The realisation sub-processes (1) The actual data acquisition, whatever the source or the method used, including data entry The traditional phases of data treatment till the macrodata estimates are produced It includes the production of complex statistics (e.g. indices), macrodata validation, confidentiality treatment

The realisation sub-processes (2) The release of statistical outputs to users The quality evaluation done at the end of a specific edition of a statistical business process Quality Management The overarching process on Quality represents quality assurance system implemented across the business process

Uses of GSBPM Standardisation of terminology in international context Support to statistical process documentation Analyse processes in order to identify common subprocesses Make inventory of available IT tools and application to rationalise and identify gaps Make inventory of available methodological tools to rationalise and identify gaps Reference model to support audit and self assessment procedures …

Example 1. Business Process Model at Statistics Sweden http://www1.unece.org/stat/platform/display/GSBPM/Implementing+the+GSBPM

Example 2. Spreadsheet for process documentation (v4.0) http://www1.unece.org/stat/platform/display/GSBPM/Uses+of+GSBPM

Example 3. ABS Prices processes mapped to GSBPM (V4.0)

Example 4. Istat repository of methods and tools organised by GSBPM phases https://www.istat.it/en/methods-and-tools/methods-and-it-tools

Quality indicators for GSBPM A wide set of quality indicators (partly qualitative and partly quantitative) has been mapped to each subprocess of GSBPM version 5.0 with the aim of expanding the quality management layer for the GSBPM UNECE (2017) Quality Indicators for the Generic Statistical Business Process Model (GSBPM) - For Statistics derived from Surveys and Administrative Data Sources. Version 2.0 October 2017

Principles used in mapping the QIs to the GSBPM Generic indicators as GSBPM is; Consistent with existing quality assurance frameworks; No formulas, only descriptions or explanations; Quantitative indicators whenever possible; Qualitative indicators in the form of yes/no or large/medium/low when appropriate; Map indicators to the phase they measure even if they might be calculated at a later stage; Allow for a certain degree of redundancy by mentioning the same indicators in different phases or sub-processes Personalisation of the indicators left to NSIs

Example Quality Dimension Indicator Notes Soundness of implementation Has the questionnaire been tested using appropriate methods (e.g. questionnaire pre-test, pilot in real situation, in depth - interviews, focus groups, interviewer support,…)? Corresponds to the appropriate statistical procedures principle in the ES Code of Practice Have the test results been taken into account in the process of implementing the final questionnaire, and documented in a report? Corresponds to… Has the data collection tool/instrument (electronic questionnaire, acquisition web site, SDMX hub) been tested and how? This indicator refers to the tests of the IT instruments used for data collection (e.g. functionality test, stress test…) Corresponds to … To what extent have the test results been taken into account in the process of implementing the final data collection tools Corresponds to…..

Example Quality Dimension Indicator Notes Accuracy and reliability   Edit failure rates can be calculated for key variables and by domains of interest. A sub-class of edits could be those designed to detect outlier observations. A high/very high edit failure rate for a given variable would be suggest possible errors in previous phases (e.g. in the questionnaire or in data collection). ………. ……….. Accessibility and clarity Percentage of metadata adequately archived (easily retrievable; properly labelled; retention period indicated)

On-going activities on GSBPM Further revision of GSBPM: a consultation is ongoing on the UNECE wiki. Relationships between GSBPM and other standards are being discussed Improvement to specific phases and sub-processes description

The Istat experience on business process documentation: SIDI/SIQual Istat official information system for documenting quality and reference metadata of the statistical production processes SIDI «input» system, SIQual for consultation SIDI first implementation in 2001, later converted in a web-based architecture SIQual first release in 2005, English version released in 2008 SIDI/SIQual approach to documentation is highly structured and standardised, descriptive additional fields are available to better describe standard items

SIDI/SIQual contents IN-DEPTH DOCUMENTATION: Sampling design Index production methodology … CONCEPTUAL METADATA Themes Units …. OPERATIONS Phases Operations Sub-operations DOCUMENTS REPOSITORY Regulations, Manuals Questionnaires Documents (field operations, methods, standards, …) GENERALISED SOFTWARE Phases Software QUALITY CONTROL ACTIONS Phase/Non sampling Errors Preventing Monitoring Evaluating Standard quality indicators Process oriented Coverage Unit non response Coding Editing and imputation Costs Product oriented Revision analysis Timeliness and punctuality Coherence preliminar/final results Coherence with other sources Lenght of comparable time series

SIDI/SIQual example of documentation

SIDI/SIQual example of documentation

Mapping GSBPM and SIDI/SIQual model ISTAT SIDI/SIQual 1. Specify needs Planning 2. Design Frame Development 3. Build Re-planning 4. Collect Data Collection Data Pre-processing 5. Process Editing and Imputation Data Processing 6. Analyse Data Validation Statistical Disclosure Control Dissemination 7. Disseminate Data Storage Documentation 8. Evaluate Evaluate (QIs and Quality Control)

Reference UNECE (2013) Generic Statistical Business Process Model GSBPM (Version 5.0, December 2013) http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0 UNECE (2013) Generic Statistical Information Model (GSIM): Communication Paper for a General Statistical Audience (Version 1.1, December 2013) http://www1.unece.org/stat/platform/display/metis/Generic+Statistical+Information+Model UNECE(2011) Strategic vision of the HLG http://www1.unece.org/stat/platform/display/hlgbas/Strategic+vision+of+the+HLG UNECE (2017) Quality Indicators for the Generic Statistical Business Process Model (GSBPM) - For Statistics derived from Surveys and Administrative Data Sources. Version 2.0 October 2017 https://statswiki.unece.org/display/GSBPM/Quality+Indicators+Home