1. Theme (iv): Standards and international collaboration
UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Work Session on Statistical Data Editing April 2017 The Hague, Netherlands
Theme (iv): Standards and international collaboration
Discussants: Agnes Andics, Simona Rosati, Sander Scholtus

2. Introduction to theme (iv)
Full title: Standards and international collaboration – including implementation of the new and emerging standards: VTL, GSDEMs, CSPA
Development of methods, processes and software tools for data editing is often complex, time-consuming and costly
Statistical organisations face similar challenges
It seems natural to collaborate and share methods and tools
However, sharing requires agreement on standards

3. Introduction to theme (iv)
New and emerging standards:
Generic Statistical Business Process Model (GSBPM)
Generic Statistical Data Editing Models (GSDEMs)
Generic Statistical Information Model (GSIM)
Common Statistical Production Architecture (CSPA)
Validation and Transformation Language (VTL)
In this theme: presentations on
developing data editing infrastructure based on these standards
initiatives by individual statistical institutes and international projects

4. Agenda for theme (iv) Monday afternoon: Tuesday morning:
Eurostat: The modernisation of validation in the ESS – a multidimensional approach
Norway: Methods library as part of the modernization of the statistical production in Norway
Poland: Technical aspects of VTL to SQL translation
Tuesday morning:
Norway: A new GSDEM of multisource data for multiple statistics
Germany: Validation, Shared Services and Enterprise Architecture: How it fits
Germany: The ESSnet ValiDat Integration
General discussion

5. Summary: presentations in theme (iv)
WP.13 The modernisation of validation in the ESS – a multidimensional approach (Eurostat)
The ESS and the ambitious programme to improve the way data sent to Eurostat are validated.
A multi-pronged approach chosen by the ESS to achieve its medium-term goals for validation:
Ensure the transparency of the validation procedures through a common validation policy.
Improve the interoperability between Eurostat and Member States.
A concrete example of implementation of this approach in the National Account domain.
ESS Validation principles

6. Summary: presentations in theme (iv)
WP.14 Methods library as part of the modernization of the statistical production in Norway (Norway)
A method library to offer common methods throughout the production process:
CSPA-compatible
R language
R packages
Wiki page for user documentation
Separate graphics module
Methods included:
Macro-editing
Imputation
Confidentiality

7. Summary: presentations in theme (iv)
WP.15 Technical aspects of VTL to SQL translation (Poland)
VTL as a new international standard language for defining validation and transformation rules on statistical data:
Different version of VTL.
Translation from VTL to typical data processing languages.
A source-to-source compiler – VtlProcessingLib – was developed to validate data and aggregate:
A programming library that allows for translating source code of VTL to target code of different programming languages.
The efforts are concentrated on developing overall software architecture independent on the language specification’s details.

8. Summary: presentations in theme (iv)
WP.16 A new GSDEM of multisource data for multiple statistics (Norway)
Editing multisource data for producing multiple statistics
“Parallel editing” versus “coordinated editing”
Advantages of coordinated editing:
Avoids duplication of data editing functions
Ensures harmonisation of related statistical outputs
Possibility for generating new statistics
Application at Statistics Norway: employment and wage statistics

9. Summary: presentations in theme (iv)
WP.17 Validation, Shared Services and Enterprise Architecture: How it fits (Germany)
Different types of architecture relevant to the production of statistics:
Business Architecture (GSBPM, GSDEMs)
Information Architecture (GSIM, LIM)
Application Architecture (CSPA)
Infrastructure Architecture
HLG-MOS and ESS Vision 2020: move towards a service-oriented architecture (CSPA)
Application of these ideas to validation (ESSnet ValiDat Integration)

10. Summary: presentations in theme (iv)
WP.18 The ESSnet ValiDat Integration (Germany)
ESSnet project of six national statistical institutes
Builds on results of previous ESSnet ValiDat Foundation
Handbook on methodology for data validation (2016)
Goals for this project:
Improve handbook
Develop better validation metrics (to evaluate rules and data)
Investigate costs and benefits of different strategies to implement validation services within the ESS

11. Questions / Points for discussion
New GSDEM: coordinated editing
Do other institutes have experiences with parallel editing / coordinated editing of multisource data?
How to maintain coordination in practice between the statistic-specific editing functions beyond the intermediate statistical data?

12. Questions / Points for discussion
Several initiatives presented to create new tools for data editing and validation that comply with standards (CSPA, VTL). What about re-use?
Are there institutes already (planning to) re-use some of these tools in their own statistical production?
Or are many institutes (planning to) develop their own shareable tools, which may lead to many competing services that offer a similar functionality?