Workshop on the Validation of Waste Statistics

Slides:



Advertisements
Similar presentations
Eurostat The ESS.VIP Validation and its implementation in waste statistics Q2014 – Session 13 4 June 2014 Hartmut Schrör, Eurostat.
Advertisements

ESS VIP project on Validation
Supplement 02CASE Tools1 Supplement 02 - Case Tools And Franchise Colleges By MANSHA NAWAZ.
Development plan and quality plan for your Project
LEVERAGING THE ENTERPRISE INFORMATION ENVIRONMENT Louise Edmonds Senior Manager Information Management ACT Health.
Release & Deployment ITIL Version 3
Background Data validation, a critical issue for the E.S.S.
What is Business Analysis Planning & Monitoring?
WP.5 - DDI-SDMX Integration
WP.5 - DDI-SDMX Integration E.S.S. cross-cutting project on Information Models and Standards Marco Pellegrino, Denis Grofils Eurostat METIS Work Session6-8.
Eurostat Expression language (EL) in Eurostat SDMX - TWG Luxembourg, 5 Jun 2013 Adam Wroński.
Proposals on standardisation process in ESS, The Hague_ ESS net Preparation of Standardisation 1 Proposals on standardisation process.
Slide 1 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Agenda Item 8 Preliminary ideas for a 2011 census hub Giuseppe.
ESS-VIP ICT Project Work Package III Task Force Meeting, Luxembourg, 5 March 2013.
Metadata Working Group Jean HELLER EUROSTAT Directorate A: Statistical Information System Unit A-3: Reference data bases.
2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.
Statistical data editing - UNECE work session – OSLO September 2012 Proposal of a revised approach for data validation within the European Statistical.
EDIT – Eurostat’s editing tool
Standardisation in the European Statistical System inventory of normative documents and the standard-setting process – results of the ESSnet on Standardisation.
Eurostat 1 3.An overview of the SDMX implementation process Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course,
IT Directors’ Group Meeting October 2010 Sharing data validation tools in the ESS Christine WIRTZ – Head of Unit B3 Georges PONGAS – Unit B3 Daniel.
Eurostat Sharing data validation services Item 5.1 of the agenda.
Validation Architecture in the ESS CSPA Workshop, Geneva June 2016 Geneva June 2016 Eurostat, Vincent TRONET, Unit B1.
M&CML: A Monitoring & Control Specification Modeling Language
Theme (iv): Standards and international collaboration
UNECE-CES Work session on Statistical Data Editing
Monitoring and Evaluation Systems for NARS Organisations in Papua New Guinea Day 2. Session 5. Review of the organizational strategy.
The ESS vision, ESSnets and SDMX
IEEE Std 1074: Standard for Software Lifecycle
ESS Validation State of Play and next steps
Implementing the ESS Vision 2020
EUROSTAT Unit B3 IT for statistical production Ewa Stacewicz
DIME ITDG, Luxembourg 28 June 2016
"IT principles" Context, roadmap
ESTP TRAINING ON EGR Luxembourg – December 2014
ESS Vision 2020 Validation: Implementation of deliverables
SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION
Validation Break-out sessions
22 February, ITDG/DIME Item 2 – Progress and deployment
ESSnet Projects Pascal JACQUES Unit/B5 Methodology and research
2. An overview of SDMX (What is SDMX? Part I)
ESS Vision 2020: ESS.VIP Validation
SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION
2. An overview of SDMX (What is SDMX? Part I)
Data Validation in the ESS Context
ESS.VIP VALIDATION An ESS.VIP project for mutual benefits
Business and IT Architecture for ESS validation
Working Group on Population and Housing Censuses
Data Validation in the ESS Context
Statistical Information Technology
Giuliano Amerini Unit E6 (Transport)
ITDG meeting of of October 2011
Sharing data validation activities in the ESS.
ESS VIP ICT Project Task Force Meeting 5-6 March 2013.
ESS Validation Project State of Play and next steps
3. An overview of the SDMX implementation process
Towards a new tool for seasonal adjustments: Demetra+
SDMX : General introduction H. Linden, Eurostat, Unit B5
Task Force Household Budget Survey Innovative tools and sources
ESS.VIP Validation Item 5.1
AICT5 – eProject Project Planning for ICT
Assessment of quality of standards
Actions on Data validation
SDMX Implementation The National Accounts use case
(Project) SIGN OFF PROCESS MONTH DAY, YEAR
3. An overview of the SDMX implementation process
Modernisation of Validation in the ESS Collaboration with countries
OBSERVER DATA MANAGEMENT PRINCIPLES AND BEST PRACTICE (Agenda Item 4)
ESS Vision and VALIDATION
Validation Activities in the ESS What you will hear today…
Presentation transcript:

Workshop on the Validation of Waste Statistics Item 2: ESS.VIP VALIDATION: Objectives, scope & concepts Angel Simón Delgado EUROSTAT Email: ESTAT-ESSVIP-VALIDATION@ec.europa.eu

ESS.VIP VALIDATION VIP VALIDATION – First Phase VIP VALIDATION – Deliverables in First Phase ESS.VIP VALIDATON – Definition of the project Concept of Excellence in Validation

ESS VIP-V First Phase Overall goal: To develop validation solutions to be used by different production chains (horizontal integration), within the ESS (vertical integration) Bottom-up approach: Extensive consultation of all possible stakeholders Participative management Business driven approach From pilots experience to general principles

Scope, objectives and outputs ESS VIP-V First Phase Scope, objectives and outputs Documentation / Standardisation: template and guidelines for process description template and guidelines for a standard documentation of the validation process Methodological analysis of Data Validation Typology of validation rules Standard definition of validation levels Standard formalised “syntax” (understandable by business users) to express validation rules Distribution of responsibilities in the production chain Guidelines to be used for the attribution of responsibility in the whole production chain (MSs and Eurostat) by the WG. Guidelines based on efficiency principles (Validation  Corrections: “the sooner, the better”) Preparing for IS/IT solutions and architecture

Scope, objectives and outputs ESS VIP-V First Phase Scope, objectives and outputs Towards IT/IS solutions and architecture: Users’ requirements to develop a new software to allow business users to input validation rules and the corresponding error messages in a shared Central Repository of Validation Rules. The new software should be able to generate the rules in the Validation syntax developed by the project. b) Validation Architecture defining the elements and their relationships in an integrated validation system ("common platform") to be used by: Internal users, in an appropriate IS architecture to facilitate horizontal integration of IT/IS systems All stakeholders in the production chain

Contribution to the VISION More efficient production chain with clear attribution of responsibilities Standard definitions, guidelines and validation syntax Development of a common validation procedure Common solutions to be shared within the ESS

Templates & Guidelines ESS.VIP Validation First Phase - Deliverables Documentation Examples Methodology 1.1 Inventory of documents 1.2 Analysis of inventory 1.5 Inventory of validation rules 1.6 Inventory of error messages 1.3 Validation & statistical processes 2.4 Analysis of validation typologies 3.1 Validation rules by typology 3.3 Error messages 1.4 Validation typologies 2.4 Analysis of validation typologies 2.5 Levels of validation Solutions Templates & Guidelines 3.2 Validation syntax 4.1 Functional specifications for GUI 2.1 Documentation of validation process 2.2 Documentation of statistical process 3.3 Error messages 3.4 Selection of validation rules 3.5 Improvement actions 3.6 Attribution of responsibilities

Data Deliverables: Validation levels Validation complexity Same file Within an organisation Within a domain From the same source Same dataset Same file Level 0: Format & file structure Level 1: Cells, records, file Between files Level 2: Revisions and Time series Between datasets Level 2: Between correlated datasets From different sources Level 3: Mirror checks Between domains Level 4: Consistency checks Between different organisations Level 5: Consistency checks Validation complexity

Deliverables: Typology of validation rules File Structure Filename File type Delimiters Format >1 file checks Referential integrity Code list Cardinality Mirror Time series Revised data integrity Model-based consistency 1 file checks Type Length Presence Allowed character Uniqueness Range 1n file checks Consistency Control Conditional

Validation Controls – Different actors – Different responsibilities Deliverables: Guidelines for the attribution of responsibility of validation activities in the whole production chain Guidelines for the allocation of responsibility, for the implementation of validation rules within the ESS based on an AGREEMENT Eurostat-NSI's with periodic performance revisions from both sides Proposal for a generic business architecture of data validation: Validation Controls – Different actors – Different responsibilities Step 1 Step 2 Step 3 Step 4 Data preparation by the NSI's Transmission of data and validation report Loading data to production database Additional processing & dissemination

Deliverables: Standard template for error/warning messages Standard templates for error/warning messages and for validation report Validation report structure: Body Footer Rules applied Error/warning messages Total failures No. errors No. warnings Header Total records Time stamp Records failed User ID Sum of weights Data checked Maximum admissible error weight (dataset name…) Rate of acceptance Maximum possible amount of error Rate of performance Error/warning message structure: Rule ID Severity Rule type ID Message text Action Failing data

Deliverables: VALS - Validation syntax Standard syntax for validation language To define a meta-language for the domain of statistical data validation to express, document and communicate validation rules Trade-off between Human-understandable and Computer-parseable language Implementation through Graphic User Interface to support business users to input and maintain validation rules and rule-sets Types Examples Type check validate ( type(A1.Rcount)='ΤΕΧΤ') Range & math validate( Table2.C_5 between (Table2.C_5_1 + Table2.C_5_2 + Table2.C_5_3 - tolerance) and (Table2.C_5_1 + Table2.C_5_2 + Table2.C_5_3 + tolerance)) Code list check validate ( match_codelist (A1.Quarter, CL_QRT) ) Range check validate ( H.HB050 between 1 and 12 ) Rules & Metarules validate ( age_range_rule.result = true and country_codes_rule.result = true) …

ESS.VIP VALIDATION - Package 1: Proposed approach ESS.VIP VALIDATION - Package 1: IMPLEMENTATION Goals Implementation of the methodological developments of VIP-V Phase I in the statistical domains/WGs Maintenance and refinement of standards developed User requirements for further developments Evaluation, monitoring and reporting

ESS.VIP VALIDATION - Package 2: Proposed approach ESS.VIP VALIDATION - Package 2: MICRO DATA Goals Vertical integration of the micro data validation within the ESS production processes taking into account the results of the first phase of ESS VIP-V Extension of the functional specifications to apply to micro data validation Integrated solutions for micro data validation

ESS.VIP VALIDATION - Package 3: Proposed approach ESS.VIP VALIDATION - Package 3: SOLUTIONS Goals Adaptation of existing validation tools to the functional specifications issued from ESS VIP-V Phase I Deployment of validation solutions to MSs Distribution of validation rules in agreed language Building-Blocks in an adequate web services architecture Provision of web services validation solutions to be used by Member States before transmission to Eurostat

ESS.VIP VALIDATION - Package 4: EXTENSIONS AND GENERAL COORDINATION Proposed approach ESS.VIP VALIDATION - Package 4: EXTENSIONS AND GENERAL COORDINATION Goals Overall coordination of the project Coherence of validation approaches within ESS Implementation of the meta-language within ESS Analysis of links with other VIP & ESS VIP projects Good practices identification More sophisticated validation solutions: Longitudinal validation Mirror checks validation ESS wide shared final warehouses

Elements in a validation system: ESS production chain Eurostat Member States Single Entry Point Statistical domain 1 Member States statistical production and validation Statistical domain … Validation Services Web Inteface Statistical domain n Eurostat

Validation Service Validation Service Data Definition registry Validation rules repository Validation Service Data Errors Validation report Metrics

Good practices in validation Processes with good validation development Document providing detailed validation process and rules, agreed by and available to Data Providers Regular discussion in the WP and update of the document Easy access to the latest version and to previous versions if relevant Domain managers know which validation rules have been applied; they don’t need to know the details of HOW validation rules are technically implemented A clear Validation summary report is provided to Data Providers; a detailed report is available IT tools manage the feedback to users Limits Coherence between maturity of validation and maturity of process Relative effort for harmonisation can be significant for small processes [/small teams] Validation is dynamic (improvement) and needs resources to evolve

Towards the idea of "excellence" in validation Review loop Assess Efficiency and completeness of rules are checked against error metrics and feedback from users Adapt Validation checks are re-designed if needed Execute methods and tools Process Production process integrates the standard validation tools Feedback Standardised feedback to data providers on errors and corrections sent fast Control Validation metrics discussed with the data providers Design validation checks Optimal Validation checks are sufficient, complete, efficient Transparent Validation checks are expressed in an understandable way and written in a document Sound Actions "if error" are defined Implement within ESS What? Validation checks discussed and agreed with data providers Where/who? Optimal distribution of validation tasks amongst Eurostat and the data providers How? All validation checks in a centrally supported tool Validation metrics are established

Current actions and next steps Assessment of Eurostat domains in the field of validation Functional specifications for Validation services to be accordingly adapted to the findings of the project Tools development: System to create/maintain a central repository of validation rules Development and/or adaptation of IT Tools (EDIT, eDAMIS) Set of standard documentation for domain managers for harmonisation of communication with data providers Task Force to: Identify best practices in ESS Advice on implementation in the ESS Optimisation of the validation process

Thank you More information on: ESTAT-ESSVIP-VALIDATION@ec.europe.eu