Kristina Dourmashkin Eurostat Unit E4

Slides:



Advertisements
Similar presentations
Online, Web-Based Data Collection System February, 2011.
Advertisements

Hearth Bulk System Divisional Secretaries’ Briefing 2012.
Eurostat Unit B3 – Statistical Information Technologies Data transmission tools and services 15/05/ eDAMIS The standard solution for transmitting.
Editing Building Block (EBB) Validation Tool for FDI and ITS Balance of Payments Working Group 02 April 2012 Unit B4, IT for Statistical Production Georges.
Eurostat 1 7a. Practical use case 1: Pesticides Use Project Blanaru Cristina Eurostat Unit B5: “Central data and metadata services” SDMX Basics course,
IS4STAT Vittorio Viaggi Eurostat
7b. SDMX practical use case: Census Hub
ESTP Course on the EGR November b. Identification of LEU: data flows to/from the EGR IS.
IT Directors’ Group Meeting October 2010 Sharing data validation tools in the ESS Christine WIRTZ – Head of Unit B3 Georges PONGAS – Unit B3 Daniel.
EDIT validation tool item 8 of the agenda Structural Business Statistics Working Group 14 April 2015, Luxembourg Arlind Dobërdolani.
State of play – Urban Audit data collection
Training Course on EDIT 2013
Catch and Landings statistics
Catch and Landings statistics
New datasets by regional typologies based on multi-dimensional tables
Data validation rules Item 3b Eurostat Task Force on Annual Financial Accounts Frankfurt, 4 March 2016.
EUROSTAT Unit B3 IT for statistical production Ewa Stacewicz
Catch and Landings statistics
Workshop on the Validation of Waste Statistics
NUAC meeting June Validation – where are we?
Kristina Dourmashkin Eurostat Unit E4
8 EGR preliminary frame and validation tasks
Data collection of 2012: Data transmission standards and tools
5 NSI data to EGR ESTP Training on EGR 6-7 September 2016.
Data exchange between ENP-South countries and Eurostat
Kristina Dourmashkin Eurostat Unit E4
Progress report on eDamis Regional and Urban Statistics
Hearth Bulk System Divisional Secretaries Briefing 2011.
Integrated management of LAU based territorial classifications
The 2016 update of the Manual on City statistics
Development of production routines for Crime & Criminal justice statistics Arsela Sturc SOGETI.
Data availability – Urban Audit data collection
The future of annual LAU data exchange Abolishing CONC/MODA
IT Directors Group October 2006
Catch and Landings statistics
Data Validation in the ESS Context
State of play of Urban Audit
6. EGR Identification Service
City statistics – state of play of data collection
eDAMIS Status for UA collection
ESS.VIP VALIDATION An ESS.VIP project for mutual benefits
ETS WG meeting 6-7 September 2006
Data Validation in the ESS Context
LAMAS Working Group June 2017
Sharing data validation activities in the ESS.
Data validation Urban Audit Theory and practice
Education and Training Statistics Working Group – 2-3 June 2016
Working Party on Fisheries Statistics 14 October 2013
Unit D2: Regional Statistics New Eurostat rules Data Transmissions
Degree of urbanisation
Recent update of cities and FUAs – metadata of the process
Kristina Dourmashkin Eurostat, Unit E4
Education and Training Statistics Working Group – 1-2 June 2017
Tools for transmitting data to Eurostat The Single Entry Point (SEP)
EGR Identification service
9. Practical use case 3: Pesticides Use Project
provision of statistics in Germany
The migration to the new EDAMIS
Progress Meeting on sub-national statistics Items 2_3_4_6_8 Luxembourg, 7 November Eurostat.
European Statistical System Metadata Handler ESS MH (Super) Providers
eDAMIS – Statistics of usage
Data compilation and pre-validation
The migration to the new EDAMIS
eDAMIS Validation Possibilities
Item 5 Modernisation of the EU-SILC Production
Validation Activities in the ESS What you will hear today…
Validation at Insee.
5 NSI data to EGR ESTP Training on EGR May 2019.
Presentation transcript:

Kristina Dourmashkin Eurostat Unit E4 Overview of data Kristina Dourmashkin Eurostat Unit E4 NUAC meeting 10/05/2017, Brussels

eDamis EDIT + .csv Error report Import + publication Data delivered by eDamis to our directory EDIT + .csv EDIT scans for .csv files, imports to EDIT, validates Error report Errors returned to NSI Import + publication This is what happens when a datafile is sent by eDamis.

EDIT + CSV Check structure Check duplicates Extract relevant data from production database Merged datafile validated EDIT + CSV The validation goes through three steps: Rough checks for right structure, right type, numbers where expected etc. Then it checks for duplicates in the file. If there are any errors in these steps, the datafile is rejected. Step 2: EDIT extracts relevant data from the production database Step 3:Validates the merged file for consistency etc, also time series checks.

eDamis EDIT + .csv Error report Import + publication Data delivered by eDamis to our directory EDIT + .csv EDIT scans for .csv files, imports to EDIT, validates Error report Errors returned to NSI Import + publication Back to the workflow chart. The EDIT + .csv part can be accessed online by you.

Web-access You can do it to! If you ask for access .

To validate a file, upload it to EDIT using "File selection" Parameters are set to a default, but you can (I think) Change max accepted difference per year Check only the data file uploaded Validate all data in the database for one country

EDIT – now coming soon Structure check Inter dataset checks with Type check other regional data Length check Range check Presence check Allowed characters check Code list check  Uniqueness check Control check Spatial level control check Time series check  

All data with reference year 2008 or later has been checked All data received validated using the program

What you can do Send datafiles in comma separated value format Validate data using EDIT Check "old" data using EDIT You are not forced to, but it would help if you could: Send datafiles through eDamis, only in .csv format. Instead of setting up your own validation system, use ours, EDIT is easy to access online, just ask. If you want you can check all the exiting data, also before 2008

https://webgate. ec. europa. eu/fpfis/mwikis/ESSValidServ/index https://webgate.ec.europa.eu/fpfis/mwikis/ESSValidServ/index.php/Main_Page For more information on Eurostat's validation and the EDIT tools, look up the ESS page on Editing and Validation standards

Thank you for your attention! Any comments? Contact: kristina.dourmashkin@ec.europa.eu, E4 ESTAT-Urban-Audit@ec.europa.eu