1. Data validation rules
Item 3b Eurostat Task Force on Annual Financial Accounts Frankfurt, 4 March 2016

2. Data transmission/validation procedure
SDMX template or DB extraction XML files eDamis notification production database (FINA2010) structural and data check reports contact countries validation dissemination

3. Transmission errors
Errors leading to file rejection (no data check report produced)
Quality issues with the data (presented in the data check reports

4. Coding: embargo dates WRONG CORRECT
A country may forget to delete or update an old embargo date
Use of embargo dates with flags other than "N" (e.g. "F" or "C")
dates are taken into account only with flag "N"
WRONG
CORRECT

5. Coding: flags (1) WRONG CORRECT
"NaN" values with OBS_STATUS "A" or "E" – wrong!
"NaN" can only be used with flags such as "L", "M", "J" etc.
Numerical values (including "0") with flags "L", "M" or "J" – wrong!
Numerical values can only be used with flags "A", "E", "P", etc.
WRONG
CORRECT

6. Coding: flags (2) WRONG Value "0" used as OBS_STATUS and CONF_STATUS
"J" or "M” used as a CONF_STATUS
"v" used as OBS_STATUS
Value "L" used instead of "NaN"
Value with two dots inside
"Nan" values instead of NaN (it is case-sensitive).
WRONG

7. Coding: flags (3)
Flag "L": data should exist but are not collected (not available)
Flag "M": data do not exist by definition; not applicable
Zero value is not a substitute for either "L" or "M"
WRONG

8. Coding errors: reference area
Use of "LU" as Reference Area
This happens because "LU" is used in the sample questionnaires and countries forget to change the field
WRONG (unless you are Luxembourg…)

9. Coding errors: sector S2
Balancing items (B9f, BF90) are to be transmitted with opposite sign: S1 S2
SECTOR
SUB-SECTOR
Total economy
SECTOR ► S1
OBS_STATUS
OBS_CONF
CPAREA ► W0
B9F
457.00
SECTOR
SUB-SECTOR
Rest of the World - total (4)
SECTOR ► S1
OBS_STATUS
OBS_CONF
CPAREA ► W1
B9F
CORRECT

10. Other coding errors
Wrongly coded series probably linked to old SDMX template
Dummy figures of SDMX template left in the XML file
Some numbers transmitted with space(s) inside, probably due to conversion issues.
Obligatory parameters missing from the XML file, such as UNIT_MULT, TABLE_ID, Valuation, Maturity etc.

11. Validation checks See FINA2010 document
Structural report: checks the validity of the contents of the transmission (correct coding, series transmitted)
Data check report: checks the internal coherence of the data (mainly arithmetic checks)
(Revision report: checks size of revisions, comparing new transmission with existing validated data)

12. Eurostat task force on data validation
TF Validation met in March and October It includes NSIs and NCBs, ECB and OECD.
Objectives
To review validation checks performed in Eurostat upon reception of ESA 2010 TP tables
To clarify methodological or practical aspects underlying issues common to ESA data transmissions
To propose validation rules for an internal or external pre-validation tool
To investigate possibilities of a common process for the collection and dissemination of metadata.

13. TF Validation: current work
draft a validation handbook
introduction of automatic validation reports
Set-up in 2016 of a structural validation service: allows checking a transmitted file for SDMX compliance. Invalid files will be rejected and a report sent to the sender by eDamis.
Note: pre-validation service: part of 3-pillar strategy for quality assurance of ESA datasets, along with the quality reports and compliance monitoring.

14. ESA 2010 quality assessment framework
Regulation (EU) No 549/2013, articles 4 & 12:
MS shall provide to Eurostat a report on the quality of transmitted data [Art. 4(2)]
Eurostat shall assess quality of MS' data transmissions [Art. 4(4)]
Provision of a report on application of the regulation to Council and European Parliament by 1 July 2018 and every five years thereafter [Art. 12]