1. Task Force on Annual Financial Accounts
Item 4. NAPS-S Project & Validation Services
Frankfurt, 9 March, 2017
Eurostat Directorate C
Eurostat

2. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

3. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

4. Summary: Current ESA business process
National Production
Generate SDMX-ML
PUSH Transmission
Structural Validation
Content Validation
Processing & Calculation
Data Sharing & Dissemination
Data provider
Eurostat
Structural Validation

5. New architecture (NAPS-S)
Shared / Replicated Services
For NA
and others
Structural Validation
Content Validation
Statistical Service A
Statistical Service B
SDMX Registry
VTL Repository
Common Repository
Common Repository

6. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

7. 3 possible ways to achieve the goal…
Connect to the process
Transmission to Eurostat required
Eurostat runs service and delivers report
Connect to the service
Integration of service into own workflow
Connect to the repository
Installation of service or tool in own environment
Only access to repository (SDMX DSDs, validation rules)

8. 3 possible ways to achieve the goal…
Connect to the process
Transmission to Eurostat required
Eurostat runs service and delivers report
Connect to the service
Integration of service into own workflow
Connect to the repository
Installation of service or tool in own environment
Only access to repository (SDMX DSDs, validation rules)

9. Transmission flows: Eurostat process
“Production transmission”
NASEC_T0610_A
Eurostat production storage
EDAMIS
Manual review
“Validation only”
VNASEC_T00610_A
Nothing 
Shared / Replicated Services
For NA
and others
Structural Validation
Content Validation
SDMX Registry
Ruleset Repository

10. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

11. Structural Validation SDMX Registry
SDMX compliance
Valid SDMX-ML file
Coded according to the DSD
Mandatory fields present
Correct data types
Dataflow definition
Basic logical checks
Sender ID and REF_AREA
Table ID is present
Value "NaN" and OBS_STATUS
EMBARGO_DATE and CONF_STATUS
PRICES and REF_YEAR_PRICE
Basic content checks
Missing or unexpected series
Hole in series
Zero values
Negative values
General plausibility and consistency (within file)
Additivity of breakdowns
Outliers
Consistency between prices
Unadjusted and adjusted series
Advanced plausability and consistency (across files)
Revisions
Quarterly versus Annual
Same series across tables
Cross-domain or source checks
Balance of Payments
Trade statistics
Labour market statistics
Data pulished by NSI or IO
Structural Validation
SDMX Registry
Content Validation
Ruleset Repository

12. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

13. Example of the STRUVAL error report
SDMX compliance
Valid SDMX-ML file
Coded according to the DSD
Mandatory fields present
Correct data types
Dataflow definition
Example of a mistake
Example of the STRUVAL error report

14. Content Validation Service
Basic logical checks
Sender ID and REF_AREA
Table ID is present
Value "NaN" and OBS_STATUS
EMBARGO_DATE and CONF_STATUS
PRICES and REF_YEAR_PRICE
Content Validation Service
Reports are…
… not the same layout as Struval Struval: technical Conval: statistical … grouped by rule in the ruleset
… CSV files  best opened with Excel

15. NaN is used with flags: M, L, J  used
Basic logical checks
Sender ID and REF_AREA
Table ID is present
Value "NaN" and OBS_STATUS
EMBARGO_DATE and CONF_STATUS
PRICES and REF_YEAR_PRICE
NaN is used with flags: M, L, J  used
(H, Q, S)  not really used in ESA

16. Further details: webinars warning: URLs to change in April!
Understanding validation reports
Understanding Struval, validation grids and country metadata files
Conval introduction

17. Contents National Accounts business process flow Before After
Implementation scenarios for data providers
Scope: Struval & Conval
Report examples
Timeline

18. ? Conval Phase I Done  2016 Q1-2017 Conval Phase II 2017
SDMX compliance
Valid SDMX-ML file
Coded according to the DSD
Mandatory fields present
Correct data types
Dataflow definition
Basic logical checks
Sender ID and REF_AREA
Table ID is present
Value "NaN" and OBS_STATUS
EMBARGO_DATE and CONF_STATUS
PRICES and REF_YEAR_PRICE
Basic content checks
Missing or unexpected series
Hole in series
Zero values
Negative values
General plausibility and consistency (within file)
Additivity of breakdowns
Outliers
Consistency between prices
Unadjusted and adjusted series
Advanced plausability and consistency (across files)
Revisions
Quarterly versus Annual
Same series across tables
Cross-domain or source checks
Balance of Payments
Trade statistics
Labour market statistics
Data pulished by NSI or IO
Done 
2016
Conval Phase I
Q1-2017
Structural Validation
SDMX Registry
Conval Phase II
2017 ?
Content Validation
VTL Repository
Conval Phase III
Well…

19. Timeline for structural validation
Report error messages currently under review
Struval available in production flows:
NAMAIN: all flows
NASEC: table 8 (tables 6, 7 from mid-March)
Struval available in validation flows:
VNAMAIN: all flows
VNASEC: all flows
Struval to be configured:
(V)NASU, (V)NAPENS, (V)NAREG

20. Timeline for content validation
Pilot of Conval: mid-March to mid-April 2017
Parts of tables 1 and 8
Improvements following suggestions from pilot
Rollout to all V-flows for basic content checks
Phase II for selected V-flows & testing
Staged go-live as decided within sub-domains