1. ESS.VIP ADMIN EssNet on Quality in Multi-source Statistics, progress report
19TH WORKING GROUP ON QUALITY IN STATISTICS, 6 December 2016 Fabrice Gras, Unit B1 – methodology and corporate architecture

2. ESS VIP.ADMIN Quality of multisource statistics
Three specific grant agreements (SGA) foreseen under a framework partnership agreement
(DK, IE, IT, LT, HU, NL, AT, NO):
SGA1: exploratory work and gap analysis, development of some indicators (signed in 2015)
SGA2: proposal of a structure for the guidelines and development of some chapters, including proper methods and indicators (to be signed in autumn 2016)
SGA3: completion of the guidelines (to be signed in autumn )

3. ESSnet on quality of multisource statistics estimated timetable
Framework Partnership Agreement
08/ /2019
1st Specific Grant Agreement
01/2016 – 04/2017
2nd Specific Grant Agreement
05/ /2018
3rd Specific Grant Agreement
06/2018 – 08/2019

4. Quality of multisource statistics
On-going first SGA of the ESSnet
WP1: Quality of input data
WP2: Frame Quality
WP3: Quality of statistical output
WP4: Communication, dissemination and implementation

5. ESSnet on quality of multisource statistics -goals
Work Packages
Contents
WP1. Checklist for evaluating the quality of input data
Review and testing of existing checklists
Commented repository
Recommendation on further action
WP2. Methodology for the assessment of the quality of frames for social statistics
Indicators
Sufficiency criteria
Metadata requirements
Recommendation for a monitoring process
WP3. Framework for the quality evaluation of statistical output based on multiple sources
Critical review of existing indicators and approaches; testing
Proposal of new approaches and indicators
Implementation guidelines
Recommendation for updating the for quality reports
WP4. Dissemination and implementation
Produce some quality reports
Workshop – stock taking, exchange of experience
Promotion of results

6. Quality of multisource statistics
Input quality
Quality of frames for social statistics
Output quality

7. 1st SGA Milestones: January 2016 – first ESSnet meeting
April 2016 – workshop on quality aspects of using administrative sources in the production of statistics
June 2016: final report for WP1,intermediate report for WP2 and WP3 and summary document on the workshop.
September 2016: intermediate ESSnet meeting.
November 2016: intermediate report for WP2 and WP3
March 2017: final ESSnet meeting
April 2017: Final report for WP2 and WP3

8. WP1 objective: Creation of checklists for evaluating the quality of input data
Subtasks:
Critical review and testing of existing methodology
Commented repository
Consolidated version of checklist
Identification of possible gaps

9. The work so far, WP1:
A gross list was created with more than 500 indicators
Members of the ESSNET selected indicators for testing
Clear indicators with precise definitions
Six dimensions are used (similarly to the ESSnet Admin project)
At least 2 indicators for each dimension
Our list uses 16 quantitative indicators
Tests were carried out by 3 countries
Report of WP1 has been circulated in the ESS and is under approval

10. First Specific Grant Agreement –WP1 on input data
Gaps:
Pre-checklist to use before data becomes available
Check-list to compare two sources
Deliverable
Expected delivery
Actual delivery
Status
Draft report on checklists for evaluating the quality of input data
May 2016
June 2016
Circulated to MS
Final report
June 2016
August 2016
Under revision

11. WP2 objective: Produce guidelines on how to assess frame quality for social statistics
Subtasks:
Literature review
Comparative analysis
Gap analysis
Proposal of quality measures
Development and test

13. First Specific Grant Agreement –WP2 on frames for social statistics
Gaps:
The analysis list the existing approaches and levels of maturity
Less mature approaches should be developed
Deliverable
Expected delivery
Actual delivery
Status
Intermediate report on the quality of frames for social statistics
June 2016
July 2016
Under revision
Intermediate report
Draft action plan
November 2016
Will be circulated to MS
Final action plan
Final report
April 2017

14. WP3 objective: Produce relevant measures for the quality of the output
Subtasks:
Critical review on eight topics: classification error, linkage error, under coverage error, measurement errors, macro-integration, repeated weighting, benchmarking and time series/small area estimation
Suitability tests: 7 basic data configuration to be considered
Action plan

15. The work so far, WP3:
Definition of basic data configurations to be tested (Komuso classification)
Literature review on: quality assessment of imputations in administrative data, unit non-response, coverage (frame) errors, capture-recapture techniques, and classification errors
11 suitability tests already carried out (use of latent variable, use of simulation, estimation of classification errors).
Action plan:
Making indicators more suitable for application in practice
Existing gap: assessment of the coherence dimension of quality (difference in concepts or in methods)

16. First Specific Grant Agreement –WP3 on output
Gaps:
Several data configurations have been analysed
Gaps for each configuration are identified
Other data configurations could be covered
Deliverable
Expected delivery
Actual delivery
Status
Intermediate report on the quality of statistical output based on multiple sources
June 2016
July 2016
Under revision
Draft action plan
Intermediate report
November 2016
Will be circulated to MS
Final action plan
Final report
April 2017

17. Example of basic data configuration (1)

18. Example of basic data configuration (2)

19. First Specific Grant Agreement –WP4 on dissemination
Deliverable
Expected delivery
Actual delivery
Status
Communication plan
March 2016
June 2016
Accepted
Workshop on quality of multisource statistics
April/May 2016
April 2016
Workshop documentation
May 2016
CROS portal page
April 2016
On-going updates
Workshop:
Validation of the ESSnet approach
Request to produce guidelines
Communication: VIG, DIME, DSS, WG Methodology, WG Quality, ESS.VIP BIGD, ESS.VIP ESBRs, UNECE Data Integration, UNECE TF on census, Q2016, CESS 2016, NTTS 2017, ISI 2017.

20. Focus on the workshop:
Knowledge of the ESSnet – its members and its workplan
Presentation of good practices
Discussion of the tasks and objectives of the ESSnet
Discussion in general on quality
Presentations available on the CROS portal at:

21. Next steps Second SGA: Third SGA Synergies Structure of guidelines
Complete some parts of it
Third SGA
Finalise guidelines
Practical examples/quality reports
Synergies
Other ADMIN WPs (applications, guidance on estimation methods)
Planned implementing regulation on frames for social statistics
ESS.VIP BIGD

22. Thank you for your attention!
CONTACT:
ESS. VIP ADMIN
Sorina Vâju (Eurostat):
Fabrice Gras –
ESSnet on quality of multisource statistics
Niels Ploug (Statistics Denmark):