1 European Conference on Quality in Official Statistics Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey Nicola Massarelli

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 2 Checking the quality of the EU-LFS Ad-hoc quality initiatives –Task force on the quality of the LFS –Workshop on LFS quality assurance Standard quality-assessment tools –Annual and quarterly quality reports of national surveys –Summary: EU-LFS quality report –Summary: Report from the Commission to the Council and the Parliament (every 3 years) –Large extent of metadata

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 3 Labour Force Survey: background Main statistical source on the labour market –Estimates of employment and unemployment (ILO concepts) –Labour market characteristics Carried out in all MS + EFTA + candidate countries –In older MS, EU micro-data available since 1983 ILO concept used also outside the EU (worldwide comparability)

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 4 LFS improvements Increased comparability –LFS is continuous in all MS from 2003, except for IT, AT and MT(2004) and DE (2005) (Council Regulation 1991/2002) –Clearer explanatory notes (from 2006) Increased availability of quarterly micro-data –FR (2003); CY (2004); DE (2005) Increased relevance –6 new variables added in 2006; income from 2009 (Council Regulation 2257/2003; Council Regulation 1372/2007)

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 5 Higher visibility, higher concerns Coherence (national accounts, registered unemployment) Accuracy Comparability (Over space, Over time) Need to review the quality of the LFS and reassure users Workshop on LFS quality assurance Need to streamline the standard tools for quality assessment Task Force on the Quality of the LFS

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 6 Ultimate goal of the two initiatives: consolidate the gains from having adopted an EU-LFS Reassuring users –by highlighting the already good quality standards of the LFS –by improving the survey in areas where weaknesses are detected –by making improvements visible –by improving the standard tools for quality assessment, also promoting a quality assurance framework

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 7 Task Force on the Quality of the LFS 6 meetings between June 2007 and mid-2009 Tactical goals –Assess issues –Discuss how to improve the situation –Recommend good practices Focus: employment and unemployment Priority: coherence and accuracy, then relevance and comparability Composition: 9 MS + Eurostat

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 8 Work Organization: 2 pillars Working program –Meetings schedule –Meetings contents Outline of the final report –Introduction –Review, and (possibly) quantification of quality issues Relevance of ILO concepts Accuracy (sampling and non sampling errors) Coherence Comparability Timeliness and punctuality, Accessibility and clarity (secondary issues) –Recommendations of good practices

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 9 Working tools Assessment of each quality issue based on –TF members expertise –Existing documents –Ad hoc documents on national experiences e.g. flowcharts of national questionnaires Special questionnaires (when necessary) –Introductory questionnaire on main quality concerns (general; all countries) –Reconciliation table (coherence; TF countries) –Questionnaire on communication issues (coherence; TF countries)

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 10 Questionnaire on main quality concerns Especially addressed to countries not in the TF The questionnaire is structured by quality dimension For each quality dimension each paragraph contains only one quality issue Comments cover –a description of seriousness of the problem –if possible, a numeric assessment –(in case) description of how issues were overcome

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 11 Questionnaire on main quality concerns – examples of different kinds of assessment Statement: In Germany, e.g., the level of employment according to the LFS is about 1.5 million less than the estimates from the National Accounts –Answer: In the Czech Republic the level of employment published by National Accounts exceeds approx. 5% the LFS figures Statement: Although most users are aware of the large conceptual and methodological differences between the ILO based and the registered unemployment, any disruption of the assumption of stable relationship between the two concepts will cause perceived problems of coherence –Answer: Methodological changes in 2002 and 2004 improved coherence between LFS and registered unemployment in the CZ

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 12 Reconciliation table

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 13 Questionnaire on communication of coherence issues Questions on seriousness of the problem at national level 1.Has (in)coherence between LFS and NA employment estimates emerged as a problem for users? 2.Who explicitly manifested concern? 3.Which concern? 4.Did it raise a public debate? Questions on actions to reassure users 5.What action has your NSI undertaken to solve the issue and reassure users? 6.What kind of metadata on the incoherence between LFS and NA employment are available to users? 7.Which guidance is provided to users on which data best fits certain purposes? 8.What kind of users are such metadata thought for? 9.How are these metadata disseminated?

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 14 The workshop on LFS Quality Assurance It’s the 2 nd initiative in the field of quality of the LFS in place at the moment Need to review and streamline the scheme currently used Need to broaden the scope of quality evaluation towards quality assurance Need to agree on the calculation of specific indicators –non-response rates –coefficients of variation

Q 2008, Rome, 8-11 July 2008 Improving the quality and the quality assessment of the Labour Force Survey 15 Expected follow-up TF on the Quality of the LFS –General improvement of LFS quality at national and (consequently) EU level Workshop on LFS Quality Assurance –Improving regular assessment at EU level at national level –Increasing international comparability of quality assessment