Photis Stavropoulos AGILIS SA Draft quality report of the implementation of the 2003 LFS ad hoc module on lifelong learning Photis Stavropoulos AGILIS SA ETS-WG meeting, 31/01-02/02, 2005

Introduction 30 countries included an ad hoc module on Lifelong Learning (LLL) in the 2003 Labour Force Survey (LFS) A draft quality report about the implementation of the module has been prepared, based on information provided by the countries until 7/1/2005 This talk presents the main findings contained in the draft report The final report is scheduled for 31/3

Overview Collection of information about the implementation of the module Main aspects of the implementation of the module Draft quality assessment Planned improvements for the final quality report

Information collection Countries submitted MEGs, Grant reports, Self evaluation reports, etc. A metadata questionnaire was prepared Questionnaires were pre-filled with submitted information and sent back to countries Questionnaires returned until 7/1/2005 and available reports were used for the preparation of the draft quality report

Implementation of the module Adoption of the questionnaire Most variables were adopted as defined in Commission Regulation 1313/2002 A few of them however were either modified or not included at all, e.g. HATFIELD, LLLFIELD were considered covered by HATFIELD and EDUCFIELD of the core questionnaire (ES, FI, SE, DK, HU, IT) Registers were used for LLLLEVEL and LLLFIELD (NO) and for HATFIELD (SE and FI) UK constrained questions about non-regular education to the most recent activity and Norway to the most recent job-related activity Most extended modifications in Norway, UK and France

Implementation of the module Target population Regulation 1313/2002: persons aged 15 years and over Exceptions 15-74: DK, EE, FI, HU, LV, NL SE, 16-74: CH, IS, 16 - … : ES, UK, 15-64: BE Norway used a target population of 15 to 59 years for questions on regular education (registers for older persons) and 16 to 74 for learning outside the regular education system. Sampling scheme Two-stage stratified sampling of individuals or households in the majority of cases

Implementation of the module Mode of data collection PAPI or CAPI, often with CATI Only CATI in the Nordic countries Postal survey (Denmark, in addition to CATI) Proxy interviews allowed in most countries Response rates

Implementation of the module Reported users and uses of the module’s results Users: Ministries, media, academia, research organizations Uses: Indicators on LLL, linkage of information on training activities with information on their cost, comparisons at international level Dissemination of results Publications in paper and electronic format, mailing lists, provision of anonymized microdata to researchers

Implementation of the module Main reported problems Recalling of short term learning activities Inefficient coding of field of training High percentage of “General programs” in field-of education questions Difficulty in distinction between formal and non-formal education High non-response about taught activities Inaccurate information Lack of coherence of results with other sources High cost of the survey

Implementation of the module Suggestions for improvement, made by countries Reduce reference period Limit detailed investigation of taught activities to the most recent one Ask about duration of taught activities in closed form (pre-defined time intervals) Include additional checks during data entry Improve training of interviewers on coding of field of education Provide more explanations in the questionnaire about the questions

Quality assessment Relevance LFS regulation restricts the size of the module Consequence: not all potential needs can be covered Coverage of needs for indicators of LLL Participation rates and time spent in LLL (covered) Perceived interest in learning (partly covered - LLLCOURPURP) Sources of financing and employer support for learning (partly covered - LLLCOURWORH) Outcome of learning activities (not covered) Barriers to learning (not covered) Investment in LLL (not covered) Conformity of countries with Commission Regulation Modification or non-inclusion of certain variables Reduction of target population in Belgium (15-64)

Quality assessment Accuracy Direct assessment (variability of results) Sweden reports rates with s.e. < 1.5% in most cases Latvia reports main rates of participation in LLL with c.v < 5.85% Indirect assessment Coverage errors: reported at below 2%, except EE (5%) and ES (17%) Measurement error sources: difficulty to distinguish formal from non-formal education, comprehension problems for informal training activities, long reference period, proxy interviews, coding errors during the interview Processing errors: Coding errors at the statistical agency Non-response: the rate of non-response was non-negligible in several countries, no information about differences between respondents and non-respondents

Quality assessment Comparability Concepts Measurement processes Variables used: Countries with greatest deviation from the regulation were France, UK and Norway Statistical unit: Individual and educational activity (common for all countries) Target population: Major discrepancies were found in Belgium(15-64) Reference period of results: Sweden, France and Iceland implement the module throughout 2003, extending the reference period of results Measurement processes Various sampling schemes, not common data collection mode, difficulties in education field coding

Quality assessment Coherence HATFIELD is in agreement with data from administrative registers in Denmark LLLLEVEL compared with data from educational institutes in Bulgaria Overestimates participants up to and including upper secondary education  due to different reference periods Underestimates participants to higher education  difference possibly within limits of sampling error LLLCOURWORH compared with Staff Training Survey (STS) in Sweden Underestimates training financed by the employers Consistent patterns of participation in employer-financed activities by sex and level of education Inconsistent differences in rates between subpopulations LLLLEVEL, LLLCOURLENP, LLLCOURPURP incoherent with core LFS questionnaire

Quality assessment Timeliness and punctuality Distance between end of reference period and time of delivery to Eurostat ranges from 6 to 11 months Punctuality: Sweden delivered data 2 months after scheduled time, Hungary was on time

Accessibility and clarity Quality assessment Accessibility and clarity Users can obtain statistics in the form of results publications (paper and electronic) and anonymized microdata (Ireland, Hungary, Latvia) Methodological information (Ireland and Hungary) Information in MEGs, Grant reports, self evaluation reports  not widely available

Planned improvements for final quality report Presentation of data processing and analysis Detailed description on data editing, treatment of non-response, presentation of the results Problems and suggestions The section will be revised with new information Relevance More information on user needs, users and uses of the results of the module Accuracy Coefficients of variation and non-sampling errors Comparability Visual representations of comparability using multidimensional scaling and clustering techniques on the complete metadata Coherence Comparison of the EU results and similar EU-wide results

… your comments are welcome