Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of May 2014

StatFin/EssNet web-project: 1. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22) 2. Pilot survey (N = 8 000), experimental design (”split- plot”) with DK-option, working time and job search questions. Comparison to CAWI data from the same period. Paradata, respondent feedback.

Content of the presentation: 1. Questionnaire design: results & recommendations 2. Data collection process 3. Mode-effects: WSTATOR (employment status) 4. Mode-effects: HWACTUAL (actual working hours) 5. Qualitative feedback

1. Questionnaire design: layout BlaiseIS default layout was not perceived as attractive Number of question / page Navigation bars Instructions Error messages Don’t know –option (DK)

Error message, DK Instructions, info-button Status bar Number of questions, updating the remaining questions Scrolling down for navigation bars

When the cursor points the info- symbol, the instructions appear in a box below.

Navigation bars: Takaisin = Back Jatka = Continue Jatka myöhemmin = Continue later/pause Kyllä = Yes En = No (Don’t know not visible)

Kyllä = yes Ei = no En osaa sanoa = Don’t know An error message: Please respond to this question or choose the “Don’t know” option. DK-option, version 1

Kyllä = yes Ei = no En osaa sanoa = Don’t know DK-option, version 2

2. Data collection process Response rate 30 %  potential for savings in data collection costs when the sample is large enough CAWI-data more ”distorted” ? CAPI non-response more evenly distriputed with regard to educational background, gender and age than CAWI-data. Speed: 40 % responses were received withing 3 days  after that switch to CAPI?

3. Mode-effects: WSTATOR (employment status)

No significant mode-effects, but…. media attention remains if unemployment rate 8,2 % instead of 7,4 % sample for pilot study was small CAWI respondents answer more often that they have been looking for job = mode effect or selection bias?

4. Mode-effects: HWUSUAL (actual working hours) ”One question” –format for HWACTUAL: “How many hours did you work in your main job during the week ?” ___ hours ___ minutes

”Grid” –format for HWACTUAL: “How many hours did you work in your main job during the week ?” Date Hours Minutes Monday ___ ___ Tuesday ___ ___ Wednesday ___ ___ Thursday ___ ___ Friday ___ ___ Saturday ___ ___ Sunday ___ ___ Total ___ ___

Average number of hours worked (CATI-data and two different CAWI-datas)

5. Qualitative feedback Too intimate information on employer Questions do not fit for people with multiple jobs without main job or with scholarship All kind of entrepreneurial activity is not covered Working time items do not fit for people in atypical employment Pensioners do not understand point of reference week

Conclusions and lessons learnt Response rate of 30 % and speed of data collection encourage to continue towards mixed-mode LFS. Visual design is important: balanced layout, intuitive navigation bars, informative error messages, a few questions per page, hidden instructions, hidden DK. CAWI data more ”distorted” In working time questions different format for CATI and CAWI may be needed to reduce mode-effects (”functional equivalency”). Differences in WSTATOR need to be further analysed.

Thank you! Contact: