1. Building Web Collection Capability for ONS Social Surveys Fiona Dawe and Laura Wilson

2. Outline Background Study methodology Results Way forward

3. Background 2008 online pilot 2010 Lound & Betts review of mixed modes Survey environment: Falling response rates Increasing costs of data collection Respondent burden Labour Force Survey: Shortest & most straightforward ONS social survey Afford the largest cost savings I

4. Issues to address Household data – can this be collected? Questionnaire length Complex questions and routing Will LFS specific questions work in this mode?

5. 2010/11 Online pilot (a) Opinions Online Survey Mode effect test Comparison of online self completion vs interviewer coding (b) LFS Online Survey (wave 6) Test of LFS questionnaire online Comparison of LFS responses with online responses SIC and SOC (c) Cognitive testing Real time usability testing Investigation of response issues

6. 2010/11 Online pilot : Response (a) November Opinions Online Survey (complete) 8% (b) December Opinions Online Survey (complete) 17% (c) LFS Online Survey (complete)  Total sample 25% - Non-email sample 13% - Email sample 40%

7. Questionnaire content - LFS Questions on… Accommodation details Nationality Country of birth Ethnicity Religion Sexual identity Government training schemes Main job Home workers Employment pattern Second job Looking for work Benefit entitlement Employment 12 months ago Earnings Included: Short household section Basic socio-demographic information about each household member Relationships to each other Collected detailed information on one person only per household Key Labour Market Division questions ‘Unimode’ question design

8. Questionnaire content - OPN Questions on… LFS questions: -Country of birth -Nationality -Sexual identity -Health problems -Employment (including workplace and second job) -Benefits -SIC and SOC coding frames -Workplace coding frames -Nationality coding frames -Country of birth coding frames Sponsored OPN client Modules (with permission) Included: Short household section Basic socio-demographic information about each household member Relationships to each other Interviewed one person in the household who was aged 16 or over ‘Unimode’ question design

9. Questionnaire length LFS Av. Completion - 18 minutes (estimate was 20) Av. Partial - 9 minutes Only 25 % of partial respondents spent more than 14 minutes on the survey. OPN Nov: Av. Completion – 21 minutes (estimate was 20) Av. Partial – 4 minutes Dec Av. Completion – 29 minutes (estimate was 20; more modules) Av. Partial – 7 minutes Abandoned the questionnaire quite soon, within only a few minutes.

10. Coding frames - OPN OPN – face-to-face: – self-coded using laptop and interviewer coded also OPN – FTF self-complete frame: – 10% refused to complete frame – 53% reported SIC issues and 60% reported issues SOC issues – 18% SIC and 21% SOC differed at level 1 when compared

11. LFS – online: – free text description for industry (SIC) and occupation (SOC) – self-coded for workplace, nationality and country of birth LFS – coded at office: – Automated and SOC coders in Titchfield – compared these with wave 5 LFS interviewer codes – found discrepancies of 50% for SIC and 60% for SOC Conclusion: OPN and LFS were inconclusive in recommending whether to: – include frames for respondents to use themselves – or, to collect the free text and code in-house coding Coding frames - LFS

12. Mode Effects Opinions Survey (OPN) and Opinions Online Survey (OPNO) Online and face-to-face compared 2-tailed t-test used with 1% significance level Results: - Online sample comprised of:  sig. more men  sig. more people aged between 45 and 64  sig. less 16 to 24 year olds Due to significance, we calibrated the data to age and sex population totals Results: - Some significant differences observed on ‘non-sensitive’ questions - Average weekly income higher in online respondents

13. Way forward I Identify software and hardware solution Further testing of LFS suitability for online data collection Learn more from the 2011 Census Target: December 2013 to recommend extending the LFS mixed mode data collection to include online questionnaires

14. Thank you Contact: Fiona.dawe@ons.gov.uk Laura.wilson@ons.gov.uk