Does mixing F2F and web lead to cost savings? Mari 18 July 2013 ESRA2013

1 Contents Background About the study Response Potential for cost savings Actual cost savings Cost model Conclusions

Background 1.

3 Context Increasing attention on mixed modes Potential for cost savings Potential for increasing response Unclear to what extent this potential realised in reality Especially if face-to-face involved

4 Survey I NatCen ran for ISER at University of Essex Main survey largest household panel in the world IPs test innovations and carry experiments IP5 – first test of F2F/Web in a large F2F panel survey

5 Survey II Household survey: Household questionnaire Individual questionnaire Adult self-completion Youth self-completion Split off households Sample: original W1, W4 refresh, W4 non-respondents 15 experiments Interview length 60 minutes per household Incentives: £5, £10, £20, £30

6 Design Experimental group F2F phase, web open Web only phase Control groupF2F only phaseNo web

Response 2.

8 Web take-up 23% Households completed fully online Another 13% completed partially

9 Household response by sample type, original responding sample Base: All issued households in original sample (responding) Source: Jäckle, Lynn, Burton (2013)

10 Household response by sample type, refreshment sample Base: All issued households in refreshment sample Source: Jäckle, Lynn, Burton (2013)

11 Web response by incentive amount Base: All households in mixed mode sample Source: Jäckle, Lynn, Burton (2013)

Cost 3.

13 Expected effect on cost Decrease No interviewing fees for web completes No trips required to web completes Increase More programming: web and F2F questionnaires Set up and manage s Man a helpline for the web group

14 One-off costs Research time to develop new procedures and support the mixed modes Develop sample management system Update response monitoring tools Field management cost to support interviewers

15 Effect on costs Total costsup by third Fielwork costsdown by tenth Programming costup three times Research costup by two-thirds

16 Fieldwork costs Main source of savings: Fees for productive cases Marginal savings on: Travel time and mileage – 3% Why travel less affected: Main source of cost – travel to and from PSUs Smaller points – more travel Web cases in F2F more difficult – require more calls

17 Cost of achieved interview by incentive amount Base: All issued households

18 Simulated costs per achieved sample size

Conclusion 4.

20 Conclusions Mixed modes take up is higher than expected No evidence that response in mixed modes is higher Risk of more refusals and partial households Costs increase in the first year Fieldwork costs do not decrease as much as might be expected Increase in web response with higher incentives is not enough to offset the incentive value Mixed modes design becomes cost effective at higher achieved sample sizes

If you want further information or would like to contact the author, Mari Toomse-Smith Senior Research Director T E. Visit us online, natcen.ac.uk Thank you