1. Timing Isn’t Everything, But Money Talks How to encourage a face-to-face household panel to go online? University of Essex, July 2013 Gerry Nicolaas Carl Cullinane

2. 1 Contents Background Design of Experiment Results Summary of Results and Conclusions

3. Background 1.

4. 3 Background Switch to mixed mode data collection at wave 7 of Understanding Society Potential for cost savings Potential for reducing attrition

5. 4 Context Longitudinal household panel Face-to-face interviewing of all household members at waves 1 to 6 Greatest potential for reducing data collection costs when an interviewer does not have to visit the household Previous experiment mixing telephone & face-to-face: Costs can be reduced BUT response rates suffer

6. 5 Innovation Panel – wave 5 Vehicle for methodological development & testing About 2,500 individuals in 1,500 households Main objective of IP5 = Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data quality Sequential mixed mode design starting with web

7. 66 IP5 Design Experimental group F2F phase, web open Web only phase Control groupF2F only phaseNo web

8. 7 Research question Is it possible to boost the proportion of whole households completing web questionnaires by Timing the arrival of the invitation to go online? Offering a web bonus?

9. Design of experiment within mixed-mode sample 2.

10. 9 Design of experiment within mixed-mode sample (1) Timing of invitation to go online Random allocation of households to: Letter (+ email) to arrive on Friday Letter (+ email) to arrive on Monday Reminder letters (+ emails) sent 2 and 4 days later (2) Web bonus Random allocation of households to: No web bonus £5 per household member conditional on all completing online questionnaire

11. 10 Design of experiment within mixed-mode sample 270 households 265 households 266 households 276 households No web bonus Cond. £5 per hhld member Invite arrival on Friday Invite arrival on Monday

12. Results 3.

13. 12 Experimental effects on Web Response Base: IP5 WEB Sample (n=1077)

14. 13 Experimental effects on Web Response Base: IP5 WEB Sample (n=1077)

15. 14 Effect of Bonus by Sample Type Base: IP5 WEB Sample (n=1077)

16. 15 Effect of Bonus by Upfront Incentives Base: IP5 WEB Sample (n=1077)

17. 16 Unconditional Incentives Original Sample Refreshment Sample £5 £10 £30£20

18. 17 Effect of Bonus by Children in Household Base: IP5 WEB Sample (n=1077)

19. 18 Effect of Bonus by Advance Mailing Base: IP5 WEB Sample (n=1077)

20. 19 Effect of timing by Advance Mailing Base: IP5 WEB Sample (n=1077)

21. 20 Experimental effects on Final Response Base: IP5 WEB Sample (n=1077) 75.6 72.8

22. 21 Experimental effects on Final Response Base: IP5 WEB Sample (n=1077) 75.6 72.8 7474.4

23. 22 Estimation of cost savings Web bonus of £5 compared to no web bonus: Small reduction in travel and mileage costs 15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5%

24. 23 Summary of results and conclusions 4.

25. 24 Summary of results Friday mailing had a small but diluted effect Web bonus increased full household web response Effect of bonus varied by a number of factors, e.g. Diminishing returns with larger upfront incentives Greater effect for households with children Greater effect when household contacted by email Web bonus did not increase final response rate But potential for cost savings

26. 25 Limitations Small sample sizes Confounding of sample type and value of unconditional incentives sent with advance letter Estimation of costs

27. 26 Points for discussion Timing to be looked at further? Potential for greater cost savings through targeting Large upfront incentives vs conditional web bonus- costs

28. If you want further information or would like to contact the author, Carl Cullinane T. 020 7549 7158 E. carl.cullinane@natcen.ac.uk Visit us online, natcen.ac.uk Thank you