1. Using the web to survey the UK general population
@GenPopWeb
Using the web to survey the UK general population
ESRA conference, Ljubljana, July 2013

2. What 1.

3. @GenPopWeb
A network to explore how the web can be used to survey the UK general population
Purpose is to exchange knowledge, generate new thinking and work towards resolving the challenge
One year grant funded by the National Centre for Research Methods 2 2

4. @GenPopWeb Organising team: Gerry Nicolaas, NatCen Social Research
Peter Lynn, ISER
Lisa Calderwood, CLS
Caroline Roberts, UNIL
UK Core Group:
Survey organisations
Universities running grant funded social surveys
Academic grant funding bodies (ESRC, NCRM)
Purpose of UK Core Group is to steer the direction of the network 3 3

5. Focus of the network General population Large scale social surveys
Complex measures and high quality data
New surveys as well as existing surveys
Cross-sectional surveys as well as longitudinal surveys
Web as stand-alone mode or in mixed mode design 4 4

6. Opening Conference (Feb)
Activities in 2013
Map out what we know, debate key issues,
identify questions to be answered
Opening Conference (Feb)
Catalyse problem solving
& new thinking
Second conference
(Jun)
3 full sessions!
ESRA conference
(Jul)
Summary
Research agenda
Recommendations for infrastructure
Final report
(Dec) 5 5

7. Main headlines from opening conference2.

8. Opening conference Presentations on 3 themes
Coverage and sampling:
Peter Lynn, Mario Callegaro, Doug Rivers, Louise Morris
Participation and engagement:
Caroline Roberts, Annette Scherpenzeel, Annemieke Luiten, Jon Puleston
Measurement challenges:
Lisa Calderwood, Tina Glasner, Peter Lugtig, Mick Couper
No detail about the presentations – refer to website (following slide)
Workshop with speakers and UK Core group

9. @GenPopWeb website

10. Sampling and coverage 1st mode of contact cannot be web
Dependency on Postcode Address File as a sampling frame
Digital divide
A main barrier to using web is selection bias
Multiple modes of data collection needed
Can we enhance address sampling frames with commercial data?
Dependency on PAF:
No pop registers
RDD not feasible (low rr)
Can we enhance PAF using commercial data (refer to US work in this area)
1st mode of contact cannot be web:
Postal contact introduces selection bias (refer to ESS web experiment presentation) – perhaps need to move away from using respondent selection
Digital divide:
Offline households and low web response rates suggest that web cannot be used as a standalone mode but may be feasible in mixed mode design

11. Participation and engagement
A main barrier to using web is low level of participation
We need to know more about the participant experience in web
Mention experiments that use postal contact and incentives to encourage online participation (refer to other ESRA presentations) – appears to be a ceiling of 20%-25% participating online
Sequential design strategy with web first and then f2f – final response rate lower than f2f only designs
Participant engagement = not just starting the web but putting sufficient effort into providing good quality data and completing it without an interviewer who would otherwise be able to motivate and to help with difficulties

12. Measurement issues Are we looking to replicate or improve?
Measurement is an issue but not a barrier
Can we correct for measurement differences using experiments, parallel runs, etc?
Mobile web

13. Two distinct issues highlighted:
How to use web in cross-sectional surveys of UK general population?
Does the UK need a probability-based web panel?

14. Main conclusions from June workshop3.

15. 2-day workshop in June Focus on two key areas: Format of workshop:
Web data collection for x-sectional surveys of UK general pop
A probability-based web panel for the UK
Format of workshop:
Presentations
Panel discussions
Open workshop discussions

16. Cross-sectional surveys of UK general population
Web unlikely to be used as a stand- alone mode but mixed mode design worth exploring further
Rethink how we design & conduct surveys
Getting respondents to go online when contacting them in a different mode
Respondent selection or whole households
Measurement effects & selection effects
Cost savings are marginal
Clients’ tolerance for error versus cost

17. A probability-based web panel for the UK
Random probability sample:
Fresh sample of addresses
Piggy-back on existing RPS surveys
Panel recruitment:
Face-to-face
Incentives
Coverage of offline households:
Provide hardware and broadband
Other mode

18. A probability-based web panel for the UK
Is this the web solution for cross-sectional & longitudinal surveys?
No commercial incentive to invest in prob-based web panel
Is there an unmet demand/need for this?
How can it be funded to ensure long-term sustainability?
Does web mean mobile?

19. Next steps 4.

20. Main issues for the UK (and elsewhere?)
Dependency on address file for sampling – enhancement?
Using web in cross-sectional surveys - could we & should we?
Are probability-based mobile web panels the solution for both cross-sectional and longitudinal data collection in the UK?
Need to rethink how we design surveys
Clients’ tolerance for error versus cost 19 19

21. Next steps Online outputs Now UK Core Group September WebDataNet
Twitter feed
Synthesis papers Presentations
Event reports
Now
UK Core Group
Discuss research agenda and recommendations
September
WebDataNet
GenPopWeb will become a WebDataNet Task Force
September
Final report
Research agenda & recommendations to facilitate transition to web data collection
December

22. Thank you For further info Email Gerry.Nicolaas@natcen.ac.uk
Go to the website