1. Dr. Vera Toepoel v.toepoel@uvt.nl Leisure Sciences GOR, March 2011
Response Quantity, Quality, and Costs of Building an Online Panel via Social Contacts
Dr. Vera Toepoel
Leisure Sciences
GOR, March 2011

2. Research Problem
More than 50% of all survey data in the Netherlands are collected via Internet
Problem 1: majority of the Dutch people are not willing to join a web panel
Problem 2: the minority (20%) of the web panellists fills out the majority (80%) of the questionnaires
(NOPVO, 2006)
Are results representative of the Dutch population?!
NO! E.g. Internet panels contain too many (heavy) Internet users and SP-voters and too few CDA-voters, church-goers, and ethnic minorities
21 juli 2018

3. Traditional Approach to Building a Web Panel
Volunteer panel
Recruitment via , banners, self-selection
Pro: cheap, fast, many panel members
Con: no sampling frame available
2. Probability-based panel
Recruitment via probability sampling (address-based sampling)
Pro: representativeness
Con: Unaffordable for most people
Still problem of reluctance of being a panel member
21 juli 2018

4. Unconventional Approach to Building a Web Panel
Snowball-effect via social networks
Leisure Panel
Sample base: administrative records of Breda University of Applied Sciences(7000 students with a national spread)
Each student is asked to recruit panel members (each year new students to recruit new panel members)
Experiment with instruction to recruit new panel members (those not in other panels, ethnic minorities etc.)
21 juli 2018

5. Recruiting via social networks
Student
Father
Social Network Sites
Neighbor
Friend
21 juli 2018

6. Mail chain We
Student
relative
Friend/
21 juli 2018

7. Mail chain We
Students
7000
Panel members
???
21 juli 2018

8. Literature
Majority of Dutch people is not willing to be in a web panel (NOPVO, 2006)
Announcement boosts response rates (Dillman, 2007)
augmentation boosts response rates (Dillman, 2010)
Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010)
Cash is more powerful than gifts (Goritz, 2010)
Response depends on relevance
21 juli 2018

9. Experiment/design 1
Majority of Dutch people is not willing to be in a web panel
Request to fill out single questionnaire->request to be a panel member
Request to be a panel member ->request to fill out single questionnaire
Announcement boosts response rates (Dillman, 2007)
Announcement letter: mail/ /no
augmentation boosts response rates (Dillman, 2010)
Reminder
21 juli 2018

10. Experiment/design 2
Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010)
Unconditional incentive versus no incentive
Cash is more powerful than gifts (Goritz, 2010)
5 euro versus lottery ticket
Response depends on relevance
Response to “Leisure Panel” dependent on Academy for Leisure, Tourism, Media
21 juli 2018

11. Recruitment Week 48 2011: pre-announcement
Week : recruitment mail
Week : marketing (newsletter, tv)
Week 51: 2011: reminder
21 juli 2018

12. Marketing
21 juli 2018

13. Design: 3 different academies
Frequency
Tourism
2045
Leisure
1453
Media
1009
Total students
4507
21 juli 2018

14. Design: announcement Across 3 academies Frequency Letter
1000 (600 euro)
1000
No announcement
2507
Total students
4507
Incentives in letter
Frequency
5 euro
333 (1700 euro)
Lottery ticket (costs 3 euro)
333 (1000 euro)
No incentive
334
Total students
4507
Across 3 academies
21 juli 2018

15. Research questions
When should we time the request to be a panel member?
Are the costs associated with sending an announcement letter worth it?
Does a reminder mail increase response rates?
Does an unconditional incentive boosts response rates and is it worth the costs?
Is cash more powerful than a lottery ticket?
Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)?
21 juli 2018

16. Response??? We
Students
4507
Panel members
???
21 juli 2018

17. Response??? We
Students
4507
Panel members
120
21 juli 2018

18. Response??? 21 juli 2018 We Students 4507 Panel members (45 students)
120
21 juli 2018

19. When should we time the request to be a panel member?no
Panel member yes
1st questionnaire
23%
77%
1st panel request 9%
91% N
203
Chi Square
7.79 p
.005
21 juli 2018

20. Are the costs associated with sending an announcement letter worth it?
Costs 600 euro
Students
Recruited panel members %
Letter
1000 (334)
56 (9)
6% (3%)
Mail
1000 32 3%
No announcement
2498 41 2% N
4498
Chi Square
40.68 p
.000
Letter without incentives between parentheses
Costs per panel member: 11 euro
Announcement does not significantly boost response rates and is not worth the costs
21 juli 2018

21. Does a reminder mail increase response rates?
Recruitment mail: 93 panelmembers
Reminder: 27 panelmembers
Increase=23%
21 juli 2018

22. 2700 euro (333*5 euro+333*lottery ticket)
Does an unconditional incentive boosts response rates and is it worth the costs?
2700 euro (333*5 euro+333*lottery ticket)
Students
Recruited panel member %
No Incentive
334 9 3%
Incentive
666 46 7% N
1000
Chi Square
7.59 p
.01
Note: incentive is provided with announcement letter
Unconditional incentive boosts response rate significantly
Costs per panel member: 59 euro
21 juli 2018

23. Is cash more powerful than a lottery ticket?
1700 euro (333*5 euro)
1000 euro (333*lottery ticket)
Students
Recruited panel member %
5 euro
333 25 8%
Lottery ticket 22 7%
No incentive
334 9 3% N
1000
Chi Square
8.26 p
.02
Costs per panel member: 68 euro/cash; 45 euro/lottery
Difference between cash and lottery not significant
21 juli 2018

24. The higher the relevance, the more panel members are recruited
Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)?
Academies
Students
Recruited panel member %
Leisure
1453 57 4%
Tourism
2045 55 3%
Media
1000 15 2% N
4498
Chi Square
12.92 p
.00
Academy for Leisure produces significantly more panel members than Academy for Media (difference with Academy for Tourism not significant)
The higher the relevance, the more panel members are recruited
21 juli 2018

25. Conclusion
Response quantity is poor. Students are not really motivated to recruit panel members
Quality is poor (mostly students)
Students do not understand request/Students do not know the sender of the request
First panel request, then questionnaire yields most panel members (response actual waves?)
Announcement letter does not work
Reminder works
Incentives work, but are not really worth the costs (difference between cash and lottery ticket not significant)
Relevance matters
21 juli 2018

26. Future actions Lecturers are ask to discuss Leisure Panel and request
Mail is send via directors of academies
Different types of incentives
50*2 tickets for event
2*250 euro in cash
Does this work???
21 juli 2018