1. SSI Confidential © 2013 Survey Sampling International | 1 | With the global leader in sampling and data services Interest Bias Kathrin Reichert Pete Cape

2. SSI Confidential © 2013 Survey Sampling International | 2 | Good morning Mrs Respondent!

3. SSI Confidential © 2013 Survey Sampling International | 3 | Basis for survey choice Favoured sender — Multiple panel membership makes this a random factor Biggest reward — Response is not related reward Shortest interview — Response is not (much) related to interview length Biggest reward per minute of work — Humans are not Homo Economicus! Top of the list — Time of send out makes this a random factor Next to arrive….. — Time of survey taking makes this a random factor

4. SSI Confidential © 2013 Survey Sampling International | 4 | Basis for survey choice Or.. The one I’m most likely to qualify for Or The one I’m most interested it…. A systematic choice Therefore a potential bias

5. SSI Confidential © 2013 Survey Sampling International | 5 | You can learn a lot from the invite…. Scientific purpose Voluntary participation Anonymity Data only for research Agency contacts Technical hotline Subject?

6. SSI Confidential © 2013 Survey Sampling International | 6 | Hypotheses Mentions of the subject matter will tend to attract people who think they will qualify — Ownership/Usage of subject matter will be higher Interest in the subject will be higher when the subject matter is mentioned The more specific the subject matter the greater will be the interest in the subject Survey estimates will be biased as a result Non-related subjects may also be affected Where the subject matter is not mentioned; the results will be more accurate

7. SSI Confidential © 2013 Survey Sampling International | 7 | Treatments & Design HiddenGeneralSpecific “Different subjects”CarsSports Cars “Different subjects”PetsDogs “Different subjects”SportsFootball HiddenGeneralSpecific “Different subjects”CarsSports Cars “Different subjects”PetsDogs “Different subjects”SportsFootball Direct survey invitation sent to General and Specific samples Survey Router used to drive respondents to the “Hidden” survey — Sample of respondents ready and willing to take a survey now c.200 respondents per treatment

8. SSI Confidential © 2013 Survey Sampling International Pets & Dogs | 8 |

9. SSI Confidential © 2013 Survey Sampling International | 9 | Pet Ownership Do you currently own a Pet?

10. SSI Confidential © 2013 Survey Sampling International | 10 | Why raised Pet Ownership with the Router? Because the introduction reads “Survey about Pets” What if all drop-outs were non-pet owners?

11. SSI Confidential © 2013 Survey Sampling International | 11 | Pet Ownership Do you currently own a Pet?

12. SSI Confidential © 2013 Survey Sampling International | 12 | Dog Ownership Do you own a Dog?

13. SSI Confidential © 2013 Survey Sampling International | 13 | Summary For a subject like Pets (or Dogs) mentioning the subject brings more owners The owners are not especially different People are self-selecting out on the basis of ownership Non-owners are more likely to have some relevant knowledge from past ownership Or interest from future ownership

14. SSI Confidential © 2013 Survey Sampling International Cars & Sports Cars | 14 |

15. SSI Confidential © 2013 Survey Sampling International | 15 | Car Driving Do you drive a Car?

16. SSI Confidential © 2013 Survey Sampling International | 16 | Interested in the subject? How interested are you in Cars? Top 2 Box – Extremely, Very

17. SSI Confidential © 2013 Survey Sampling International | 17 | Bias from subject matter High incidence targets are more difficult to bias Some people (Sports Car drivers?) like the opportunity to talk What sort of people are attracted by specific subject matters?

18. SSI Confidential © 2013 Survey Sampling International | 18 | How intense is the interest? How interested are you in Sports Cars? Top 2 Box – Extremely, Very

19. SSI Confidential © 2013 Survey Sampling International | 19 | Attitudes to Sports Cars

20. SSI Confidential © 2013 Survey Sampling International | 20 | Summary Some topics maybe polarising and attract devotees The samples may look representative — Or not far off But attitudes and behaviours may be wildly different Because of self-selection in of people who have an active interest in the subject

21. SSI Confidential © 2013 Survey Sampling International Sports & Football | 21 |

22. SSI Confidential © 2013 Survey Sampling International | 22 | The same interest bias

23. SSI Confidential © 2013 Survey Sampling International | 23 | Attracting the same “ultras” How fanatical are you about football? Top 2 Box Extremely, Very

24. SSI Confidential © 2013 Survey Sampling International | 24 | Displaying different behaviours Do you have a team you support?

25. SSI Confidential © 2013 Survey Sampling International | 25 | Attitudes are sometimes unaffected

26. SSI Confidential © 2013 Survey Sampling International | 26 | But not all the time…..

27. SSI Confidential © 2013 Survey Sampling International | 27 | Weighting to correct “interest”

28. SSI Confidential © 2013 Survey Sampling International | 28 | Market Estimations Should market research be able to estimate the numbers of members of the Deutscher Fußball-Bund? — Football players and club members… — Would an online panel do any better? MR - Expect underestimate (better things to do ) Online – further underestimate (correlation with working class?)

29. SSI Confidential © 2013 Survey Sampling International | 29 | Market Estimations

30. SSI Confidential © 2013 Survey Sampling International Summary | 30 |

31. SSI Confidential © 2013 Survey Sampling International | 31 | Summary There is a Qualification bias — Mentioning the subject can bring raised levels of ownership There is an Interest bias — Some subjects attract people who have strong views The two can work together Qualification bias may help you be efficient if you want users only But the Interest bias may ruin your sample Data can be affected In ways you do not expect

32. SSI Confidential © 2013 Survey Sampling International | 32 | Take aways You are not required to mention the subject Think about how you would encourage potential non-users to take the survey Router samples that avoid mentioning subject may produce the best results

33. SSI Confidential © 2013 Survey Sampling International | 33 | surveysampling.com info@surveysampling.com kathrin.reichert@surveysampling.com pete.cape@surveysampling.com