1. 1 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Response Rate in Surveys Key resource: Dillman, D.A., Smyth, J.D., Christian, L.M., 2009. Internet, Mail and Mixed-Mode survey: The tailored design method, 3 rd ed. John Wiley & Sons. Chapter 7, pgs 234-299.

2. 2 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Check your understanding of response rate 1.Response rate means that everyone answered the survey who was supposed to AgreeDisagree 2. There is one standard response rate AgreeDisagree 3. A higher response helps us be more sure the findings represent everyone in the study population AgreeDisagree 4. People will respond or not; there is little we can do about it. AgreeDisagree 5. If your response rate is low, explain your results relative to those who responded AgreeDisagree Check answers on next slide Select agree or disagree for each item:

3. 3 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Answers – find explanations in the slides that follow 1. Response rate means that everyone answered the survey who was supposed to AgreeDisagree 2. There is one standard response rate AgreeDisagree 3. A higher response helps us be more sure the findings represent everyone in the study population AgreeDisagree 4. People will respond or not; there is little we can do about it. AgreeDisagree 5. If your response rate is low, explain your results relative to those who responded AgreeDisagree

4. 4 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Response rate is… The proportion of people who return the survey questionnaire. It is calculated by dividing the number of returned surveys by the total number of surveys distributed. Example: If you distribute 250 questionnaires and you get 85 questionnaires back, your response rate is 34%. # that answered # you contacted

5. 5 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Low response to our surveys We often send out surveys and find that few are returned. Response rates of 30% and lower are common in Extension. Often, the number of returned surveys is too small to aggregate in a meaningful way or make any comparisons. Best practice says that a response rate under 70% should be a warning.

6. 6 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Why is response rate important? Its the only way to know if your survey results are representative. High response rate promotes confidence in results. Low response rate increases the likelihood of biased results.

7. 7 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation The higher, the better. While it is not actually the % that matters but WHO responds, a higher response rate means that you can be more sure that the answers reflect the population. So, we want to remove barriers and motivate as many people as possible to complete and return the questionnaire. How can we do that? There is no standard response rate

8. 8 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Most important things that influence response rate: Salience of the topic – interest in the topic of the survey Personalized request and communications Multiple follow-up contacts Sponsor of the survey is respected, trusted Questionnaire is concise and easy to complete

9. 9 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation 10 Ways to increase response rate 1.Generate positive publicity for your survey. 2.Appeal to peoples helping tendencies – ask them to help by providing their input. 3.Make the survey topic salient – important Ensure that respondents see the value of the survey and their response. Point out their personal connection to the topic 4.Tailor, personalize communications 5.Make the questionnaire attractive and easy to complete AND easy to return

10. 10 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation 10 ways to increase response rate, cont. 6.Provide incentives (token of appreciation) 7.Show positive regard; Say thank you 8.For mail survey, provide 1st class postage/return postage 9.Make (multiple) follow-up contacts – by mail, email, telephone, in person… 10.Use a combination of survey modes – telephone plus mail; internet plus mail.

11. 11 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Personalize communications Purpose: to motivate people to answer; appeal to their helping tendencies Use name, if appropriate and possible Personalize the importance of the survey to the respondent For mail surveys, use of letterhead printed on quality paper Use recognizable logos and graphics Personalize salutations as appropriate

12. 12 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Incentives In Extension, we may be limited to what we can offer as incentives, but research shows that incentives do increase response rates. Prepaid incentives work better than promising something if the questionnaire is returned. Incentives can be cash ($1 as smallest amount) or nonmonetary, e.g., a tea bag inserted with a mail questionnaire; a gift card; state park or movie pass; etc. Or, a lottery, raffle, or prize drawing (e.g., offer to put names of all completed and return questionnaires into a raffle for a $50 gift certificate to the areas best restaurant); or, offer to donate $1 for every questionnaire submitted to the local food pantry.

13. 13 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Multiple contacts Consider 3-5 contacts –Brief pre-notification –Questionnaire mailing with cover letter –Thank you follow-up (post card) –Reminder with replacement questionnaire –Final contact More important than number of contacts is that each has a different look and appeal Strategically time all contacts Use different modes of contact –different ways of contacting people (email, mail, individual) –different delivery of the questionnaire (url, email attachment, postal mail, in-person)

14. 14 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation

15. 15 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Response rate may be affected by how the survey is distributed. In Zoomerang, there are two options: 1.Use email addresses. you need individual names with current email addresses. Send invitation plus other communications - reminders, track respondents, and receive detailed deployment information. –This is the only option that allows you to calculate the response rate 2.Post the url in a public place – when you dont have individual addresses. E.g., in newsletter, on web site, bulletin board; or send to an email list (not individual addresses). –You may not know who sees/received the url – you do not know the total number of people who received the questionnaire.

16. 16 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Address low response Determine how people who responded are different from those who didnt respond. Describe your results in terms of who did respond. Dont imply that the results apply to anyone other than those who responded.

17. 17 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation When response rate is low… Use language that is suggestive rather than decisive. Examples: The data suggests vs. The data prove; It appears vs. It shows Dont generalize findings to the entire group. Indicate who responded, i.e., who the data represents.

18. 18 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation When response rate is low… Use a summary statement such as: The response rate was only 45% so care should be taken when reviewing these results. The results should be applied to those who responded, not all participants.

19. 19 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation How would you rate your understanding: Now after reviewing these slides to Before you started? NOW Very poor Poor Fair Good Very good BEFORE Very poor Poor Fair Good Very good

20. 20 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation