1. Presented at the 2006 American Educational Research Association Annual Meeting April 7 – 11 San Francisco, CA Techniques for Generating Higher Response Rates Presented at the 2006 American Educational Research Association Annual Meeting April 7 – 11 San Francisco, CA Kevin Kenward, PhD Director of Research National Council of State Boards of Nursing Lynn C. Webb, Ed.D. Testing Consultant

3. Increasingly, [people] are refusing to participate in consumer surveys. A truly representative survey may be unattainable… The New York Times June 2004

4. Status of Survey Nonresponse Falling cooperation rates in the developed world (de Leeuw and de Heer, 2002) Greater decline in one-time surveys vs. longitudinal surveys (AAPOR, 2003) Greater decline in unit nonresponse than item nonresponse

6. An electronic survey was conducted to compare the roles of two types of Advanced Practice Registered Nurses Nurse Practitioners (NP) Clinical Nurse Specialists (CNS) Job Analysis

8. The professional sophistication of Advanced Practice RNs led us to believe that an electronic survey would be appropriate, and the electronic format offered advantages over paper.

9. The history of nurse response rates 40% led us to believe we would have strong participation.

10. The response rate was disappointingly low. 13% In desperation, nurses were randomly selected from the mailing list and asked why they did not participate in the study. TOO BUSY was the recurring theme in the explanations offered.

11. How do We Get People to Complete Our Surveys and Provide us with Good Information ??

12. Techniques for Generating Higher Response Rates l Preliminary or Advance Notification l Follow-ups l Appeals l Postage and Mailing Classes l Personalization l Incentives

13. Decreasing Response Rates QUESTIONNAIRE LENGTH - MAIL 4 PAGES OR LESS No Effect MORE THAN 4 PAGES - 8%

14. MONETARY INCENTIVES Prepaid - Creates Trust and Obligation Amount - Symbolic Token of Appreciation Understanding the Techniques

15. Phone Call Reminder Mail Survey Long Form Version 1 Mail Survey Long Form Version 2 Mail Survey Short Form N=1,500 N=1,000

16. Mail Survey Long Form Version 1 + Photo on Cover Letter Mail Survey Long Form Version 1 + Photo on Cover Letter

17. Mail Survey + $1 Incentive Mail Survey + $1 Incentive Mail Survey + $2 Incentive Mail Survey + $2 Incentive Mail Survey + $5 Incentive Mail Survey + $5 Incentive Monetary Incentives N=250

18. Mail Survey + Starbucks $3 Gift Certificate Mail Survey + $250 Drawing N=250 N=1,000

19. 2% 26% Results

20. Mail Survey Long Form Version 1 Mail Survey Long Form Version 2 Mail Survey Short Form 24% 26% 30% Results

21. Increasing Response Rates Monetary Incentives RESPONSE AMOUNT RATE $1.0043% $2.0042% $5.0054% SOURCE: Yu and Cooper, JMR.

22. Increasing Response Rates NON-MONETARY INCENTIVES Starbucks $3 Gift Certificate42% $250 Drawing 28%

23. Effective Nonresponse Rate Reduction Methods Incentives reduce nonresponse rates higher effects with increasing incentives prepaid incentives more powerful than promised cash more powerful than in-kind incentives can reduce total survey costs

24. Sample Size Mailing cost Response Rate Cost per Complete 10,000 $14,000 $5 Incentive No Incentive $64,000 14%54% $10.00$11.85

25. 2 nd Mailing 10,000 – 1,400 = 8,660 5% Response Rate Cost = $12,040

26. Mailing cost Response Rate Cost per Complete 18,60010,000 $26,040 $5 Incentive No Incentive $64,000 18%54% $14.23$11.85 Total Mailed To

27. Returned Incentives $1 $2 $5 Starbucks # People 13 5 3 2