1. Use of Incentives in Surveys Supported by Federal Grants
Sandra H. Berry, Jennifer S. Pevar, and
Megan Zander-Cotugno
CDC PRAMS National Meeting
December 9, 2008

2. Overview
What we know about incentives from the survey methods literature
IRB issues in providing incentives
Results of a survey of NIH grant recipients who planned to do surveys
Conclusions

3. Survey Literature Findings on Use of Financial Incentives
Incentives improve response across modes
Effects appear more or less linear, size matters
Prepaid incentives more effective than promised
Lotteries less effective than payments
Lower income respondents more responsive than higher income respondents
Incentives can help increase representativeness of the sample

4. How Financial Incentives Fit Into Survey Response
Respondents consider “costs” of survey participation
Time, inconvenience
Loss of privacy, concerns about other possible consequences
How it will feel to be interviewed or complete the survey
How others might view them for taking part
Whether the purpose of the survey is a potential loss for them

5. Non-Financial Reasons for Survey Participation
Possible “benefits” other than payment
Belief that the survey will be beneficial to them or to others, desire to help
Interest in the survey, desire to talk about the topic
Helping or being associated with a survey sponsor
Belief that participating in the survey will be pleasant
Desire to help and/or talk with interviewer
Prestige from being a study participant

6. Institutional Review Boards
All federally supported surveys should be reviewed by an IRB
IRBs operate under guidelines from DHHS Office for Human Research Protections
Requirements for approval:
Risks are minimized
Subject selection is equitable, vulnerable populations protected
Informed consent
Protection for subjects’ privacy and data confidentiality
Each IRB can and does interpret guidelines in light of local circumstances

7. IRBs Wrestle With Incentives
Do they “coerce” people who would not otherwise freely agree to participate in research to do so?
How should the level of incentives be determined?
As “wages” for time and effort of participation?
Based on the value researchers place on participation?
Equitably for all participants or recognizing different values participants may place on their effort and time?

8. Special Problems with Incentives for IRBs
Lotteries
Provide unequal rewards across participants
Undermine informed decision since chances of winning are overvalued
Disadvantaged populations
Especially vulnerable to coercion
May be induced to lie or conceal information in order to participate
Higher payments for people who refuse to participate

9. Survey Literature on Incentives Does Not Address Ethical Issues
Focus is on practical concern with effectiveness in terms of response rates and/or MSE
“AAPOR Best Practices” suggests considering use of incentives to stimulate cooperation
Ability to enhance participation of reluctant respondents and under-represented groups is a plus
Not part of AAPOR disclosure requirements
Not addressed in AAPOR advice on dealing with IRBs

10. Web Survey of NIH Grantees Doing Surveys
Sampling frame: NIH CRISP database of grants
Lists about 250 in each year
482 unique grants in
Reviewed abstracts:
Included those with mention of survey data collection
Excluded methodological studies
145 grants selected for data collection

11. Incentives Web Survey
Sent invitation to participate and three follow ups to Principal Investigator listed in CRISP
Provided link to COPAFS and Incentives Conference web pages
Promised copy of the paper as “incentive”
Allowed PI or designated proxy to fill out the survey
Asked PI to select one survey, the “most important” in terms of research goals
Received 92 responses - 63%

12. Most Surveys Used Incentives
Reason for no survey did not include anything to do with incentives.

13. Why Didn’t Surveys Use Incentives?
Shorter surveys
40% under 15 minutes, nearly all under an hour
Mean time to complete 27 minutes vs. 45 minutes
Reasons:
60% expected good response without incentives
60% did not have budget for incentives
27% survey team did not want to pay incentives
None reported IRB was a factor

14. How Did Surveys Use Incentives?
Mix of expectations and timing
44% for completed survey
32% for partial complete
13% for considering participation
11% prepaid incentive
About half the surveys included tasks other than the survey
Half of those provided a separate payment

15. Kinds and Amounts of Incentives
Kinds of Incentives
Cash - 31%
Gift cards or certificates - 27%
Checks - 25%
Amounts of monetary incentives
$10 or less - 34%
$ %
$50 or more - 14%
Other kinds of incentives
Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc.

16. Why Are Incentives Used?
73% said to increase response rates was main reason
Rated as main or very important reason:
Reduce non-response bias - 71%
Reward participants for research participation - 56%
Not rated as important reasons:
Reduce data collection time or follow up costs
Help interviewers feel more comfortable
IRB wanted incentives

17. What Kinds of Surveys Were These?
Potentially Sensitive Topics
Health status or health conditions - 74%
Personal financial information - 44%
Sexual behavior - 31%
Drug use or drug use history - 25%
Immigration status - 14%
Special requests
Linkage to other databases (e.g. Medicare) 7%

18. Modes Used With Incentives

19. How Were Amounts Determined?
Based on open-ended comments:
Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges
Time and contribution of personal information
Going rate for surveys of this kind of subject
Accounting or safety issues for interviewers or subjects or budget issues as constraints
Experiments to determine effective amounts

20. What Was the Role of IRBs?
For paying incentives or not:
Only one participant cited IRB preferences as a main reason for paying incentives
88% said IRB preferences were not important in making this decision
For kind or amount of incentives:
23% said IRBs raised questions or placed limitations
IRBs capped amount or required same incentive for all
Limited how and when incentives could be mentioned

21. Conclusions
At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50
75% of telephone surveys paid incentives, range was $15-20
Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests
Most paid incentives to increase response rates
IRBs were a factor, but not a major limitation