Use of Incentives in Surveys Supported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December 9, 2008

Overview What we know about incentives from the survey methods literature IRB issues in providing incentives Results of a survey of NIH grant recipients who planned to do surveys Conclusions

Survey Literature Findings on Use of Financial Incentives Incentives improve response across modes Effects appear more or less linear, size matters Prepaid incentives more effective than promised Lotteries less effective than payments Lower income respondents more responsive than higher income respondents Incentives can help increase representativeness of the sample

How Financial Incentives Fit Into Survey Response Respondents consider “costs” of survey participation Time, inconvenience Loss of privacy, concerns about other possible consequences How it will feel to be interviewed or complete the survey How others might view them for taking part Whether the purpose of the survey is a potential loss for them

Non-Financial Reasons for Survey Participation Possible “benefits” other than payment Belief that the survey will be beneficial to them or to others, desire to help Interest in the survey, desire to talk about the topic Helping or being associated with a survey sponsor Belief that participating in the survey will be pleasant Desire to help and/or talk with interviewer Prestige from being a study participant

Institutional Review Boards All federally supported surveys should be reviewed by an IRB IRBs operate under guidelines from DHHS Office for Human Research Protections Requirements for approval: Risks are minimized Subject selection is equitable, vulnerable populations protected Informed consent Protection for subjects’ privacy and data confidentiality Each IRB can and does interpret guidelines in light of local circumstances

IRBs Wrestle With Incentives Do they “coerce” people who would not otherwise freely agree to participate in research to do so? How should the level of incentives be determined? As “wages” for time and effort of participation? Based on the value researchers place on participation? Equitably for all participants or recognizing different values participants may place on their effort and time?

Special Problems with Incentives for IRBs Lotteries Provide unequal rewards across participants Undermine informed decision since chances of winning are overvalued Disadvantaged populations Especially vulnerable to coercion May be induced to lie or conceal information in order to participate Higher payments for people who refuse to participate

Survey Literature on Incentives Does Not Address Ethical Issues Focus is on practical concern with effectiveness in terms of response rates and/or MSE “AAPOR Best Practices” suggests considering use of incentives to stimulate cooperation Ability to enhance participation of reluctant respondents and under-represented groups is a plus Not part of AAPOR disclosure requirements Not addressed in AAPOR advice on dealing with IRBs

Web Survey of NIH Grantees Doing Surveys Sampling frame: NIH CRISP database of grants Lists about 250 in each year 482 unique grants in 2004-2006 Reviewed abstracts: Included those with mention of survey data collection Excluded methodological studies 145 grants selected for data collection

Incentives Web Survey Sent e-mail invitation to participate and three follow ups to Principal Investigator listed in CRISP Provided link to COPAFS and Incentives Conference web pages Promised copy of the paper as “incentive” Allowed PI or designated proxy to fill out the survey Asked PI to select one survey, the “most important” in terms of research goals Received 92 responses - 63%

Most Surveys Used Incentives Reason for no survey did not include anything to do with incentives.

Why Didn’t Surveys Use Incentives? Shorter surveys 40% under 15 minutes, nearly all under an hour Mean time to complete 27 minutes vs. 45 minutes Reasons: 60% expected good response without incentives 60% did not have budget for incentives 27% survey team did not want to pay incentives None reported IRB was a factor

How Did Surveys Use Incentives? Mix of expectations and timing 44% for completed survey 32% for partial complete 13% for considering participation 11% prepaid incentive About half the surveys included tasks other than the survey Half of those provided a separate payment

Kinds and Amounts of Incentives Kinds of Incentives Cash - 31% Gift cards or certificates - 27% Checks - 25% Amounts of monetary incentives $10 or less - 34% $20-49 - 48% $50 or more - 14% Other kinds of incentives Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc.

Why Are Incentives Used? 73% said to increase response rates was main reason Rated as main or very important reason: Reduce non-response bias - 71% Reward participants for research participation - 56% Not rated as important reasons: Reduce data collection time or follow up costs Help interviewers feel more comfortable IRB wanted incentives

What Kinds of Surveys Were These? Potentially Sensitive Topics Health status or health conditions - 74% Personal financial information - 44% Sexual behavior - 31% Drug use or drug use history - 25% Immigration status - 14% Special requests Linkage to other databases (e.g. Medicare) 7%

Modes Used With Incentives

How Were Amounts Determined? Based on open-ended comments: Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges Time and contribution of personal information Going rate for surveys of this kind of subject Accounting or safety issues for interviewers or subjects or budget issues as constraints Experiments to determine effective amounts

What Was the Role of IRBs? For paying incentives or not: Only one participant cited IRB preferences as a main reason for paying incentives 88% said IRB preferences were not important in making this decision For kind or amount of incentives: 23% said IRBs raised questions or placed limitations IRBs capped amount or required same incentive for all Limited how and when incentives could be mentioned

Conclusions At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50 75% of telephone surveys paid incentives, range was $15-20 Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests Most paid incentives to increase response rates IRBs were a factor, but not a major limitation